2024-05-11T22:50:46.020 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-11T22:50:46.020 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2024-05-11_21:16:03-rbd-squid-distro-default-smithi --verbose --archive /home/teuthworker/archive/teuthology-2024-05-11_21:16:03-rbd-squid-distro-default-smithi/7702761 --owner scheduled_teuthology@teuthology --description rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} -- /home/teuthworker/archive/teuthology-2024-05-11_21:16:03-rbd-squid-distro-default-smithi/7702761/orig.config.yaml 2024-05-11T22:50:46.053 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-11T22:50:46.134 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-11_21:16:03-rbd-squid-distro-default-smithi/7702761 branch: squid description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} email: ceph-qa@ceph.com first_in_suite: false job_id: '7702761' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-05-11_21:16:03-rbd-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: rbd cache: true rbd cache policy: writearound global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED sha1: f72fecff68e1d400c4568684327c900485c20d6a ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: f72fecff68e1d400c4568684327c900485c20d6a selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: f72fecff68e1d400c4568684327c900485c20d6a owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 2019 sha1: f72fecff68e1d400c4568684327c900485c20d6a sleep_before_teardown: 0 subset: 102/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_f72fecff68e1d400c4568684327c900485c20d6a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: f72fecff68e1d400c4568684327c900485c20d6a targets: smithi005.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL4A8qSqk2ZegQA45a0axYnvKI+qA8FsrZHVhlCcuJuOoj0AsoMSNbKCaR32V5DB7V0ndrcTj65u9zl4bADaEDg= smithi115.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLu1gWFUp+C2xGXhq1u5sOl299Z7d4k6HYoNw4jxI1WcmBbAExYoR60Y2NOBN0zOlA3BHeCiOfQtu5UmnRU5+aw= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-11_21:16:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-11T22:50:46.134 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_f72fecff68e1d400c4568684327c900485c20d6a/qa; will attempt to use it 2024-05-11T22:50:46.135 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_f72fecff68e1d400c4568684327c900485c20d6a/qa/tasks 2024-05-11T22:50:46.135 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-11T22:50:46.137 INFO:teuthology.task.internal:Checking packages... 2024-05-11T22:50:46.161 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'f72fecff68e1d400c4568684327c900485c20d6a' 2024-05-11T22:50:46.161 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-11T22:50:46.161 INFO:teuthology.packaging:ref: None 2024-05-11T22:50:46.161 INFO:teuthology.packaging:tag: None 2024-05-11T22:50:46.161 INFO:teuthology.packaging:branch: squid 2024-05-11T22:50:46.161 INFO:teuthology.packaging:sha1: f72fecff68e1d400c4568684327c900485c20d6a 2024-05-11T22:50:46.161 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-05-11T22:50:46.490 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2555.gf72fecff 2024-05-11T22:50:46.491 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-11T22:50:46.502 INFO:teuthology.task.internal:no buildpackages task found 2024-05-11T22:50:46.502 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-11T22:50:46.520 INFO:teuthology.task.internal:Saving configuration 2024-05-11T22:50:46.532 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-11T22:50:46.541 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-11T22:50:46.566 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-11_21:16:03-rbd-squid-distro-default-smithi/7702761', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-11 22:45:44.209860', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL4A8qSqk2ZegQA45a0axYnvKI+qA8FsrZHVhlCcuJuOoj0AsoMSNbKCaR32V5DB7V0ndrcTj65u9zl4bADaEDg='} 2024-05-11T22:50:46.585 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-11_21:16:03-rbd-squid-distro-default-smithi/7702761', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-11 22:45:44.208831', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLu1gWFUp+C2xGXhq1u5sOl299Z7d4k6HYoNw4jxI1WcmBbAExYoR60Y2NOBN0zOlA3BHeCiOfQtu5UmnRU5+aw='} 2024-05-11T22:50:46.585 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-11T22:50:46.597 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-05-11T22:50:46.597 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-05-11T22:50:46.597 INFO:teuthology.run_tasks:Running task console_log... 2024-05-11T22:50:46.678 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc78d2d0e50>, signals=[15]) 2024-05-11T22:50:46.678 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-11T22:50:46.690 INFO:teuthology.task.internal:Opening connections... 2024-05-11T22:50:46.690 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi005.front.sepia.ceph.com 2024-05-11T22:50:46.692 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T22:50:46.772 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2024-05-11T22:50:46.773 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T22:50:46.847 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-11T22:50:46.860 DEBUG:teuthology.orchestra.run.smithi005:> uname -m 2024-05-11T22:50:46.879 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2024-05-11T22:50:46.879 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2024-05-11T22:50:46.937 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="CentOS Stream" 2024-05-11T22:50:46.937 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="9" 2024-05-11T22:50:46.937 INFO:teuthology.orchestra.run.smithi005.stdout:ID="centos" 2024-05-11T22:50:46.937 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="rhel fedora" 2024-05-11T22:50:46.937 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="9" 2024-05-11T22:50:46.937 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el9" 2024-05-11T22:50:46.937 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-11T22:50:46.937 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31" 2024-05-11T22:50:46.937 INFO:teuthology.orchestra.run.smithi005.stdout:LOGO="fedora-logo-icon" 2024-05-11T22:50:46.937 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-11T22:50:46.937 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://centos.org/" 2024-05-11T22:50:46.938 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-11T22:50:46.938 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-11T22:50:46.938 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-11T22:50:46.938 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2024-05-11T22:50:46.961 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2024-05-11T22:50:46.979 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2024-05-11T22:50:46.980 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2024-05-11T22:50:47.037 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="CentOS Stream" 2024-05-11T22:50:47.037 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="9" 2024-05-11T22:50:47.037 INFO:teuthology.orchestra.run.smithi115.stdout:ID="centos" 2024-05-11T22:50:47.037 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="rhel fedora" 2024-05-11T22:50:47.037 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="9" 2024-05-11T22:50:47.037 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el9" 2024-05-11T22:50:47.037 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-11T22:50:47.037 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31" 2024-05-11T22:50:47.037 INFO:teuthology.orchestra.run.smithi115.stdout:LOGO="fedora-logo-icon" 2024-05-11T22:50:47.038 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-11T22:50:47.038 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://centos.org/" 2024-05-11T22:50:47.038 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-11T22:50:47.038 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-11T22:50:47.038 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-11T22:50:47.038 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2024-05-11T22:50:47.061 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-11T22:50:47.074 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-11T22:50:47.086 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-11T22:50:47.087 DEBUG:teuthology.orchestra.run.smithi005:> test '!' -e /home/ubuntu/cephtest 2024-05-11T22:50:47.089 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2024-05-11T22:50:47.105 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-11T22:50:47.113 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-11T22:50:47.113 DEBUG:teuthology.orchestra.run.smithi005:> test -z $(ls -A /var/lib/ceph) 2024-05-11T22:50:47.147 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2024-05-11T22:50:47.170 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-11T22:50:47.231 INFO:teuthology.run_tasks:Running task kernel... 2024-05-11T22:50:47.249 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-11T22:50:47.249 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-11T22:50:47.249 DEBUG:teuthology.orchestra.run.smithi005:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-11T22:50:47.250 DEBUG:teuthology.orchestra.run.smithi115:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-11T22:50:47.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T22:50:47.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T22:50:47.267 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-05-11T22:50:47.268 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-05-11T22:50:47.324 INFO:teuthology.orchestra.run.smithi115.stdout:5.14.0-437.el9.x86_64 2024-05-11T22:50:47.324 INFO:teuthology.task.kernel:Running kernel on smithi115: 5.14.0-437.el9.x86_64 2024-05-11T22:50:47.324 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum install -y kernel 2024-05-11T22:50:47.325 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-437.el9.x86_64 2024-05-11T22:50:47.326 INFO:teuthology.task.kernel:Running kernel on smithi005: 5.14.0-437.el9.x86_64 2024-05-11T22:50:47.326 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum install -y kernel 2024-05-11T22:50:49.814 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 9.8 MB/s | 8.1 MB 00:00 2024-05-11T22:50:50.750 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.1 MB 00:01 2024-05-11T22:50:51.856 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-05-11T22:50:52.467 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 31 MB/s | 19 MB 00:00 2024-05-11T22:50:56.701 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-05-11T22:50:57.465 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-05-11T22:50:58.285 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 16 kB 00:00 2024-05-11T22:50:58.932 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 16 kB 00:00 2024-05-11T22:50:59.356 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-05-11T22:51:00.095 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:01 2024-05-11T22:51:04.436 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-11T22:51:05.502 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-05-11T22:51:06.425 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-11T22:51:06.425 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-11T22:51:06.525 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T22:51:06.532 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:51:06.532 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-05-11T22:51:06.532 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:51:06.532 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-05-11T22:51:06.533 INFO:teuthology.orchestra.run.smithi115.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-11T22:51:06.533 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-05-11T22:51:06.533 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-11T22:51:06.533 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-11T22:51:06.533 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-11T22:51:06.533 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:51:06.533 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-05-11T22:51:06.533 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:51:06.533 INFO:teuthology.orchestra.run.smithi115.stdout:Install 4 Packages 2024-05-11T22:51:06.533 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:51:06.534 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 78 M 2024-05-11T22:51:06.534 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 123 M 2024-05-11T22:51:06.534 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-05-11T22:51:06.729 INFO:teuthology.orchestra.run.smithi115.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 3.0 MB/s | 484 kB 00:00 2024-05-11T22:51:07.121 INFO:teuthology.orchestra.run.smithi115.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 28 MB/s | 15 MB 00:00 2024-05-11T22:51:07.607 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-11T22:51:07.607 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-11T22:51:07.702 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T22:51:07.707 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:51:07.708 INFO:teuthology.orchestra.run.smithi005.stdout: Package Architecture Version Repository Size 2024-05-11T22:51:07.708 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:51:07.708 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-11T22:51:07.708 INFO:teuthology.orchestra.run.smithi005.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-11T22:51:07.708 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-05-11T22:51:07.708 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-11T22:51:07.708 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-11T22:51:07.708 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-11T22:51:07.708 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:51:07.708 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-11T22:51:07.708 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:51:07.708 INFO:teuthology.orchestra.run.smithi005.stdout:Install 4 Packages 2024-05-11T22:51:07.709 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:51:07.709 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 78 M 2024-05-11T22:51:07.709 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 123 M 2024-05-11T22:51:07.709 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-11T22:51:07.763 INFO:teuthology.orchestra.run.smithi115.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 28 MB/s | 34 MB 00:01 2024-05-11T22:51:08.027 INFO:teuthology.orchestra.run.smithi005.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 2.5 MB/s | 484 kB 00:00 2024-05-11T22:51:08.146 INFO:teuthology.orchestra.run.smithi115.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 20 MB/s | 28 MB 00:01 2024-05-11T22:51:08.146 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:51:08.146 INFO:teuthology.orchestra.run.smithi115.stdout:Total 48 MB/s | 78 MB 00:01 2024-05-11T22:51:08.535 INFO:teuthology.orchestra.run.smithi005.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 22 MB/s | 15 MB 00:00 2024-05-11T22:51:08.829 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-05-11T22:51:09.144 INFO:teuthology.orchestra.run.smithi005.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 26 MB/s | 34 MB 00:01 2024-05-11T22:51:09.162 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-05-11T22:51:09.162 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-05-11T22:51:09.427 INFO:teuthology.orchestra.run.smithi005.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 20 MB/s | 28 MB 00:01 2024-05-11T22:51:09.428 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:51:09.428 INFO:teuthology.orchestra.run.smithi005.stdout:Total 45 MB/s | 78 MB 00:01 2024-05-11T22:51:10.064 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-05-11T22:51:10.064 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-05-11T22:51:10.133 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-11T22:51:10.544 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-11T22:51:10.544 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-11T22:51:11.481 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-11T22:51:11.482 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-11T22:51:12.447 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-05-11T22:51:13.110 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-11T22:51:13.196 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T22:51:14.022 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-11T22:51:14.693 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-11T22:51:14.786 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T22:51:15.873 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T22:51:16.000 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T22:51:17.564 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T22:51:17.713 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T22:51:20.718 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T22:51:20.760 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-11T22:51:22.791 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T22:51:22.832 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-11T22:51:25.491 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T22:51:27.929 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T22:51:54.654 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T22:51:54.691 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-11T22:51:55.002 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-11T22:51:55.002 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-11T22:51:55.002 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T22:51:55.002 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T22:51:55.815 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T22:51:55.815 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:51:55.815 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-05-11T22:51:55.815 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-11T22:51:55.815 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-11T22:51:55.815 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-11T22:51:55.815 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-11T22:51:55.816 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:51:55.816 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T22:51:56.156 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q kernel | sort -rV | head -n 1 2024-05-11T22:51:56.246 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-11T22:51:56.246 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-11T22:51:56.247 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-11T22:51:56.247 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-11T22:51:56.247 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-11T22:51:56.247 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-11T22:51:56.247 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi115.front.sepia.ceph.com, path=None, version=distro) 2024-05-11T22:51:56.247 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q kernel | sort -rV | head -n 1 2024-05-11T22:51:56.296 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-11T22:51:56.296 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-11T22:51:56.297 DEBUG:teuthology.orchestra.run.smithi115:> sudo rpm -qi grub2-tools 2024-05-11T22:51:56.326 INFO:teuthology.orchestra.run.smithi115.stdout:Name : grub2-tools 2024-05-11T22:51:56.326 INFO:teuthology.orchestra.run.smithi115.stdout:Epoch : 1 2024-05-11T22:51:56.326 INFO:teuthology.orchestra.run.smithi115.stdout:Version : 2.06 2024-05-11T22:51:56.327 INFO:teuthology.orchestra.run.smithi115.stdout:Release : 68.el9 2024-05-11T22:51:56.327 INFO:teuthology.orchestra.run.smithi115.stdout:Architecture: x86_64 2024-05-11T22:51:56.327 INFO:teuthology.orchestra.run.smithi115.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-11T22:51:56.327 INFO:teuthology.orchestra.run.smithi115.stdout:Group : Unspecified 2024-05-11T22:51:56.327 INFO:teuthology.orchestra.run.smithi115.stdout:Size : 8264254 2024-05-11T22:51:56.327 INFO:teuthology.orchestra.run.smithi115.stdout:License : GPLv3+ 2024-05-11T22:51:56.327 INFO:teuthology.orchestra.run.smithi115.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-11T22:51:56.327 INFO:teuthology.orchestra.run.smithi115.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-11T22:51:56.327 INFO:teuthology.orchestra.run.smithi115.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-11T22:51:56.327 INFO:teuthology.orchestra.run.smithi115.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-11T22:51:56.327 INFO:teuthology.orchestra.run.smithi115.stdout:Packager : builder@centos.org 2024-05-11T22:51:56.327 INFO:teuthology.orchestra.run.smithi115.stdout:Vendor : CentOS 2024-05-11T22:51:56.327 INFO:teuthology.orchestra.run.smithi115.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-11T22:51:56.327 INFO:teuthology.orchestra.run.smithi115.stdout:Summary : Support tools for GRUB. 2024-05-11T22:51:56.328 INFO:teuthology.orchestra.run.smithi115.stdout:Description : 2024-05-11T22:51:56.328 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:51:56.328 INFO:teuthology.orchestra.run.smithi115.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-11T22:51:56.328 INFO:teuthology.orchestra.run.smithi115.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-11T22:51:56.328 INFO:teuthology.orchestra.run.smithi115.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-11T22:51:56.328 INFO:teuthology.orchestra.run.smithi115.stdout:hardware devices. 2024-05-11T22:51:56.328 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:51:56.328 INFO:teuthology.orchestra.run.smithi115.stdout:This subpackage provides tools for support of all platforms. 2024-05-11T22:51:56.329 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-11T22:51:56.329 INFO:teuthology.task.kernel:Updating grub on smithi115 to boot 5.14.0-444.el9.x86_64 2024-05-11T22:51:56.329 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-11T22:51:56.946 INFO:teuthology.orchestra.run.smithi115.stderr:Generating grub configuration file ... 2024-05-11T22:51:58.405 INFO:teuthology.orchestra.run.smithi115.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-11T22:51:58.443 INFO:teuthology.orchestra.run.smithi115.stderr:done 2024-05-11T22:51:58.445 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-05-11T22:51:58.461 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.rXzprc4j63 2024-05-11T22:51:58.461 DEBUG:teuthology.orchestra.run.smithi115:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.rXzprc4j63 2024-05-11T22:51:58.526 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.rXzprc4j63 2024-05-11T22:51:58.663 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.rXzprc4j63 is 6KB 2024-05-11T22:51:58.712 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.rXzprc4j63 2024-05-11T22:51:58.726 DEBUG:teuthology.orchestra.run.smithi115:> sudo /bin/ls /boot/loader/entries || true 2024-05-11T22:51:58.788 INFO:teuthology.orchestra.run.smithi115.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-11T22:51:58.788 INFO:teuthology.orchestra.run.smithi115.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-11T22:51:58.788 INFO:teuthology.orchestra.run.smithi115.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-11T22:51:58.788 INFO:teuthology.orchestra.run.smithi115.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-11T22:51:58.789 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-11T22:51:58.894 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T22:51:58.935 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-11T22:51:58.937 DEBUG:teuthology.orchestra.run.smithi115:> sudo shutdown -r now 2024-05-11T22:51:59.252 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-11T22:51:59.253 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-11T22:51:59.253 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T22:51:59.253 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T22:52:00.051 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T22:52:00.051 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:52:00.051 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-11T22:52:00.051 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-11T22:52:00.051 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-11T22:52:00.051 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-11T22:52:00.051 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-11T22:52:00.051 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:52:00.051 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T22:52:00.409 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q kernel | sort -rV | head -n 1 2024-05-11T22:52:00.483 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-11T22:52:00.483 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-11T22:52:00.483 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-11T22:52:00.483 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-11T22:52:00.483 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-11T22:52:00.483 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-11T22:52:00.483 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi005.front.sepia.ceph.com, path=None, version=distro) 2024-05-11T22:52:00.483 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q kernel | sort -rV | head -n 1 2024-05-11T22:52:00.534 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-11T22:52:00.535 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-11T22:52:00.535 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -qi grub2-tools 2024-05-11T22:52:00.566 INFO:teuthology.orchestra.run.smithi005.stdout:Name : grub2-tools 2024-05-11T22:52:00.566 INFO:teuthology.orchestra.run.smithi005.stdout:Epoch : 1 2024-05-11T22:52:00.566 INFO:teuthology.orchestra.run.smithi005.stdout:Version : 2.06 2024-05-11T22:52:00.566 INFO:teuthology.orchestra.run.smithi005.stdout:Release : 68.el9 2024-05-11T22:52:00.566 INFO:teuthology.orchestra.run.smithi005.stdout:Architecture: x86_64 2024-05-11T22:52:00.566 INFO:teuthology.orchestra.run.smithi005.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-11T22:52:00.567 INFO:teuthology.orchestra.run.smithi005.stdout:Group : Unspecified 2024-05-11T22:52:00.567 INFO:teuthology.orchestra.run.smithi005.stdout:Size : 8264254 2024-05-11T22:52:00.567 INFO:teuthology.orchestra.run.smithi005.stdout:License : GPLv3+ 2024-05-11T22:52:00.567 INFO:teuthology.orchestra.run.smithi005.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-11T22:52:00.567 INFO:teuthology.orchestra.run.smithi005.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-11T22:52:00.567 INFO:teuthology.orchestra.run.smithi005.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-11T22:52:00.567 INFO:teuthology.orchestra.run.smithi005.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-11T22:52:00.567 INFO:teuthology.orchestra.run.smithi005.stdout:Packager : builder@centos.org 2024-05-11T22:52:00.567 INFO:teuthology.orchestra.run.smithi005.stdout:Vendor : CentOS 2024-05-11T22:52:00.567 INFO:teuthology.orchestra.run.smithi005.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-11T22:52:00.567 INFO:teuthology.orchestra.run.smithi005.stdout:Summary : Support tools for GRUB. 2024-05-11T22:52:00.567 INFO:teuthology.orchestra.run.smithi005.stdout:Description : 2024-05-11T22:52:00.567 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:52:00.567 INFO:teuthology.orchestra.run.smithi005.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-11T22:52:00.568 INFO:teuthology.orchestra.run.smithi005.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-11T22:52:00.568 INFO:teuthology.orchestra.run.smithi005.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-11T22:52:00.568 INFO:teuthology.orchestra.run.smithi005.stdout:hardware devices. 2024-05-11T22:52:00.568 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:52:00.568 INFO:teuthology.orchestra.run.smithi005.stdout:This subpackage provides tools for support of all platforms. 2024-05-11T22:52:00.569 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-11T22:52:00.569 INFO:teuthology.task.kernel:Updating grub on smithi005 to boot 5.14.0-444.el9.x86_64 2024-05-11T22:52:00.569 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-11T22:52:01.181 INFO:teuthology.orchestra.run.smithi005.stderr:Generating grub configuration file ... 2024-05-11T22:52:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-11T22:52:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:done 2024-05-11T22:52:02.604 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-05-11T22:52:02.618 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.yMJdT07stw 2024-05-11T22:52:02.619 DEBUG:teuthology.orchestra.run.smithi005:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.yMJdT07stw 2024-05-11T22:52:02.698 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.yMJdT07stw 2024-05-11T22:52:02.838 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.yMJdT07stw is 6KB 2024-05-11T22:52:02.886 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.yMJdT07stw 2024-05-11T22:52:02.901 DEBUG:teuthology.orchestra.run.smithi005:> sudo /bin/ls /boot/loader/entries || true 2024-05-11T22:52:02.963 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-11T22:52:02.963 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-11T22:52:02.963 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-11T22:52:02.964 INFO:teuthology.orchestra.run.smithi005.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-11T22:52:02.965 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-11T22:52:03.118 DEBUG:teuthology.orchestra.run.smithi005:> sudo shutdown -r now 2024-05-11T22:52:28.965 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-11T22:52:28.965 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-05-11T22:52:28.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T22:52:33.125 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-11T22:52:33.125 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-11T22:52:33.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T22:52:47.519 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-05-11T22:52:56.526 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-05-11T22:52:56.527 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T22:52:59.586 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-05-11T22:53:11.597 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-05-11T22:53:11.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T22:53:30.014 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-05-11T22:53:33.130 DEBUG:teuthology.orchestra.remote:timed out 2024-05-11T22:53:42.133 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-11T22:53:42.134 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T22:53:42.549 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-05-11T22:53:42.917 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-11T22:53:42.917 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-444.el9.x86_64"... 2024-05-11T22:53:42.917 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-05-11T22:53:42.932 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-444.el9.x86_64 2024-05-11T22:53:42.933 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-11T22:53:42.933 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-11T22:53:42.933 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-11T22:53:43.934 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-11T22:53:43.934 DEBUG:teuthology.orchestra.run.smithi005:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-11T22:53:44.005 INFO:teuthology.orchestra.run.smithi005.stdout:ttyS1 2024-05-11T22:53:44.044 DEBUG:teuthology.parallel:result is None 2024-05-11T22:53:45.015 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-05-11T22:53:45.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T22:53:45.774 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-05-11T22:53:46.169 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-05-11T22:53:46.169 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-444.el9.x86_64"... 2024-05-11T22:53:46.169 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-05-11T22:53:46.184 INFO:teuthology.orchestra.run.smithi115.stdout:5.14.0-444.el9.x86_64 2024-05-11T22:53:46.184 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-11T22:53:46.184 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-11T22:53:46.184 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-11T22:53:47.185 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-11T22:53:47.185 DEBUG:teuthology.orchestra.run.smithi115:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-11T22:53:47.274 INFO:teuthology.orchestra.run.smithi115.stdout:ttyS1 2024-05-11T22:53:47.325 DEBUG:teuthology.parallel:result is None 2024-05-11T22:53:47.325 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-11T22:53:47.334 INFO:teuthology.task.internal:Creating test directory... 2024-05-11T22:53:47.334 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-11T22:53:47.337 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-11T22:53:47.363 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-11T22:53:47.369 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-11T22:53:47.379 INFO:teuthology.task.internal:Creating archive directory... 2024-05-11T22:53:47.379 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-11T22:53:47.406 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-11T22:53:47.437 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-11T22:53:47.445 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-11T22:53:47.446 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-11T22:53:47.475 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-11T22:53:47.529 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T22:53:47.538 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T22:53:47.547 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T22:53:47.549 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T22:53:47.550 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-11T22:53:47.557 INFO:teuthology.task.internal:Configuring sudo... 2024-05-11T22:53:47.557 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-11T22:53:47.560 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-11T22:53:47.603 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-11T22:53:47.612 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-11T22:53:47.612 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-11T22:53:47.615 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-11T22:53:47.640 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T22:53:47.723 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T22:53:47.772 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T22:53:47.847 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T22:53:47.896 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T22:53:47.896 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-11T22:53:47.963 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T22:53:48.007 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T22:53:48.052 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T22:53:48.135 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T22:53:48.174 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-11T22:53:48.174 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-11T22:53:48.251 DEBUG:teuthology.orchestra.run.smithi005:> sudo service rsyslog restart 2024-05-11T22:53:48.253 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2024-05-11T22:53:48.331 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T22:53:48.336 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T22:53:48.687 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-11T22:53:48.695 INFO:teuthology.task.internal:Starting timer... 2024-05-11T22:53:48.695 INFO:teuthology.run_tasks:Running task pcp... 2024-05-11T22:53:48.705 INFO:teuthology.run_tasks:Running task selinux... 2024-05-11T22:53:48.712 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-11T22:53:48.713 DEBUG:teuthology.orchestra.run.smithi005:> sudo service auditd rotate 2024-05-11T22:53:48.793 INFO:teuthology.orchestra.run.smithi005.stdout:Rotating logs: 2024-05-11T22:53:48.795 DEBUG:teuthology.orchestra.run.smithi115:> sudo service auditd rotate 2024-05-11T22:53:48.846 INFO:teuthology.orchestra.run.smithi115.stdout:Rotating logs: 2024-05-11T22:53:48.848 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-11T22:53:48.848 DEBUG:teuthology.orchestra.run.smithi005:> /usr/sbin/getenforce 2024-05-11T22:53:48.887 INFO:teuthology.orchestra.run.smithi005.stdout:Permissive 2024-05-11T22:53:48.888 DEBUG:teuthology.orchestra.run.smithi115:> /usr/sbin/getenforce 2024-05-11T22:53:48.917 INFO:teuthology.orchestra.run.smithi115.stdout:Permissive 2024-05-11T22:53:48.918 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi005.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi115.front.sepia.ceph.com': 'permissive'} 2024-05-11T22:53:48.918 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T22:53:48.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T22:53:48.954 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T22:53:48.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T22:53:48.984 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-11T22:53:48.985 DEBUG:teuthology.orchestra.run.smithi005:> sudo /usr/sbin/setenforce permissive 2024-05-11T22:53:49.020 DEBUG:teuthology.orchestra.run.smithi115:> sudo /usr/sbin/setenforce permissive 2024-05-11T22:53:49.049 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-11T22:53:49.064 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-11T22:53:49.081 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-11T22:53:49.083 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi005.front.sepia.ceph.com,smithi115.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-11T22:57:50.797 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi005.front.sepia.ceph.com'), Remote(name='ubuntu@smithi115.front.sepia.ceph.com')] 2024-05-11T22:57:50.798 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-11T22:57:50.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T22:57:50.882 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-05-11T22:57:50.970 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-05-11T22:57:50.970 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-05-11T22:57:50.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T22:57:51.050 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-05-11T22:57:51.127 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-05-11T22:57:51.127 INFO:teuthology.run_tasks:Running task clock... 2024-05-11T22:57:51.139 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-11T22:57:51.139 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-11T22:57:51.139 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T22:57:51.142 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-11T22:57:51.142 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T22:57:51.170 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-11T22:57:51.186 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-11T22:57:51.198 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-11T22:57:51.211 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: ntpd: command not found 2024-05-11T22:57:51.213 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-11T22:57:51.223 INFO:teuthology.orchestra.run.smithi005.stdout:506 Cannot talk to daemon 2024-05-11T22:57:51.239 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-11T22:57:51.239 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: ntpd: command not found 2024-05-11T22:57:51.251 INFO:teuthology.orchestra.run.smithi115.stdout:506 Cannot talk to daemon 2024-05-11T22:57:51.252 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-11T22:57:51.264 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-11T22:57:51.278 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-11T22:57:51.311 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-05-11T22:57:51.315 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T22:57:51.315 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-05-11T22:57:51.315 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T22:57:51.315 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T22:57:51.315 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T22:57:51.315 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T22:57:51.332 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-05-11T22:57:51.336 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T22:57:51.336 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-05-11T22:57:51.336 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T22:57:51.336 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T22:57:51.336 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T22:57:51.336 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T22:57:51.337 INFO:teuthology.run_tasks:Running task install... 2024-05-11T22:57:51.346 DEBUG:teuthology.task.install:project ceph 2024-05-11T22:57:51.346 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f72fecff68e1d400c4568684327c900485c20d6a'}} 2024-05-11T22:57:51.346 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'f72fecff68e1d400c4568684327c900485c20d6a'} 2024-05-11T22:57:51.346 INFO:teuthology.task.install:Using flavor: default 2024-05-11T22:57:51.352 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-11T22:57:51.352 INFO:teuthology.task.install:extra packages: [] 2024-05-11T22:57:51.352 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f72fecff68e1d400c4568684327c900485c20d6a', 'tag': None, 'wait_for_package': False} 2024-05-11T22:57:51.353 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f72fecff68e1d400c4568684327c900485c20d6a 2024-05-11T22:57:51.355 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f72fecff68e1d400c4568684327c900485c20d6a', 'tag': None, 'wait_for_package': False} 2024-05-11T22:57:51.355 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f72fecff68e1d400c4568684327c900485c20d6a 2024-05-11T22:57:51.501 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/f72fecff68e1d400c4568684327c900485c20d6a/centos/9/flavors/default/ 2024-05-11T22:57:51.501 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2555.gf72fecff 2024-05-11T22:57:51.509 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/f72fecff68e1d400c4568684327c900485c20d6a/centos/9/flavors/default/ 2024-05-11T22:57:51.509 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2555.gf72fecff 2024-05-11T22:57:51.616 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/f72fecff68e1d400c4568684327c900485c20d6a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/f72fecff68e1d400c4568684327c900485c20d6a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/f72fecff68e1d400c4568684327c900485c20d6a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-11T22:57:51.616 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T22:57:51.616 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-11T22:57:51.644 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-11T22:57:51.644 DEBUG:teuthology.orchestra.run.smithi005:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f72fecff68e1d400c4568684327c900485c20d6a/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-11T22:57:51.682 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/f72fecff68e1d400c4568684327c900485c20d6a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/f72fecff68e1d400c4568684327c900485c20d6a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/f72fecff68e1d400c4568684327c900485c20d6a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-11T22:57:51.682 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-11T22:57:51.682 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-11T22:57:51.709 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-11T22:57:51.710 DEBUG:teuthology.orchestra.run.smithi115:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f72fecff68e1d400c4568684327c900485c20d6a/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-11T22:57:51.713 DEBUG:teuthology.orchestra.run.smithi005:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-11T22:57:51.775 DEBUG:teuthology.orchestra.run.smithi115:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-11T22:57:51.789 DEBUG:teuthology.orchestra.run.smithi005:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-11T22:57:51.850 DEBUG:teuthology.orchestra.run.smithi115:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-11T22:57:51.855 INFO:teuthology.orchestra.run.smithi005.stdout:check_obsoletes = 1 2024-05-11T22:57:51.857 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum clean all 2024-05-11T22:57:51.915 INFO:teuthology.orchestra.run.smithi115.stdout:check_obsoletes = 1 2024-05-11T22:57:51.916 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum clean all 2024-05-11T22:57:52.103 INFO:teuthology.orchestra.run.smithi005.stdout:54 files removed 2024-05-11T22:57:52.131 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-radosgw 2024-05-11T22:57:52.144 INFO:teuthology.orchestra.run.smithi115.stdout:54 files removed 2024-05-11T22:57:52.171 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-radosgw 2024-05-11T22:57:52.797 INFO:teuthology.orchestra.run.smithi115.stdout:ceph packages for x86_64 236 kB/s | 83 kB 00:00 2024-05-11T22:57:52.845 INFO:teuthology.orchestra.run.smithi005.stdout:ceph packages for x86_64 196 kB/s | 83 kB 00:00 2024-05-11T22:57:53.181 INFO:teuthology.orchestra.run.smithi115.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-05-11T22:57:53.279 INFO:teuthology.orchestra.run.smithi005.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-05-11T22:57:53.515 INFO:teuthology.orchestra.run.smithi115.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-05-11T22:57:53.696 INFO:teuthology.orchestra.run.smithi005.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-05-11T22:57:54.040 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.1 MB 00:00 2024-05-11T22:57:54.138 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 8.1 MB 00:00 2024-05-11T22:57:56.016 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-05-11T22:57:56.030 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-05-11T22:57:59.992 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.2 MB 00:00 2024-05-11T22:58:00.698 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 6.5 MB/s | 6.2 MB 00:00 2024-05-11T22:58:01.651 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-05-11T22:58:02.199 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 16 kB 00:00 2024-05-11T22:58:08.134 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 3.6 MB/s | 21 MB 00:05 2024-05-11T22:58:13.692 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-05-11T22:58:16.153 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T22:58:16.162 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:58:16.162 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-05-11T22:58:16.162 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:58:16.162 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-11T22:58:16.162 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 10 M 2024-05-11T22:58:16.162 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrading: 2024-05-11T22:58:16.162 INFO:teuthology.orchestra.run.smithi005.stdout: librados2 x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 3.4 M 2024-05-11T22:58:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1 x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 3.1 M 2024-05-11T22:58:16.163 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-05-11T22:58:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-11T22:58:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 5.4 M 2024-05-11T22:58:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 21 M 2024-05-11T22:58:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 25 k 2024-05-11T22:58:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-11T22:58:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-11T22:58:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-11T22:58:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2 x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 745 k 2024-05-11T22:58:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-11T22:58:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-11T22:58:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-11T22:58:16.163 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1 x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 500 k 2024-05-11T22:58:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-11T22:58:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2 x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 5.3 M 2024-05-11T22:58:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-11T22:58:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-11T22:58:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-11T22:58:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 45 k 2024-05-11T22:58:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 135 k 2024-05-11T22:58:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 161 k 2024-05-11T22:58:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-11T22:58:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 321 k 2024-05-11T22:58:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 301 k 2024-05-11T22:58:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 99 k 2024-05-11T22:58:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-11T22:58:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-11T22:58:16.164 INFO:teuthology.orchestra.run.smithi005.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-11T22:58:16.165 INFO:teuthology.orchestra.run.smithi005.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-11T22:58:16.165 INFO:teuthology.orchestra.run.smithi005.stdout:Installing weak dependencies: 2024-05-11T22:58:16.165 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-11T22:58:16.165 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:58:16.165 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-11T22:58:16.165 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:58:16.165 INFO:teuthology.orchestra.run.smithi005.stdout:Install 30 Packages 2024-05-11T22:58:16.165 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrade 2 Packages 2024-05-11T22:58:16.165 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:58:16.165 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 60 M 2024-05-11T22:58:16.166 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-11T22:58:17.443 INFO:teuthology.orchestra.run.smithi005.stdout:(1/32): ceph-radosgw-19.0.0-2555.gf72fecff.el9. 8.6 MB/s | 10 MB 00:01 2024-05-11T22:58:17.502 INFO:teuthology.orchestra.run.smithi005.stdout:(2/32): ceph-selinux-19.0.0-2555.gf72fecff.el9. 432 kB/s | 25 kB 00:00 2024-05-11T22:58:17.644 INFO:teuthology.orchestra.run.smithi005.stdout:(3/32): libcephfs2-19.0.0-2555.gf72fecff.el9.x8 5.1 MB/s | 745 kB 00:00 2024-05-11T22:58:17.727 INFO:teuthology.orchestra.run.smithi005.stdout:(4/32): libradosstriper1-19.0.0-2555.gf72fecff. 5.9 MB/s | 500 kB 00:00 2024-05-11T22:58:17.944 INFO:teuthology.orchestra.run.smithi005.stdout:(5/32): ceph-common-19.0.0-2555.gf72fecff.el9.x 12 MB/s | 21 MB 00:01 2024-05-11T22:58:18.003 INFO:teuthology.orchestra.run.smithi005.stdout:(6/32): python3-ceph-argparse-19.0.0-2555.gf72f 774 kB/s | 45 kB 00:00 2024-05-11T22:58:18.111 INFO:teuthology.orchestra.run.smithi005.stdout:(7/32): ceph-base-19.0.0-2555.gf72fecff.el9.x86 2.9 MB/s | 5.4 MB 00:01 2024-05-11T22:58:18.153 INFO:teuthology.orchestra.run.smithi005.stdout:(8/32): python3-ceph-common-19.0.0-2555.gf72fec 900 kB/s | 135 kB 00:00 2024-05-11T22:58:18.179 INFO:teuthology.orchestra.run.smithi005.stdout:(9/32): python3-cephfs-19.0.0-2555.gf72fecff.el 2.4 MB/s | 161 kB 00:00 2024-05-11T22:58:18.204 INFO:teuthology.orchestra.run.smithi005.stdout:(10/32): python3-rados-19.0.0-2555.gf72fecff.el 6.2 MB/s | 321 kB 00:00 2024-05-11T22:58:18.279 INFO:teuthology.orchestra.run.smithi005.stdout:(11/32): librgw2-19.0.0-2555.gf72fecff.el9.x86_ 9.6 MB/s | 5.3 MB 00:00 2024-05-11T22:58:18.304 INFO:teuthology.orchestra.run.smithi005.stdout:(12/32): python3-rgw-19.0.0-2555.gf72fecff.el9. 991 kB/s | 99 kB 00:00 2024-05-11T22:58:18.330 INFO:teuthology.orchestra.run.smithi005.stdout:(13/32): python3-rbd-19.0.0-2555.gf72fecff.el9. 2.0 MB/s | 301 kB 00:00 2024-05-11T22:58:18.405 INFO:teuthology.orchestra.run.smithi005.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 852 kB/s | 107 kB 00:00 2024-05-11T22:58:18.447 INFO:teuthology.orchestra.run.smithi005.stdout:(15/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-05-11T22:58:18.481 INFO:teuthology.orchestra.run.smithi005.stdout:(16/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 300 kB/s | 45 kB 00:00 2024-05-11T22:58:18.531 INFO:teuthology.orchestra.run.smithi005.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 847 kB/s | 42 kB 00:00 2024-05-11T22:58:18.565 INFO:teuthology.orchestra.run.smithi005.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-05-11T22:58:18.590 INFO:teuthology.orchestra.run.smithi005.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-05-11T22:58:18.615 INFO:teuthology.orchestra.run.smithi005.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-05-11T22:58:18.657 INFO:teuthology.orchestra.run.smithi005.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 389 kB/s | 36 kB 00:00 2024-05-11T22:58:18.682 INFO:teuthology.orchestra.run.smithi005.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 514 kB/s | 47 kB 00:00 2024-05-11T22:58:19.033 INFO:teuthology.orchestra.run.smithi005.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 74 kB/s | 26 kB 00:00 2024-05-11T22:58:19.316 INFO:teuthology.orchestra.run.smithi005.stdout:(24/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 441 kB/s | 309 kB 00:00 2024-05-11T22:58:19.333 INFO:teuthology.orchestra.run.smithi005.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 162 kB/s | 49 kB 00:00 2024-05-11T22:58:19.425 INFO:teuthology.orchestra.run.smithi005.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 624 kB/s | 67 kB 00:00 2024-05-11T22:58:19.592 INFO:teuthology.orchestra.run.smithi005.stdout:(27/32): libarrow-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 4.4 MB 00:00 2024-05-11T22:58:19.642 INFO:teuthology.orchestra.run.smithi005.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.7 MB/s | 839 kB 00:00 2024-05-11T22:58:19.668 INFO:teuthology.orchestra.run.smithi005.stdout:(29/32): re2-20211101-3.el9.x86_64.rpm 791 kB/s | 192 kB 00:00 2024-05-11T22:58:19.960 INFO:teuthology.orchestra.run.smithi005.stdout:(30/32): librbd1-19.0.0-2555.gf72fecff.el9.x86_ 11 MB/s | 3.1 MB 00:00 2024-05-11T22:58:20.118 INFO:teuthology.orchestra.run.smithi005.stdout:(31/32): thrift-0.15.0-2.el9.x86_64.rpm 3.0 MB/s | 1.6 MB 00:00 2024-05-11T22:58:20.243 INFO:teuthology.orchestra.run.smithi005.stdout:(32/32): librados2-19.0.0-2555.gf72fecff.el9.x8 5.6 MB/s | 3.4 MB 00:00 2024-05-11T22:58:20.248 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:58:20.248 INFO:teuthology.orchestra.run.smithi005.stdout:Total 15 MB/s | 60 MB 00:04 2024-05-11T22:58:20.355 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-11T22:58:20.396 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-11T22:58:20.396 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-11T22:58:20.469 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:18 2024-05-11T22:58:21.057 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-11T22:58:21.058 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-11T22:58:22.504 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-11T22:58:22.579 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-05-11T22:58:22.607 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-05-11T22:58:22.640 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-05-11T22:58:22.663 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-11T22:58:22.686 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-11T22:58:22.866 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-11T22:58:22.920 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-11T22:58:23.313 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-05-11T22:58:23.367 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librados2-2:19.0.0-2555.gf72fecff.el9.x86_64 7/34 2024-05-11T22:58:23.443 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:19.0.0-2555.gf72fecff.el9.x86_64 7/34 2024-05-11T22:58:23.533 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rados-2:19.0.0-2555.gf72fecff.el9.x86_64 8/34 2024-05-11T22:58:23.560 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs2-2:19.0.0-2555.gf72fecff.el9.x86_64 9/34 2024-05-11T22:58:23.606 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephfs2-2:19.0.0-2555.gf72fecff.el9.x86_64 9/34 2024-05-11T22:58:23.635 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-05-11T22:58:23.679 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-argparse-2:19.0.0-2555.gf72fecff.el9. 11/34 2024-05-11T22:58:23.757 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cephfs-2:19.0.0-2555.gf72fecff.el9.x86_64 12/34 2024-05-11T22:58:23.787 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libradosstriper1-2:19.0.0-2555.gf72fecff.el9.x86_6 13/34 2024-05-11T22:58:23.835 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2555.gf72fecff.el9.x86_6 13/34 2024-05-11T22:58:23.870 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-05-11T22:58:23.980 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-05-11T22:58:24.012 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-05-11T22:58:24.053 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-05-11T22:58:24.085 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-05-11T22:58:24.114 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-05-11T22:58:24.147 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-05-11T22:58:24.471 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-05-11T22:58:24.518 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librbd1-2:19.0.0-2555.gf72fecff.el9.x86_64 22/34 2024-05-11T22:58:24.593 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:19.0.0-2555.gf72fecff.el9.x86_64 22/34 2024-05-11T22:58:24.623 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rbd-2:19.0.0-2555.gf72fecff.el9.x86_64 23/34 2024-05-11T22:58:24.659 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-05-11T22:58:25.158 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-05-11T22:58:25.187 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librgw2-2:19.0.0-2555.gf72fecff.el9.x86_64 26/34 2024-05-11T22:58:25.259 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librgw2-2:19.0.0-2555.gf72fecff.el9.x86_64 26/34 2024-05-11T22:58:25.339 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rgw-2:19.0.0-2555.gf72fecff.el9.x86_64 27/34 2024-05-11T22:58:25.365 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-common-2:19.0.0-2555.gf72fecff.el9.x8 28/34 2024-05-11T22:58:25.599 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-05-11T22:58:27.714 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.0.0-2555.gf72fecff.el9.x86_64 29/34 2024-05-11T22:58:27.753 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-common-2:19.0.0-2555.gf72fecff.el9.x86_64 29/34 2024-05-11T22:58:27.999 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T22:58:28.009 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:58:28.009 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-05-11T22:58:28.009 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:58:28.009 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-05-11T22:58:28.009 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 10 M 2024-05-11T22:58:28.009 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2024-05-11T22:58:28.009 INFO:teuthology.orchestra.run.smithi115.stdout: librados2 x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 3.4 M 2024-05-11T22:58:28.009 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1 x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 3.1 M 2024-05-11T22:58:28.009 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-05-11T22:58:28.009 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-11T22:58:28.009 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 5.4 M 2024-05-11T22:58:28.009 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 21 M 2024-05-11T22:58:28.010 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 25 k 2024-05-11T22:58:28.010 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-11T22:58:28.010 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-11T22:58:28.010 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-11T22:58:28.010 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2 x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 745 k 2024-05-11T22:58:28.010 INFO:teuthology.orchestra.run.smithi115.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-11T22:58:28.010 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-11T22:58:28.010 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-11T22:58:28.010 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1 x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 500 k 2024-05-11T22:58:28.010 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-11T22:58:28.010 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2 x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 5.3 M 2024-05-11T22:58:28.011 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-11T22:58:28.011 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-11T22:58:28.011 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-11T22:58:28.011 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 45 k 2024-05-11T22:58:28.011 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 135 k 2024-05-11T22:58:28.011 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 161 k 2024-05-11T22:58:28.011 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-11T22:58:28.011 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 321 k 2024-05-11T22:58:28.011 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 301 k 2024-05-11T22:58:28.011 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 99 k 2024-05-11T22:58:28.011 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-11T22:58:28.011 INFO:teuthology.orchestra.run.smithi115.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-11T22:58:28.011 INFO:teuthology.orchestra.run.smithi115.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-11T22:58:28.011 INFO:teuthology.orchestra.run.smithi115.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-11T22:58:28.011 INFO:teuthology.orchestra.run.smithi115.stdout:Installing weak dependencies: 2024-05-11T22:58:28.011 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-11T22:58:28.012 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:58:28.012 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-05-11T22:58:28.012 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:58:28.012 INFO:teuthology.orchestra.run.smithi115.stdout:Install 30 Packages 2024-05-11T22:58:28.012 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 2 Packages 2024-05-11T22:58:28.012 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:58:28.012 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 60 M 2024-05-11T22:58:28.012 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-05-11T22:58:28.337 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.0.0-2555.gf72fecff.el9.x86_64 29/34 2024-05-11T22:58:28.373 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-base-2:19.0.0-2555.gf72fecff.el9.x86_64 30/34 2024-05-11T22:58:28.465 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-base-2:19.0.0-2555.gf72fecff.el9.x86_64 30/34 2024-05-11T22:58:28.465 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-11T22:58:28.466 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-11T22:58:28.466 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:58:28.508 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-selinux-2:19.0.0-2555.gf72fecff.el9.x86_64 31/34 2024-05-11T22:58:30.076 INFO:teuthology.orchestra.run.smithi115.stdout:(1/32): ceph-radosgw-19.0.0-2555.gf72fecff.el9. 6.0 MB/s | 10 MB 00:01 2024-05-11T22:58:30.125 INFO:teuthology.orchestra.run.smithi115.stdout:(2/32): ceph-selinux-19.0.0-2555.gf72fecff.el9. 507 kB/s | 25 kB 00:00 2024-05-11T22:58:30.226 INFO:teuthology.orchestra.run.smithi115.stdout:(3/32): libcephfs2-19.0.0-2555.gf72fecff.el9.x8 7.3 MB/s | 745 kB 00:00 2024-05-11T22:58:30.301 INFO:teuthology.orchestra.run.smithi115.stdout:(4/32): libradosstriper1-19.0.0-2555.gf72fecff. 6.5 MB/s | 500 kB 00:00 2024-05-11T22:58:30.376 INFO:teuthology.orchestra.run.smithi115.stdout:(5/32): ceph-base-19.0.0-2555.gf72fecff.el9.x86 2.7 MB/s | 5.4 MB 00:02 2024-05-11T22:58:30.435 INFO:teuthology.orchestra.run.smithi115.stdout:(6/32): python3-ceph-argparse-19.0.0-2555.gf72f 776 kB/s | 45 kB 00:00 2024-05-11T22:58:30.510 INFO:teuthology.orchestra.run.smithi115.stdout:(7/32): python3-ceph-common-19.0.0-2555.gf72fec 1.8 MB/s | 135 kB 00:00 2024-05-11T22:58:30.594 INFO:teuthology.orchestra.run.smithi115.stdout:(8/32): python3-cephfs-19.0.0-2555.gf72fecff.el 1.9 MB/s | 161 kB 00:00 2024-05-11T22:58:30.703 INFO:teuthology.orchestra.run.smithi115.stdout:(9/32): librgw2-19.0.0-2555.gf72fecff.el9.x86_6 13 MB/s | 5.3 MB 00:00 2024-05-11T22:58:30.753 INFO:teuthology.orchestra.run.smithi115.stdout:(10/32): python3-rados-19.0.0-2555.gf72fecff.el 2.0 MB/s | 321 kB 00:00 2024-05-11T22:58:30.778 INFO:teuthology.orchestra.run.smithi115.stdout:(11/32): python3-rbd-19.0.0-2555.gf72fecff.el9. 3.9 MB/s | 301 kB 00:00 2024-05-11T22:58:30.820 INFO:teuthology.orchestra.run.smithi115.stdout:(12/32): python3-rgw-19.0.0-2555.gf72fecff.el9. 1.4 MB/s | 99 kB 00:00 2024-05-11T22:58:30.854 INFO:teuthology.orchestra.run.smithi115.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 1.4 MB/s | 107 kB 00:00 2024-05-11T22:58:30.896 INFO:teuthology.orchestra.run.smithi115.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-05-11T22:58:30.921 INFO:teuthology.orchestra.run.smithi115.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 675 kB/s | 45 kB 00:00 2024-05-11T22:58:30.996 INFO:teuthology.orchestra.run.smithi115.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 662 kB 00:00 2024-05-11T22:58:31.021 INFO:teuthology.orchestra.run.smithi115.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2024-05-11T22:58:31.080 INFO:teuthology.orchestra.run.smithi115.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 725 kB/s | 42 kB 00:00 2024-05-11T22:58:31.106 INFO:teuthology.orchestra.run.smithi115.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-05-11T22:58:31.148 INFO:teuthology.orchestra.run.smithi115.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 536 kB/s | 36 kB 00:00 2024-05-11T22:58:31.173 INFO:teuthology.orchestra.run.smithi115.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 710 kB/s | 47 kB 00:00 2024-05-11T22:58:31.431 INFO:teuthology.orchestra.run.smithi115.stdout:(22/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.1 MB/s | 309 kB 00:00 2024-05-11T22:58:31.490 INFO:teuthology.orchestra.run.smithi115.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 447 kB/s | 26 kB 00:00 2024-05-11T22:58:31.548 INFO:teuthology.orchestra.run.smithi115.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 837 kB/s | 49 kB 00:00 2024-05-11T22:58:31.741 INFO:teuthology.orchestra.run.smithi115.stdout:(25/32): ceph-common-19.0.0-2555.gf72fecff.el9. 6.2 MB/s | 21 MB 00:03 2024-05-11T22:58:31.766 INFO:teuthology.orchestra.run.smithi115.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 310 kB/s | 67 kB 00:00 2024-05-11T22:58:31.833 INFO:teuthology.orchestra.run.smithi115.stdout:(27/32): libarrow-9.0.0-11.el9.x86_64.rpm 6.7 MB/s | 4.4 MB 00:00 2024-05-11T22:58:31.858 INFO:teuthology.orchestra.run.smithi115.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-05-11T22:58:31.958 INFO:teuthology.orchestra.run.smithi115.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-05-11T22:58:32.133 INFO:teuthology.orchestra.run.smithi115.stdout:(30/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.1 MB/s | 839 kB 00:00 2024-05-11T22:58:32.192 INFO:teuthology.orchestra.run.smithi115.stdout:(31/32): librbd1-19.0.0-2555.gf72fecff.el9.x86_ 13 MB/s | 3.1 MB 00:00 2024-05-11T22:58:32.501 INFO:teuthology.orchestra.run.smithi115.stdout:(32/32): librados2-19.0.0-2555.gf72fecff.el9.x8 5.3 MB/s | 3.4 MB 00:00 2024-05-11T22:58:32.503 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:58:32.504 INFO:teuthology.orchestra.run.smithi115.stdout:Total 13 MB/s | 60 MB 00:04 2024-05-11T22:58:32.609 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-05-11T22:58:32.646 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-05-11T22:58:32.646 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-05-11T22:58:33.376 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-05-11T22:58:33.377 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-05-11T22:58:34.784 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-05-11T22:58:34.863 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-05-11T22:58:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-05-11T22:58:34.923 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-05-11T22:58:34.945 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-11T22:58:34.969 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-11T22:58:35.164 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-11T22:58:35.218 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-11T22:58:35.560 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-05-11T22:58:35.674 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librados2-2:19.0.0-2555.gf72fecff.el9.x86_64 7/34 2024-05-11T22:58:35.767 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:19.0.0-2555.gf72fecff.el9.x86_64 7/34 2024-05-11T22:58:35.858 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rados-2:19.0.0-2555.gf72fecff.el9.x86_64 8/34 2024-05-11T22:58:35.884 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs2-2:19.0.0-2555.gf72fecff.el9.x86_64 9/34 2024-05-11T22:58:35.947 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs2-2:19.0.0-2555.gf72fecff.el9.x86_64 9/34 2024-05-11T22:58:35.976 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-05-11T22:58:36.029 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-argparse-2:19.0.0-2555.gf72fecff.el9. 11/34 2024-05-11T22:58:36.098 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cephfs-2:19.0.0-2555.gf72fecff.el9.x86_64 12/34 2024-05-11T22:58:36.121 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libradosstriper1-2:19.0.0-2555.gf72fecff.el9.x86_6 13/34 2024-05-11T22:58:36.168 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2555.gf72fecff.el9.x86_6 13/34 2024-05-11T22:58:36.202 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-05-11T22:58:36.313 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-05-11T22:58:36.345 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-05-11T22:58:36.386 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-05-11T22:58:36.426 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-05-11T22:58:36.455 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-05-11T22:58:36.488 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-05-11T22:58:36.800 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-05-11T22:58:36.843 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librbd1-2:19.0.0-2555.gf72fecff.el9.x86_64 22/34 2024-05-11T22:58:36.925 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:19.0.0-2555.gf72fecff.el9.x86_64 22/34 2024-05-11T22:58:36.984 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rbd-2:19.0.0-2555.gf72fecff.el9.x86_64 23/34 2024-05-11T22:58:37.025 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-05-11T22:58:37.501 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-05-11T22:58:37.529 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librgw2-2:19.0.0-2555.gf72fecff.el9.x86_64 26/34 2024-05-11T22:58:37.600 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librgw2-2:19.0.0-2555.gf72fecff.el9.x86_64 26/34 2024-05-11T22:58:37.668 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rgw-2:19.0.0-2555.gf72fecff.el9.x86_64 27/34 2024-05-11T22:58:37.698 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-common-2:19.0.0-2555.gf72fecff.el9.x8 28/34 2024-05-11T22:58:38.510 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2555.gf72fecff.el9.x86_64 31/34 2024-05-11T22:58:38.510 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /sys 2024-05-11T22:58:38.510 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /proc 2024-05-11T22:58:38.511 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /mnt 2024-05-11T22:58:38.511 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /var/tmp 2024-05-11T22:58:38.511 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /home 2024-05-11T22:58:38.511 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /root 2024-05-11T22:58:38.511 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /tmp 2024-05-11T22:58:38.511 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:58:39.493 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-radosgw-2:19.0.0-2555.gf72fecff.el9.x86_64 32/34 2024-05-11T22:58:39.520 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2555.gf72fecff.el9.x86_64 32/34 2024-05-11T22:58:39.520 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T22:58:39.520 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-11T22:58:39.520 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T22:58:39.520 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T22:58:39.520 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:58:39.522 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-11T22:58:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-11T22:58:39.598 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-11T22:58:39.913 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:19.0.0-2555.gf72fecff.el9.x86_64 29/34 2024-05-11T22:58:39.952 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-common-2:19.0.0-2555.gf72fecff.el9.x86_64 29/34 2024-05-11T22:58:40.511 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:19.0.0-2555.gf72fecff.el9.x86_64 29/34 2024-05-11T22:58:40.548 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-base-2:19.0.0-2555.gf72fecff.el9.x86_64 30/34 2024-05-11T22:58:40.639 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-base-2:19.0.0-2555.gf72fecff.el9.x86_64 30/34 2024-05-11T22:58:40.639 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-11T22:58:40.639 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-11T22:58:40.639 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:58:40.699 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-selinux-2:19.0.0-2555.gf72fecff.el9.x86_64 31/34 2024-05-11T22:58:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-11T22:58:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-base-2:19.0.0-2555.gf72fecff.el9.x86_64 1/34 2024-05-11T22:58:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-common-2:19.0.0-2555.gf72fecff.el9.x86_64 2/34 2024-05-11T22:58:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-radosgw-2:19.0.0-2555.gf72fecff.el9.x86_64 3/34 2024-05-11T22:58:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-selinux-2:19.0.0-2555.gf72fecff.el9.x86_64 4/34 2024-05-11T22:58:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs2-2:19.0.0-2555.gf72fecff.el9.x86_64 5/34 2024-05-11T22:58:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libradosstriper1-2:19.0.0-2555.gf72fecff.el9.x86_6 6/34 2024-05-11T22:58:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librgw2-2:19.0.0-2555.gf72fecff.el9.x86_64 7/34 2024-05-11T22:58:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2555.gf72fecff.el9. 8/34 2024-05-11T22:58:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-common-2:19.0.0-2555.gf72fecff.el9.x8 9/34 2024-05-11T22:58:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cephfs-2:19.0.0-2555.gf72fecff.el9.x86_64 10/34 2024-05-11T22:58:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rados-2:19.0.0-2555.gf72fecff.el9.x86_64 11/34 2024-05-11T22:58:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rbd-2:19.0.0-2555.gf72fecff.el9.x86_64 12/34 2024-05-11T22:58:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rgw-2:19.0.0-2555.gf72fecff.el9.x86_64 13/34 2024-05-11T22:58:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-05-11T22:58:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-05-11T22:58:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-05-11T22:58:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-05-11T22:58:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-05-11T22:58:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-05-11T22:58:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-05-11T22:58:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-05-11T22:58:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-05-11T22:58:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-05-11T22:58:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-05-11T22:58:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-05-11T22:58:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-05-11T22:58:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-05-11T22:58:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-05-11T22:58:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-05-11T22:58:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-05-11T22:58:40.790 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:19.0.0-2555.gf72fecff.el9.x86_64 31/34 2024-05-11T22:58:40.790 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-05-11T22:58:40.790 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:19.0.0-2555.gf72fecff.el9.x86_64 33/34 2024-05-11T22:58:41.525 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-05-11T22:58:41.525 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:58:41.525 INFO:teuthology.orchestra.run.smithi005.stdout:Upgraded: 2024-05-11T22:58:41.525 INFO:teuthology.orchestra.run.smithi005.stdout: librados2-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:41.525 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:41.525 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-11T22:58:41.525 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-11T22:58:41.526 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:41.526 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:41.526 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:41.526 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:41.526 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-11T22:58:41.526 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-11T22:58:41.526 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-11T22:58:41.526 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:41.526 INFO:teuthology.orchestra.run.smithi005.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-11T22:58:41.526 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-11T22:58:41.526 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-11T22:58:41.526 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:41.526 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-11T22:58:41.526 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:41.527 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-11T22:58:41.527 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-11T22:58:41.527 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-11T22:58:41.527 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:41.527 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:41.527 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:41.527 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-11T22:58:41.527 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:41.527 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:41.527 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:41.527 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-11T22:58:41.527 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-11T22:58:41.527 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-11T22:58:41.527 INFO:teuthology.orchestra.run.smithi005.stdout: re2-1:20211101-3.el9.x86_64 2024-05-11T22:58:41.527 INFO:teuthology.orchestra.run.smithi005.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-11T22:58:41.528 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:58:41.528 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T22:58:41.860 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-test 2024-05-11T22:58:42.448 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:00:29 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T22:58:42.577 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T22:58:42.577 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:58:42.578 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-05-11T22:58:42.578 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:58:42.578 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-11T22:58:42.578 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 48 M 2024-05-11T22:58:42.578 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-05-11T22:58:42.578 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 163 k 2024-05-11T22:58:42.578 INFO:teuthology.orchestra.run.smithi005.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-11T22:58:42.578 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-11T22:58:42.578 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:58:42.578 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-11T22:58:42.578 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:58:42.578 INFO:teuthology.orchestra.run.smithi005.stdout:Install 4 Packages 2024-05-11T22:58:42.578 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:58:42.579 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 49 M 2024-05-11T22:58:42.579 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 207 M 2024-05-11T22:58:42.579 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-11T22:58:43.017 INFO:teuthology.orchestra.run.smithi005.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.8 MB/s | 305 kB 00:00 2024-05-11T22:58:43.059 INFO:teuthology.orchestra.run.smithi005.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-05-11T22:58:43.117 INFO:teuthology.orchestra.run.smithi005.stdout:(3/4): libcephsqlite-19.0.0-2555.gf72fecff.el9. 622 kB/s | 163 kB 00:00 2024-05-11T22:58:45.926 INFO:teuthology.orchestra.run.smithi005.stdout:(4/4): ceph-test-19.0.0-2555.gf72fecff.el9.x86_ 16 MB/s | 48 MB 00:03 2024-05-11T22:58:45.928 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:58:45.928 INFO:teuthology.orchestra.run.smithi005.stdout:Total 15 MB/s | 49 MB 00:03 2024-05-11T22:58:45.956 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-11T22:58:45.979 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-11T22:58:45.979 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-11T22:58:46.365 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-11T22:58:46.366 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-11T22:58:46.913 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-11T22:58:47.001 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-11T22:58:47.042 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-11T22:58:47.069 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephsqlite-2:19.0.0-2555.gf72fecff.el9.x86_64 3/4 2024-05-11T22:58:50.040 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2555.gf72fecff.el9.x86_64 31/34 2024-05-11T22:58:50.040 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /sys 2024-05-11T22:58:50.040 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /proc 2024-05-11T22:58:50.040 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /mnt 2024-05-11T22:58:50.040 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /var/tmp 2024-05-11T22:58:50.040 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /home 2024-05-11T22:58:50.040 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /root 2024-05-11T22:58:50.040 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /tmp 2024-05-11T22:58:50.040 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:58:51.002 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-radosgw-2:19.0.0-2555.gf72fecff.el9.x86_64 32/34 2024-05-11T22:58:51.027 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2555.gf72fecff.el9.x86_64 32/34 2024-05-11T22:58:51.028 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T22:58:51.028 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-11T22:58:51.028 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T22:58:51.028 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T22:58:51.028 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:58:51.030 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-11T22:58:51.112 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-11T22:58:51.112 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-11T22:58:51.912 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2555.gf72fecff.el9.x86_64 3/4 2024-05-11T22:58:51.996 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-test-2:19.0.0-2555.gf72fecff.el9.x86_64 4/4 2024-05-11T22:58:52.270 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-11T22:58:52.270 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-base-2:19.0.0-2555.gf72fecff.el9.x86_64 1/34 2024-05-11T22:58:52.270 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-common-2:19.0.0-2555.gf72fecff.el9.x86_64 2/34 2024-05-11T22:58:52.270 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-radosgw-2:19.0.0-2555.gf72fecff.el9.x86_64 3/34 2024-05-11T22:58:52.270 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-selinux-2:19.0.0-2555.gf72fecff.el9.x86_64 4/34 2024-05-11T22:58:52.270 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs2-2:19.0.0-2555.gf72fecff.el9.x86_64 5/34 2024-05-11T22:58:52.271 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libradosstriper1-2:19.0.0-2555.gf72fecff.el9.x86_6 6/34 2024-05-11T22:58:52.271 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librgw2-2:19.0.0-2555.gf72fecff.el9.x86_64 7/34 2024-05-11T22:58:52.271 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2555.gf72fecff.el9. 8/34 2024-05-11T22:58:52.271 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-common-2:19.0.0-2555.gf72fecff.el9.x8 9/34 2024-05-11T22:58:52.271 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cephfs-2:19.0.0-2555.gf72fecff.el9.x86_64 10/34 2024-05-11T22:58:52.271 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rados-2:19.0.0-2555.gf72fecff.el9.x86_64 11/34 2024-05-11T22:58:52.271 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rbd-2:19.0.0-2555.gf72fecff.el9.x86_64 12/34 2024-05-11T22:58:52.271 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rgw-2:19.0.0-2555.gf72fecff.el9.x86_64 13/34 2024-05-11T22:58:52.271 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-05-11T22:58:52.271 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-05-11T22:58:52.271 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-05-11T22:58:52.271 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-05-11T22:58:52.271 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-05-11T22:58:52.271 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-05-11T22:58:52.271 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-05-11T22:58:52.272 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-05-11T22:58:52.272 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-05-11T22:58:52.272 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-05-11T22:58:52.272 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-05-11T22:58:52.272 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-05-11T22:58:52.273 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-05-11T22:58:52.273 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-05-11T22:58:52.273 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-05-11T22:58:52.273 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-05-11T22:58:52.273 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-05-11T22:58:52.273 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:19.0.0-2555.gf72fecff.el9.x86_64 31/34 2024-05-11T22:58:52.273 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-05-11T22:58:52.273 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:19.0.0-2555.gf72fecff.el9.x86_64 33/34 2024-05-11T22:58:53.065 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-05-11T22:58:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:58:53.066 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2024-05-11T22:58:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: librados2-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:53.066 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-05-11T22:58:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-11T22:58:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-11T22:58:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-11T22:58:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-11T22:58:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-11T22:58:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-11T22:58:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-11T22:58:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-11T22:58:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-11T22:58:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-11T22:58:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-11T22:58:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-11T22:58:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-11T22:58:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-11T22:58:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-11T22:58:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: re2-1:20211101-3.el9.x86_64 2024-05-11T22:58:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-11T22:58:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:58:53.068 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T22:58:53.411 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-test 2024-05-11T22:58:53.974 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:28 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T22:58:54.093 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T22:58:54.094 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:58:54.094 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-05-11T22:58:54.094 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:58:54.095 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-05-11T22:58:54.095 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 48 M 2024-05-11T22:58:54.095 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-05-11T22:58:54.095 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 163 k 2024-05-11T22:58:54.095 INFO:teuthology.orchestra.run.smithi115.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-11T22:58:54.095 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-11T22:58:54.095 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:58:54.095 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-05-11T22:58:54.095 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:58:54.095 INFO:teuthology.orchestra.run.smithi115.stdout:Install 4 Packages 2024-05-11T22:58:54.095 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:58:54.096 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 49 M 2024-05-11T22:58:54.096 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 207 M 2024-05-11T22:58:54.096 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-05-11T22:58:54.397 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-test-2:19.0.0-2555.gf72fecff.el9.x86_64 4/4 2024-05-11T22:58:54.397 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-test-2:19.0.0-2555.gf72fecff.el9.x86_64 1/4 2024-05-11T22:58:54.397 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephsqlite-2:19.0.0-2555.gf72fecff.el9.x86_64 2/4 2024-05-11T22:58:54.397 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-11T22:58:54.458 INFO:teuthology.orchestra.run.smithi115.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 305 kB 00:00 2024-05-11T22:58:54.501 INFO:teuthology.orchestra.run.smithi115.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-05-11T22:58:54.664 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-11T22:58:54.664 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:58:54.664 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-11T22:58:54.664 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:54.664 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:58:54.664 INFO:teuthology.orchestra.run.smithi005.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-11T22:58:54.664 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-11T22:58:54.664 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:58:54.664 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T22:58:54.725 INFO:teuthology.orchestra.run.smithi115.stdout:(3/4): libcephsqlite-19.0.0-2555.gf72fecff.el9. 324 kB/s | 163 kB 00:00 2024-05-11T22:58:54.926 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph 2024-05-11T22:58:55.511 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:00:42 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T22:58:55.645 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T22:58:55.648 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:58:55.648 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-05-11T22:58:55.649 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:58:55.649 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-11T22:58:55.649 INFO:teuthology.orchestra.run.smithi005.stdout: ceph x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 6.6 k 2024-05-11T22:58:55.649 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-05-11T22:58:55.649 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 2.4 M 2024-05-11T22:58:55.649 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 1.5 M 2024-05-11T22:58:55.649 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2555.gf72fecff.el9 ceph-noarch 249 k 2024-05-11T22:58:55.649 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 4.7 M 2024-05-11T22:58:55.649 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 17 M 2024-05-11T22:58:55.649 INFO:teuthology.orchestra.run.smithi005.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-11T22:58:55.649 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-11T22:58:55.649 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-11T22:58:55.649 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-11T22:58:55.649 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-11T22:58:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-11T22:58:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-11T22:58:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-11T22:58:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-11T22:58:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-11T22:58:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections 2024-05-11T22:58:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-11T22:58:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-11T22:58:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-11T22:58:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-11T22:58:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-11T22:58:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-11T22:58:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-11T22:58:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-11T22:58:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-11T22:58:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-11T22:58:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-11T22:58:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-11T22:58:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-11T22:58:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-11T22:58:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-11T22:58:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-11T22:58:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-11T22:58:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-11T22:58:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-11T22:58:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-11T22:58:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:58:55.651 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-11T22:58:55.652 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:58:55.652 INFO:teuthology.orchestra.run.smithi005.stdout:Install 36 Packages 2024-05-11T22:58:55.652 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:58:55.652 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 30 M 2024-05-11T22:58:55.652 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 107 M 2024-05-11T22:58:55.652 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-11T22:58:56.828 INFO:teuthology.orchestra.run.smithi005.stdout:(1/36): ceph-19.0.0-2555.gf72fecff.el9.x86_64.r 17 kB/s | 6.6 kB 00:00 2024-05-11T22:58:58.419 INFO:teuthology.orchestra.run.smithi005.stdout:(2/36): ceph-mgr-19.0.0-2555.gf72fecff.el9.x86_ 774 kB/s | 1.5 MB 00:01 2024-05-11T22:58:59.078 INFO:teuthology.orchestra.run.smithi005.stdout:(3/36): ceph-mds-19.0.0-2555.gf72fecff.el9.x86_ 917 kB/s | 2.4 MB 00:02 2024-05-11T22:58:59.362 INFO:teuthology.orchestra.run.smithi005.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2555.gf72f 879 kB/s | 249 kB 00:00 2024-05-11T22:58:59.495 INFO:teuthology.orchestra.run.smithi005.stdout:(5/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-05-11T22:58:59.587 INFO:teuthology.orchestra.run.smithi005.stdout:(6/36): python3-cryptography-36.0.1-4.el9.x86_6 13 MB/s | 1.2 MB 00:00 2024-05-11T22:58:59.629 INFO:teuthology.orchestra.run.smithi005.stdout:(7/36): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-05-11T22:58:59.671 INFO:teuthology.orchestra.run.smithi005.stdout:(8/36): python3-pycparser-2.20-6.el9.noarch.rpm 3.2 MB/s | 135 kB 00:00 2024-05-11T22:58:59.713 INFO:teuthology.orchestra.run.smithi005.stdout:(9/36): python3-requests-2.25.1-8.el9.noarch.rp 2.9 MB/s | 125 kB 00:00 2024-05-11T22:58:59.755 INFO:teuthology.orchestra.run.smithi005.stdout:(10/36): python3-urllib3-1.26.5-5.el9.noarch.rp 5.1 MB/s | 215 kB 00:00 2024-05-11T22:59:00.222 INFO:teuthology.orchestra.run.smithi005.stdout:(11/36): ceph-mon-19.0.0-2555.gf72fecff.el9.x86 1.4 MB/s | 4.7 MB 00:03 2024-05-11T22:59:00.531 INFO:teuthology.orchestra.run.smithi005.stdout:(12/36): lua-5.4.4-4.el9.x86_64.rpm 243 kB/s | 188 kB 00:00 2024-05-11T22:59:01.006 INFO:teuthology.orchestra.run.smithi005.stdout:(13/36): python3-mako-1.1.4-6.el9.noarch.rpm 220 kB/s | 172 kB 00:00 2024-05-11T22:59:01.131 INFO:teuthology.orchestra.run.smithi005.stdout:(14/36): python3-markupsafe-1.1.1-12.el9.x86_64 58 kB/s | 35 kB 00:00 2024-05-11T22:59:01.350 INFO:teuthology.orchestra.run.smithi115.stdout:(4/4): ceph-test-19.0.0-2555.gf72fecff.el9.x86_ 6.8 MB/s | 48 MB 00:07 2024-05-11T22:59:01.352 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:59:01.352 INFO:teuthology.orchestra.run.smithi115.stdout:Total 6.7 MB/s | 49 MB 00:07 2024-05-11T22:59:01.376 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-05-11T22:59:01.394 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-05-11T22:59:01.395 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-05-11T22:59:01.656 INFO:teuthology.orchestra.run.smithi005.stdout:(15/36): python3-pytz-2021.1-5.el9.noarch.rpm 78 kB/s | 51 kB 00:00 2024-05-11T22:59:01.748 INFO:teuthology.orchestra.run.smithi005.stdout:(16/36): python3-toml-0.10.2-6.el9.noarch.rpm 67 kB/s | 42 kB 00:00 2024-05-11T22:59:01.759 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-05-11T22:59:01.760 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-05-11T22:59:01.815 INFO:teuthology.orchestra.run.smithi005.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 140 kB/s | 22 kB 00:00 2024-05-11T22:59:02.115 INFO:teuthology.orchestra.run.smithi005.stdout:(18/36): luarocks-3.9.2-1.el9.noarch.rpm 412 kB/s | 151 kB 00:00 2024-05-11T22:59:02.132 INFO:teuthology.orchestra.run.smithi005.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 137 kB/s | 43 kB 00:00 2024-05-11T22:59:02.282 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-05-11T22:59:02.307 INFO:teuthology.orchestra.run.smithi005.stdout:(20/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 896 kB/s | 172 kB 00:00 2024-05-11T22:59:02.357 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-11T22:59:02.398 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-11T22:59:02.407 INFO:teuthology.orchestra.run.smithi005.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 107 kB/s | 11 kB 00:00 2024-05-11T22:59:02.418 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephsqlite-2:19.0.0-2555.gf72fecff.el9.x86_64 3/4 2024-05-11T22:59:02.508 INFO:teuthology.orchestra.run.smithi005.stdout:(22/36): python3-jaraco-classes-3.2.1-5.el9.noa 176 kB/s | 18 kB 00:00 2024-05-11T22:59:02.633 INFO:teuthology.orchestra.run.smithi005.stdout:(23/36): python3-jaraco-collections-3.0.0-8.el9 186 kB/s | 23 kB 00:00 2024-05-11T22:59:02.658 INFO:teuthology.orchestra.run.smithi005.stdout:(24/36): python3-cherrypy-18.6.1-2.el9.noarch.r 681 kB/s | 358 kB 00:00 2024-05-11T22:59:02.725 INFO:teuthology.orchestra.run.smithi005.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 211 kB/s | 19 kB 00:00 2024-05-11T22:59:02.759 INFO:teuthology.orchestra.run.smithi005.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 196 kB/s | 20 kB 00:00 2024-05-11T22:59:02.826 INFO:teuthology.orchestra.run.smithi005.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 462 kB/s | 46 kB 00:00 2024-05-11T22:59:02.868 INFO:teuthology.orchestra.run.smithi005.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 727 kB/s | 79 kB 00:00 2024-05-11T22:59:02.951 INFO:teuthology.orchestra.run.smithi005.stdout:(29/36): python3-portend-3.1.0-2.el9.noarch.rpm 197 kB/s | 16 kB 00:00 2024-05-11T22:59:02.993 INFO:teuthology.orchestra.run.smithi005.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-05-11T22:59:03.052 INFO:teuthology.orchestra.run.smithi005.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 898 kB/s | 90 kB 00:00 2024-05-11T22:59:03.078 INFO:teuthology.orchestra.run.smithi005.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 426 kB/s | 36 kB 00:00 2024-05-11T22:59:03.294 INFO:teuthology.orchestra.run.smithi005.stdout:(33/36): python3-webob-1.8.7-6.el9.noarch.rpm 948 kB/s | 230 kB 00:00 2024-05-11T22:59:03.328 INFO:teuthology.orchestra.run.smithi005.stdout:(34/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.7 MB/s | 427 kB 00:00 2024-05-11T22:59:03.386 INFO:teuthology.orchestra.run.smithi005.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 217 kB/s | 20 kB 00:00 2024-05-11T22:59:03.978 INFO:teuthology.orchestra.run.smithi005.stdout:(36/36): ceph-osd-19.0.0-2555.gf72fecff.el9.x86 3.0 MB/s | 17 MB 00:05 2024-05-11T22:59:03.981 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:59:03.981 INFO:teuthology.orchestra.run.smithi005.stdout:Total 3.6 MB/s | 30 MB 00:08 2024-05-11T22:59:04.129 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-11T22:59:04.174 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-11T22:59:04.174 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-11T22:59:04.650 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-11T22:59:04.651 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-11T22:59:05.957 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-11T22:59:06.023 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-05-11T22:59:06.056 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-05-11T22:59:06.083 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-05-11T22:59:06.159 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-05-11T22:59:06.208 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-05-11T22:59:06.240 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-05-11T22:59:06.337 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-05-11T22:59:06.414 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-05-11T22:59:06.460 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-05-11T22:59:06.496 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-05-11T22:59:06.525 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-05-11T22:59:06.559 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-05-11T22:59:06.593 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-05-11T22:59:06.634 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-05-11T22:59:06.677 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-05-11T22:59:06.711 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-05-11T22:59:06.749 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-05-11T22:59:06.826 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-05-11T22:59:06.988 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-05-11T22:59:07.069 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-05-11T22:59:07.128 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-05-11T22:59:07.143 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2555.gf72fecff.el9.x86_64 3/4 2024-05-11T22:59:07.166 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-05-11T22:59:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-test-2:19.0.0-2555.gf72fecff.el9.x86_64 4/4 2024-05-11T22:59:07.225 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-05-11T22:59:07.288 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-05-11T22:59:07.502 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-05-11T22:59:07.545 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-05-11T22:59:07.627 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-05-11T22:59:07.724 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-05-11T22:59:07.774 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-05-11T22:59:07.901 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-05-11T22:59:08.091 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2555.gf72fecff.el9. 31/36 2024-05-11T22:59:08.120 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-2:19.0.0-2555.gf72fecff.el9.x86_64 32/36 2024-05-11T22:59:08.148 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2555.gf72fecff.el9.x86_64 32/36 2024-05-11T22:59:08.148 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T22:59:08.148 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-11T22:59:08.148 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T22:59:08.148 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T22:59:08.148 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:09.640 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-test-2:19.0.0-2555.gf72fecff.el9.x86_64 4/4 2024-05-11T22:59:09.640 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-test-2:19.0.0-2555.gf72fecff.el9.x86_64 1/4 2024-05-11T22:59:09.640 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephsqlite-2:19.0.0-2555.gf72fecff.el9.x86_64 2/4 2024-05-11T22:59:09.640 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-11T22:59:09.671 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-osd-2:19.0.0-2555.gf72fecff.el9.x86_64 33/36 2024-05-11T22:59:09.700 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-osd-2:19.0.0-2555.gf72fecff.el9.x86_64 33/36 2024-05-11T22:59:09.700 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T22:59:09.700 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-11T22:59:09.700 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T22:59:09.701 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T22:59:09.701 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:09.921 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-11T22:59:09.921 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:09.921 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-05-11T22:59:09.921 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:59:09.921 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:59:09.922 INFO:teuthology.orchestra.run.smithi115.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-11T22:59:09.922 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-11T22:59:09.922 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:09.922 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T22:59:10.163 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mon-2:19.0.0-2555.gf72fecff.el9.x86_64 34/36 2024-05-11T22:59:10.188 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mon-2:19.0.0-2555.gf72fecff.el9.x86_64 34/36 2024-05-11T22:59:10.189 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T22:59:10.189 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-11T22:59:10.189 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T22:59:10.189 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T22:59:10.189 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:10.202 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph 2024-05-11T22:59:10.422 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mds-2:19.0.0-2555.gf72fecff.el9.x86_64 35/36 2024-05-11T22:59:10.447 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mds-2:19.0.0-2555.gf72fecff.el9.x86_64 35/36 2024-05-11T22:59:10.447 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T22:59:10.448 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-11T22:59:10.448 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T22:59:10.448 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T22:59:10.448 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:10.514 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-2:19.0.0-2555.gf72fecff.el9.x86_64 36/36 2024-05-11T22:59:10.762 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:45 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T22:59:10.887 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T22:59:10.890 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:59:10.890 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-05-11T22:59:10.890 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:59:10.890 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-05-11T22:59:10.890 INFO:teuthology.orchestra.run.smithi115.stdout: ceph x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 6.6 k 2024-05-11T22:59:10.890 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-05-11T22:59:10.890 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 2.4 M 2024-05-11T22:59:10.890 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 1.5 M 2024-05-11T22:59:10.890 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2555.gf72fecff.el9 ceph-noarch 249 k 2024-05-11T22:59:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 4.7 M 2024-05-11T22:59:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 17 M 2024-05-11T22:59:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-11T22:59:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-11T22:59:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-11T22:59:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-11T22:59:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-11T22:59:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-11T22:59:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-11T22:59:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-11T22:59:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-11T22:59:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-11T22:59:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections 2024-05-11T22:59:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-11T22:59:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-11T22:59:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-11T22:59:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-11T22:59:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-11T22:59:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-11T22:59:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-11T22:59:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-11T22:59:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-11T22:59:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-11T22:59:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-11T22:59:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-11T22:59:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-11T22:59:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-11T22:59:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-11T22:59:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-11T22:59:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-11T22:59:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-11T22:59:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-11T22:59:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-11T22:59:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:10.893 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-05-11T22:59:10.893 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:59:10.893 INFO:teuthology.orchestra.run.smithi115.stdout:Install 36 Packages 2024-05-11T22:59:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:10.894 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 30 M 2024-05-11T22:59:10.894 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 107 M 2024-05-11T22:59:10.894 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-05-11T22:59:11.735 INFO:teuthology.orchestra.run.smithi115.stdout:(1/36): ceph-19.0.0-2555.gf72fecff.el9.x86_64.r 28 kB/s | 6.6 kB 00:00 2024-05-11T22:59:11.885 INFO:teuthology.orchestra.run.smithi115.stdout:(2/36): ceph-mgr-19.0.0-2555.gf72fecff.el9.x86_ 3.9 MB/s | 1.5 MB 00:00 2024-05-11T22:59:11.944 INFO:teuthology.orchestra.run.smithi115.stdout:(3/36): ceph-mds-19.0.0-2555.gf72fecff.el9.x86_ 5.3 MB/s | 2.4 MB 00:00 2024-05-11T22:59:12.002 INFO:teuthology.orchestra.run.smithi115.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2555.gf72f 4.2 MB/s | 249 kB 00:00 2024-05-11T22:59:12.292 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-2:19.0.0-2555.gf72fecff.el9.x86_64 36/36 2024-05-11T22:59:12.292 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-2:19.0.0-2555.gf72fecff.el9.x86_64 1/36 2024-05-11T22:59:12.293 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mds-2:19.0.0-2555.gf72fecff.el9.x86_64 2/36 2024-05-11T22:59:12.293 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-2:19.0.0-2555.gf72fecff.el9.x86_64 3/36 2024-05-11T22:59:12.293 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mon-2:19.0.0-2555.gf72fecff.el9.x86_64 4/36 2024-05-11T22:59:12.293 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-osd-2:19.0.0-2555.gf72fecff.el9.x86_64 5/36 2024-05-11T22:59:12.293 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2555.gf72fecff.el9. 6/36 2024-05-11T22:59:12.293 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-05-11T22:59:12.293 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-05-11T22:59:12.293 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-05-11T22:59:12.293 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-05-11T22:59:12.293 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-05-11T22:59:12.293 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-05-11T22:59:12.293 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-05-11T22:59:12.293 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-05-11T22:59:12.293 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-05-11T22:59:12.293 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-05-11T22:59:12.294 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-05-11T22:59:12.294 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-05-11T22:59:12.294 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-05-11T22:59:12.294 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-05-11T22:59:12.294 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-05-11T22:59:12.294 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-05-11T22:59:12.294 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-05-11T22:59:12.294 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-05-11T22:59:12.294 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-05-11T22:59:12.294 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-05-11T22:59:12.294 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-05-11T22:59:12.294 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-05-11T22:59:12.294 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-05-11T22:59:12.294 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-05-11T22:59:12.295 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-05-11T22:59:12.295 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-05-11T22:59:12.295 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-05-11T22:59:12.295 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-05-11T22:59:12.295 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-05-11T22:59:12.761 INFO:teuthology.orchestra.run.smithi115.stdout:(5/36): ceph-osd-19.0.0-2555.gf72fecff.el9.x86_ 19 MB/s | 17 MB 00:00 2024-05-11T22:59:12.878 INFO:teuthology.orchestra.run.smithi115.stdout:(6/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 289 kB/s | 253 kB 00:00 2024-05-11T22:59:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-05-11T22:59:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:13.275 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-11T22:59:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:59:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:59:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:59:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core-2:19.0.0-2555.gf72fecff.el9.noarch 2024-05-11T22:59:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:59:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:59:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: lua-5.4.4-4.el9.x86_64 2024-05-11T22:59:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-11T22:59:13.275 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-11T22:59:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-11T22:59:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-11T22:59:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-11T22:59:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-11T22:59:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-11T22:59:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-11T22:59:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-11T22:59:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-11T22:59:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-11T22:59:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-11T22:59:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-11T22:59:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-11T22:59:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-11T22:59:13.276 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-11T22:59:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-11T22:59:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply-3.11-14.el9.noarch 2024-05-11T22:59:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-11T22:59:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-11T22:59:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-11T22:59:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-11T22:59:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-11T22:59:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-11T22:59:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-11T22:59:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-11T22:59:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-11T22:59:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-11T22:59:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-11T22:59:13.277 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:13.277 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T22:59:13.278 INFO:teuthology.orchestra.run.smithi115.stdout:(7/36): ceph-mon-19.0.0-2555.gf72fecff.el9.x86_ 3.0 MB/s | 4.7 MB 00:01 2024-05-11T22:59:13.661 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-base 2024-05-11T22:59:13.678 INFO:teuthology.orchestra.run.smithi115.stdout:(8/36): python3-ply-3.11-14.el9.noarch.rpm 133 kB/s | 106 kB 00:00 2024-05-11T22:59:13.803 INFO:teuthology.orchestra.run.smithi115.stdout:(9/36): python3-cryptography-36.0.1-4.el9.x86_6 1.2 MB/s | 1.2 MB 00:01 2024-05-11T22:59:14.029 INFO:teuthology.orchestra.run.smithi115.stdout:(10/36): python3-pycparser-2.20-6.el9.noarch.rp 180 kB/s | 135 kB 00:00 2024-05-11T22:59:14.212 INFO:teuthology.orchestra.run.smithi115.stdout:(11/36): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-05-11T22:59:14.250 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:01 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T22:59:14.263 INFO:teuthology.orchestra.run.smithi115.stdout:(12/36): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-11T22:59:14.313 INFO:teuthology.orchestra.run.smithi115.stdout:(13/36): python3-markupsafe-1.1.1-12.el9.x86_64 696 kB/s | 35 kB 00:00 2024-05-11T22:59:14.333 INFO:teuthology.orchestra.run.smithi005.stdout:Package ceph-base-2:19.0.0-2555.gf72fecff.el9.x86_64 is already installed. 2024-05-11T22:59:14.363 INFO:teuthology.orchestra.run.smithi115.stdout:(14/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-11T22:59:14.377 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T22:59:14.379 INFO:teuthology.orchestra.run.smithi005.stdout:Nothing to do. 2024-05-11T22:59:14.379 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T22:59:14.405 INFO:teuthology.orchestra.run.smithi115.stdout:(15/36): python3-requests-2.25.1-8.el9.noarch.r 172 kB/s | 125 kB 00:00 2024-05-11T22:59:14.422 INFO:teuthology.orchestra.run.smithi115.stdout:(16/36): python3-toml-0.10.2-6.el9.noarch.rpm 713 kB/s | 42 kB 00:00 2024-05-11T22:59:14.462 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install cephadm 2024-05-11T22:59:14.498 INFO:teuthology.orchestra.run.smithi115.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-05-11T22:59:14.631 INFO:teuthology.orchestra.run.smithi115.stdout:(18/36): python3-urllib3-1.26.5-5.el9.noarch.rp 259 kB/s | 215 kB 00:00 2024-05-11T22:59:14.940 INFO:teuthology.orchestra.run.smithi115.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 98 kB/s | 43 kB 00:00 2024-05-11T22:59:15.040 INFO:teuthology.orchestra.run.smithi115.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 245 kB/s | 151 kB 00:00 2024-05-11T22:59:15.042 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:02 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T22:59:15.170 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T22:59:15.171 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:15.171 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-05-11T22:59:15.172 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:15.172 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-11T22:59:15.172 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm noarch 2:19.0.0-2555.gf72fecff.el9 ceph-noarch 763 k 2024-05-11T22:59:15.172 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:15.172 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-11T22:59:15.172 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:15.172 INFO:teuthology.orchestra.run.smithi005.stdout:Install 1 Package 2024-05-11T22:59:15.172 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:15.172 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 763 k 2024-05-11T22:59:15.172 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 769 k 2024-05-11T22:59:15.172 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-11T22:59:15.174 INFO:teuthology.orchestra.run.smithi115.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 80 kB/s | 11 kB 00:00 2024-05-11T22:59:15.215 INFO:teuthology.orchestra.run.smithi115.stdout:(22/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 295 kB/s | 172 kB 00:00 2024-05-11T22:59:15.282 INFO:teuthology.orchestra.run.smithi115.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 163 kB/s | 18 kB 00:00 2024-05-11T22:59:15.316 INFO:teuthology.orchestra.run.smithi115.stdout:(24/36): python3-cherrypy-18.6.1-2.el9.noarch.r 954 kB/s | 358 kB 00:00 2024-05-11T22:59:15.341 INFO:teuthology.orchestra.run.smithi115.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 184 kB/s | 23 kB 00:00 2024-05-11T22:59:15.392 INFO:teuthology.orchestra.run.smithi115.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 178 kB/s | 19 kB 00:00 2024-05-11T22:59:15.425 INFO:teuthology.orchestra.run.smithi115.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 181 kB/s | 20 kB 00:00 2024-05-11T22:59:15.467 INFO:teuthology.orchestra.run.smithi115.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 368 kB/s | 46 kB 00:00 2024-05-11T22:59:15.479 INFO:teuthology.orchestra.run.smithi005.stdout:cephadm-19.0.0-2555.gf72fecff.el9.noarch.rpm 2.4 MB/s | 763 kB 00:00 2024-05-11T22:59:15.480 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:59:15.480 INFO:teuthology.orchestra.run.smithi005.stdout:Total 2.4 MB/s | 763 kB 00:00 2024-05-11T22:59:15.480 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-11T22:59:15.486 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-11T22:59:15.486 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-11T22:59:15.493 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-11T22:59:15.493 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-11T22:59:15.534 INFO:teuthology.orchestra.run.smithi115.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 554 kB/s | 79 kB 00:00 2024-05-11T22:59:15.565 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-11T22:59:15.585 INFO:teuthology.orchestra.run.smithi115.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-05-11T22:59:15.609 INFO:teuthology.orchestra.run.smithi115.stdout:(31/36): python3-portend-3.1.0-2.el9.noarch.rpm 115 kB/s | 16 kB 00:00 2024-05-11T22:59:15.668 INFO:teuthology.orchestra.run.smithi115.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 672 kB/s | 90 kB 00:00 2024-05-11T22:59:15.702 INFO:teuthology.orchestra.run.smithi115.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 306 kB/s | 36 kB 00:00 2024-05-11T22:59:15.802 INFO:teuthology.orchestra.run.smithi115.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-05-11T22:59:15.827 INFO:teuthology.orchestra.run.smithi115.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 159 kB/s | 20 kB 00:00 2024-05-11T22:59:15.958 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephadm-2:19.0.0-2555.gf72fecff.el9.noarch 1/1 2024-05-11T22:59:15.961 INFO:teuthology.orchestra.run.smithi115.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.4 MB/s | 427 kB 00:00 2024-05-11T22:59:15.966 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:59:15.966 INFO:teuthology.orchestra.run.smithi115.stdout:Total 5.9 MB/s | 30 MB 00:05 2024-05-11T22:59:16.053 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephadm-2:19.0.0-2555.gf72fecff.el9.noarch 1/1 2024-05-11T22:59:16.110 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-05-11T22:59:16.151 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-05-11T22:59:16.151 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-05-11T22:59:16.405 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephadm-2:19.0.0-2555.gf72fecff.el9.noarch 1/1 2024-05-11T22:59:16.594 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-05-11T22:59:16.594 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-05-11T22:59:16.663 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephadm-2:19.0.0-2555.gf72fecff.el9.noarch 1/1 2024-05-11T22:59:16.663 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:16.663 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-11T22:59:16.664 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm-2:19.0.0-2555.gf72fecff.el9.noarch 2024-05-11T22:59:16.664 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:16.664 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T22:59:16.877 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-immutable-object-cache 2024-05-11T22:59:17.467 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:04 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T22:59:17.597 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T22:59:17.598 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:17.598 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repo Size 2024-05-11T22:59:17.598 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:17.598 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-11T22:59:17.598 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 145 k 2024-05-11T22:59:17.598 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:17.598 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-11T22:59:17.598 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:17.598 INFO:teuthology.orchestra.run.smithi005.stdout:Install 1 Package 2024-05-11T22:59:17.599 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:17.599 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 145 k 2024-05-11T22:59:17.599 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 435 k 2024-05-11T22:59:17.599 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-11T22:59:17.811 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-immutable-object-cache-19.0.0-2555.gf72fec 684 kB/s | 145 kB 00:00 2024-05-11T22:59:17.811 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:59:17.812 INFO:teuthology.orchestra.run.smithi005.stdout:Total 679 kB/s | 145 kB 00:00 2024-05-11T22:59:17.812 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-11T22:59:17.820 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-11T22:59:17.820 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-11T22:59:17.839 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-05-11T22:59:17.880 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-11T22:59:17.880 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-11T22:59:17.913 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-05-11T22:59:17.948 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-05-11T22:59:18.004 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-05-11T22:59:18.029 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-11T22:59:18.081 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-05-11T22:59:18.103 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2555.gf72fecff. 1/1 2024-05-11T22:59:18.125 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-05-11T22:59:18.131 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2555.gf72fecff. 1/1 2024-05-11T22:59:18.131 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T22:59:18.131 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-11T22:59:18.131 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:18.156 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-05-11T22:59:18.249 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-05-11T22:59:18.327 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-05-11T22:59:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-05-11T22:59:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-05-11T22:59:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-05-11T22:59:18.466 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-05-11T22:59:18.500 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-05-11T22:59:18.540 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-05-11T22:59:18.575 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-05-11T22:59:18.614 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-05-11T22:59:18.655 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-05-11T22:59:18.731 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-05-11T22:59:18.869 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-05-11T22:59:18.949 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-05-11T22:59:19.002 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-05-11T22:59:19.038 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2555.gf72fecff. 1/1 2024-05-11T22:59:19.038 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:19.038 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-11T22:59:19.038 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:59:19.039 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:19.039 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T22:59:19.048 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-05-11T22:59:19.105 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-05-11T22:59:19.161 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-05-11T22:59:19.275 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-mgr 2024-05-11T22:59:19.357 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-05-11T22:59:19.410 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-05-11T22:59:19.506 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-05-11T22:59:19.602 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-05-11T22:59:19.646 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-05-11T22:59:19.768 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-05-11T22:59:19.866 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:06 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T22:59:19.923 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2555.gf72fecff.el9. 31/36 2024-05-11T22:59:19.950 INFO:teuthology.orchestra.run.smithi005.stdout:Package ceph-mgr-2:19.0.0-2555.gf72fecff.el9.x86_64 is already installed. 2024-05-11T22:59:19.953 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-2:19.0.0-2555.gf72fecff.el9.x86_64 32/36 2024-05-11T22:59:19.980 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2555.gf72fecff.el9.x86_64 32/36 2024-05-11T22:59:19.980 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T22:59:19.980 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-11T22:59:19.980 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T22:59:19.980 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T22:59:19.980 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:19.994 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T22:59:19.995 INFO:teuthology.orchestra.run.smithi005.stdout:Nothing to do. 2024-05-11T22:59:19.995 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T22:59:20.079 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-mgr-dashboard 2024-05-11T22:59:20.663 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:07 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T22:59:20.793 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T22:59:20.794 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:20.794 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-05-11T22:59:20.795 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:20.795 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-11T22:59:20.795 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2555.gf72fecff.el9 ceph-noarch 3.5 M 2024-05-11T22:59:20.795 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-05-11T22:59:20.795 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-11T22:59:20.795 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2555.gf72fecff.el9 ceph-noarch 32 k 2024-05-11T22:59:20.795 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2555.gf72fecff.el9 ceph-noarch 16 k 2024-05-11T22:59:20.795 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-11T22:59:20.795 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-11T22:59:20.795 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-11T22:59:20.795 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-11T22:59:20.795 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-11T22:59:20.795 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-11T22:59:20.795 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-11T22:59:20.796 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-11T22:59:20.796 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:20.796 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-11T22:59:20.796 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:20.796 INFO:teuthology.orchestra.run.smithi005.stdout:Install 12 Packages 2024-05-11T22:59:20.796 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:20.796 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 8.6 M 2024-05-11T22:59:20.796 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 103 M 2024-05-11T22:59:20.797 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-11T22:59:21.412 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-osd-2:19.0.0-2555.gf72fecff.el9.x86_64 33/36 2024-05-11T22:59:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-osd-2:19.0.0-2555.gf72fecff.el9.x86_64 33/36 2024-05-11T22:59:21.440 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T22:59:21.440 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-11T22:59:21.440 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T22:59:21.440 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T22:59:21.440 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:21.489 INFO:teuthology.orchestra.run.smithi005.stdout:(1/12): ceph-grafana-dashboards-19.0.0-2555.gf7 159 kB/s | 32 kB 00:00 2024-05-11T22:59:21.514 INFO:teuthology.orchestra.run.smithi005.stdout:(2/12): ceph-prometheus-alerts-19.0.0-2555.gf72 73 kB/s | 16 kB 00:00 2024-05-11T22:59:21.715 INFO:teuthology.orchestra.run.smithi005.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.3 MB/s | 269 kB 00:00 2024-05-11T22:59:21.773 INFO:teuthology.orchestra.run.smithi005.stdout:(4/12): ceph-mgr-dashboard-19.0.0-2555.gf72fecf 7.3 MB/s | 3.5 MB 00:00 2024-05-11T22:59:21.865 INFO:teuthology.orchestra.run.smithi005.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.7 MB/s | 1.0 MB 00:00 2024-05-11T22:59:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mon-2:19.0.0-2555.gf72fecff.el9.x86_64 34/36 2024-05-11T22:59:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mon-2:19.0.0-2555.gf72fecff.el9.x86_64 34/36 2024-05-11T22:59:21.904 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T22:59:21.904 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-11T22:59:21.904 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T22:59:21.904 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T22:59:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:21.965 INFO:teuthology.orchestra.run.smithi005.stdout:(6/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.4 MB/s | 863 kB 00:00 2024-05-11T22:59:22.130 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mds-2:19.0.0-2555.gf72fecff.el9.x86_64 35/36 2024-05-11T22:59:22.149 INFO:teuthology.orchestra.run.smithi005.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 68 kB/s | 19 kB 00:00 2024-05-11T22:59:22.154 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mds-2:19.0.0-2555.gf72fecff.el9.x86_64 35/36 2024-05-11T22:59:22.154 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T22:59:22.154 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-11T22:59:22.155 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T22:59:22.155 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T22:59:22.155 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:22.219 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-2:19.0.0-2555.gf72fecff.el9.x86_64 36/36 2024-05-11T22:59:22.374 INFO:teuthology.orchestra.run.smithi005.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 912 kB/s | 548 kB 00:00 2024-05-11T22:59:22.583 INFO:teuthology.orchestra.run.smithi005.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 333 kB/s | 144 kB 00:00 2024-05-11T22:59:22.641 INFO:teuthology.orchestra.run.smithi005.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 115 kB/s | 31 kB 00:00 2024-05-11T22:59:22.908 INFO:teuthology.orchestra.run.smithi005.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 2.2 MB/s | 2.0 MB 00:00 2024-05-11T22:59:23.008 INFO:teuthology.orchestra.run.smithi005.stdout:(12/12): python3-routes-2.5.1-5.el9.noarch.rpm 443 kB/s | 188 kB 00:00 2024-05-11T22:59:23.011 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:59:23.012 INFO:teuthology.orchestra.run.smithi005.stdout:Total 3.9 MB/s | 8.6 MB 00:02 2024-05-11T22:59:23.067 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-11T22:59:23.087 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-11T22:59:23.087 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-11T22:59:23.277 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-11T22:59:23.277 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-11T22:59:23.925 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-11T22:59:23.972 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-2:19.0.0-2555.gf72fecff.el9.x86_64 36/36 2024-05-11T22:59:23.972 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-2:19.0.0-2555.gf72fecff.el9.x86_64 1/36 2024-05-11T22:59:23.972 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mds-2:19.0.0-2555.gf72fecff.el9.x86_64 2/36 2024-05-11T22:59:23.972 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-2:19.0.0-2555.gf72fecff.el9.x86_64 3/36 2024-05-11T22:59:23.972 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mon-2:19.0.0-2555.gf72fecff.el9.x86_64 4/36 2024-05-11T22:59:23.972 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-osd-2:19.0.0-2555.gf72fecff.el9.x86_64 5/36 2024-05-11T22:59:23.972 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2555.gf72fecff.el9. 6/36 2024-05-11T22:59:23.972 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-05-11T22:59:23.972 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-05-11T22:59:23.973 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-05-11T22:59:23.973 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-05-11T22:59:23.973 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-05-11T22:59:23.973 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-05-11T22:59:23.973 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-05-11T22:59:23.973 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-05-11T22:59:23.973 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-05-11T22:59:23.973 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-05-11T22:59:23.973 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-05-11T22:59:23.973 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-05-11T22:59:23.973 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-05-11T22:59:23.973 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-05-11T22:59:23.974 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-05-11T22:59:23.974 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-05-11T22:59:23.974 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-05-11T22:59:23.974 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-05-11T22:59:23.974 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-05-11T22:59:23.974 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-05-11T22:59:23.974 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-05-11T22:59:23.974 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-05-11T22:59:23.974 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-05-11T22:59:23.974 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-05-11T22:59:23.974 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-05-11T22:59:23.974 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-05-11T22:59:23.974 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-05-11T22:59:23.975 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-05-11T22:59:23.975 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-05-11T22:59:24.027 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-05-11T22:59:24.061 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-05-11T22:59:24.118 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-05-11T22:59:24.146 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-05-11T22:59:24.247 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-05-11T22:59:24.379 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-05-11T22:59:24.513 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-05-11T22:59:24.578 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-05-11T22:59:24.605 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-05-11T22:59:24.636 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2555.gf72fecff.el9 10/12 2024-05-11T22:59:25.023 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-05-11T22:59:25.024 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:25.024 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-05-11T22:59:25.024 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:59:25.024 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:59:25.024 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:59:25.024 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core-2:19.0.0-2555.gf72fecff.el9.noarch 2024-05-11T22:59:25.024 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:59:25.024 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:59:25.024 INFO:teuthology.orchestra.run.smithi115.stdout: lua-5.4.4-4.el9.x86_64 2024-05-11T22:59:25.024 INFO:teuthology.orchestra.run.smithi115.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-11T22:59:25.024 INFO:teuthology.orchestra.run.smithi115.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-11T22:59:25.024 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-11T22:59:25.025 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-11T22:59:25.025 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-11T22:59:25.025 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-11T22:59:25.025 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-11T22:59:25.025 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-11T22:59:25.025 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-11T22:59:25.025 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-11T22:59:25.025 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-11T22:59:25.025 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-11T22:59:25.025 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-11T22:59:25.025 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-11T22:59:25.025 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-11T22:59:25.025 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-11T22:59:25.025 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-11T22:59:25.025 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply-3.11-14.el9.noarch 2024-05-11T22:59:25.025 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-11T22:59:25.026 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-11T22:59:25.026 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-11T22:59:25.026 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-11T22:59:25.026 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-11T22:59:25.026 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-11T22:59:25.026 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-11T22:59:25.026 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-11T22:59:25.026 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-11T22:59:25.026 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-11T22:59:25.026 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-11T22:59:25.026 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:25.026 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T22:59:25.384 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-base 2024-05-11T22:59:25.738 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2555.gf72fecff.el 11/12 2024-05-11T22:59:25.770 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2555.gf72fecff.el9.noa 12/12 2024-05-11T22:59:25.950 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:00 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T22:59:26.031 INFO:teuthology.orchestra.run.smithi115.stdout:Package ceph-base-2:19.0.0-2555.gf72fecff.el9.x86_64 is already installed. 2024-05-11T22:59:26.072 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T22:59:26.073 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-05-11T22:59:26.074 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T22:59:26.148 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install cephadm 2024-05-11T22:59:26.704 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:01 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T22:59:26.827 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T22:59:26.828 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:59:26.828 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-05-11T22:59:26.828 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:59:26.828 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-05-11T22:59:26.828 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm noarch 2:19.0.0-2555.gf72fecff.el9 ceph-noarch 763 k 2024-05-11T22:59:26.828 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:26.828 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-05-11T22:59:26.829 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:59:26.829 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-05-11T22:59:26.829 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:26.829 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 763 k 2024-05-11T22:59:26.829 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 769 k 2024-05-11T22:59:26.829 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-05-11T22:59:27.004 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2555.gf72fecff.el9.noa 12/12 2024-05-11T22:59:27.004 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2555.gf72fecff.el 1/12 2024-05-11T22:59:27.004 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2555.gf72fecff.el9.noa 2/12 2024-05-11T22:59:27.005 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2555.gf72fecff.el9 3/12 2024-05-11T22:59:27.005 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-05-11T22:59:27.005 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-05-11T22:59:27.005 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-05-11T22:59:27.005 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-05-11T22:59:27.005 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-05-11T22:59:27.005 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-05-11T22:59:27.005 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-05-11T22:59:27.005 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-05-11T22:59:27.113 INFO:teuthology.orchestra.run.smithi115.stdout:cephadm-19.0.0-2555.gf72fecff.el9.noarch.rpm 2.6 MB/s | 763 kB 00:00 2024-05-11T22:59:27.113 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:59:27.114 INFO:teuthology.orchestra.run.smithi115.stdout:Total 2.6 MB/s | 763 kB 00:00 2024-05-11T22:59:27.114 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-05-11T22:59:27.119 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-05-11T22:59:27.119 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-05-11T22:59:27.125 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-05-11T22:59:27.125 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-05-11T22:59:27.202 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-05-11T22:59:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-05-11T22:59:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:27.313 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-11T22:59:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-11T22:59:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards-2:19.0.0-2555.gf72fecff.el9.noarch 2024-05-11T22:59:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard-2:19.0.0-2555.gf72fecff.el9.noarch 2024-05-11T22:59:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts-2:19.0.0-2555.gf72fecff.el9.noarch 2024-05-11T22:59:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-11T22:59:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-11T22:59:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-11T22:59:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-11T22:59:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-11T22:59:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-11T22:59:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-11T22:59:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-11T22:59:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:27.314 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T22:59:27.555 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-11T22:59:27.657 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:19.0.0-2555.gf72fecff.el9.noarch 1/1 2024-05-11T22:59:27.750 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephadm-2:19.0.0-2555.gf72fecff.el9.noarch 1/1 2024-05-11T22:59:28.110 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:19.0.0-2555.gf72fecff.el9.noarch 1/1 2024-05-11T22:59:28.146 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:15 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T22:59:28.273 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T22:59:28.275 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-05-11T22:59:28.275 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-05-11T22:59:28.275 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-05-11T22:59:28.275 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-11T22:59:28.275 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2555.gf72fecff.el9 ceph-noarch 7.4 M 2024-05-11T22:59:28.275 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-05-11T22:59:28.275 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-11T22:59:28.275 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-11T22:59:28.275 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-11T22:59:28.275 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-11T22:59:28.275 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-11T22:59:28.275 INFO:teuthology.orchestra.run.smithi005.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-11T22:59:28.275 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-11T22:59:28.276 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-11T22:59:28.276 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-11T22:59:28.276 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-11T22:59:28.276 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-11T22:59:28.276 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:28.276 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-11T22:59:28.276 INFO:teuthology.orchestra.run.smithi005.stdout:======================================================================================= 2024-05-11T22:59:28.276 INFO:teuthology.orchestra.run.smithi005.stdout:Install 12 Packages 2024-05-11T22:59:28.276 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:28.277 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 38 M 2024-05-11T22:59:28.277 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 209 M 2024-05-11T22:59:28.277 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-11T22:59:28.376 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephadm-2:19.0.0-2555.gf72fecff.el9.noarch 1/1 2024-05-11T22:59:28.376 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:28.376 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-05-11T22:59:28.376 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm-2:19.0.0-2555.gf72fecff.el9.noarch 2024-05-11T22:59:28.376 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:28.376 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T22:59:28.592 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-immutable-object-cache 2024-05-11T22:59:28.667 INFO:teuthology.orchestra.run.smithi005.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 880 kB/s | 195 kB 00:00 2024-05-11T22:59:28.750 INFO:teuthology.orchestra.run.smithi005.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 393 kB/s | 33 kB 00:00 2024-05-11T22:59:28.783 INFO:teuthology.orchestra.run.smithi005.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 803 kB 00:00 2024-05-11T22:59:28.867 INFO:teuthology.orchestra.run.smithi005.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 218 kB/s | 18 kB 00:00 2024-05-11T22:59:28.926 INFO:teuthology.orchestra.run.smithi005.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 605 kB/s | 35 kB 00:00 2024-05-11T22:59:28.984 INFO:teuthology.orchestra.run.smithi005.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 13 MB/s | 3.0 MB 00:00 2024-05-11T22:59:29.029 INFO:teuthology.orchestra.run.smithi005.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 5.4 MB/s | 244 kB 00:00 2024-05-11T22:59:29.137 INFO:teuthology.orchestra.run.smithi005.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 22 MB/s | 4.7 MB 00:00 2024-05-11T22:59:29.170 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:04 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T22:59:29.204 INFO:teuthology.orchestra.run.smithi005.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-05-11T22:59:29.229 INFO:teuthology.orchestra.run.smithi005.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.7 MB/s | 250 kB 00:00 2024-05-11T22:59:29.294 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T22:59:29.295 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:59:29.295 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-05-11T22:59:29.295 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:59:29.295 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-05-11T22:59:29.295 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 145 k 2024-05-11T22:59:29.295 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:29.295 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-05-11T22:59:29.295 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:59:29.295 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-05-11T22:59:29.295 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:29.296 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 145 k 2024-05-11T22:59:29.296 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 435 k 2024-05-11T22:59:29.296 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-05-11T22:59:29.488 INFO:teuthology.orchestra.run.smithi005.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 57 MB/s | 16 MB 00:00 2024-05-11T22:59:29.528 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-immutable-object-cache-19.0.0-2555.gf72fec 625 kB/s | 145 kB 00:00 2024-05-11T22:59:29.528 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:59:29.529 INFO:teuthology.orchestra.run.smithi115.stdout:Total 621 kB/s | 145 kB 00:00 2024-05-11T22:59:29.529 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-05-11T22:59:29.537 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-05-11T22:59:29.537 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-05-11T22:59:29.593 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-05-11T22:59:29.593 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-05-11T22:59:29.605 INFO:teuthology.orchestra.run.smithi005.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-2 6.4 MB/s | 7.4 MB 00:01 2024-05-11T22:59:29.606 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:59:29.606 INFO:teuthology.orchestra.run.smithi005.stdout:Total 29 MB/s | 38 MB 00:01 2024-05-11T22:59:29.745 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-05-11T22:59:29.821 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2555.gf72fecff. 1/1 2024-05-11T22:59:29.846 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2555.gf72fecff. 1/1 2024-05-11T22:59:29.846 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T22:59:29.846 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-11T22:59:29.846 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:29.971 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-11T22:59:29.999 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-11T22:59:29.999 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-11T22:59:30.453 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-11T22:59:30.453 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-11T22:59:30.700 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2555.gf72fecff. 1/1 2024-05-11T22:59:30.701 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:30.701 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-05-11T22:59:30.701 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:59:30.701 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:30.701 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T22:59:30.932 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr 2024-05-11T22:59:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-11T22:59:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-11T22:59:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-11T22:59:31.461 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-11T22:59:31.501 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:06 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T22:59:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-11T22:59:31.585 INFO:teuthology.orchestra.run.smithi115.stdout:Package ceph-mgr-2:19.0.0-2555.gf72fecff.el9.x86_64 is already installed. 2024-05-11T22:59:31.628 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T22:59:31.629 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-05-11T22:59:31.629 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T22:59:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-11T22:59:31.699 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-dashboard 2024-05-11T22:59:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-11T22:59:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-11T22:59:32.259 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:07 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T22:59:32.381 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T22:59:32.382 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:59:32.383 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-05-11T22:59:32.383 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:59:32.383 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-05-11T22:59:32.383 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2555.gf72fecff.el9 ceph-noarch 3.5 M 2024-05-11T22:59:32.383 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-05-11T22:59:32.383 INFO:teuthology.orchestra.run.smithi115.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-11T22:59:32.383 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2555.gf72fecff.el9 ceph-noarch 32 k 2024-05-11T22:59:32.383 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2555.gf72fecff.el9 ceph-noarch 16 k 2024-05-11T22:59:32.383 INFO:teuthology.orchestra.run.smithi115.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-11T22:59:32.383 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-11T22:59:32.383 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-11T22:59:32.383 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-11T22:59:32.383 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-11T22:59:32.383 INFO:teuthology.orchestra.run.smithi115.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-11T22:59:32.384 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-11T22:59:32.384 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-11T22:59:32.384 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:32.384 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-05-11T22:59:32.384 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:59:32.384 INFO:teuthology.orchestra.run.smithi115.stdout:Install 12 Packages 2024-05-11T22:59:32.384 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:32.384 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 8.6 M 2024-05-11T22:59:32.384 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 103 M 2024-05-11T22:59:32.385 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-05-11T22:59:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-11T22:59:32.630 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-11T22:59:33.107 INFO:teuthology.orchestra.run.smithi115.stdout:(1/12): ceph-grafana-dashboards-19.0.0-2555.gf7 166 kB/s | 32 kB 00:00 2024-05-11T22:59:33.132 INFO:teuthology.orchestra.run.smithi115.stdout:(2/12): ceph-prometheus-alerts-19.0.0-2555.gf72 76 kB/s | 16 kB 00:00 2024-05-11T22:59:33.341 INFO:teuthology.orchestra.run.smithi115.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.3 MB/s | 269 kB 00:00 2024-05-11T22:59:33.416 INFO:teuthology.orchestra.run.smithi115.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.3 MB/s | 1.0 MB 00:00 2024-05-11T22:59:33.800 INFO:teuthology.orchestra.run.smithi115.stdout:(5/12): ceph-mgr-dashboard-19.0.0-2555.gf72fecf 4.0 MB/s | 3.5 MB 00:00 2024-05-11T22:59:33.961 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-11T22:59:34.083 INFO:teuthology.orchestra.run.smithi115.stdout:(6/12): protobuf-compiler-3.14.0-13.el9.x86_64. 1.1 MB/s | 863 kB 00:00 2024-05-11T22:59:34.159 INFO:teuthology.orchestra.run.smithi115.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 54 kB/s | 19 kB 00:00 2024-05-11T22:59:34.184 INFO:teuthology.orchestra.run.smithi115.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 714 kB/s | 548 kB 00:00 2024-05-11T22:59:34.384 INFO:teuthology.orchestra.run.smithi115.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 154 kB/s | 31 kB 00:00 2024-05-11T22:59:34.434 INFO:teuthology.orchestra.run.smithi115.stdout:(10/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 5.8 MB/s | 2.0 MB 00:00 2024-05-11T22:59:34.460 INFO:teuthology.orchestra.run.smithi115.stdout:(11/12): python3-grpcio-tools-1.46.7-10.el9.x86 481 kB/s | 144 kB 00:00 2024-05-11T22:59:34.501 INFO:teuthology.orchestra.run.smithi115.stdout:(12/12): python3-routes-2.5.1-5.el9.noarch.rpm 1.6 MB/s | 188 kB 00:00 2024-05-11T22:59:34.504 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:59:34.504 INFO:teuthology.orchestra.run.smithi115.stdout:Total 4.1 MB/s | 8.6 MB 00:02 2024-05-11T22:59:34.553 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-05-11T22:59:34.570 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-05-11T22:59:34.570 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-05-11T22:59:34.745 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-05-11T22:59:34.745 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-05-11T22:59:35.043 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-11T22:59:35.077 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2555.gf72fe 12/12 2024-05-11T22:59:35.366 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-05-11T22:59:35.453 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-05-11T22:59:35.529 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-05-11T22:59:35.575 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-05-11T22:59:35.607 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-05-11T22:59:35.708 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-05-11T22:59:35.832 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-05-11T22:59:35.934 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-05-11T22:59:35.994 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-05-11T22:59:36.025 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-05-11T22:59:36.055 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2555.gf72fecff.el9 10/12 2024-05-11T22:59:37.097 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2555.gf72fecff.el 11/12 2024-05-11T22:59:37.139 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2555.gf72fecff.el9.noa 12/12 2024-05-11T22:59:37.621 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2555.gf72fe 12/12 2024-05-11T22:59:37.621 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2555.gf72fe 1/12 2024-05-11T22:59:37.621 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-11T22:59:37.621 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-11T22:59:37.621 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-11T22:59:37.622 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-11T22:59:37.622 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-11T22:59:37.622 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-11T22:59:37.622 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-11T22:59:37.622 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-11T22:59:37.622 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-11T22:59:37.622 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-11T22:59:37.979 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-11T22:59:37.979 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:37.980 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-11T22:59:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2555.gf72fecff.el9.noarch 2024-05-11T22:59:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-11T22:59:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-11T22:59:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-11T22:59:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-11T22:59:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-11T22:59:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-11T22:59:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-11T22:59:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-11T22:59:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-11T22:59:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-11T22:59:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-11T22:59:37.980 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:37.981 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T22:59:38.310 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-mgr-rook 2024-05-11T22:59:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2555.gf72fecff.el9.noa 12/12 2024-05-11T22:59:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2555.gf72fecff.el 1/12 2024-05-11T22:59:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2555.gf72fecff.el9.noa 2/12 2024-05-11T22:59:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2555.gf72fecff.el9 3/12 2024-05-11T22:59:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-05-11T22:59:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-05-11T22:59:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-05-11T22:59:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-05-11T22:59:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-05-11T22:59:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-05-11T22:59:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-05-11T22:59:38.438 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-05-11T22:59:38.777 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-05-11T22:59:38.777 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:38.777 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-05-11T22:59:38.777 INFO:teuthology.orchestra.run.smithi115.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-11T22:59:38.777 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards-2:19.0.0-2555.gf72fecff.el9.noarch 2024-05-11T22:59:38.777 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard-2:19.0.0-2555.gf72fecff.el9.noarch 2024-05-11T22:59:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts-2:19.0.0-2555.gf72fecff.el9.noarch 2024-05-11T22:59:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-11T22:59:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-11T22:59:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-11T22:59:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-11T22:59:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-11T22:59:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-11T22:59:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-11T22:59:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-11T22:59:38.778 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:38.778 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T22:59:38.905 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:25 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T22:59:39.032 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T22:59:39.033 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:39.033 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-05-11T22:59:39.033 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:39.033 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-11T22:59:39.033 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook noarch 2:19.0.0-2555.gf72fecff.el9 ceph-noarch 49 k 2024-05-11T22:59:39.034 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-05-11T22:59:39.034 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-11T22:59:39.034 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-11T22:59:39.034 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-11T22:59:39.034 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-11T22:59:39.034 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-11T22:59:39.034 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-11T22:59:39.034 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-11T22:59:39.034 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-11T22:59:39.034 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-11T22:59:39.034 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-11T22:59:39.034 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-11T22:59:39.034 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-11T22:59:39.035 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:39.035 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-11T22:59:39.035 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:39.035 INFO:teuthology.orchestra.run.smithi005.stdout:Install 13 Packages 2024-05-11T22:59:39.035 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:39.035 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 2.2 M 2024-05-11T22:59:39.035 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 26 M 2024-05-11T22:59:39.035 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-11T22:59:39.042 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-11T22:59:39.385 INFO:teuthology.orchestra.run.smithi005.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 278 kB/s | 26 kB 00:00 2024-05-11T22:59:39.402 INFO:teuthology.orchestra.run.smithi005.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 176 kB/s | 19 kB 00:00 2024-05-11T22:59:39.469 INFO:teuthology.orchestra.run.smithi005.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.4 MB/s | 159 kB 00:00 2024-05-11T22:59:39.494 INFO:teuthology.orchestra.run.smithi005.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.4 MB/s | 222 kB 00:00 2024-05-11T22:59:39.511 INFO:teuthology.orchestra.run.smithi005.stdout:(5/13): ceph-mgr-rook-19.0.0-2555.gf72fecff.el9 225 kB/s | 49 kB 00:00 2024-05-11T22:59:39.553 INFO:teuthology.orchestra.run.smithi005.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 912 kB/s | 54 kB 00:00 2024-05-11T22:59:39.604 INFO:teuthology.orchestra.run.smithi005.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.0 MB/s | 279 kB 00:00 2024-05-11T22:59:39.647 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:14 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T22:59:39.696 INFO:teuthology.orchestra.run.smithi005.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 175 kB/s | 32 kB 00:00 2024-05-11T22:59:39.746 INFO:teuthology.orchestra.run.smithi005.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 90 kB/s | 15 kB 00:00 2024-05-11T22:59:39.773 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T22:59:39.775 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================= 2024-05-11T22:59:39.775 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-05-11T22:59:39.775 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================= 2024-05-11T22:59:39.775 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-05-11T22:59:39.775 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2555.gf72fecff.el9 ceph-noarch 7.4 M 2024-05-11T22:59:39.775 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-05-11T22:59:39.775 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-11T22:59:39.776 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-11T22:59:39.776 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-11T22:59:39.776 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-11T22:59:39.776 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-11T22:59:39.776 INFO:teuthology.orchestra.run.smithi115.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-11T22:59:39.776 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-11T22:59:39.776 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-11T22:59:39.776 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-11T22:59:39.776 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-11T22:59:39.776 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-11T22:59:39.776 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:39.776 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-05-11T22:59:39.776 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================= 2024-05-11T22:59:39.776 INFO:teuthology.orchestra.run.smithi115.stdout:Install 12 Packages 2024-05-11T22:59:39.776 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:39.777 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 38 M 2024-05-11T22:59:39.777 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 209 M 2024-05-11T22:59:39.777 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-05-11T22:59:39.796 INFO:teuthology.orchestra.run.smithi005.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-05-11T22:59:39.821 INFO:teuthology.orchestra.run.smithi005.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 1.1 MB/s | 206 kB 00:00 2024-05-11T22:59:39.855 INFO:teuthology.orchestra.run.smithi005.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-05-11T22:59:39.955 INFO:teuthology.orchestra.run.smithi005.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 4.0 MB/s | 1.0 MB 00:00 2024-05-11T22:59:39.958 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:59:39.959 INFO:teuthology.orchestra.run.smithi005.stdout:Total 2.4 MB/s | 2.2 MB 00:00 2024-05-11T22:59:40.025 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-11T22:59:40.041 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-11T22:59:40.041 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-11T22:59:40.136 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-11T22:59:40.136 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-11T22:59:40.396 INFO:teuthology.orchestra.run.smithi115.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 379 kB/s | 195 kB 00:00 2024-05-11T22:59:40.680 INFO:teuthology.orchestra.run.smithi115.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-05-11T22:59:40.778 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-11T22:59:40.880 INFO:teuthology.orchestra.run.smithi115.stdout:(3/12): ceph-mgr-diskprediction-local-19.0.0-25 7.4 MB/s | 7.4 MB 00:00 2024-05-11T22:59:40.971 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-11T22:59:41.035 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-11T22:59:41.080 INFO:teuthology.orchestra.run.smithi115.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 48 kB/s | 33 kB 00:00 2024-05-11T22:59:41.099 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-11T22:59:41.134 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-11T22:59:41.176 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-11T22:59:41.278 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-11T22:59:41.397 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-11T22:59:41.422 INFO:teuthology.orchestra.run.smithi115.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 34 kB/s | 18 kB 00:00 2024-05-11T22:59:41.463 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-11T22:59:41.697 INFO:teuthology.orchestra.run.smithi115.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 57 kB/s | 35 kB 00:00 2024-05-11T22:59:41.881 INFO:teuthology.orchestra.run.smithi115.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.5 MB/s | 3.0 MB 00:01 2024-05-11T22:59:42.275 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-11T22:59:42.324 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-11T22:59:42.383 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-11T22:59:42.433 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-11T22:59:42.460 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-rook-2:19.0.0-2555.gf72fecff.el9.noarch 13/13 2024-05-11T22:59:42.564 INFO:teuthology.orchestra.run.smithi115.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 282 kB/s | 244 kB 00:00 2024-05-11T22:59:42.623 INFO:teuthology.orchestra.run.smithi115.stdout:(9/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 3.9 MB/s | 4.7 MB 00:01 2024-05-11T22:59:43.115 INFO:teuthology.orchestra.run.smithi115.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.1 MB/s | 5.1 MB 00:01 2024-05-11T22:59:43.238 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2555.gf72fecff.el9.noarch 13/13 2024-05-11T22:59:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2555.gf72fecff.el9.noarch 1/13 2024-05-11T22:59:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-11T22:59:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-11T22:59:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-11T22:59:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-11T22:59:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-11T22:59:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-11T22:59:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-11T22:59:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-11T22:59:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-11T22:59:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-11T22:59:43.239 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-11T22:59:43.415 INFO:teuthology.orchestra.run.smithi115.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 295 kB/s | 250 kB 00:00 2024-05-11T22:59:43.560 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-11T22:59:43.560 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:43.560 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-11T22:59:43.560 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook-2:19.0.0-2555.gf72fecff.el9.noarch 2024-05-11T22:59:43.560 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-11T22:59:43.560 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-11T22:59:43.560 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-11T22:59:43.560 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-11T22:59:43.560 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-11T22:59:43.560 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-11T22:59:43.560 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-11T22:59:43.561 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-11T22:59:43.561 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-11T22:59:43.561 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-11T22:59:43.561 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-11T22:59:43.561 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-11T22:59:43.561 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:43.561 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T22:59:43.828 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-mgr-cephadm 2024-05-11T22:59:44.123 INFO:teuthology.orchestra.run.smithi115.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-05-11T22:59:44.124 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:59:44.125 INFO:teuthology.orchestra.run.smithi115.stdout:Total 8.7 MB/s | 38 MB 00:04 2024-05-11T22:59:44.424 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:31 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T22:59:44.477 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-05-11T22:59:44.504 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-05-11T22:59:44.504 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-05-11T22:59:44.554 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T22:59:44.555 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-05-11T22:59:44.555 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:44.556 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-11T22:59:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2555.gf72fecff.el9 ceph-noarch 142 k 2024-05-11T22:59:44.556 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-05-11T22:59:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-11T22:59:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-11T22:59:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-11T22:59:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-11T22:59:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-11T22:59:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:44.556 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-11T22:59:44.556 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:44.556 INFO:teuthology.orchestra.run.smithi005.stdout:Install 6 Packages 2024-05-11T22:59:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:44.557 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 7.0 M 2024-05-11T22:59:44.557 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 33 M 2024-05-11T22:59:44.557 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-11T22:59:44.936 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-05-11T22:59:44.936 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-05-11T22:59:44.974 INFO:teuthology.orchestra.run.smithi005.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.6 MB/s | 248 kB 00:00 2024-05-11T22:59:45.133 INFO:teuthology.orchestra.run.smithi005.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 6.0 MB 00:00 2024-05-11T22:59:45.233 INFO:teuthology.orchestra.run.smithi005.stdout:(3/6): ceph-mgr-cephadm-19.0.0-2555.gf72fecff.e 348 kB/s | 142 kB 00:00 2024-05-11T22:59:45.258 INFO:teuthology.orchestra.run.smithi005.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.9 MB/s | 548 kB 00:00 2024-05-11T22:59:45.308 INFO:teuthology.orchestra.run.smithi005.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 330 kB/s | 58 kB 00:00 2024-05-11T22:59:45.383 INFO:teuthology.orchestra.run.smithi005.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 339 kB/s | 51 kB 00:00 2024-05-11T22:59:45.386 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:59:45.387 INFO:teuthology.orchestra.run.smithi005.stdout:Total 8.4 MB/s | 7.0 MB 00:00 2024-05-11T22:59:45.460 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-11T22:59:45.473 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-11T22:59:45.473 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-11T22:59:45.576 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-11T22:59:45.576 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-11T22:59:45.689 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-05-11T22:59:45.757 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-11T22:59:45.805 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-11T22:59:45.888 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-11T22:59:45.914 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-11T22:59:45.947 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-11T22:59:46.056 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-11T22:59:46.111 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-11T22:59:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-11T22:59:46.146 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-11T22:59:46.227 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-11T22:59:46.665 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-11T22:59:46.759 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-11T22:59:46.829 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-11T22:59:46.856 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2555.gf72fecff.el9.noarch 6/6 2024-05-11T22:59:46.895 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-11T22:59:47.019 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-11T22:59:47.683 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2555.gf72fecff.el9.noarch 6/6 2024-05-11T22:59:47.683 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2555.gf72fecff.el9.noarch 1/6 2024-05-11T22:59:47.683 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-11T22:59:47.683 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-11T22:59:47.683 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-11T22:59:47.683 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-11T22:59:47.950 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-11T22:59:47.951 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:47.951 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-11T22:59:47.951 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm-2:19.0.0-2555.gf72fecff.el9.noarch 2024-05-11T22:59:47.951 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-11T22:59:47.951 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-11T22:59:47.951 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-11T22:59:47.951 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-11T22:59:47.951 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-11T22:59:47.951 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:47.951 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T22:59:48.191 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-fuse 2024-05-11T22:59:48.258 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-11T22:59:48.779 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:35 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T22:59:48.906 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T22:59:48.906 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-05-11T22:59:48.907 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:48.907 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-11T22:59:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 889 k 2024-05-11T22:59:48.907 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-05-11T22:59:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-11T22:59:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:48.907 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-11T22:59:48.907 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:48.907 INFO:teuthology.orchestra.run.smithi005.stdout:Install 2 Packages 2024-05-11T22:59:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:48.908 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 969 k 2024-05-11T22:59:48.908 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 2.8 M 2024-05-11T22:59:48.908 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-11T22:59:49.085 INFO:teuthology.orchestra.run.smithi005.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 743 kB/s | 80 kB 00:00 2024-05-11T22:59:49.290 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-11T22:59:49.294 INFO:teuthology.orchestra.run.smithi005.stdout:(2/2): ceph-fuse-19.0.0-2555.gf72fecff.el9.x86_ 2.7 MB/s | 889 kB 00:00 2024-05-11T22:59:49.294 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:59:49.295 INFO:teuthology.orchestra.run.smithi005.stdout:Total 2.4 MB/s | 969 kB 00:00 2024-05-11T22:59:49.307 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-11T22:59:49.320 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2555.gf72fe 12/12 2024-05-11T22:59:49.331 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-11T22:59:49.332 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-11T22:59:49.416 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-11T22:59:49.417 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-11T22:59:49.616 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-11T22:59:49.758 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-11T22:59:49.830 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-fuse-2:19.0.0-2555.gf72fecff.el9.x86_64 2/2 2024-05-11T22:59:50.518 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2555.gf72fecff.el9.x86_64 2/2 2024-05-11T22:59:50.518 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-fuse-2:19.0.0-2555.gf72fecff.el9.x86_64 1/2 2024-05-11T22:59:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-11T22:59:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:50.791 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-11T22:59:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse-2:19.0.0-2555.gf72fecff.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-11T22:59:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:50.792 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T22:59:51.107 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-volume 2024-05-11T22:59:51.705 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:38 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T22:59:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2555.gf72fe 12/12 2024-05-11T22:59:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2555.gf72fe 1/12 2024-05-11T22:59:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-11T22:59:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-11T22:59:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-11T22:59:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-11T22:59:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-11T22:59:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-11T22:59:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-11T22:59:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-11T22:59:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-11T22:59:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-11T22:59:51.834 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T22:59:51.835 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:51.835 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-05-11T22:59:51.835 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:51.835 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-11T22:59:51.835 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume noarch 2:19.0.0-2555.gf72fecff.el9 ceph-noarch 264 k 2024-05-11T22:59:51.835 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-05-11T22:59:51.835 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-11T22:59:51.835 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:51.835 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-11T22:59:51.835 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:51.836 INFO:teuthology.orchestra.run.smithi005.stdout:Install 2 Packages 2024-05-11T22:59:51.836 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:51.836 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 341 k 2024-05-11T22:59:51.836 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 1.5 M 2024-05-11T22:59:51.836 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-11T22:59:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-11T22:59:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:52.107 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-05-11T22:59:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2555.gf72fecff.el9.noarch 2024-05-11T22:59:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-11T22:59:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-11T22:59:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-11T22:59:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-11T22:59:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-11T22:59:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-11T22:59:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-11T22:59:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-11T22:59:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-11T22:59:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-11T22:59:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-11T22:59:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:52.108 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T22:59:52.441 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-rook 2024-05-11T22:59:52.457 INFO:teuthology.orchestra.run.smithi005.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 225 kB/s | 77 kB 00:00 2024-05-11T22:59:52.691 INFO:teuthology.orchestra.run.smithi005.stdout:(2/2): ceph-volume-19.0.0-2555.gf72fecff.el9.no 457 kB/s | 264 kB 00:00 2024-05-11T22:59:52.691 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:59:52.692 INFO:teuthology.orchestra.run.smithi005.stdout:Total 398 kB/s | 341 kB 00:00 2024-05-11T22:59:52.706 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-11T22:59:52.717 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-11T22:59:52.718 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-11T22:59:52.759 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-11T22:59:52.759 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-11T22:59:53.021 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:28 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T22:59:53.036 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-11T22:59:53.144 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T22:59:53.145 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:59:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-05-11T22:59:53.145 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:59:53.145 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-05-11T22:59:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook noarch 2:19.0.0-2555.gf72fecff.el9 ceph-noarch 49 k 2024-05-11T22:59:53.145 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-05-11T22:59:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-11T22:59:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-11T22:59:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-11T22:59:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-11T22:59:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-11T22:59:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-11T22:59:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-11T22:59:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-11T22:59:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-11T22:59:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-11T22:59:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-11T22:59:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-11T22:59:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:53.146 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-05-11T22:59:53.146 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:59:53.146 INFO:teuthology.orchestra.run.smithi115.stdout:Install 13 Packages 2024-05-11T22:59:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:53.147 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 2.2 M 2024-05-11T22:59:53.147 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 26 M 2024-05-11T22:59:53.147 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-05-11T22:59:53.235 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-05-11T22:59:53.276 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-volume-2:19.0.0-2555.gf72fecff.el9.noarch 2/2 2024-05-11T22:59:53.293 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-volume-2:19.0.0-2555.gf72fecff.el9.noarch 2/2 2024-05-11T22:59:53.293 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T22:59:53.294 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-11T22:59:53.294 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:53.561 INFO:teuthology.orchestra.run.smithi115.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 235 kB/s | 26 kB 00:00 2024-05-11T22:59:53.578 INFO:teuthology.orchestra.run.smithi115.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 152 kB/s | 19 kB 00:00 2024-05-11T22:59:53.695 INFO:teuthology.orchestra.run.smithi115.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-05-11T22:59:53.720 INFO:teuthology.orchestra.run.smithi115.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 222 kB 00:00 2024-05-11T22:59:53.771 INFO:teuthology.orchestra.run.smithi115.stdout:(5/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-05-11T22:59:53.796 INFO:teuthology.orchestra.run.smithi115.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 279 kB 00:00 2024-05-11T22:59:53.846 INFO:teuthology.orchestra.run.smithi115.stdout:(7/13): ceph-mgr-rook-19.0.0-2555.gf72fecff.el9 125 kB/s | 49 kB 00:00 2024-05-11T22:59:53.871 INFO:teuthology.orchestra.run.smithi115.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 199 kB/s | 15 kB 00:00 2024-05-11T22:59:53.896 INFO:teuthology.orchestra.run.smithi115.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 256 kB/s | 32 kB 00:00 2024-05-11T22:59:53.989 INFO:teuthology.orchestra.run.smithi115.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 1.4 MB/s | 206 kB 00:00 2024-05-11T22:59:54.014 INFO:teuthology.orchestra.run.smithi115.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 505 kB/s | 59 kB 00:00 2024-05-11T22:59:54.047 INFO:teuthology.orchestra.run.smithi115.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-05-11T22:59:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-volume-2:19.0.0-2555.gf72fecff.el9.noarch 1/2 2024-05-11T22:59:54.181 INFO:teuthology.orchestra.run.smithi115.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.3 MB/s | 1.0 MB 00:00 2024-05-11T22:59:54.181 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:59:54.182 INFO:teuthology.orchestra.run.smithi115.stdout:Total 2.1 MB/s | 2.2 MB 00:01 2024-05-11T22:59:54.246 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-05-11T22:59:54.260 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-05-11T22:59:54.261 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-05-11T22:59:54.341 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-05-11T22:59:54.341 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:54.341 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-11T22:59:54.341 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume-2:19.0.0-2555.gf72fecff.el9.noarch 2024-05-11T22:59:54.341 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-11T22:59:54.341 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:54.341 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T22:59:54.345 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-05-11T22:59:54.346 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-05-11T22:59:54.646 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install librados-devel 2024-05-11T22:59:54.845 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-05-11T22:59:55.010 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-11T22:59:55.065 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-11T22:59:55.120 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-11T22:59:55.158 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-11T22:59:55.199 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-11T22:59:55.244 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:42 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T22:59:55.293 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-11T22:59:55.375 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T22:59:55.376 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:55.376 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repo Size 2024-05-11T22:59:55.376 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:55.376 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-11T22:59:55.376 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 127 k 2024-05-11T22:59:55.376 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:55.376 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-11T22:59:55.376 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:55.377 INFO:teuthology.orchestra.run.smithi005.stdout:Install 1 Package 2024-05-11T22:59:55.377 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:55.377 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 127 k 2024-05-11T22:59:55.377 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 456 k 2024-05-11T22:59:55.377 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-11T22:59:55.403 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-11T22:59:55.459 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-11T22:59:55.605 INFO:teuthology.orchestra.run.smithi005.stdout:librados-devel-19.0.0-2555.gf72fecff.el9.x86_64 557 kB/s | 127 kB 00:00 2024-05-11T22:59:55.606 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:59:55.606 INFO:teuthology.orchestra.run.smithi005.stdout:Total 552 kB/s | 127 kB 00:00 2024-05-11T22:59:55.607 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-11T22:59:55.613 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-11T22:59:55.613 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-11T22:59:55.681 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-11T22:59:55.681 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-11T22:59:55.839 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-11T22:59:55.948 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librados-devel-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T22:59:56.170 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-11T22:59:56.223 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-11T22:59:56.256 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-11T22:59:56.304 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-11T22:59:56.326 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-rook-2:19.0.0-2555.gf72fecff.el9.noarch 13/13 2024-05-11T22:59:56.406 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados-devel-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T22:59:56.682 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados-devel-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T22:59:56.682 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:56.682 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-11T22:59:56.682 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:59:56.682 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:56.682 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T22:59:56.930 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install libcephfs2 2024-05-11T22:59:57.060 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2555.gf72fecff.el9.noarch 13/13 2024-05-11T22:59:57.060 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2555.gf72fecff.el9.noarch 1/13 2024-05-11T22:59:57.060 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-11T22:59:57.060 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-11T22:59:57.060 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-11T22:59:57.060 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-11T22:59:57.060 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-11T22:59:57.060 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-11T22:59:57.060 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-11T22:59:57.060 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-11T22:59:57.060 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-11T22:59:57.061 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-11T22:59:57.061 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-11T22:59:57.379 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-11T22:59:57.379 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:57.379 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-05-11T22:59:57.379 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook-2:19.0.0-2555.gf72fecff.el9.noarch 2024-05-11T22:59:57.379 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-11T22:59:57.379 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-11T22:59:57.379 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-11T22:59:57.379 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-11T22:59:57.379 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-11T22:59:57.379 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-11T22:59:57.380 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-11T22:59:57.380 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-11T22:59:57.380 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-11T22:59:57.380 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-11T22:59:57.380 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-11T22:59:57.380 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-11T22:59:57.380 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:57.380 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T22:59:57.523 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:44 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T22:59:57.608 INFO:teuthology.orchestra.run.smithi005.stdout:Package libcephfs2-2:19.0.0-2555.gf72fecff.el9.x86_64 is already installed. 2024-05-11T22:59:57.623 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-cephadm 2024-05-11T22:59:57.654 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T22:59:57.655 INFO:teuthology.orchestra.run.smithi005.stdout:Nothing to do. 2024-05-11T22:59:57.655 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T22:59:57.739 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install libcephfs-devel 2024-05-11T22:59:58.181 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:33 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T22:59:58.302 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T22:59:58.303 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:59:58.303 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-05-11T22:59:58.303 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:59:58.303 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-05-11T22:59:58.303 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2555.gf72fecff.el9 ceph-noarch 142 k 2024-05-11T22:59:58.303 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-05-11T22:59:58.303 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-11T22:59:58.303 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-11T22:59:58.303 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-11T22:59:58.303 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-11T22:59:58.303 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-11T22:59:58.303 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:58.304 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-05-11T22:59:58.304 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T22:59:58.304 INFO:teuthology.orchestra.run.smithi115.stdout:Install 6 Packages 2024-05-11T22:59:58.304 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T22:59:58.304 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 7.0 M 2024-05-11T22:59:58.304 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 33 M 2024-05-11T22:59:58.304 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-05-11T22:59:58.328 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:45 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T22:59:58.455 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T22:59:58.455 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:58.455 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repo Size 2024-05-11T22:59:58.455 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:58.455 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-11T22:59:58.455 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 32 k 2024-05-11T22:59:58.456 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:58.456 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-11T22:59:58.456 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T22:59:58.456 INFO:teuthology.orchestra.run.smithi005.stdout:Install 1 Package 2024-05-11T22:59:58.456 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:58.456 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 32 k 2024-05-11T22:59:58.456 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 142 k 2024-05-11T22:59:58.457 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-11T22:59:58.607 INFO:teuthology.orchestra.run.smithi115.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-05-11T22:59:58.641 INFO:teuthology.orchestra.run.smithi115.stdout:(2/6): ceph-mgr-cephadm-19.0.0-2555.gf72fecff.e 627 kB/s | 142 kB 00:00 2024-05-11T22:59:58.704 INFO:teuthology.orchestra.run.smithi005.stdout:libcephfs-devel-19.0.0-2555.gf72fecff.el9.x86_6 128 kB/s | 32 kB 00:00 2024-05-11T22:59:58.704 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:59:58.705 INFO:teuthology.orchestra.run.smithi005.stdout:Total 127 kB/s | 32 kB 00:00 2024-05-11T22:59:58.705 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-11T22:59:58.709 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-11T22:59:58.709 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-11T22:59:58.723 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-11T22:59:58.724 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-11T22:59:58.806 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-11T22:59:58.841 INFO:teuthology.orchestra.run.smithi115.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-05-11T22:59:58.906 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs-devel-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T22:59:58.983 INFO:teuthology.orchestra.run.smithi115.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 169 kB/s | 58 kB 00:00 2024-05-11T22:59:59.008 INFO:teuthology.orchestra.run.smithi115.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 308 kB/s | 51 kB 00:00 2024-05-11T22:59:59.075 INFO:teuthology.orchestra.run.smithi115.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.1 MB/s | 548 kB 00:00 2024-05-11T22:59:59.077 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-05-11T22:59:59.078 INFO:teuthology.orchestra.run.smithi115.stdout:Total 9.0 MB/s | 7.0 MB 00:00 2024-05-11T22:59:59.148 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-05-11T22:59:59.160 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-05-11T22:59:59.160 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-05-11T22:59:59.255 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-05-11T22:59:59.256 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-05-11T22:59:59.275 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T22:59:59.499 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs-devel-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T22:59:59.499 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:59.499 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-11T22:59:59.499 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T22:59:59.499 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T22:59:59.499 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T22:59:59.556 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-05-11T22:59:59.678 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install librados2 2024-05-11T22:59:59.722 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-11T22:59:59.768 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-11T23:00:00.270 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:47 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T23:00:00.288 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-11T23:00:00.353 INFO:teuthology.orchestra.run.smithi005.stdout:Package librados2-2:19.0.0-2555.gf72fecff.el9.x86_64 is already installed. 2024-05-11T23:00:00.380 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-11T23:00:00.397 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T23:00:00.398 INFO:teuthology.orchestra.run.smithi005.stdout:Nothing to do. 2024-05-11T23:00:00.398 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T23:00:00.442 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-11T23:00:00.472 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2555.gf72fecff.el9.noarch 6/6 2024-05-11T23:00:00.480 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install librbd1 2024-05-11T23:00:01.068 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:48 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T23:00:01.154 INFO:teuthology.orchestra.run.smithi005.stdout:Package librbd1-2:19.0.0-2555.gf72fecff.el9.x86_64 is already installed. 2024-05-11T23:00:01.198 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T23:00:01.199 INFO:teuthology.orchestra.run.smithi005.stdout:Nothing to do. 2024-05-11T23:00:01.199 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T23:00:01.281 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install python3-rados 2024-05-11T23:00:01.325 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2555.gf72fecff.el9.noarch 6/6 2024-05-11T23:00:01.325 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2555.gf72fecff.el9.noarch 1/6 2024-05-11T23:00:01.325 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-11T23:00:01.325 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-11T23:00:01.325 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-11T23:00:01.325 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-11T23:00:01.601 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-11T23:00:01.601 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:01.602 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-05-11T23:00:01.602 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm-2:19.0.0-2555.gf72fecff.el9.noarch 2024-05-11T23:00:01.602 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-11T23:00:01.602 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-11T23:00:01.602 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-11T23:00:01.602 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-11T23:00:01.602 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-11T23:00:01.602 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:01.602 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T23:00:01.848 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-fuse 2024-05-11T23:00:01.868 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:48 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T23:00:01.953 INFO:teuthology.orchestra.run.smithi005.stdout:Package python3-rados-2:19.0.0-2555.gf72fecff.el9.x86_64 is already installed. 2024-05-11T23:00:01.996 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T23:00:01.997 INFO:teuthology.orchestra.run.smithi005.stdout:Nothing to do. 2024-05-11T23:00:01.997 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T23:00:02.081 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install python3-rgw 2024-05-11T23:00:02.409 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:37 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T23:00:02.532 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T23:00:02.532 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-05-11T23:00:02.532 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:02.532 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-05-11T23:00:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 889 k 2024-05-11T23:00:02.533 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-05-11T23:00:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-11T23:00:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:02.533 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-05-11T23:00:02.533 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:02.533 INFO:teuthology.orchestra.run.smithi115.stdout:Install 2 Packages 2024-05-11T23:00:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:02.533 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 969 k 2024-05-11T23:00:02.534 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 2.8 M 2024-05-11T23:00:02.534 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-05-11T23:00:02.671 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:49 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T23:00:02.754 INFO:teuthology.orchestra.run.smithi005.stdout:Package python3-rgw-2:19.0.0-2555.gf72fecff.el9.x86_64 is already installed. 2024-05-11T23:00:02.798 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T23:00:02.799 INFO:teuthology.orchestra.run.smithi005.stdout:Nothing to do. 2024-05-11T23:00:02.800 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T23:00:02.881 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install python3-cephfs 2024-05-11T23:00:02.943 INFO:teuthology.orchestra.run.smithi115.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 222 kB/s | 80 kB 00:00 2024-05-11T23:00:03.169 INFO:teuthology.orchestra.run.smithi115.stdout:(2/2): ceph-fuse-19.0.0-2555.gf72fecff.el9.x86_ 1.5 MB/s | 889 kB 00:00 2024-05-11T23:00:03.169 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-05-11T23:00:03.169 INFO:teuthology.orchestra.run.smithi115.stdout:Total 1.5 MB/s | 969 kB 00:00 2024-05-11T23:00:03.181 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-05-11T23:00:03.203 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-05-11T23:00:03.203 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-05-11T23:00:03.281 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-05-11T23:00:03.281 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-05-11T23:00:03.472 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:50 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T23:00:03.476 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-05-11T23:00:03.556 INFO:teuthology.orchestra.run.smithi005.stdout:Package python3-cephfs-2:19.0.0-2555.gf72fecff.el9.x86_64 is already installed. 2024-05-11T23:00:03.600 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T23:00:03.602 INFO:teuthology.orchestra.run.smithi005.stdout:Nothing to do. 2024-05-11T23:00:03.602 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T23:00:03.611 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-11T23:00:03.683 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-fuse-2:19.0.0-2555.gf72fecff.el9.x86_64 2/2 2024-05-11T23:00:03.686 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install python3-rbd 2024-05-11T23:00:04.275 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:51 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T23:00:04.292 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2555.gf72fecff.el9.x86_64 2/2 2024-05-11T23:00:04.292 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-fuse-2:19.0.0-2555.gf72fecff.el9.x86_64 1/2 2024-05-11T23:00:04.359 INFO:teuthology.orchestra.run.smithi005.stdout:Package python3-rbd-2:19.0.0-2555.gf72fecff.el9.x86_64 is already installed. 2024-05-11T23:00:04.403 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T23:00:04.404 INFO:teuthology.orchestra.run.smithi005.stdout:Nothing to do. 2024-05-11T23:00:04.404 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T23:00:04.481 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install rbd-fuse 2024-05-11T23:00:04.534 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-11T23:00:04.534 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:04.534 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-05-11T23:00:04.534 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse-2:19.0.0-2555.gf72fecff.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-11T23:00:04.534 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:04.534 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T23:00:04.787 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-volume 2024-05-11T23:00:05.073 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:52 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T23:00:05.201 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T23:00:05.202 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T23:00:05.202 INFO:teuthology.orchestra.run.smithi005.stdout: Package Architecture Version Repository Size 2024-05-11T23:00:05.202 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T23:00:05.202 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-11T23:00:05.202 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 85 k 2024-05-11T23:00:05.202 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:05.202 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-11T23:00:05.202 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T23:00:05.203 INFO:teuthology.orchestra.run.smithi005.stdout:Install 1 Package 2024-05-11T23:00:05.203 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:05.203 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 85 k 2024-05-11T23:00:05.203 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 231 k 2024-05-11T23:00:05.203 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-11T23:00:05.358 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:40 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T23:00:05.474 INFO:teuthology.orchestra.run.smithi005.stdout:rbd-fuse-19.0.0-2555.gf72fecff.el9.x86_64.rpm 315 kB/s | 85 kB 00:00 2024-05-11T23:00:05.474 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-11T23:00:05.474 INFO:teuthology.orchestra.run.smithi005.stdout:Total 313 kB/s | 85 kB 00:00 2024-05-11T23:00:05.475 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-11T23:00:05.482 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-11T23:00:05.482 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-11T23:00:05.485 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T23:00:05.486 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:05.486 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-05-11T23:00:05.486 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:05.486 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-05-11T23:00:05.486 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume noarch 2:19.0.0-2555.gf72fecff.el9 ceph-noarch 264 k 2024-05-11T23:00:05.486 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-05-11T23:00:05.486 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-11T23:00:05.486 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:05.486 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-05-11T23:00:05.486 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:05.486 INFO:teuthology.orchestra.run.smithi115.stdout:Install 2 Packages 2024-05-11T23:00:05.486 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:05.487 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 341 k 2024-05-11T23:00:05.487 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 1.5 M 2024-05-11T23:00:05.487 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-05-11T23:00:05.549 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-11T23:00:05.549 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-11T23:00:05.699 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-11T23:00:05.791 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-fuse-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:05.864 INFO:teuthology.orchestra.run.smithi115.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 705 kB/s | 77 kB 00:00 2024-05-11T23:00:06.015 INFO:teuthology.orchestra.run.smithi115.stdout:(2/2): ceph-volume-19.0.0-2555.gf72fecff.el9.no 1.0 MB/s | 264 kB 00:00 2024-05-11T23:00:06.015 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-05-11T23:00:06.016 INFO:teuthology.orchestra.run.smithi115.stdout:Total 644 kB/s | 341 kB 00:00 2024-05-11T23:00:06.027 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-05-11T23:00:06.037 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-05-11T23:00:06.037 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-05-11T23:00:06.070 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-05-11T23:00:06.070 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-05-11T23:00:06.137 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:06.211 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-05-11T23:00:06.383 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-05-11T23:00:06.406 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-fuse-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:06.406 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:06.406 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-11T23:00:06.406 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T23:00:06.406 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:06.406 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T23:00:06.417 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-volume-2:19.0.0-2555.gf72fecff.el9.noarch 2/2 2024-05-11T23:00:06.432 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-volume-2:19.0.0-2555.gf72fecff.el9.noarch 2/2 2024-05-11T23:00:06.432 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T23:00:06.433 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-11T23:00:06.433 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:06.648 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install rbd-mirror 2024-05-11T23:00:07.154 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-volume-2:19.0.0-2555.gf72fecff.el9.noarch 1/2 2024-05-11T23:00:07.239 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:54 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T23:00:07.369 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T23:00:07.370 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T23:00:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repo Size 2024-05-11T23:00:07.370 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T23:00:07.370 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-11T23:00:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 3.1 M 2024-05-11T23:00:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:07.370 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-11T23:00:07.370 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T23:00:07.370 INFO:teuthology.orchestra.run.smithi005.stdout:Install 1 Package 2024-05-11T23:00:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:07.370 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 3.1 M 2024-05-11T23:00:07.370 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 13 M 2024-05-11T23:00:07.370 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-11T23:00:07.443 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-05-11T23:00:07.443 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:07.443 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-05-11T23:00:07.443 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume-2:19.0.0-2555.gf72fecff.el9.noarch 2024-05-11T23:00:07.443 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-11T23:00:07.443 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:07.443 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T23:00:07.762 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librados-devel 2024-05-11T23:00:07.769 INFO:teuthology.orchestra.run.smithi005.stdout:rbd-mirror-19.0.0-2555.gf72fecff.el9.x86_64.rpm 7.7 MB/s | 3.1 MB 00:00 2024-05-11T23:00:07.770 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-11T23:00:07.770 INFO:teuthology.orchestra.run.smithi005.stdout:Total 7.7 MB/s | 3.1 MB 00:00 2024-05-11T23:00:07.771 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-11T23:00:07.780 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-11T23:00:07.781 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-11T23:00:07.866 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-11T23:00:07.866 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-11T23:00:08.314 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-11T23:00:08.329 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:43 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T23:00:08.379 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-mirror-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:08.406 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:08.406 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T23:00:08.406 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-11T23:00:08.406 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T23:00:08.406 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T23:00:08.406 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:08.456 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T23:00:08.457 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:08.457 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-05-11T23:00:08.457 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:08.457 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-05-11T23:00:08.457 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 127 k 2024-05-11T23:00:08.457 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:08.457 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-05-11T23:00:08.457 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:08.457 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-05-11T23:00:08.458 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:08.458 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 127 k 2024-05-11T23:00:08.458 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 456 k 2024-05-11T23:00:08.458 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-05-11T23:00:08.829 INFO:teuthology.orchestra.run.smithi115.stdout:librados-devel-19.0.0-2555.gf72fecff.el9.x86_64 343 kB/s | 127 kB 00:00 2024-05-11T23:00:08.830 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-05-11T23:00:08.830 INFO:teuthology.orchestra.run.smithi115.stdout:Total 341 kB/s | 127 kB 00:00 2024-05-11T23:00:08.830 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-05-11T23:00:08.836 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-05-11T23:00:08.837 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-05-11T23:00:08.901 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-05-11T23:00:08.901 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-05-11T23:00:09.060 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-05-11T23:00:09.166 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librados-devel-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:09.555 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-mirror-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:09.556 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:09.556 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-11T23:00:09.556 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T23:00:09.556 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:09.556 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T23:00:09.615 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados-devel-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:09.825 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install rbd-nbd 2024-05-11T23:00:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados-devel-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:09.867 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-05-11T23:00:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T23:00:09.867 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:09.867 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T23:00:10.138 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install libcephfs2 2024-05-11T23:00:10.423 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:01:57 ago on Sat 11 May 2024 10:58:13 PM UTC. 2024-05-11T23:00:10.551 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-05-11T23:00:10.552 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T23:00:10.552 INFO:teuthology.orchestra.run.smithi005.stdout: Package Architecture Version Repository Size 2024-05-11T23:00:10.552 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T23:00:10.552 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-05-11T23:00:10.552 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 171 k 2024-05-11T23:00:10.552 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:10.552 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-05-11T23:00:10.552 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-05-11T23:00:10.552 INFO:teuthology.orchestra.run.smithi005.stdout:Install 1 Package 2024-05-11T23:00:10.552 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:10.553 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 171 k 2024-05-11T23:00:10.553 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 494 k 2024-05-11T23:00:10.553 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-05-11T23:00:10.707 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:45 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T23:00:10.789 INFO:teuthology.orchestra.run.smithi115.stdout:Package libcephfs2-2:19.0.0-2555.gf72fecff.el9.x86_64 is already installed. 2024-05-11T23:00:10.831 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T23:00:10.832 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-05-11T23:00:10.832 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T23:00:10.904 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install libcephfs-devel 2024-05-11T23:00:10.932 INFO:teuthology.orchestra.run.smithi005.stdout:rbd-nbd-19.0.0-2555.gf72fecff.el9.x86_64.rpm 452 kB/s | 171 kB 00:00 2024-05-11T23:00:10.933 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-05-11T23:00:10.933 INFO:teuthology.orchestra.run.smithi005.stdout:Total 450 kB/s | 171 kB 00:00 2024-05-11T23:00:10.933 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-05-11T23:00:10.942 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-05-11T23:00:10.942 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-05-11T23:00:11.011 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-05-11T23:00:11.011 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-05-11T23:00:11.174 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-05-11T23:00:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-nbd-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:11.472 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:46 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T23:00:11.595 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T23:00:11.596 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:11.596 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-05-11T23:00:11.596 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:11.596 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-05-11T23:00:11.596 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 32 k 2024-05-11T23:00:11.596 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:11.596 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-05-11T23:00:11.596 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:11.596 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-05-11T23:00:11.596 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:11.597 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 32 k 2024-05-11T23:00:11.597 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 142 k 2024-05-11T23:00:11.597 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-05-11T23:00:11.705 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:11.855 INFO:teuthology.orchestra.run.smithi115.stdout:libcephfs-devel-19.0.0-2555.gf72fecff.el9.x86_6 123 kB/s | 32 kB 00:00 2024-05-11T23:00:11.855 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-05-11T23:00:11.855 INFO:teuthology.orchestra.run.smithi115.stdout:Total 122 kB/s | 32 kB 00:00 2024-05-11T23:00:11.856 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-05-11T23:00:11.859 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-05-11T23:00:11.859 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-05-11T23:00:11.871 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-05-11T23:00:11.872 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-05-11T23:00:11.948 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-nbd-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:11.948 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:11.948 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-05-11T23:00:11.948 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T23:00:11.948 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:11.949 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-05-11T23:00:11.956 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-05-11T23:00:12.062 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs-devel-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:12.210 DEBUG:teuthology.parallel:result is None 2024-05-11T23:00:12.425 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:12.674 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs-devel-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:12.674 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:12.674 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-05-11T23:00:12.674 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T23:00:12.675 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:12.675 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T23:00:12.854 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librados2 2024-05-11T23:00:13.419 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:48 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T23:00:13.500 INFO:teuthology.orchestra.run.smithi115.stdout:Package librados2-2:19.0.0-2555.gf72fecff.el9.x86_64 is already installed. 2024-05-11T23:00:13.542 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T23:00:13.543 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-05-11T23:00:13.543 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T23:00:13.612 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librbd1 2024-05-11T23:00:14.178 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:49 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T23:00:14.259 INFO:teuthology.orchestra.run.smithi115.stdout:Package librbd1-2:19.0.0-2555.gf72fecff.el9.x86_64 is already installed. 2024-05-11T23:00:14.301 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T23:00:14.302 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-05-11T23:00:14.302 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T23:00:14.374 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rados 2024-05-11T23:00:14.940 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:49 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T23:00:15.021 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rados-2:19.0.0-2555.gf72fecff.el9.x86_64 is already installed. 2024-05-11T23:00:15.063 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T23:00:15.064 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-05-11T23:00:15.064 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T23:00:15.138 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rgw 2024-05-11T23:00:15.707 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:50 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T23:00:15.788 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rgw-2:19.0.0-2555.gf72fecff.el9.x86_64 is already installed. 2024-05-11T23:00:15.829 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T23:00:15.830 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-05-11T23:00:15.830 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T23:00:15.907 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-cephfs 2024-05-11T23:00:16.469 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:51 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T23:00:16.549 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-cephfs-2:19.0.0-2555.gf72fecff.el9.x86_64 is already installed. 2024-05-11T23:00:16.590 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T23:00:16.592 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-05-11T23:00:16.592 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T23:00:16.665 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rbd 2024-05-11T23:00:17.231 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:52 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T23:00:17.313 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rbd-2:19.0.0-2555.gf72fecff.el9.x86_64 is already installed. 2024-05-11T23:00:17.355 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T23:00:17.356 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-05-11T23:00:17.356 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T23:00:17.432 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-fuse 2024-05-11T23:00:17.998 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:52 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T23:00:18.121 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T23:00:18.121 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:18.122 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-05-11T23:00:18.122 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:18.122 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-05-11T23:00:18.122 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 85 k 2024-05-11T23:00:18.122 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:18.122 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-05-11T23:00:18.122 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:18.122 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-05-11T23:00:18.122 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:18.122 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 85 k 2024-05-11T23:00:18.122 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 231 k 2024-05-11T23:00:18.122 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-05-11T23:00:18.699 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-fuse-19.0.0-2555.gf72fecff.el9.x86_64.rpm 148 kB/s | 85 kB 00:00 2024-05-11T23:00:18.700 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-05-11T23:00:18.700 INFO:teuthology.orchestra.run.smithi115.stdout:Total 148 kB/s | 85 kB 00:00 2024-05-11T23:00:18.700 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-05-11T23:00:18.707 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-05-11T23:00:18.707 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-05-11T23:00:18.769 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-05-11T23:00:18.770 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-05-11T23:00:18.910 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-05-11T23:00:19.012 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-fuse-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:19.376 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:19.663 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-fuse-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:19.664 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:19.664 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-05-11T23:00:19.664 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T23:00:19.664 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:19.664 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T23:00:19.910 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-mirror 2024-05-11T23:00:20.483 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:55 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T23:00:20.606 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T23:00:20.607 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:20.607 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-05-11T23:00:20.607 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:20.608 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-05-11T23:00:20.608 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 3.1 M 2024-05-11T23:00:20.608 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:20.608 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-05-11T23:00:20.608 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:20.608 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-05-11T23:00:20.608 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:20.608 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 3.1 M 2024-05-11T23:00:20.608 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 13 M 2024-05-11T23:00:20.608 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-05-11T23:00:22.069 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-mirror-19.0.0-2555.gf72fecff.el9.x86_64.rpm 2.1 MB/s | 3.1 MB 00:01 2024-05-11T23:00:22.070 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-05-11T23:00:22.070 INFO:teuthology.orchestra.run.smithi115.stdout:Total 2.1 MB/s | 3.1 MB 00:01 2024-05-11T23:00:22.071 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-05-11T23:00:22.081 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-05-11T23:00:22.082 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-05-11T23:00:22.167 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-05-11T23:00:22.167 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-05-11T23:00:22.593 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-05-11T23:00:22.645 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-mirror-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T23:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-11T23:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T23:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T23:00:22.671 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:23.716 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-mirror-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:23.716 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:23.716 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-05-11T23:00:23.716 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T23:00:23.716 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:23.717 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T23:00:23.965 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-nbd 2024-05-11T23:00:24.552 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:59 ago on Sat 11 May 2024 10:58:25 PM UTC. 2024-05-11T23:00:24.679 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-05-11T23:00:24.680 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-05-11T23:00:24.680 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:24.680 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-05-11T23:00:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd x86_64 2:19.0.0-2555.gf72fecff.el9 ceph 171 k 2024-05-11T23:00:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:24.680 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-05-11T23:00:24.680 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-05-11T23:00:24.680 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-05-11T23:00:24.680 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:24.680 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 171 k 2024-05-11T23:00:24.680 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 494 k 2024-05-11T23:00:24.680 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-05-11T23:00:25.306 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-nbd-19.0.0-2555.gf72fecff.el9.x86_64.rpm 273 kB/s | 171 kB 00:00 2024-05-11T23:00:25.307 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-05-11T23:00:25.307 INFO:teuthology.orchestra.run.smithi115.stdout:Total 273 kB/s | 171 kB 00:00 2024-05-11T23:00:25.307 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-05-11T23:00:25.315 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-05-11T23:00:25.315 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-05-11T23:00:25.378 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-05-11T23:00:25.378 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-05-11T23:00:25.534 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-05-11T23:00:25.628 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-nbd-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:26.041 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:26.287 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-nbd-2:19.0.0-2555.gf72fecff.el9.x86_64 1/1 2024-05-11T23:00:26.288 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:26.288 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-05-11T23:00:26.288 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd-2:19.0.0-2555.gf72fecff.el9.x86_64 2024-05-11T23:00:26.288 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:26.288 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-05-11T23:00:26.585 DEBUG:teuthology.parallel:result is None 2024-05-11T23:00:26.586 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f72fecff68e1d400c4568684327c900485c20d6a 2024-05-11T23:00:26.723 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-11T23:00:26.750 INFO:teuthology.orchestra.run.smithi005.stdout:19.0.0-2555.gf72fecff.el9 2024-05-11T23:00:26.750 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2555.gf72fecff.el9 2024-05-11T23:00:26.750 INFO:teuthology.task.install:The correct ceph version 19.0.0-2555.gf72fecff is installed. 2024-05-11T23:00:26.753 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f72fecff68e1d400c4568684327c900485c20d6a 2024-05-11T23:00:26.970 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-11T23:00:26.991 INFO:teuthology.orchestra.run.smithi115.stdout:19.0.0-2555.gf72fecff.el9 2024-05-11T23:00:26.992 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2555.gf72fecff.el9 2024-05-11T23:00:26.992 INFO:teuthology.task.install:The correct ceph version 19.0.0-2555.gf72fecff is installed. 2024-05-11T23:00:26.993 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-11T23:00:26.993 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T23:00:26.994 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-11T23:00:27.025 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-11T23:00:27.025 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-11T23:00:27.056 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-11T23:00:27.056 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T23:00:27.056 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/daemon-helper 2024-05-11T23:00:27.095 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-11T23:00:27.164 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-11T23:00:27.165 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2024-05-11T23:00:27.189 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-11T23:00:27.253 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-11T23:00:27.254 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T23:00:27.254 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-11T23:00:27.284 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-11T23:00:27.352 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-11T23:00:27.352 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-11T23:00:27.378 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-11T23:00:27.442 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-11T23:00:27.442 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T23:00:27.442 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/stdin-killer 2024-05-11T23:00:27.468 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-11T23:00:27.534 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-11T23:00:27.534 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/stdin-killer 2024-05-11T23:00:27.559 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-11T23:00:27.624 INFO:teuthology.run_tasks:Running task ceph... 2024-05-11T23:00:27.722 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-11T23:00:27.723 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 777 /var/log/ceph 2024-05-11T23:00:27.725 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /var/log/ceph 2024-05-11T23:00:27.749 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-11T23:00:27.750 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-11T23:00:27.791 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-11T23:00:27.816 INFO:tasks.ceph:Creating extra log directories... 2024-05-11T23:00:27.834 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-11T23:00:27.859 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-11T23:00:27.893 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-11T23:00:27.893 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-11T23:00:27.893 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-05-11_21:16:03-rbd-squid-distro-default-smithi/7702761', 'branch': 'squid', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7702761', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-05-11_21:16:03-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': 'f72fecff68e1d400c4568684327c900485c20d6a'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'f72fecff68e1d400c4568684327c900485c20d6a'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'f72fecff68e1d400c4568684327c900485c20d6a'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 2019, 'sha1': 'f72fecff68e1d400c4568684327c900485c20d6a', 'sleep_before_teardown': 0, 'subset': '102/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_f72fecff68e1d400c4568684327c900485c20d6a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'f72fecff68e1d400c4568684327c900485c20d6a', 'targets': {'smithi005.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL4A8qSqk2ZegQA45a0axYnvKI+qA8FsrZHVhlCcuJuOoj0AsoMSNbKCaR32V5DB7V0ndrcTj65u9zl4bADaEDg=', 'smithi115.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLu1gWFUp+C2xGXhq1u5sOl299Z7d4k6HYoNw4jxI1WcmBbAExYoR60Y2NOBN0zOlA3BHeCiOfQtu5UmnRU5+aw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-11_21:16:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-11T23:00:27.894 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-11T23:00:27.926 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-11T23:00:27.941 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m0777 -- /var/run/ceph 2024-05-11T23:00:27.982 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m0777 -- /var/run/ceph 2024-05-11T23:00:28.007 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T23:00:28.007 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/scratch_devs of=/dev/stdout 2024-05-11T23:00:28.062 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-11T23:00:28.062 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_1 2024-05-11T23:00:28.118 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-11T23:00:28.118 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T23:00:28.118 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-11T23:00:28.118 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T23:00:28.118 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T23:00:28.118 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-11 23:00:08.659400535 +0000 2024-05-11T23:00:28.118 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-11 22:57:31.855194856 +0000 2024-05-11T23:00:28.118 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-11 22:57:31.855194856 +0000 2024-05-11T23:00:28.118 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-05-11 22:57:31.855194856 +0000 2024-05-11T23:00:28.119 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-11T23:00:28.181 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-05-11T23:00:28.182 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-05-11T23:00:28.182 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 9.5856e-05 s, 5.3 MB/s 2024-05-11T23:00:28.183 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-11T23:00:28.239 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_2 2024-05-11T23:00:28.295 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-11T23:00:28.295 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T23:00:28.295 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-11T23:00:28.295 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T23:00:28.295 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T23:00:28.296 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-11 23:00:08.660400516 +0000 2024-05-11T23:00:28.296 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-11 22:57:32.199188098 +0000 2024-05-11T23:00:28.296 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-11 22:57:32.199188098 +0000 2024-05-11T23:00:28.296 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-05-11 22:57:32.199188098 +0000 2024-05-11T23:00:28.296 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-11T23:00:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-05-11T23:00:28.359 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-05-11T23:00:28.359 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000102797 s, 5.0 MB/s 2024-05-11T23:00:28.360 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-11T23:00:28.416 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_3 2024-05-11T23:00:28.471 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-11T23:00:28.471 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T23:00:28.471 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-11T23:00:28.471 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T23:00:28.471 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T23:00:28.471 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-11 23:00:08.660400516 +0000 2024-05-11T23:00:28.471 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-11 22:57:32.552181163 +0000 2024-05-11T23:00:28.472 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-11 22:57:32.552181163 +0000 2024-05-11T23:00:28.472 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-05-11 22:57:32.552181163 +0000 2024-05-11T23:00:28.472 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-11T23:00:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-05-11T23:00:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-05-11T23:00:28.535 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 8.9083e-05 s, 5.7 MB/s 2024-05-11T23:00:28.536 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-11T23:00:28.593 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_4 2024-05-11T23:00:28.649 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-11T23:00:28.649 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T23:00:28.649 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-11T23:00:28.650 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T23:00:28.650 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T23:00:28.650 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-05-11 23:00:08.661400497 +0000 2024-05-11T23:00:28.650 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-05-11 22:57:32.887174582 +0000 2024-05-11T23:00:28.650 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-05-11 22:57:32.887174582 +0000 2024-05-11T23:00:28.650 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-05-11 22:57:32.887174582 +0000 2024-05-11T23:00:28.650 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-11T23:00:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-05-11T23:00:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-05-11T23:00:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 8.9987e-05 s, 5.7 MB/s 2024-05-11T23:00:28.715 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-11T23:00:28.772 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-11T23:00:28.772 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-11T23:00:28.772 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2024-05-11T23:00:28.792 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-11T23:00:28.792 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 2024-05-11T23:00:28.850 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-11T23:00:28.851 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T23:00:28.851 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-05-11T23:00:28.851 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T23:00:28.851 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T23:00:28.851 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-05-11 23:00:22.913863446 +0000 2024-05-11T23:00:28.851 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-05-11 22:57:26.899589252 +0000 2024-05-11T23:00:28.851 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-05-11 22:57:26.899589252 +0000 2024-05-11T23:00:28.851 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-05-11 22:57:26.899589252 +0000 2024-05-11T23:00:28.851 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-11T23:00:28.918 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-05-11T23:00:28.918 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-05-11T23:00:28.918 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000220929 s, 2.3 MB/s 2024-05-11T23:00:28.919 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-11T23:00:28.978 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 2024-05-11T23:00:29.035 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-11T23:00:29.035 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T23:00:29.035 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-11T23:00:29.035 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T23:00:29.035 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T23:00:29.036 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-05-11 23:00:22.914863428 +0000 2024-05-11T23:00:29.036 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-05-11 22:57:27.245582725 +0000 2024-05-11T23:00:29.036 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-05-11 22:57:27.245582725 +0000 2024-05-11T23:00:29.036 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-05-11 22:57:27.245582725 +0000 2024-05-11T23:00:29.036 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-11T23:00:29.101 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-05-11T23:00:29.101 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-05-11T23:00:29.101 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000176626 s, 2.9 MB/s 2024-05-11T23:00:29.103 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-11T23:00:29.160 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 2024-05-11T23:00:29.217 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-11T23:00:29.217 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T23:00:29.217 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-11T23:00:29.217 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T23:00:29.217 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T23:00:29.217 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-05-11 23:00:22.914863428 +0000 2024-05-11T23:00:29.217 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-05-11 22:57:27.591576198 +0000 2024-05-11T23:00:29.217 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-05-11 22:57:27.591576198 +0000 2024-05-11T23:00:29.218 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-05-11 22:57:27.591576198 +0000 2024-05-11T23:00:29.218 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-11T23:00:29.282 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-05-11T23:00:29.282 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-05-11T23:00:29.282 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000221179 s, 2.3 MB/s 2024-05-11T23:00:29.283 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-11T23:00:29.340 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 2024-05-11T23:00:29.397 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-11T23:00:29.397 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T23:00:29.397 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-11T23:00:29.397 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T23:00:29.397 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T23:00:29.397 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-05-11 23:00:22.914863428 +0000 2024-05-11T23:00:29.397 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-05-11 22:57:27.934569727 +0000 2024-05-11T23:00:29.397 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-05-11 22:57:27.934569727 +0000 2024-05-11T23:00:29.398 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-05-11 22:57:27.934569727 +0000 2024-05-11T23:00:29.398 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-11T23:00:29.461 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-05-11T23:00:29.461 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-05-11T23:00:29.462 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000206028 s, 2.5 MB/s 2024-05-11T23:00:29.463 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-11T23:00:29.520 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-11T23:00:29.520 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi005.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi115.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-11T23:00:29.520 INFO:tasks.ceph:Generating config... 2024-05-11T23:00:29.521 INFO:tasks.ceph:[client] rbd cache = True 2024-05-11T23:00:29.521 INFO:tasks.ceph:[client] rbd cache policy = writearound 2024-05-11T23:00:29.521 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-05-11T23:00:29.521 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-05-11T23:00:29.521 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-05-11T23:00:29.521 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-11T23:00:29.521 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-11T23:00:29.521 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-11T23:00:29.521 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-11T23:00:29.522 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-11T23:00:29.522 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-11T23:00:29.522 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2024-05-11T23:00:29.522 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-05-11T23:00:29.522 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-11T23:00:29.522 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-11T23:00:29.522 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-11T23:00:29.522 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-11T23:00:29.522 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-11T23:00:29.522 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-11T23:00:29.522 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-11T23:00:29.522 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-11T23:00:29.522 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-11T23:00:29.522 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-05-11T23:00:29.523 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-11T23:00:29.523 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-05-11T23:00:29.523 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-11T23:00:29.523 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-05-11T23:00:29.523 INFO:tasks.ceph:Setting up mon.a... 2024-05-11T23:00:29.523 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-11T23:00:29.576 INFO:teuthology.orchestra.run.smithi005.stdout:creating /etc/ceph/ceph.keyring 2024-05-11T23:00:29.579 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-11T23:00:29.628 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-11T23:00:29.654 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.5'), ('mon.c', '[v2:172.21.15.5:3301,v1:172.21.15.5:6790]'), ('mon.b', '172.21.15.115')] 2024-05-11T23:00:29.655 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.5,[v2:172.21.15.5:3301,v1:172.21.15.5:6790],172.21.15.115', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd cache policy': 'writearound'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-05-11T23:00:29.655 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T23:00:29.655 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-11T23:00:29.712 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.5 --addv c '[v2:172.21.15.5:3301,v1:172.21.15.5:6790]' --add b 172.21.15.115 --print /home/ubuntu/cephtest/ceph.monmap 2024-05-11T23:00:29.800 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-11T23:00:29.801 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: generated fsid 34b20022-f11b-4169-a5f6-6386270a2f8f 2024-05-11T23:00:29.801 INFO:teuthology.orchestra.run.smithi005.stdout:setting min_mon_release = quincy 2024-05-11T23:00:29.801 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 0 2024-05-11T23:00:29.801 INFO:teuthology.orchestra.run.smithi005.stdout:fsid 34b20022-f11b-4169-a5f6-6386270a2f8f 2024-05-11T23:00:29.801 INFO:teuthology.orchestra.run.smithi005.stdout:last_changed 2024-05-11T23:00:29.801385+0000 2024-05-11T23:00:29.801 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-05-11T23:00:29.801385+0000 2024-05-11T23:00:29.801 INFO:teuthology.orchestra.run.smithi005.stdout:min_mon_release 17 (quincy) 2024-05-11T23:00:29.801 INFO:teuthology.orchestra.run.smithi005.stdout:election_strategy: 1 2024-05-11T23:00:29.801 INFO:teuthology.orchestra.run.smithi005.stdout:0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-05-11T23:00:29.801 INFO:teuthology.orchestra.run.smithi005.stdout:1: [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon.b 2024-05-11T23:00:29.801 INFO:teuthology.orchestra.run.smithi005.stdout:2: [v2:172.21.15.5:3301/0,v1:172.21.15.5:6790/0] mon.c 2024-05-11T23:00:29.801 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-11T23:00:29.802 DEBUG:teuthology.orchestra.run.smithi005:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-11T23:00:29.819 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 34b20022-f11b-4169-a5f6-6386270a2f8f... 2024-05-11T23:00:29.820 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-11T23:00:29.861 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-11T23:00:29.904 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-05-11T23:00:29.904 INFO:teuthology.orchestra.run.smithi005.stdout: chdir = "" 2024-05-11T23:00:29.904 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-11T23:00:29.904 INFO:teuthology.orchestra.run.smithi005.stdout: auth supported = cephx 2024-05-11T23:00:29.904 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: filestore xattr use omap = true 2024-05-11T23:00:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: mon clock drift allowed = 1.000 2024-05-11T23:00:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: osd crush chooseleaf type = 0 2024-05-11T23:00:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: auth debug = true 2024-05-11T23:00:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on old message = true 2024-05-11T23:00:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on bug = true 2024-05-11T23:00:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-11T23:00:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: mon pg warn max object skew = 0 2024-05-11T23:00:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: # disable pg_autoscaler by default for new pools 2024-05-11T23:00:29.905 INFO:teuthology.orchestra.run.smithi005.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-11T23:00:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default size = 2 2024-05-11T23:00:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow primary affinity = true 2024-05-11T23:00:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow pg remap = true 2024-05-11T23:00:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on legacy crush tunables = false 2024-05-11T23:00:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on crush straw calc version zero = false 2024-05-11T23:00:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on no sortbitwise = false 2024-05-11T23:00:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on osd down out interval zero = false 2024-05-11T23:00:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on too few osds = false 2024-05-11T23:00:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-11T23:00:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-11T23:00:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: mon_allow_pool_size_one = true 2024-05-11T23:00:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-11T23:00:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.906 INFO:teuthology.orchestra.run.smithi005.stdout: osd default data pool replay window = 5 2024-05-11T23:00:29.907 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.907 INFO:teuthology.orchestra.run.smithi005.stdout: mon allow pool delete = true 2024-05-11T23:00:29.907 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.907 INFO:teuthology.orchestra.run.smithi005.stdout: mon cluster log file level = debug 2024-05-11T23:00:29.907 INFO:teuthology.orchestra.run.smithi005.stdout: debug asserts on shutdown = true 2024-05-11T23:00:29.907 INFO:teuthology.orchestra.run.smithi005.stdout: mon health detail to clog = false 2024-05-11T23:00:29.907 INFO:teuthology.orchestra.run.smithi005.stdout: mon host = "172.21.15.5,[v2:172.21.15.5:3301,v1:172.21.15.5:6790],172.21.15.115" 2024-05-11T23:00:29.907 INFO:teuthology.orchestra.run.smithi005.stdout: mon client directed command retry = 5 2024-05-11T23:00:29.907 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on pool no app = False 2024-05-11T23:00:29.907 INFO:teuthology.orchestra.run.smithi005.stdout: ms inject socket failures = 5000 2024-05-11T23:00:29.907 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 34b20022-f11b-4169-a5f6-6386270a2f8f 2024-05-11T23:00:29.907 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.907 INFO:teuthology.orchestra.run.smithi005.stdout:[osd] 2024-05-11T23:00:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: osd journal size = 100 2024-05-11T23:00:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub load threshold = 5.0 2024-05-11T23:00:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub max interval = 600 2024-05-11T23:00:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: osd mclock profile = high_recovery_ops 2024-05-11T23:00:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: osd recover clone overlap = true 2024-05-11T23:00:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: osd recovery max chunk = 1048576 2024-05-11T23:00:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug shutdown = true 2024-05-11T23:00:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug op order = true 2024-05-11T23:00:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug verify stray on activate = true 2024-05-11T23:00:29.908 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug trim objects = true 2024-05-11T23:00:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: osd open classes on start = true 2024-05-11T23:00:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug pg log writeout = true 2024-05-11T23:00:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: osd deep scrub update digest min age = 30 2024-05-11T23:00:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: osd map max advance = 10 2024-05-11T23:00:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: journal zero on create = true 2024-05-11T23:00:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: filestore ondisk finisher threads = 3 2024-05-11T23:00:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: filestore apply finisher threads = 3 2024-05-11T23:00:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: bdev debug aio = true 2024-05-11T23:00:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug misdirected ops = true 2024-05-11T23:00:29.909 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore block size = 96636764160 2024-05-11T23:00:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore compression algorithm = snappy 2024-05-11T23:00:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore compression mode = aggressive 2024-05-11T23:00:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore fsck on mount = True 2024-05-11T23:00:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluefs = 1/20 2024-05-11T23:00:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluestore = 1/20 2024-05-11T23:00:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-11T23:00:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: debug osd = 20 2024-05-11T23:00:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: debug rocksdb = 4/10 2024-05-11T23:00:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd backfillfull_ratio = 0.85 2024-05-11T23:00:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd full ratio = 0.9 2024-05-11T23:00:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd nearfull ratio = 0.8 2024-05-11T23:00:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: osd blocked scrub grace period = 3600 2024-05-11T23:00:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: osd failsafe full ratio = 0.95 2024-05-11T23:00:29.910 INFO:teuthology.orchestra.run.smithi005.stdout: osd max markdown count = 1000 2024-05-11T23:00:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: osd objectstore = bluestore 2024-05-11T23:00:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: osd shutdown pgref assert = True 2024-05-11T23:00:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.911 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr] 2024-05-11T23:00:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-11T23:00:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: debug mgr = 20 2024-05-11T23:00:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-05-11T23:00:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-05-11T23:00:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-05-11T23:00:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-05-11T23:00:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: mgr/telemetry/nag = false 2024-05-11T23:00:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.911 INFO:teuthology.orchestra.run.smithi005.stdout:[mon] 2024-05-11T23:00:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-05-11T23:00:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-05-11T23:00:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: debug paxos = 20 2024-05-11T23:00:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-05-11T23:00:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: mon data avail warn = 5 2024-05-11T23:00:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: mon mgr mkfs grace = 240 2024-05-11T23:00:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-05-11T23:00:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd reporter subtree level = osd 2024-05-11T23:00:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd prime pg temp = true 2024-05-11T23:00:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-05-11T23:00:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-11T23:00:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: auth mon ticket ttl = 660 # 11m 2024-05-11T23:00:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: auth service ticket ttl = 240 # 4m 2024-05-11T23:00:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: # don't complain about insecure global_id in the test suite 2024-05-11T23:00:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-11T23:00:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-11T23:00:29.912 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.913 INFO:teuthology.orchestra.run.smithi005.stdout: # 1m isn't quite enough 2024-05-11T23:00:29.913 INFO:teuthology.orchestra.run.smithi005.stdout: mon_down_mkfs_grace = 2m 2024-05-11T23:00:29.913 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.913 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_filestore_osds = false 2024-05-11T23:00:29.913 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:00:29.913 INFO:teuthology.orchestra.run.smithi005.stdout:[client] 2024-05-11T23:00:29.913 INFO:teuthology.orchestra.run.smithi005.stdout: rgw cache enabled = true 2024-05-11T23:00:29.913 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable ops log = true 2024-05-11T23:00:29.913 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable usage log = true 2024-05-11T23:00:29.913 INFO:teuthology.orchestra.run.smithi005.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-11T23:00:29.913 INFO:teuthology.orchestra.run.smithi005.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-11T23:00:29.913 INFO:teuthology.orchestra.run.smithi005.stdout: rbd cache = True 2024-05-11T23:00:29.913 INFO:teuthology.orchestra.run.smithi005.stdout: rbd cache policy = writearound 2024-05-11T23:00:29.913 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.a] 2024-05-11T23:00:29.913 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.c] 2024-05-11T23:00:29.913 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.b] 2024-05-11T23:00:29.914 INFO:teuthology.orchestra.run.smithi115.stdout:[global] 2024-05-11T23:00:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: chdir = "" 2024-05-11T23:00:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-11T23:00:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: auth supported = cephx 2024-05-11T23:00:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: filestore xattr use omap = true 2024-05-11T23:00:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: mon clock drift allowed = 1.000 2024-05-11T23:00:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: osd crush chooseleaf type = 0 2024-05-11T23:00:29.914 INFO:teuthology.orchestra.run.smithi115.stdout: auth debug = true 2024-05-11T23:00:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: ms die on old message = true 2024-05-11T23:00:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: ms die on bug = true 2024-05-11T23:00:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-11T23:00:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: mon pg warn max object skew = 0 2024-05-11T23:00:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: # disable pg_autoscaler by default for new pools 2024-05-11T23:00:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-11T23:00:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: osd pool default size = 2 2024-05-11T23:00:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd allow primary affinity = true 2024-05-11T23:00:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd allow pg remap = true 2024-05-11T23:00:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on legacy crush tunables = false 2024-05-11T23:00:29.915 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on crush straw calc version zero = false 2024-05-11T23:00:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on no sortbitwise = false 2024-05-11T23:00:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on osd down out interval zero = false 2024-05-11T23:00:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on too few osds = false 2024-05-11T23:00:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-11T23:00:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-11T23:00:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: mon_allow_pool_size_one = true 2024-05-11T23:00:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-11T23:00:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: osd default data pool replay window = 5 2024-05-11T23:00:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: mon allow pool delete = true 2024-05-11T23:00:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: mon cluster log file level = debug 2024-05-11T23:00:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: debug asserts on shutdown = true 2024-05-11T23:00:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: mon health detail to clog = false 2024-05-11T23:00:29.916 INFO:teuthology.orchestra.run.smithi115.stdout: mon host = "172.21.15.5,[v2:172.21.15.5:3301,v1:172.21.15.5:6790],172.21.15.115" 2024-05-11T23:00:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: mon client directed command retry = 5 2024-05-11T23:00:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: mon warn on pool no app = False 2024-05-11T23:00:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: ms inject socket failures = 5000 2024-05-11T23:00:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: fsid = 34b20022-f11b-4169-a5f6-6386270a2f8f 2024-05-11T23:00:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.917 INFO:teuthology.orchestra.run.smithi115.stdout:[osd] 2024-05-11T23:00:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: osd journal size = 100 2024-05-11T23:00:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: osd scrub load threshold = 5.0 2024-05-11T23:00:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: osd scrub max interval = 600 2024-05-11T23:00:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: osd mclock profile = high_recovery_ops 2024-05-11T23:00:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: osd recover clone overlap = true 2024-05-11T23:00:29.917 INFO:teuthology.orchestra.run.smithi115.stdout: osd recovery max chunk = 1048576 2024-05-11T23:00:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug shutdown = true 2024-05-11T23:00:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug op order = true 2024-05-11T23:00:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug verify stray on activate = true 2024-05-11T23:00:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug trim objects = true 2024-05-11T23:00:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: osd open classes on start = true 2024-05-11T23:00:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug pg log writeout = true 2024-05-11T23:00:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: osd deep scrub update digest min age = 30 2024-05-11T23:00:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: osd map max advance = 10 2024-05-11T23:00:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: journal zero on create = true 2024-05-11T23:00:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.918 INFO:teuthology.orchestra.run.smithi115.stdout: filestore ondisk finisher threads = 3 2024-05-11T23:00:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: filestore apply finisher threads = 3 2024-05-11T23:00:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: bdev debug aio = true 2024-05-11T23:00:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: osd debug misdirected ops = true 2024-05-11T23:00:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore block size = 96636764160 2024-05-11T23:00:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore compression algorithm = snappy 2024-05-11T23:00:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore compression mode = aggressive 2024-05-11T23:00:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: bluestore fsck on mount = True 2024-05-11T23:00:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: debug bluefs = 1/20 2024-05-11T23:00:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: debug bluestore = 1/20 2024-05-11T23:00:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-05-11T23:00:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: debug osd = 20 2024-05-11T23:00:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: debug rocksdb = 4/10 2024-05-11T23:00:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd backfillfull_ratio = 0.85 2024-05-11T23:00:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd full ratio = 0.9 2024-05-11T23:00:29.919 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd nearfull ratio = 0.8 2024-05-11T23:00:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: osd blocked scrub grace period = 3600 2024-05-11T23:00:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: osd failsafe full ratio = 0.95 2024-05-11T23:00:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: osd max markdown count = 1000 2024-05-11T23:00:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: osd objectstore = bluestore 2024-05-11T23:00:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: osd shutdown pgref assert = True 2024-05-11T23:00:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.920 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr] 2024-05-11T23:00:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-05-11T23:00:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: debug mgr = 20 2024-05-11T23:00:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: debug mon = 20 2024-05-11T23:00:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: debug auth = 20 2024-05-11T23:00:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min pgs per osd = 4 2024-05-11T23:00:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min bytes per osd = 10 2024-05-11T23:00:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: mgr/telemetry/nag = false 2024-05-11T23:00:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.920 INFO:teuthology.orchestra.run.smithi115.stdout:[mon] 2024-05-11T23:00:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: debug ms = 1 2024-05-11T23:00:29.920 INFO:teuthology.orchestra.run.smithi115.stdout: debug mon = 20 2024-05-11T23:00:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: debug paxos = 20 2024-05-11T23:00:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: debug auth = 20 2024-05-11T23:00:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: mon data avail warn = 5 2024-05-11T23:00:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: mon mgr mkfs grace = 240 2024-05-11T23:00:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min pgs per osd = 4 2024-05-11T23:00:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd reporter subtree level = osd 2024-05-11T23:00:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: mon osd prime pg temp = true 2024-05-11T23:00:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: mon reweight min bytes per osd = 10 2024-05-11T23:00:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-11T23:00:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: auth mon ticket ttl = 660 # 11m 2024-05-11T23:00:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: auth service ticket ttl = 240 # 4m 2024-05-11T23:00:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: # don't complain about insecure global_id in the test suite 2024-05-11T23:00:29.921 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-11T23:00:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-11T23:00:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: # 1m isn't quite enough 2024-05-11T23:00:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: mon_down_mkfs_grace = 2m 2024-05-11T23:00:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: mon_warn_on_filestore_osds = false 2024-05-11T23:00:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-05-11T23:00:29.922 INFO:teuthology.orchestra.run.smithi115.stdout:[client] 2024-05-11T23:00:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: rgw cache enabled = true 2024-05-11T23:00:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: rgw enable ops log = true 2024-05-11T23:00:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: rgw enable usage log = true 2024-05-11T23:00:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-11T23:00:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-11T23:00:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: rbd cache = True 2024-05-11T23:00:29.922 INFO:teuthology.orchestra.run.smithi115.stdout: rbd cache policy = writearound 2024-05-11T23:00:29.922 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.a] 2024-05-11T23:00:29.922 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.c] 2024-05-11T23:00:29.923 INFO:teuthology.orchestra.run.smithi115.stdout:[mon.b] 2024-05-11T23:00:29.924 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-11T23:00:29.924 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-11T23:00:30.020 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-11T23:00:30.020 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T23:00:30.021 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-11T23:00:30.036 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T23:00:30.036 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-11T23:00:30.092 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi005.front.sepia.ceph.com 2024-05-11T23:00:30.092 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T23:00:30.092 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-11T23:00:30.092 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-11T23:00:30.167 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T23:00:30.168 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-11T23:00:30.224 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi115.front.sepia.ceph.com 2024-05-11T23:00:30.224 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-11T23:00:30.224 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-11T23:00:30.224 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-11T23:00:30.260 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-11T23:00:30.261 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-11T23:00:30.316 INFO:tasks.ceph:Setting up mon nodes... 2024-05-11T23:00:30.316 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-11T23:00:30.316 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-11T23:00:30.368 INFO:teuthology.orchestra.run.smithi005.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-11T23:00:30.371 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-11T23:00:30.454 INFO:teuthology.orchestra.run.smithi115.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-11T23:00:30.456 INFO:tasks.ceph:Setting up mds nodes... 2024-05-11T23:00:30.457 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-11T23:00:30.457 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-11T23:00:30.499 INFO:teuthology.orchestra.run.smithi005.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-11T23:00:30.512 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-11T23:00:30.552 INFO:teuthology.orchestra.run.smithi115.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-11T23:00:30.565 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-11T23:00:30.565 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi005.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi115.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-11T23:00:30.565 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-11T23:00:30.590 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-11T23:00:30.591 INFO:tasks.ceph:role: osd.0 2024-05-11T23:00:30.591 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-11T23:00:30.591 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-11T23:00:30.656 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T23:00:30.656 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T23:00:30.656 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T23:00:30.656 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T23:00:30.656 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T23:00:30.656 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-05-11T23:00:30.656 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T23:00:30.656 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T23:00:30.656 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T23:00:30.656 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T23:00:30.689 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-05-11T23:00:30.691 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-05-11T23:00:30.691 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-11T23:00:30.762 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-11T23:00:30.832 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-11T23:00:30.899 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-11T23:00:30.899 INFO:tasks.ceph:role: osd.1 2024-05-11T23:00:30.899 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-11T23:00:30.899 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-11T23:00:30.963 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T23:00:30.964 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T23:00:30.964 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T23:00:30.964 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T23:00:30.964 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T23:00:30.964 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-05-11T23:00:30.964 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T23:00:30.964 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T23:00:30.964 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T23:00:30.964 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T23:00:30.996 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-05-11T23:00:30.998 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-05-11T23:00:30.998 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-11T23:00:31.067 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-11T23:00:31.138 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-11T23:00:31.203 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-11T23:00:31.203 INFO:tasks.ceph:role: osd.2 2024-05-11T23:00:31.203 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-11T23:00:31.203 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-11T23:00:31.269 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T23:00:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T23:00:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T23:00:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T23:00:31.269 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T23:00:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-05-11T23:00:31.270 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T23:00:31.270 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T23:00:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T23:00:31.270 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T23:00:31.302 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-05-11T23:00:31.304 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-05-11T23:00:31.304 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-11T23:00:31.381 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-11T23:00:31.450 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-11T23:00:31.517 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-11T23:00:31.517 INFO:tasks.ceph:role: osd.3 2024-05-11T23:00:31.517 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-11T23:00:31.517 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-11T23:00:31.583 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T23:00:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T23:00:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T23:00:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T23:00:31.583 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T23:00:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-05-11T23:00:31.583 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T23:00:31.583 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T23:00:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T23:00:31.583 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T23:00:31.615 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-05-11T23:00:31.617 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-05-11T23:00:31.618 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-11T23:00:31.687 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-11T23:00:31.756 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T23:00:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:31.875+0000 7f8708aae640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-11T23:00:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:31.875+0000 7f8708aae640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-11T23:00:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:31.875+0000 7f8708aae640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-11T23:00:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:31.875+0000 7f8708aae640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-11T23:00:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:31.875+0000 7f8708aae640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-11T23:00:31.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:31.875+0000 7f8708aae640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-11T23:00:34.724 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-11T23:00:34.754 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T23:00:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:34.846+0000 7fe7e1e05640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-11T23:00:34.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:34.846+0000 7fe7e1e05640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-11T23:00:34.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:34.846+0000 7fe7e1e05640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-11T23:00:34.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:34.846+0000 7fe7e1e05640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-11T23:00:34.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:34.846+0000 7fe7e1e05640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-11T23:00:34.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:34.846+0000 7fe7e1e05640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-11T23:00:37.696 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-11T23:00:37.726 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T23:00:37.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:37.817+0000 7f2b08607640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-11T23:00:37.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:37.817+0000 7f2b08607640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-11T23:00:37.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:37.817+0000 7f2b08607640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-11T23:00:37.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:37.817+0000 7f2b08607640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-11T23:00:37.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:37.817+0000 7f2b08607640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-11T23:00:37.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:37.817+0000 7f2b08607640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-11T23:00:40.669 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-11T23:00:40.700 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T23:00:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:40.791+0000 7f1643615640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-11T23:00:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:40.791+0000 7f1643615640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-11T23:00:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:40.791+0000 7f1643615640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-11T23:00:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:40.791+0000 7f1643615640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-11T23:00:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:40.791+0000 7f1643615640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-11T23:00:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-05-11T23:00:40.792+0000 7f1643615640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-11T23:00:43.637 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-11T23:00:43.669 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-11T23:00:43.698 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-11T23:00:43.698 INFO:tasks.ceph:role: osd.4 2024-05-11T23:00:43.698 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi115.front.sepia.ceph.com 2024-05-11T23:00:43.698 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-11T23:00:43.774 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T23:00:43.774 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T23:00:43.774 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T23:00:43.774 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T23:00:43.774 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T23:00:43.774 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-05-11T23:00:43.775 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T23:00:43.775 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T23:00:43.775 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T23:00:43.775 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T23:00:44.287 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-05-11T23:00:44.289 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-05-11T23:00:44.290 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-11T23:00:44.410 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-11T23:00:44.440 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-11T23:00:44.505 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-11T23:00:44.505 INFO:tasks.ceph:role: osd.5 2024-05-11T23:00:44.505 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi115.front.sepia.ceph.com 2024-05-11T23:00:44.505 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-11T23:00:44.578 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T23:00:44.578 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T23:00:44.578 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T23:00:44.578 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T23:00:44.578 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T23:00:44.578 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-05-11T23:00:44.579 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T23:00:44.579 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T23:00:44.579 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T23:00:44.579 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T23:00:45.045 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-05-11T23:00:45.048 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-05-11T23:00:45.048 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-11T23:00:45.143 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-11T23:00:45.171 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-11T23:00:45.236 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-11T23:00:45.236 INFO:tasks.ceph:role: osd.6 2024-05-11T23:00:45.236 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi115.front.sepia.ceph.com 2024-05-11T23:00:45.236 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-11T23:00:45.307 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T23:00:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T23:00:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T23:00:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T23:00:45.308 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T23:00:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-05-11T23:00:45.308 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T23:00:45.308 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T23:00:45.308 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T23:00:45.308 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T23:00:45.794 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-05-11T23:00:45.796 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-05-11T23:00:45.797 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-11T23:00:45.945 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-11T23:00:45.978 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-11T23:00:46.044 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-11T23:00:46.044 INFO:tasks.ceph:role: osd.7 2024-05-11T23:00:46.044 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi115.front.sepia.ceph.com 2024-05-11T23:00:46.044 DEBUG:teuthology.orchestra.run.smithi115:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-11T23:00:46.116 INFO:teuthology.orchestra.run.smithi115.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-11T23:00:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-11T23:00:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-11T23:00:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-11T23:00:46.117 INFO:teuthology.orchestra.run.smithi115.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-11T23:00:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: = sunit=0 swidth=0 blks 2024-05-11T23:00:46.117 INFO:teuthology.orchestra.run.smithi115.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-11T23:00:46.117 INFO:teuthology.orchestra.run.smithi115.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-11T23:00:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-11T23:00:46.117 INFO:teuthology.orchestra.run.smithi115.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-11T23:00:46.637 INFO:teuthology.orchestra.run.smithi115.stdout:Discarding blocks...Done. 2024-05-11T23:00:46.639 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi115.front.sepia.ceph.com -o noatime 2024-05-11T23:00:46.639 DEBUG:teuthology.orchestra.run.smithi115:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-11T23:00:46.782 DEBUG:teuthology.orchestra.run.smithi115:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-11T23:00:46.814 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T23:00:46.913 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:46.912+0000 7fdb6a611640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-11T23:00:46.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:46.913+0000 7fdb6a611640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-11T23:00:46.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:46.913+0000 7fdb6a611640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-11T23:00:46.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:46.913+0000 7fdb6a611640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-11T23:00:46.914 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:46.913+0000 7fdb6a611640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-11T23:00:46.915 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:46.915+0000 7fdb6a611640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-11T23:00:49.779 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-11T23:00:49.808 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T23:00:49.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:49.897+0000 7fa89f218640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-11T23:00:49.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:49.897+0000 7fa89f218640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-11T23:00:49.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:49.898+0000 7fa89f218640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-11T23:00:49.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:49.898+0000 7fa89f218640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-11T23:00:49.898 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:49.898+0000 7fa89f218640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-11T23:00:49.899 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:49.898+0000 7fa89f218640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-11T23:00:52.747 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-11T23:00:52.777 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T23:00:52.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:52.867+0000 7f480bc11640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-11T23:00:52.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:52.867+0000 7f480bc11640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-11T23:00:52.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:52.867+0000 7f480bc11640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-11T23:00:52.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:52.867+0000 7f480bc11640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-11T23:00:52.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:52.867+0000 7f480bc11640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-11T23:00:52.868 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:52.867+0000 7f480bc11640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-11T23:00:55.716 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-11T23:00:55.741 DEBUG:teuthology.orchestra.run.smithi115:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-11T23:00:55.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:55.826+0000 7f4e05a1f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-11T23:00:55.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:55.826+0000 7f4e05a1f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-11T23:00:55.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:55.826+0000 7f4e05a1f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-11T23:00:55.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:55.826+0000 7f4e05a1f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-11T23:00:55.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:55.826+0000 7f4e05a1f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-11T23:00:55.827 INFO:teuthology.orchestra.run.smithi115.stderr:2024-05-11T23:00:55.827+0000 7f4e05a1f640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-11T23:00:58.677 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-11T23:00:58.705 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-11T23:00:58.705 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T23:00:58.705 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-11T23:00:58.736 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T23:00:58.736 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-11T23:00:58.807 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T23:00:58.807 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-11T23:00:58.876 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T23:00:58.876 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-11T23:00:58.945 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T23:00:58.945 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-11T23:00:59.014 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-11T23:00:59.015 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-11T23:00:59.045 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-11T23:00:59.046 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-11T23:00:59.113 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-11T23:00:59.114 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-11T23:00:59.181 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-11T23:00:59.181 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-11T23:00:59.248 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-11T23:00:59.248 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-11T23:00:59.315 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T23:00:59.316 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-11T23:00:59.335 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-11T23:00:59.335 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-11T23:00:59.372 INFO:tasks.ceph:Adding keys to all mons... 2024-05-11T23:00:59.372 DEBUG:teuthology.orchestra.run.smithi005:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-11T23:00:59.378 DEBUG:teuthology.orchestra.run.smithi115:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-11T23:00:59.415 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr.y] 2024-05-11T23:00:59.416 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCO+D9mBgsGFhAA/I4twJqHjhwbsv3xAs9Ijw== 2024-05-11T23:00:59.416 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.0] 2024-05-11T23:00:59.416 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCP+D9m0K5LNBAA1QnpD0o9yfSxJOwtvcVwFw== 2024-05-11T23:00:59.416 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.1] 2024-05-11T23:00:59.416 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCS+D9ml8V8MhAATnst+XNXMgFSIV45doYsIA== 2024-05-11T23:00:59.416 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.2] 2024-05-11T23:00:59.416 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCV+D9mmS/CMBAA3omd27bRBbpvKJoQL1+I0g== 2024-05-11T23:00:59.416 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.3] 2024-05-11T23:00:59.416 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCY+D9mpiU6LxAAUOsbeFsow2fTzGB+OZWwaw== 2024-05-11T23:00:59.416 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr.x] 2024-05-11T23:00:59.416 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCO+D9mrBIdGxAAibsR8+epA8Ig2KHJNRClRA== 2024-05-11T23:00:59.416 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.4] 2024-05-11T23:00:59.416 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCe+D9mMgt+NhAAQcjWjfx1La+L0ALZ+rMjXw== 2024-05-11T23:00:59.417 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.5] 2024-05-11T23:00:59.417 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCh+D9m6kqUNRAAz7+8zCj8n2NR1O09YZ0HeA== 2024-05-11T23:00:59.417 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.6] 2024-05-11T23:00:59.417 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCk+D9mzWjBMxAA5MRj5foNv3e/BtcI/OG22A== 2024-05-11T23:00:59.417 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.7] 2024-05-11T23:00:59.417 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCn+D9mynZWMRAAIFSrb6acDsk6GJ5efrSK8g== 2024-05-11T23:00:59.417 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-05-11T23:00:59.417 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCO+D9mpbPSHRAAwTjjv6T5KRSjiAA/f/C1cQ== 2024-05-11T23:00:59.417 INFO:teuthology.orchestra.run.smithi005.stdout:[client.1] 2024-05-11T23:00:59.417 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCO+D9m/98CIRAAxIR0kBW12N3dx/HJ3tHcoQ== 2024-05-11T23:00:59.437 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr.y] 2024-05-11T23:00:59.437 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQCO+D9mBgsGFhAA/I4twJqHjhwbsv3xAs9Ijw== 2024-05-11T23:00:59.437 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.0] 2024-05-11T23:00:59.437 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQCP+D9m0K5LNBAA1QnpD0o9yfSxJOwtvcVwFw== 2024-05-11T23:00:59.437 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.1] 2024-05-11T23:00:59.438 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQCS+D9ml8V8MhAATnst+XNXMgFSIV45doYsIA== 2024-05-11T23:00:59.438 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.2] 2024-05-11T23:00:59.438 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQCV+D9mmS/CMBAA3omd27bRBbpvKJoQL1+I0g== 2024-05-11T23:00:59.438 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.3] 2024-05-11T23:00:59.438 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQCY+D9mpiU6LxAAUOsbeFsow2fTzGB+OZWwaw== 2024-05-11T23:00:59.438 INFO:teuthology.orchestra.run.smithi115.stdout:[mgr.x] 2024-05-11T23:00:59.438 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQCO+D9mrBIdGxAAibsR8+epA8Ig2KHJNRClRA== 2024-05-11T23:00:59.438 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.4] 2024-05-11T23:00:59.438 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQCe+D9mMgt+NhAAQcjWjfx1La+L0ALZ+rMjXw== 2024-05-11T23:00:59.438 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.5] 2024-05-11T23:00:59.438 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQCh+D9m6kqUNRAAz7+8zCj8n2NR1O09YZ0HeA== 2024-05-11T23:00:59.438 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.6] 2024-05-11T23:00:59.438 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQCk+D9mzWjBMxAA5MRj5foNv3e/BtcI/OG22A== 2024-05-11T23:00:59.438 INFO:teuthology.orchestra.run.smithi115.stdout:[osd.7] 2024-05-11T23:00:59.438 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQCn+D9mynZWMRAAIFSrb6acDsk6GJ5efrSK8g== 2024-05-11T23:00:59.439 INFO:teuthology.orchestra.run.smithi115.stdout:[client.0] 2024-05-11T23:00:59.439 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQCO+D9mpbPSHRAAwTjjv6T5KRSjiAA/f/C1cQ== 2024-05-11T23:00:59.439 INFO:teuthology.orchestra.run.smithi115.stdout:[client.1] 2024-05-11T23:00:59.439 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQCO+D9m/98CIRAAxIR0kBW12N3dx/HJ3tHcoQ== 2024-05-11T23:00:59.440 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-11T23:00:59.461 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-11T23:00:59.532 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T23:00:59.535 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T23:00:59.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T23:00:59.590 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T23:00:59.641 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T23:00:59.643 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T23:00:59.694 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T23:00:59.696 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T23:00:59.746 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-11T23:00:59.749 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-11T23:00:59.799 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T23:00:59.801 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T23:00:59.852 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T23:00:59.854 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T23:00:59.905 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T23:00:59.908 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T23:00:59.958 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T23:00:59.961 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-11T23:01:00.011 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T23:01:00.013 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T23:01:00.066 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T23:01:00.068 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-11T23:01:00.122 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-11T23:01:00.123 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-11T23:01:00.148 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-11T23:01:00.246 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-11T23:01:00.275 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-11T23:01:00.341 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-11T23:01:00.439 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-11T23:01:00.465 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-11T23:01:00.490 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-11T23:01:00.587 DEBUG:teuthology.orchestra.run.smithi115:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-11T23:01:00.613 DEBUG:teuthology.orchestra.run.smithi005:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-11T23:01:00.616 DEBUG:teuthology.orchestra.run.smithi115:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-11T23:01:00.668 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-11T23:01:00.668 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-11T23:01:00.669 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-11T23:01:00.672 INFO:tasks.ceph.mon.a:Started 2024-05-11T23:01:00.672 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-11T23:01:00.672 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-11T23:01:00.674 INFO:tasks.ceph.mon.c:Started 2024-05-11T23:01:00.675 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-11T23:01:00.675 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-11T23:01:00.711 INFO:tasks.ceph.mon.b:Started 2024-05-11T23:01:00.711 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-11T23:01:00.712 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-11T23:01:00.712 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-11T23:01:00.715 INFO:tasks.ceph.mgr.y:Started 2024-05-11T23:01:00.715 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-11T23:01:00.716 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-11T23:01:00.718 INFO:tasks.ceph.mgr.x:Started 2024-05-11T23:01:00.718 INFO:tasks.ceph:Setting crush tunables to default 2024-05-11T23:01:00.719 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --cluster ceph osd crush tunables default 2024-05-11T23:01:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:adjusted tunables profile to default 2024-05-11T23:01:06.909 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:06.910+0000 7f637aefa040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T23:01:06.919 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:06.919+0000 7fdd51c2e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T23:01:06.921 INFO:tasks.ceph:check_enable_crimson: False 2024-05-11T23:01:06.921 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-11T23:01:06.921 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T23:01:06.922 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-11T23:01:06.950 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T23:01:06.950 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-11T23:01:07.016 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T23:01:07.017 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-11T23:01:07.067 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:07.068+0000 7f637aefa040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T23:01:07.083 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-05-11T23:01:07.083 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-11T23:01:07.091 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:07.091+0000 7fdd51c2e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T23:01:07.116 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-11T23:01:07.116 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-11T23:01:07.120 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:07.121+0000 7f637aefa040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T23:01:07.142 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-11T23:01:07.142 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-11T23:01:07.148 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:07.148+0000 7fdd51c2e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T23:01:07.207 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-11T23:01:07.207 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-11T23:01:07.272 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-05-11T23:01:07.272 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-11T23:01:07.338 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new 57ab1c70-9478-4a26-bb41-d753b4f3eb79 0 2024-05-11T23:01:07.385 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:07.386+0000 7f637aefa040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T23:01:07.416 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:07.416+0000 7fdd51c2e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T23:01:07.433 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:07.434+0000 7f637aefa040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T23:01:07.464 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:07.463+0000 7fdd51c2e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T23:01:07.566 INFO:teuthology.orchestra.run.smithi115.stdout:0 2024-05-11T23:01:07.577 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new 05ece110-4a99-46f0-b061-5980f7ef2a55 1 2024-05-11T23:01:07.808 INFO:teuthology.orchestra.run.smithi115.stdout:1 2024-05-11T23:01:07.820 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new 4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5 2 2024-05-11T23:01:08.009 INFO:teuthology.orchestra.run.smithi115.stdout:2 2024-05-11T23:01:08.022 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new 5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a 3 2024-05-11T23:01:08.194 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:08.195+0000 7f637aefa040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T23:01:08.209 INFO:teuthology.orchestra.run.smithi115.stdout:3 2024-05-11T23:01:08.219 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:08.219+0000 7fdd51c2e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T23:01:08.220 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new 4e87e4f0-288e-4ee6-919e-9337925a5d5e 4 2024-05-11T23:01:08.287 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:08.288+0000 7f637aefa040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T23:01:08.310 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:08.310+0000 7fdd51c2e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T23:01:08.332 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:08.332+0000 7f637aefa040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T23:01:08.355 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:08.355+0000 7fdd51c2e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T23:01:08.403 INFO:teuthology.orchestra.run.smithi115.stdout:4 2024-05-11T23:01:08.415 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5 5 2024-05-11T23:01:08.503 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:08.503+0000 7f637aefa040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T23:01:08.529 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:08.529+0000 7fdd51c2e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T23:01:08.556 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:08.556+0000 7f637aefa040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T23:01:08.586 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:08.586+0000 7fdd51c2e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T23:01:08.597 INFO:teuthology.orchestra.run.smithi115.stdout:5 2024-05-11T23:01:08.609 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new edb686ec-e1a2-40d6-975a-e32e0e7760ff 6 2024-05-11T23:01:08.609 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:08.610+0000 7f637aefa040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T23:01:08.642 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:08.642+0000 7fdd51c2e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T23:01:08.730 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:08.730+0000 7f637aefa040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T23:01:08.760 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:08.760+0000 7fdd51c2e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T23:01:08.791 INFO:teuthology.orchestra.run.smithi115.stdout:6 2024-05-11T23:01:08.803 DEBUG:teuthology.orchestra.run.smithi115:> sudo ceph --cluster ceph osd new 7aa14dbb-c6cb-4540-b160-3fd346f6a046 7 2024-05-11T23:01:08.987 INFO:teuthology.orchestra.run.smithi115.stdout:7 2024-05-11T23:01:08.999 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-11T23:01:08.999 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-11T23:01:09.002 INFO:tasks.ceph.osd.0:Started 2024-05-11T23:01:09.002 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-11T23:01:09.002 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-11T23:01:09.004 INFO:tasks.ceph.osd.1:Started 2024-05-11T23:01:09.004 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-11T23:01:09.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-11T23:01:09.006 INFO:tasks.ceph.osd.2:Started 2024-05-11T23:01:09.007 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-11T23:01:09.007 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-11T23:01:09.009 INFO:tasks.ceph.osd.3:Started 2024-05-11T23:01:09.009 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-11T23:01:09.009 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-11T23:01:09.012 INFO:tasks.ceph.osd.4:Started 2024-05-11T23:01:09.012 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-11T23:01:09.012 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-11T23:01:09.014 INFO:tasks.ceph.osd.5:Started 2024-05-11T23:01:09.015 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-11T23:01:09.015 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-11T23:01:09.017 INFO:tasks.ceph.osd.6:Started 2024-05-11T23:01:09.018 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-11T23:01:09.018 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-11T23:01:09.020 INFO:tasks.ceph.osd.7:Started 2024-05-11T23:01:09.021 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T23:01:09.159 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:09.160+0000 7f637aefa040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T23:01:09.205 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:09.205+0000 7fdd51c2e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T23:01:09.230 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:01:09.230 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":10,"fsid":"34b20022-f11b-4169-a5f6-6386270a2f8f","created":"2024-05-11T23:01:05.977693+0000","modified":"2024-05-11T23:01:08.985275+0000","last_up_change":"0.000000","last_in_change":"2024-05-11T23:01:08.985275+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"57ab1c70-9478-4a26-bb41-d753b4f3eb79","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"05ece110-4a99-46f0-b061-5980f7ef2a55","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4e87e4f0-288e-4ee6-919e-9337925a5d5e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"edb686ec-e1a2-40d6-975a-e32e0e7760ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7aa14dbb-c6cb-4540-b160-3fd346f6a046","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T23:01:09.241 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-11T23:01:09.241 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-11T23:01:09.417 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:09.417+0000 7f637aefa040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T23:01:09.468 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:09.469+0000 7f637aefa040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T23:01:09.477 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:09.477+0000 7fdd51c2e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T23:01:09.532 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:09.532+0000 7fdd51c2e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T23:01:09.583 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:09.583+0000 7f637aefa040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T23:01:09.620 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:09.620+0000 7f208a263640 -1 Falling back to public interface 2024-05-11T23:01:09.620 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:09.620+0000 7f66aa380640 -1 Falling back to public interface 2024-05-11T23:01:09.624 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:09.624+0000 7f8373aff640 -1 Falling back to public interface 2024-05-11T23:01:09.624 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:09.624+0000 7fee2ec0c640 -1 Falling back to public interface 2024-05-11T23:01:09.627 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:09.627+0000 7fe984b18640 -1 Falling back to public interface 2024-05-11T23:01:09.627 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:09.627+0000 7f80093dc640 -1 Falling back to public interface 2024-05-11T23:01:09.634 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:09.634+0000 7f1114799640 -1 Falling back to public interface 2024-05-11T23:01:09.636 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:09.636+0000 7fba09621640 -1 Falling back to public interface 2024-05-11T23:01:09.661 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:09.661+0000 7fdd51c2e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T23:01:09.863 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:09.864+0000 7f637aefa040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T23:01:09.911 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:09.912+0000 7f637aefa040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T23:01:09.969 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:09.969+0000 7fdd51c2e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T23:01:10.020 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:10.020+0000 7fdd51c2e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T23:01:10.046 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-11T23:01:10.067 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:10.068+0000 7f637aefa040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T23:01:10.190 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:10.190+0000 7fdd51c2e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T23:01:10.203 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:10.204+0000 7f637aefa040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T23:01:10.215 INFO:teuthology.misc.health.smithi005.stdout: 2024-05-11T23:01:10.215 INFO:teuthology.misc.health.smithi005.stdout:{"epoch":10,"fsid":"34b20022-f11b-4169-a5f6-6386270a2f8f","created":"2024-05-11T23:01:05.977693+0000","modified":"2024-05-11T23:01:08.985275+0000","last_up_change":"0.000000","last_in_change":"2024-05-11T23:01:08.985275+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"57ab1c70-9478-4a26-bb41-d753b4f3eb79","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"05ece110-4a99-46f0-b061-5980f7ef2a55","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4e87e4f0-288e-4ee6-919e-9337925a5d5e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"edb686ec-e1a2-40d6-975a-e32e0e7760ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7aa14dbb-c6cb-4540-b160-3fd346f6a046","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T23:01:10.226 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-11T23:01:10.253 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:10.253+0000 7f637aefa040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T23:01:10.321 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:10.322+0000 7f637aefa040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T23:01:10.335 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:10.335+0000 7fdd51c2e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T23:01:10.366 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:10.367+0000 7f637aefa040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T23:01:10.386 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:10.386+0000 7fdd51c2e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T23:01:10.461 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:10.460+0000 7fdd51c2e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T23:01:10.508 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:10.508+0000 7fdd51c2e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T23:01:10.837 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:10.836+0000 7f637aefa040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T23:01:10.950 INFO:tasks.ceph.mgr.x.smithi115.stderr:2024-05-11T23:01:10.949+0000 7f637aefa040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T23:01:11.008 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:11.008+0000 7fdd51c2e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T23:01:11.129 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:11.129+0000 7fdd51c2e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T23:01:12.141 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:12.141+0000 7fdcf2021640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-11T23:01:12.141 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:12.141+0000 7fdcf2021640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-11T23:01:13.088 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:13.088+0000 7f66aa380640 -1 osd.2 0 log_to_monitors true 2024-05-11T23:01:13.130 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:13.130+0000 7f208a263640 -1 osd.0 0 log_to_monitors true 2024-05-11T23:01:13.156 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:13.156+0000 7f8373aff640 -1 osd.4 0 log_to_monitors true 2024-05-11T23:01:13.163 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:13.162+0000 7fba09621640 -1 osd.6 0 log_to_monitors true 2024-05-11T23:01:13.165 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:13.165+0000 7f80093dc640 -1 osd.5 0 log_to_monitors true 2024-05-11T23:01:13.170 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:13.169+0000 7fe984b18640 -1 osd.7 0 log_to_monitors true 2024-05-11T23:01:13.172 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:13.172+0000 7f1114799640 -1 osd.1 0 log_to_monitors true 2024-05-11T23:01:13.177 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:13.177+0000 7fee2ec0c640 -1 osd.3 0 log_to_monitors true 2024-05-11T23:01:15.056 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:15.056+0000 7f66a6316640 -1 osd.2 0 waiting for initial osdmap 2024-05-11T23:01:15.070 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:15.070+0000 7f669e136640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T23:01:15.079 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:15.079+0000 7f20861f9640 -1 osd.0 0 waiting for initial osdmap 2024-05-11T23:01:15.089 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:15.089+0000 7f207e019640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T23:01:16.052 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:16.052+0000 7f111072f640 -1 osd.1 0 waiting for initial osdmap 2024-05-11T23:01:16.062 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:16.062+0000 7f110854f640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T23:01:16.066 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:16.066+0000 7fee2a4c1640 -1 osd.3 0 waiting for initial osdmap 2024-05-11T23:01:16.078 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:16.078+0000 7fee222e1640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T23:01:16.223 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:16.222+0000 7fba05064640 -1 osd.6 0 waiting for initial osdmap 2024-05-11T23:01:16.228 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:16.228+0000 7f836fa95640 -1 osd.4 0 waiting for initial osdmap 2024-05-11T23:01:16.234 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:16.234+0000 7fb9fce84640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T23:01:16.238 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:16.238+0000 7f8005372640 -1 osd.5 0 waiting for initial osdmap 2024-05-11T23:01:16.242 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:16.242+0000 7f83678b5640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T23:01:16.246 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:16.246+0000 7fe9812c2640 -1 osd.7 0 waiting for initial osdmap 2024-05-11T23:01:16.251 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:16.251+0000 7f7ffd192640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T23:01:16.257 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:16.257+0000 7fe9788ce640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T23:01:17.032 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-11T23:01:17.347 INFO:teuthology.misc.health.smithi005.stdout: 2024-05-11T23:01:17.348 INFO:teuthology.misc.health.smithi005.stdout:{"epoch":15,"fsid":"34b20022-f11b-4169-a5f6-6386270a2f8f","created":"2024-05-11T23:01:05.977693+0000","modified":"2024-05-11T23:01:17.148916+0000","last_up_change":"2024-05-11T23:01:17.148916+0000","last_in_change":"2024-05-11T23:01:08.985275+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T23:01:15.187614+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57ab1c70-9478-4a26-bb41-d753b4f3eb79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6801","nonce":3452725643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6803","nonce":3452725643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6808","nonce":3452725643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6805","nonce":3452725643}]},"public_addr":"172.21.15.5:6801/3452725643","cluster_addr":"172.21.15.5:6803/3452725643","heartbeat_back_addr":"172.21.15.5:6808/3452725643","heartbeat_front_addr":"172.21.15.5:6805/3452725643","state":["exists","up"]},{"osd":1,"uuid":"05ece110-4a99-46f0-b061-5980f7ef2a55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6825","nonce":4028727138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6827","nonce":4028727138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6831","nonce":4028727138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6829","nonce":4028727138}]},"public_addr":"172.21.15.5:6825/4028727138","cluster_addr":"172.21.15.5:6827/4028727138","heartbeat_back_addr":"172.21.15.5:6831/4028727138","heartbeat_front_addr":"172.21.15.5:6829/4028727138","state":["exists","up"]},{"osd":2,"uuid":"4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6809","nonce":2059959509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6811","nonce":2059959509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6815","nonce":2059959509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6813","nonce":2059959509}]},"public_addr":"172.21.15.5:6809/2059959509","cluster_addr":"172.21.15.5:6811/2059959509","heartbeat_back_addr":"172.21.15.5:6815/2059959509","heartbeat_front_addr":"172.21.15.5:6813/2059959509","state":["exists","up"]},{"osd":3,"uuid":"5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":296930146},{"type":"v1","addr":"172.21.15.5:6817","nonce":296930146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":296930146},{"type":"v1","addr":"172.21.15.5:6819","nonce":296930146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":296930146},{"type":"v1","addr":"172.21.15.5:6823","nonce":296930146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":296930146},{"type":"v1","addr":"172.21.15.5:6821","nonce":296930146}]},"public_addr":"172.21.15.5:6817/296930146","cluster_addr":"172.21.15.5:6819/296930146","heartbeat_back_addr":"172.21.15.5:6823/296930146","heartbeat_front_addr":"172.21.15.5:6821/296930146","state":["exists","up"]},{"osd":4,"uuid":"4e87e4f0-288e-4ee6-919e-9337925a5d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6801","nonce":495703077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6803","nonce":495703077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6807","nonce":495703077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6805","nonce":495703077}]},"public_addr":"172.21.15.115:6801/495703077","cluster_addr":"172.21.15.115:6803/495703077","heartbeat_back_addr":"172.21.15.115:6807/495703077","heartbeat_front_addr":"172.21.15.115:6805/495703077","state":["exists","up"]},{"osd":5,"uuid":"e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6811","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6813","nonce":902573489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6817","nonce":902573489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6823","nonce":902573489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6819","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6821","nonce":902573489}]},"public_addr":"172.21.15.115:6813/902573489","cluster_addr":"172.21.15.115:6817/902573489","heartbeat_back_addr":"172.21.15.115:6823/902573489","heartbeat_front_addr":"172.21.15.115:6821/902573489","state":["exists","up"]},{"osd":6,"uuid":"edb686ec-e1a2-40d6-975a-e32e0e7760ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6825","nonce":2749332272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6827","nonce":2749332272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6831","nonce":2749332272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6829","nonce":2749332272}]},"public_addr":"172.21.15.115:6825/2749332272","cluster_addr":"172.21.15.115:6827/2749332272","heartbeat_back_addr":"172.21.15.115:6831/2749332272","heartbeat_front_addr":"172.21.15.115:6829/2749332272","state":["exists","up"]},{"osd":7,"uuid":"7aa14dbb-c6cb-4540-b160-3fd346f6a046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6809","nonce":1043826337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6812","nonce":1043826337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6820","nonce":1043826337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6816","nonce":1043826337}]},"public_addr":"172.21.15.115:6809/1043826337","cluster_addr":"172.21.15.115:6812/1043826337","heartbeat_back_addr":"172.21.15.115:6820/1043826337","heartbeat_front_addr":"172.21.15.115:6816/1043826337","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.113232+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.174679+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.131118+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.207668+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T23:01:17.359 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-11T23:01:17.359 INFO:tasks.ceph:Creating RBD pool 2024-05-11T23:01:17.359 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --cluster ceph osd pool create rbd 8 2024-05-11T23:01:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'rbd' created 2024-05-11T23:01:18.176 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-05-11T23:01:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:enabled application 'rbd' on pool 'rbd' 2024-05-11T23:01:19.231 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-11T23:01:19.232 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-11T23:01:19.232 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-11T23:01:19.232 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-11T23:01:19.233 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-11T23:01:19.592 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:01:19.603 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":4,"flags":0,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1673401469},{"type":"v1","addr":"172.21.15.5:6833","nonce":1673401469}]},"active_addr":"172.21.15.5:6833/1673401469","active_change":"2024-05-11T23:01:11.131338+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":849530673}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":3184539195}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":546063188}]}]} 2024-05-11T23:01:19.608 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-11T23:01:19.608 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-11T23:01:19.609 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T23:01:19.918 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:01:19.918 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"fsid":"34b20022-f11b-4169-a5f6-6386270a2f8f","created":"2024-05-11T23:01:05.977693+0000","modified":"2024-05-11T23:01:19.155383+0000","last_up_change":"2024-05-11T23:01:17.148916+0000","last_in_change":"2024-05-11T23:01:08.985275+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T23:01:15.187614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-11T23:01:17.689285+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57ab1c70-9478-4a26-bb41-d753b4f3eb79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6801","nonce":3452725643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6803","nonce":3452725643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6808","nonce":3452725643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6805","nonce":3452725643}]},"public_addr":"172.21.15.5:6801/3452725643","cluster_addr":"172.21.15.5:6803/3452725643","heartbeat_back_addr":"172.21.15.5:6808/3452725643","heartbeat_front_addr":"172.21.15.5:6805/3452725643","state":["exists","up"]},{"osd":1,"uuid":"05ece110-4a99-46f0-b061-5980f7ef2a55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6825","nonce":4028727138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6827","nonce":4028727138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6831","nonce":4028727138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6829","nonce":4028727138}]},"public_addr":"172.21.15.5:6825/4028727138","cluster_addr":"172.21.15.5:6827/4028727138","heartbeat_back_addr":"172.21.15.5:6831/4028727138","heartbeat_front_addr":"172.21.15.5:6829/4028727138","state":["exists","up"]},{"osd":2,"uuid":"4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6809","nonce":2059959509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6811","nonce":2059959509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6815","nonce":2059959509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6813","nonce":2059959509}]},"public_addr":"172.21.15.5:6809/2059959509","cluster_addr":"172.21.15.5:6811/2059959509","heartbeat_back_addr":"172.21.15.5:6815/2059959509","heartbeat_front_addr":"172.21.15.5:6813/2059959509","state":["exists","up"]},{"osd":3,"uuid":"5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":296930146},{"type":"v1","addr":"172.21.15.5:6817","nonce":296930146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":296930146},{"type":"v1","addr":"172.21.15.5:6819","nonce":296930146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":296930146},{"type":"v1","addr":"172.21.15.5:6823","nonce":296930146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":296930146},{"type":"v1","addr":"172.21.15.5:6821","nonce":296930146}]},"public_addr":"172.21.15.5:6817/296930146","cluster_addr":"172.21.15.5:6819/296930146","heartbeat_back_addr":"172.21.15.5:6823/296930146","heartbeat_front_addr":"172.21.15.5:6821/296930146","state":["exists","up"]},{"osd":4,"uuid":"4e87e4f0-288e-4ee6-919e-9337925a5d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6801","nonce":495703077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6803","nonce":495703077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6807","nonce":495703077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6805","nonce":495703077}]},"public_addr":"172.21.15.115:6801/495703077","cluster_addr":"172.21.15.115:6803/495703077","heartbeat_back_addr":"172.21.15.115:6807/495703077","heartbeat_front_addr":"172.21.15.115:6805/495703077","state":["exists","up"]},{"osd":5,"uuid":"e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6811","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6813","nonce":902573489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6817","nonce":902573489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6823","nonce":902573489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6819","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6821","nonce":902573489}]},"public_addr":"172.21.15.115:6813/902573489","cluster_addr":"172.21.15.115:6817/902573489","heartbeat_back_addr":"172.21.15.115:6823/902573489","heartbeat_front_addr":"172.21.15.115:6821/902573489","state":["exists","up"]},{"osd":6,"uuid":"edb686ec-e1a2-40d6-975a-e32e0e7760ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6825","nonce":2749332272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6827","nonce":2749332272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6831","nonce":2749332272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6829","nonce":2749332272}]},"public_addr":"172.21.15.115:6825/2749332272","cluster_addr":"172.21.15.115:6827/2749332272","heartbeat_back_addr":"172.21.15.115:6831/2749332272","heartbeat_front_addr":"172.21.15.115:6829/2749332272","state":["exists","up"]},{"osd":7,"uuid":"7aa14dbb-c6cb-4540-b160-3fd346f6a046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6809","nonce":1043826337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6812","nonce":1043826337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6820","nonce":1043826337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6816","nonce":1043826337}]},"public_addr":"172.21.15.115:6809/1043826337","cluster_addr":"172.21.15.115:6812/1043826337","heartbeat_back_addr":"172.21.15.115:6820/1043826337","heartbeat_front_addr":"172.21.15.115:6816/1043826337","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.113232+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.174679+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.131118+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.207668+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.124542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.200832+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.169560+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.182465+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T23:01:19.931 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-11T23:01:19.931 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T23:01:20.235 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:01:20.235 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":18,"fsid":"34b20022-f11b-4169-a5f6-6386270a2f8f","created":"2024-05-11T23:01:05.977693+0000","modified":"2024-05-11T23:01:20.212450+0000","last_up_change":"2024-05-11T23:01:17.148916+0000","last_in_change":"2024-05-11T23:01:08.985275+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T23:01:15.187614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-11T23:01:17.689285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57ab1c70-9478-4a26-bb41-d753b4f3eb79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6801","nonce":3452725643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6803","nonce":3452725643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6808","nonce":3452725643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6805","nonce":3452725643}]},"public_addr":"172.21.15.5:6801/3452725643","cluster_addr":"172.21.15.5:6803/3452725643","heartbeat_back_addr":"172.21.15.5:6808/3452725643","heartbeat_front_addr":"172.21.15.5:6805/3452725643","state":["exists","up"]},{"osd":1,"uuid":"05ece110-4a99-46f0-b061-5980f7ef2a55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6825","nonce":4028727138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6827","nonce":4028727138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6831","nonce":4028727138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6829","nonce":4028727138}]},"public_addr":"172.21.15.5:6825/4028727138","cluster_addr":"172.21.15.5:6827/4028727138","heartbeat_back_addr":"172.21.15.5:6831/4028727138","heartbeat_front_addr":"172.21.15.5:6829/4028727138","state":["exists","up"]},{"osd":2,"uuid":"4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6809","nonce":2059959509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6811","nonce":2059959509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6815","nonce":2059959509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6813","nonce":2059959509}]},"public_addr":"172.21.15.5:6809/2059959509","cluster_addr":"172.21.15.5:6811/2059959509","heartbeat_back_addr":"172.21.15.5:6815/2059959509","heartbeat_front_addr":"172.21.15.5:6813/2059959509","state":["exists","up"]},{"osd":3,"uuid":"5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":296930146},{"type":"v1","addr":"172.21.15.5:6817","nonce":296930146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":296930146},{"type":"v1","addr":"172.21.15.5:6819","nonce":296930146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":296930146},{"type":"v1","addr":"172.21.15.5:6823","nonce":296930146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":296930146},{"type":"v1","addr":"172.21.15.5:6821","nonce":296930146}]},"public_addr":"172.21.15.5:6817/296930146","cluster_addr":"172.21.15.5:6819/296930146","heartbeat_back_addr":"172.21.15.5:6823/296930146","heartbeat_front_addr":"172.21.15.5:6821/296930146","state":["exists","up"]},{"osd":4,"uuid":"4e87e4f0-288e-4ee6-919e-9337925a5d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6801","nonce":495703077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6803","nonce":495703077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6807","nonce":495703077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6805","nonce":495703077}]},"public_addr":"172.21.15.115:6801/495703077","cluster_addr":"172.21.15.115:6803/495703077","heartbeat_back_addr":"172.21.15.115:6807/495703077","heartbeat_front_addr":"172.21.15.115:6805/495703077","state":["exists","up"]},{"osd":5,"uuid":"e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6811","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6813","nonce":902573489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6817","nonce":902573489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6823","nonce":902573489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6819","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6821","nonce":902573489}]},"public_addr":"172.21.15.115:6813/902573489","cluster_addr":"172.21.15.115:6817/902573489","heartbeat_back_addr":"172.21.15.115:6823/902573489","heartbeat_front_addr":"172.21.15.115:6821/902573489","state":["exists","up"]},{"osd":6,"uuid":"edb686ec-e1a2-40d6-975a-e32e0e7760ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6825","nonce":2749332272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6827","nonce":2749332272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6831","nonce":2749332272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6829","nonce":2749332272}]},"public_addr":"172.21.15.115:6825/2749332272","cluster_addr":"172.21.15.115:6827/2749332272","heartbeat_back_addr":"172.21.15.115:6831/2749332272","heartbeat_front_addr":"172.21.15.115:6829/2749332272","state":["exists","up"]},{"osd":7,"uuid":"7aa14dbb-c6cb-4540-b160-3fd346f6a046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6809","nonce":1043826337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6812","nonce":1043826337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6820","nonce":1043826337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6816","nonce":1043826337}]},"public_addr":"172.21.15.115:6809/1043826337","cluster_addr":"172.21.15.115:6812/1043826337","heartbeat_back_addr":"172.21.15.115:6820/1043826337","heartbeat_front_addr":"172.21.15.115:6816/1043826337","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.113232+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.174679+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.131118+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.207668+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.124542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.200832+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.169560+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.182465+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T23:01:20.248 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-11T23:01:20.249 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-11T23:01:20.249 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-11T23:01:20.249 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-11T23:01:20.250 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-11T23:01:20.250 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-11T23:01:20.251 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-11T23:01:20.251 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-11T23:01:20.500 INFO:teuthology.orchestra.run.smithi005.stdout:64424509442 2024-05-11T23:01:20.501 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-11T23:01:20.511 INFO:teuthology.orchestra.run.smithi005.stdout:60129542146 2024-05-11T23:01:20.512 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-11T23:01:20.513 INFO:teuthology.orchestra.run.smithi005.stdout:55834574852 2024-05-11T23:01:20.513 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-11T23:01:20.514 INFO:teuthology.orchestra.run.smithi005.stdout:60129542146 2024-05-11T23:01:20.514 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-11T23:01:20.516 INFO:teuthology.orchestra.run.smithi005.stdout:64424509443 2024-05-11T23:01:20.516 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-11T23:01:20.517 INFO:teuthology.orchestra.run.smithi005.stdout:55834574851 2024-05-11T23:01:20.517 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-11T23:01:20.518 INFO:teuthology.orchestra.run.smithi005.stdout:64424509442 2024-05-11T23:01:20.519 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-11T23:01:20.519 INFO:teuthology.orchestra.run.smithi005.stdout:64424509443 2024-05-11T23:01:20.520 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-11T23:01:21.099 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-05-11T23:01:21.120 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509442 got 1 for osd.6 2024-05-11T23:01:21.155 INFO:teuthology.orchestra.run.smithi005.stdout:64424509442 2024-05-11T23:01:21.169 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.4 2024-05-11T23:01:21.184 INFO:teuthology.orchestra.run.smithi005.stdout:55834574850 2024-05-11T23:01:21.191 INFO:teuthology.orchestra.run.smithi005.stdout:60129542145 2024-05-11T23:01:21.196 INFO:teuthology.orchestra.run.smithi005.stdout:60129542145 2024-05-11T23:01:21.196 INFO:teuthology.orchestra.run.smithi005.stdout:55834574849 2024-05-11T23:01:21.207 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574850 for osd.0 2024-05-11T23:01:21.210 INFO:teuthology.orchestra.run.smithi005.stdout:64424509441 2024-05-11T23:01:21.211 INFO:teuthology.orchestra.run.smithi005.stdout:64424509442 2024-05-11T23:01:21.212 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542146 got 60129542145 for osd.1 2024-05-11T23:01:21.213 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542146 got 60129542145 for osd.3 2024-05-11T23:01:21.214 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574849 for osd.2 2024-05-11T23:01:21.225 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509442 got 64424509441 for osd.7 2024-05-11T23:01:21.226 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.5 2024-05-11T23:01:22.121 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-11T23:01:22.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-11T23:01:22.207 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-11T23:01:22.213 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-11T23:01:22.214 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-11T23:01:22.215 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-11T23:01:22.227 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-11T23:01:22.228 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-11T23:01:22.690 INFO:teuthology.orchestra.run.smithi005.stdout:64424509442 2024-05-11T23:01:22.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509442 got 64424509442 for osd.6 2024-05-11T23:01:22.711 DEBUG:teuthology.parallel:result is None 2024-05-11T23:01:22.756 INFO:teuthology.orchestra.run.smithi005.stdout:64424509443 2024-05-11T23:01:22.774 INFO:teuthology.orchestra.run.smithi005.stdout:55834574851 2024-05-11T23:01:22.775 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.4 2024-05-11T23:01:22.775 DEBUG:teuthology.parallel:result is None 2024-05-11T23:01:22.788 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.2 2024-05-11T23:01:22.788 DEBUG:teuthology.parallel:result is None 2024-05-11T23:01:22.823 INFO:teuthology.orchestra.run.smithi005.stdout:60129542146 2024-05-11T23:01:22.824 INFO:teuthology.orchestra.run.smithi005.stdout:64424509443 2024-05-11T23:01:22.839 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.5 2024-05-11T23:01:22.839 DEBUG:teuthology.parallel:result is None 2024-05-11T23:01:22.840 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542146 got 60129542146 for osd.3 2024-05-11T23:01:22.840 DEBUG:teuthology.parallel:result is None 2024-05-11T23:01:22.855 INFO:teuthology.orchestra.run.smithi005.stdout:60129542146 2024-05-11T23:01:22.858 INFO:teuthology.orchestra.run.smithi005.stdout:55834574852 2024-05-11T23:01:22.859 INFO:teuthology.orchestra.run.smithi005.stdout:64424509442 2024-05-11T23:01:22.872 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542146 got 60129542146 for osd.1 2024-05-11T23:01:22.872 DEBUG:teuthology.parallel:result is None 2024-05-11T23:01:22.874 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509442 got 64424509442 for osd.7 2024-05-11T23:01:22.874 DEBUG:teuthology.parallel:result is None 2024-05-11T23:01:22.875 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.0 2024-05-11T23:01:22.875 DEBUG:teuthology.parallel:result is None 2024-05-11T23:01:22.875 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T23:01:22.875 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T23:01:23.213 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:01:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-11T23:01:23.226 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-05-11T23:01:23.138243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234200,"kb_used_data":1388,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754740520,"statfs":{"total":773094113280,"available":772854292480,"internally_reserved":0,"allocated":1421312,"data_stored":1584476,"data_compressed":65020,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000331"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.221043+0000","last_change":"2024-05-11T23:01:19.220007+0000","last_active":"2024-05-11T23:01:19.221043+0000","last_peered":"2024-05-11T23:01:19.221043+0000","last_clean":"2024-05-11T23:01:19.221043+0000","last_became_active":"2024-05-11T23:01:19.219681+0000","last_became_peered":"2024-05-11T23:01:19.219681+0000","last_unstale":"2024-05-11T23:01:19.221043+0000","last_undegraded":"2024-05-11T23:01:19.221043+0000","last_fullsized":"2024-05-11T23:01:19.221043+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:20:19.891140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.222055+0000","last_change":"2024-05-11T23:01:19.221663+0000","last_active":"2024-05-11T23:01:19.222055+0000","last_peered":"2024-05-11T23:01:19.222055+0000","last_clean":"2024-05-11T23:01:19.222055+0000","last_became_active":"2024-05-11T23:01:19.221440+0000","last_became_peered":"2024-05-11T23:01:19.221440+0000","last_unstale":"2024-05-11T23:01:19.222055+0000","last_undegraded":"2024-05-11T23:01:19.222055+0000","last_fullsized":"2024-05-11T23:01:19.222055+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:30:16.582232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.217814+0000","last_change":"2024-05-11T23:01:19.216700+0000","last_active":"2024-05-11T23:01:19.217814+0000","last_peered":"2024-05-11T23:01:19.217814+0000","last_clean":"2024-05-11T23:01:19.217814+0000","last_became_active":"2024-05-11T23:01:19.216413+0000","last_became_peered":"2024-05-11T23:01:19.216413+0000","last_unstale":"2024-05-11T23:01:19.217814+0000","last_undegraded":"2024-05-11T23:01:19.217814+0000","last_fullsized":"2024-05-11T23:01:19.217814+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:33:15.899460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.220973+0000","last_change":"2024-05-11T23:01:19.220232+0000","last_active":"2024-05-11T23:01:19.220973+0000","last_peered":"2024-05-11T23:01:19.220973+0000","last_clean":"2024-05-11T23:01:19.220973+0000","last_became_active":"2024-05-11T23:01:19.219917+0000","last_became_peered":"2024-05-11T23:01:19.219917+0000","last_unstale":"2024-05-11T23:01:19.220973+0000","last_undegraded":"2024-05-11T23:01:19.220973+0000","last_fullsized":"2024-05-11T23:01:19.220973+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:18:24.059625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T23:01:20.216249+0000","last_change":"2024-05-11T23:01:19.221118+0000","last_active":"2024-05-11T23:01:20.216249+0000","last_peered":"2024-05-11T23:01:20.216249+0000","last_clean":"2024-05-11T23:01:20.216249+0000","last_became_active":"2024-05-11T23:01:19.220855+0000","last_became_peered":"2024-05-11T23:01:19.220855+0000","last_unstale":"2024-05-11T23:01:20.216249+0000","last_undegraded":"2024-05-11T23:01:20.216249+0000","last_fullsized":"2024-05-11T23:01:20.216249+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:01:56.076963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.578787+0000","last_change":"2024-05-11T23:01:19.578258+0000","last_active":"2024-05-11T23:01:19.578787+0000","last_peered":"2024-05-11T23:01:19.578787+0000","last_clean":"2024-05-11T23:01:19.578787+0000","last_became_active":"2024-05-11T23:01:19.577973+0000","last_became_peered":"2024-05-11T23:01:19.577973+0000","last_unstale":"2024-05-11T23:01:19.578787+0000","last_undegraded":"2024-05-11T23:01:19.578787+0000","last_fullsized":"2024-05-11T23:01:19.578787+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:22:59.808422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.220850+0000","last_change":"2024-05-11T23:01:19.219755+0000","last_active":"2024-05-11T23:01:19.220850+0000","last_peered":"2024-05-11T23:01:19.220850+0000","last_clean":"2024-05-11T23:01:19.220850+0000","last_became_active":"2024-05-11T23:01:19.219471+0000","last_became_peered":"2024-05-11T23:01:19.219471+0000","last_unstale":"2024-05-11T23:01:19.220850+0000","last_undegraded":"2024-05-11T23:01:19.220850+0000","last_fullsized":"2024-05-11T23:01:19.220850+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:52:12.584484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T23:01:20.216253+0000","last_change":"2024-05-11T23:01:19.219128+0000","last_active":"2024-05-11T23:01:20.216253+0000","last_peered":"2024-05-11T23:01:20.216253+0000","last_clean":"2024-05-11T23:01:20.216253+0000","last_became_active":"2024-05-11T23:01:19.218846+0000","last_became_peered":"2024-05-11T23:01:19.218846+0000","last_unstale":"2024-05-11T23:01:20.216253+0000","last_undegraded":"2024-05-11T23:01:20.216253+0000","last_fullsized":"2024-05-11T23:01:20.216253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:41:09.111000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.331465+0000","last_change":"2024-05-11T23:01:18.164452+0000","last_active":"2024-05-11T23:01:19.331465+0000","last_peered":"2024-05-11T23:01:19.331465+0000","last_clean":"2024-05-11T23:01:19.331465+0000","last_became_active":"2024-05-11T23:01:18.163431+0000","last_became_peered":"2024-05-11T23:01:18.163431+0000","last_unstale":"2024-05-11T23:01:19.331465+0000","last_undegraded":"2024-05-11T23:01:19.331465+0000","last_fullsized":"2024-05-11T23:01:19.331465+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_clean_scrub_stamp":"2024-05-11T23:01:16.147564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:40:59.974639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65020,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509442,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":640117,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":159744,"data_stored":49749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":159744,"data_stored":49749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":159744,"data_stored":49749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":172032,"data_stored":55497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":159744,"data_stored":49749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":159744,"data_stored":49749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":225280,"data_stored":640117,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T23:01:23.227 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T23:01:23.525 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:01:23.526 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-11T23:01:23.538 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-05-11T23:01:23.138243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234200,"kb_used_data":1388,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754740520,"statfs":{"total":773094113280,"available":772854292480,"internally_reserved":0,"allocated":1421312,"data_stored":1584476,"data_compressed":65020,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000331"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.221043+0000","last_change":"2024-05-11T23:01:19.220007+0000","last_active":"2024-05-11T23:01:19.221043+0000","last_peered":"2024-05-11T23:01:19.221043+0000","last_clean":"2024-05-11T23:01:19.221043+0000","last_became_active":"2024-05-11T23:01:19.219681+0000","last_became_peered":"2024-05-11T23:01:19.219681+0000","last_unstale":"2024-05-11T23:01:19.221043+0000","last_undegraded":"2024-05-11T23:01:19.221043+0000","last_fullsized":"2024-05-11T23:01:19.221043+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:20:19.891140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.222055+0000","last_change":"2024-05-11T23:01:19.221663+0000","last_active":"2024-05-11T23:01:19.222055+0000","last_peered":"2024-05-11T23:01:19.222055+0000","last_clean":"2024-05-11T23:01:19.222055+0000","last_became_active":"2024-05-11T23:01:19.221440+0000","last_became_peered":"2024-05-11T23:01:19.221440+0000","last_unstale":"2024-05-11T23:01:19.222055+0000","last_undegraded":"2024-05-11T23:01:19.222055+0000","last_fullsized":"2024-05-11T23:01:19.222055+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:30:16.582232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.217814+0000","last_change":"2024-05-11T23:01:19.216700+0000","last_active":"2024-05-11T23:01:19.217814+0000","last_peered":"2024-05-11T23:01:19.217814+0000","last_clean":"2024-05-11T23:01:19.217814+0000","last_became_active":"2024-05-11T23:01:19.216413+0000","last_became_peered":"2024-05-11T23:01:19.216413+0000","last_unstale":"2024-05-11T23:01:19.217814+0000","last_undegraded":"2024-05-11T23:01:19.217814+0000","last_fullsized":"2024-05-11T23:01:19.217814+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:33:15.899460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.220973+0000","last_change":"2024-05-11T23:01:19.220232+0000","last_active":"2024-05-11T23:01:19.220973+0000","last_peered":"2024-05-11T23:01:19.220973+0000","last_clean":"2024-05-11T23:01:19.220973+0000","last_became_active":"2024-05-11T23:01:19.219917+0000","last_became_peered":"2024-05-11T23:01:19.219917+0000","last_unstale":"2024-05-11T23:01:19.220973+0000","last_undegraded":"2024-05-11T23:01:19.220973+0000","last_fullsized":"2024-05-11T23:01:19.220973+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:18:24.059625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T23:01:20.216249+0000","last_change":"2024-05-11T23:01:19.221118+0000","last_active":"2024-05-11T23:01:20.216249+0000","last_peered":"2024-05-11T23:01:20.216249+0000","last_clean":"2024-05-11T23:01:20.216249+0000","last_became_active":"2024-05-11T23:01:19.220855+0000","last_became_peered":"2024-05-11T23:01:19.220855+0000","last_unstale":"2024-05-11T23:01:20.216249+0000","last_undegraded":"2024-05-11T23:01:20.216249+0000","last_fullsized":"2024-05-11T23:01:20.216249+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:01:56.076963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.578787+0000","last_change":"2024-05-11T23:01:19.578258+0000","last_active":"2024-05-11T23:01:19.578787+0000","last_peered":"2024-05-11T23:01:19.578787+0000","last_clean":"2024-05-11T23:01:19.578787+0000","last_became_active":"2024-05-11T23:01:19.577973+0000","last_became_peered":"2024-05-11T23:01:19.577973+0000","last_unstale":"2024-05-11T23:01:19.578787+0000","last_undegraded":"2024-05-11T23:01:19.578787+0000","last_fullsized":"2024-05-11T23:01:19.578787+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:22:59.808422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.220850+0000","last_change":"2024-05-11T23:01:19.219755+0000","last_active":"2024-05-11T23:01:19.220850+0000","last_peered":"2024-05-11T23:01:19.220850+0000","last_clean":"2024-05-11T23:01:19.220850+0000","last_became_active":"2024-05-11T23:01:19.219471+0000","last_became_peered":"2024-05-11T23:01:19.219471+0000","last_unstale":"2024-05-11T23:01:19.220850+0000","last_undegraded":"2024-05-11T23:01:19.220850+0000","last_fullsized":"2024-05-11T23:01:19.220850+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:52:12.584484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T23:01:20.216253+0000","last_change":"2024-05-11T23:01:19.219128+0000","last_active":"2024-05-11T23:01:20.216253+0000","last_peered":"2024-05-11T23:01:20.216253+0000","last_clean":"2024-05-11T23:01:20.216253+0000","last_became_active":"2024-05-11T23:01:19.218846+0000","last_became_peered":"2024-05-11T23:01:19.218846+0000","last_unstale":"2024-05-11T23:01:20.216253+0000","last_undegraded":"2024-05-11T23:01:20.216253+0000","last_fullsized":"2024-05-11T23:01:20.216253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:41:09.111000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.331465+0000","last_change":"2024-05-11T23:01:18.164452+0000","last_active":"2024-05-11T23:01:19.331465+0000","last_peered":"2024-05-11T23:01:19.331465+0000","last_clean":"2024-05-11T23:01:19.331465+0000","last_became_active":"2024-05-11T23:01:18.163431+0000","last_became_peered":"2024-05-11T23:01:18.163431+0000","last_unstale":"2024-05-11T23:01:19.331465+0000","last_undegraded":"2024-05-11T23:01:19.331465+0000","last_fullsized":"2024-05-11T23:01:19.331465+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_clean_scrub_stamp":"2024-05-11T23:01:16.147564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:40:59.974639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65020,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509442,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":640117,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":159744,"data_stored":49749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":159744,"data_stored":49749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":159744,"data_stored":49749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":172032,"data_stored":55497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":159744,"data_stored":49749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":159744,"data_stored":49749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":225280,"data_stored":640117,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T23:01:23.539 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T23:01:23.539 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-11T23:01:23.539 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-11T23:01:23.539 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-11T23:01:23.880 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:01:23.880 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-11T23:01:23.893 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-11T23:01:23.893 INFO:teuthology.run_tasks:Running task thrashosds... 2024-05-11T23:01:23.904 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-05-11T23:01:23.904 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': 'f72fecff68e1d400c4568684327c900485c20d6a'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'f72fecff68e1d400c4568684327c900485c20d6a'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'f72fecff68e1d400c4568684327c900485c20d6a'}} 2024-05-11T23:01:23.904 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-05-11T23:01:23.904 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-05-11T23:01:24.225 INFO:tasks.thrashosds:Beginning thrashosds... 2024-05-11T23:01:24.225 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T23:01:24.226 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T23:01:24.522 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:01:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-11T23:01:24.535 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-05-11T23:01:23.138243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234200,"kb_used_data":1388,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754740520,"statfs":{"total":773094113280,"available":772854292480,"internally_reserved":0,"allocated":1421312,"data_stored":1584476,"data_compressed":65020,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000331"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.221043+0000","last_change":"2024-05-11T23:01:19.220007+0000","last_active":"2024-05-11T23:01:19.221043+0000","last_peered":"2024-05-11T23:01:19.221043+0000","last_clean":"2024-05-11T23:01:19.221043+0000","last_became_active":"2024-05-11T23:01:19.219681+0000","last_became_peered":"2024-05-11T23:01:19.219681+0000","last_unstale":"2024-05-11T23:01:19.221043+0000","last_undegraded":"2024-05-11T23:01:19.221043+0000","last_fullsized":"2024-05-11T23:01:19.221043+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:20:19.891140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.222055+0000","last_change":"2024-05-11T23:01:19.221663+0000","last_active":"2024-05-11T23:01:19.222055+0000","last_peered":"2024-05-11T23:01:19.222055+0000","last_clean":"2024-05-11T23:01:19.222055+0000","last_became_active":"2024-05-11T23:01:19.221440+0000","last_became_peered":"2024-05-11T23:01:19.221440+0000","last_unstale":"2024-05-11T23:01:19.222055+0000","last_undegraded":"2024-05-11T23:01:19.222055+0000","last_fullsized":"2024-05-11T23:01:19.222055+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:30:16.582232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.217814+0000","last_change":"2024-05-11T23:01:19.216700+0000","last_active":"2024-05-11T23:01:19.217814+0000","last_peered":"2024-05-11T23:01:19.217814+0000","last_clean":"2024-05-11T23:01:19.217814+0000","last_became_active":"2024-05-11T23:01:19.216413+0000","last_became_peered":"2024-05-11T23:01:19.216413+0000","last_unstale":"2024-05-11T23:01:19.217814+0000","last_undegraded":"2024-05-11T23:01:19.217814+0000","last_fullsized":"2024-05-11T23:01:19.217814+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:33:15.899460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.220973+0000","last_change":"2024-05-11T23:01:19.220232+0000","last_active":"2024-05-11T23:01:19.220973+0000","last_peered":"2024-05-11T23:01:19.220973+0000","last_clean":"2024-05-11T23:01:19.220973+0000","last_became_active":"2024-05-11T23:01:19.219917+0000","last_became_peered":"2024-05-11T23:01:19.219917+0000","last_unstale":"2024-05-11T23:01:19.220973+0000","last_undegraded":"2024-05-11T23:01:19.220973+0000","last_fullsized":"2024-05-11T23:01:19.220973+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:18:24.059625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T23:01:20.216249+0000","last_change":"2024-05-11T23:01:19.221118+0000","last_active":"2024-05-11T23:01:20.216249+0000","last_peered":"2024-05-11T23:01:20.216249+0000","last_clean":"2024-05-11T23:01:20.216249+0000","last_became_active":"2024-05-11T23:01:19.220855+0000","last_became_peered":"2024-05-11T23:01:19.220855+0000","last_unstale":"2024-05-11T23:01:20.216249+0000","last_undegraded":"2024-05-11T23:01:20.216249+0000","last_fullsized":"2024-05-11T23:01:20.216249+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:01:56.076963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.578787+0000","last_change":"2024-05-11T23:01:19.578258+0000","last_active":"2024-05-11T23:01:19.578787+0000","last_peered":"2024-05-11T23:01:19.578787+0000","last_clean":"2024-05-11T23:01:19.578787+0000","last_became_active":"2024-05-11T23:01:19.577973+0000","last_became_peered":"2024-05-11T23:01:19.577973+0000","last_unstale":"2024-05-11T23:01:19.578787+0000","last_undegraded":"2024-05-11T23:01:19.578787+0000","last_fullsized":"2024-05-11T23:01:19.578787+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:22:59.808422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.220850+0000","last_change":"2024-05-11T23:01:19.219755+0000","last_active":"2024-05-11T23:01:19.220850+0000","last_peered":"2024-05-11T23:01:19.220850+0000","last_clean":"2024-05-11T23:01:19.220850+0000","last_became_active":"2024-05-11T23:01:19.219471+0000","last_became_peered":"2024-05-11T23:01:19.219471+0000","last_unstale":"2024-05-11T23:01:19.220850+0000","last_undegraded":"2024-05-11T23:01:19.220850+0000","last_fullsized":"2024-05-11T23:01:19.220850+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:52:12.584484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T23:01:20.216253+0000","last_change":"2024-05-11T23:01:19.219128+0000","last_active":"2024-05-11T23:01:20.216253+0000","last_peered":"2024-05-11T23:01:20.216253+0000","last_clean":"2024-05-11T23:01:20.216253+0000","last_became_active":"2024-05-11T23:01:19.218846+0000","last_became_peered":"2024-05-11T23:01:19.218846+0000","last_unstale":"2024-05-11T23:01:20.216253+0000","last_undegraded":"2024-05-11T23:01:20.216253+0000","last_fullsized":"2024-05-11T23:01:20.216253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:41:09.111000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.331465+0000","last_change":"2024-05-11T23:01:18.164452+0000","last_active":"2024-05-11T23:01:19.331465+0000","last_peered":"2024-05-11T23:01:19.331465+0000","last_clean":"2024-05-11T23:01:19.331465+0000","last_became_active":"2024-05-11T23:01:18.163431+0000","last_became_peered":"2024-05-11T23:01:18.163431+0000","last_unstale":"2024-05-11T23:01:19.331465+0000","last_undegraded":"2024-05-11T23:01:19.331465+0000","last_fullsized":"2024-05-11T23:01:19.331465+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_clean_scrub_stamp":"2024-05-11T23:01:16.147564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:40:59.974639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65020,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509442,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":640117,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":159744,"data_stored":49749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":159744,"data_stored":49749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":159744,"data_stored":49749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":172032,"data_stored":55497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":159744,"data_stored":49749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":159744,"data_stored":49749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":225280,"data_stored":640117,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T23:01:24.536 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T23:01:24.831 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:01:24.832 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-11T23:01:24.844 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-05-11T23:01:23.138243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234200,"kb_used_data":1388,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754740520,"statfs":{"total":773094113280,"available":772854292480,"internally_reserved":0,"allocated":1421312,"data_stored":1584476,"data_compressed":65020,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000331"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.221043+0000","last_change":"2024-05-11T23:01:19.220007+0000","last_active":"2024-05-11T23:01:19.221043+0000","last_peered":"2024-05-11T23:01:19.221043+0000","last_clean":"2024-05-11T23:01:19.221043+0000","last_became_active":"2024-05-11T23:01:19.219681+0000","last_became_peered":"2024-05-11T23:01:19.219681+0000","last_unstale":"2024-05-11T23:01:19.221043+0000","last_undegraded":"2024-05-11T23:01:19.221043+0000","last_fullsized":"2024-05-11T23:01:19.221043+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:20:19.891140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.222055+0000","last_change":"2024-05-11T23:01:19.221663+0000","last_active":"2024-05-11T23:01:19.222055+0000","last_peered":"2024-05-11T23:01:19.222055+0000","last_clean":"2024-05-11T23:01:19.222055+0000","last_became_active":"2024-05-11T23:01:19.221440+0000","last_became_peered":"2024-05-11T23:01:19.221440+0000","last_unstale":"2024-05-11T23:01:19.222055+0000","last_undegraded":"2024-05-11T23:01:19.222055+0000","last_fullsized":"2024-05-11T23:01:19.222055+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:30:16.582232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.217814+0000","last_change":"2024-05-11T23:01:19.216700+0000","last_active":"2024-05-11T23:01:19.217814+0000","last_peered":"2024-05-11T23:01:19.217814+0000","last_clean":"2024-05-11T23:01:19.217814+0000","last_became_active":"2024-05-11T23:01:19.216413+0000","last_became_peered":"2024-05-11T23:01:19.216413+0000","last_unstale":"2024-05-11T23:01:19.217814+0000","last_undegraded":"2024-05-11T23:01:19.217814+0000","last_fullsized":"2024-05-11T23:01:19.217814+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:33:15.899460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.220973+0000","last_change":"2024-05-11T23:01:19.220232+0000","last_active":"2024-05-11T23:01:19.220973+0000","last_peered":"2024-05-11T23:01:19.220973+0000","last_clean":"2024-05-11T23:01:19.220973+0000","last_became_active":"2024-05-11T23:01:19.219917+0000","last_became_peered":"2024-05-11T23:01:19.219917+0000","last_unstale":"2024-05-11T23:01:19.220973+0000","last_undegraded":"2024-05-11T23:01:19.220973+0000","last_fullsized":"2024-05-11T23:01:19.220973+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:18:24.059625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T23:01:20.216249+0000","last_change":"2024-05-11T23:01:19.221118+0000","last_active":"2024-05-11T23:01:20.216249+0000","last_peered":"2024-05-11T23:01:20.216249+0000","last_clean":"2024-05-11T23:01:20.216249+0000","last_became_active":"2024-05-11T23:01:19.220855+0000","last_became_peered":"2024-05-11T23:01:19.220855+0000","last_unstale":"2024-05-11T23:01:20.216249+0000","last_undegraded":"2024-05-11T23:01:20.216249+0000","last_fullsized":"2024-05-11T23:01:20.216249+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:01:56.076963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.578787+0000","last_change":"2024-05-11T23:01:19.578258+0000","last_active":"2024-05-11T23:01:19.578787+0000","last_peered":"2024-05-11T23:01:19.578787+0000","last_clean":"2024-05-11T23:01:19.578787+0000","last_became_active":"2024-05-11T23:01:19.577973+0000","last_became_peered":"2024-05-11T23:01:19.577973+0000","last_unstale":"2024-05-11T23:01:19.578787+0000","last_undegraded":"2024-05-11T23:01:19.578787+0000","last_fullsized":"2024-05-11T23:01:19.578787+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:22:59.808422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.220850+0000","last_change":"2024-05-11T23:01:19.219755+0000","last_active":"2024-05-11T23:01:19.220850+0000","last_peered":"2024-05-11T23:01:19.220850+0000","last_clean":"2024-05-11T23:01:19.220850+0000","last_became_active":"2024-05-11T23:01:19.219471+0000","last_became_peered":"2024-05-11T23:01:19.219471+0000","last_unstale":"2024-05-11T23:01:19.220850+0000","last_undegraded":"2024-05-11T23:01:19.220850+0000","last_fullsized":"2024-05-11T23:01:19.220850+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:52:12.584484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-11T23:01:20.216253+0000","last_change":"2024-05-11T23:01:19.219128+0000","last_active":"2024-05-11T23:01:20.216253+0000","last_peered":"2024-05-11T23:01:20.216253+0000","last_clean":"2024-05-11T23:01:20.216253+0000","last_became_active":"2024-05-11T23:01:19.218846+0000","last_became_peered":"2024-05-11T23:01:19.218846+0000","last_unstale":"2024-05-11T23:01:20.216253+0000","last_undegraded":"2024-05-11T23:01:20.216253+0000","last_fullsized":"2024-05-11T23:01:20.216253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:41:09.111000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-11T23:01:19.331465+0000","last_change":"2024-05-11T23:01:18.164452+0000","last_active":"2024-05-11T23:01:19.331465+0000","last_peered":"2024-05-11T23:01:19.331465+0000","last_clean":"2024-05-11T23:01:19.331465+0000","last_became_active":"2024-05-11T23:01:18.163431+0000","last_became_peered":"2024-05-11T23:01:18.163431+0000","last_unstale":"2024-05-11T23:01:19.331465+0000","last_undegraded":"2024-05-11T23:01:19.331465+0000","last_fullsized":"2024-05-11T23:01:19.331465+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_clean_scrub_stamp":"2024-05-11T23:01:16.147564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:40:59.974639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65020,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509442,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":640117,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":159744,"data_stored":49749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":159744,"data_stored":49749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":159744,"data_stored":49749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542147,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":172032,"data_stored":55497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":159744,"data_stored":49749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31340,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340500,"statfs":{"total":96636764160,"available":96604672000,"internally_reserved":0,"allocated":159744,"data_stored":49749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":225280,"data_stored":640117,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T23:01:24.845 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T23:01:24.845 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-05-11T23:01:25.144 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 18 2024-05-11T23:01:25.144 INFO:teuthology.orchestra.run.smithi005.stdout:fsid 34b20022-f11b-4169-a5f6-6386270a2f8f 2024-05-11T23:01:25.144 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-05-11T23:01:05.977693+0000 2024-05-11T23:01:25.144 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2024-05-11T23:01:20.212450+0000 2024-05-11T23:01:25.144 INFO:teuthology.orchestra.run.smithi005.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-11T23:01:25.144 INFO:teuthology.orchestra.run.smithi005.stdout:crush_version 5 2024-05-11T23:01:25.144 INFO:teuthology.orchestra.run.smithi005.stdout:full_ratio 0.95 2024-05-11T23:01:25.145 INFO:teuthology.orchestra.run.smithi005.stdout:backfillfull_ratio 0.9 2024-05-11T23:01:25.145 INFO:teuthology.orchestra.run.smithi005.stdout:nearfull_ratio 0.85 2024-05-11T23:01:25.145 INFO:teuthology.orchestra.run.smithi005.stdout:require_min_compat_client luminous 2024-05-11T23:01:25.145 INFO:teuthology.orchestra.run.smithi005.stdout:min_compat_client jewel 2024-05-11T23:01:25.145 INFO:teuthology.orchestra.run.smithi005.stdout:require_osd_release squid 2024-05-11T23:01:25.145 INFO:teuthology.orchestra.run.smithi005.stdout:stretch_mode_enabled false 2024-05-11T23:01:25.145 INFO:teuthology.orchestra.run.smithi005.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-05-11T23:01:25.145 INFO:teuthology.orchestra.run.smithi005.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 application rbd read_balance_score 1.99 2024-05-11T23:01:25.145 INFO:teuthology.orchestra.run.smithi005.stdout:max_osd 8 2024-05-11T23:01:25.145 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6800/3452725643,v1:172.21.15.5:6801/3452725643] [v2:172.21.15.5:6802/3452725643,v1:172.21.15.5:6803/3452725643] exists,up 57ab1c70-9478-4a26-bb41-d753b4f3eb79 2024-05-11T23:01:25.145 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 up in weight 1 up_from 14 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6824/4028727138,v1:172.21.15.5:6825/4028727138] [v2:172.21.15.5:6826/4028727138,v1:172.21.15.5:6827/4028727138] exists,up 05ece110-4a99-46f0-b061-5980f7ef2a55 2024-05-11T23:01:25.145 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6807/2059959509,v1:172.21.15.5:6809/2059959509] [v2:172.21.15.5:6810/2059959509,v1:172.21.15.5:6811/2059959509] exists,up 4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5 2024-05-11T23:01:25.145 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6816/296930146,v1:172.21.15.5:6817/296930146] [v2:172.21.15.5:6818/296930146,v1:172.21.15.5:6819/296930146] exists,up 5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a 2024-05-11T23:01:25.145 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6800/495703077,v1:172.21.15.115:6801/495703077] [v2:172.21.15.115:6802/495703077,v1:172.21.15.115:6803/495703077] exists,up 4e87e4f0-288e-4ee6-919e-9337925a5d5e 2024-05-11T23:01:25.146 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6811/902573489,v1:172.21.15.115:6813/902573489] [v2:172.21.15.115:6815/902573489,v1:172.21.15.115:6817/902573489] exists,up e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5 2024-05-11T23:01:25.146 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6824/2749332272,v1:172.21.15.115:6825/2749332272] [v2:172.21.15.115:6826/2749332272,v1:172.21.15.115:6827/2749332272] exists,up edb686ec-e1a2-40d6-975a-e32e0e7760ff 2024-05-11T23:01:25.146 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6808/1043826337,v1:172.21.15.115:6809/1043826337] [v2:172.21.15.115:6810/1043826337,v1:172.21.15.115:6812/1043826337] exists,up 7aa14dbb-c6cb-4540-b160-3fd346f6a046 2024-05-11T23:01:25.157 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6800/3452725643,v1:172.21.15.5:6801/3452725643] [v2:172.21.15.5:6802/3452725643,v1:172.21.15.5:6803/3452725643] exists,up 57ab1c70-9478-4a26-bb41-d753b4f3eb79', 'osd.1 up in weight 1 up_from 14 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6824/4028727138,v1:172.21.15.5:6825/4028727138] [v2:172.21.15.5:6826/4028727138,v1:172.21.15.5:6827/4028727138] exists,up 05ece110-4a99-46f0-b061-5980f7ef2a55', 'osd.2 up in weight 1 up_from 13 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6807/2059959509,v1:172.21.15.5:6809/2059959509] [v2:172.21.15.5:6810/2059959509,v1:172.21.15.5:6811/2059959509] exists,up 4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5', 'osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6816/296930146,v1:172.21.15.5:6817/296930146] [v2:172.21.15.5:6818/296930146,v1:172.21.15.5:6819/296930146] exists,up 5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a', 'osd.4 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6800/495703077,v1:172.21.15.115:6801/495703077] [v2:172.21.15.115:6802/495703077,v1:172.21.15.115:6803/495703077] exists,up 4e87e4f0-288e-4ee6-919e-9337925a5d5e', 'osd.5 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6811/902573489,v1:172.21.15.115:6813/902573489] [v2:172.21.15.115:6815/902573489,v1:172.21.15.115:6817/902573489] exists,up e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5', 'osd.6 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6824/2749332272,v1:172.21.15.115:6825/2749332272] [v2:172.21.15.115:6826/2749332272,v1:172.21.15.115:6827/2749332272] exists,up edb686ec-e1a2-40d6-975a-e32e0e7760ff', 'osd.7 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.115:6808/1043826337,v1:172.21.15.115:6809/1043826337] [v2:172.21.15.115:6810/1043826337,v1:172.21.15.115:6812/1043826337] exists,up 7aa14dbb-c6cb-4540-b160-3fd346f6a046'] 2024-05-11T23:01:25.157 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-05-11T23:01:25.262 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:25.262 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mon.a", 2024-05-11T23:01:25.262 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster": "ceph", 2024-05-11T23:01:25.262 INFO:teuthology.orchestra.run.smithi005.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-05-11T23:01:25.262 INFO:teuthology.orchestra.run.smithi005.stdout: "admin_socket_mode": "", 2024-05-11T23:01:25.262 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-05-11T23:01:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_client_required": "cephx, none", 2024-05-11T23:01:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_cluster_required": "cephx", 2024-05-11T23:01:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_debug": "true", 2024-05-11T23:01:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-05-11T23:01:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-05-11T23:01:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_service_required": "cephx", 2024-05-11T23:01:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_service_ticket_ttl": "240.000000", 2024-05-11T23:01:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_supported": "cephx", 2024-05-11T23:01:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_aio": "true", 2024-05-11T23:01:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_aio_max_queue_depth": "1024", 2024-05-11T23:01:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_aio_poll_ms": "250", 2024-05-11T23:01:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_aio_reap_max": "16", 2024-05-11T23:01:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_async_discard": "false", 2024-05-11T23:01:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_block_size": "4096", 2024-05-11T23:01:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_debug_aio": "false", 2024-05-11T23:01:25.264 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-05-11T23:01:25.264 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-05-11T23:01:25.264 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_debug_inflight_ios": "false", 2024-05-11T23:01:25.264 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_enable_discard": "false", 2024-05-11T23:01:25.264 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_flock_retry": "3", 2024-05-11T23:01:25.264 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_flock_retry_interval": "0.100000", 2024-05-11T23:01:25.264 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_inject_crash": "0", 2024-05-11T23:01:25.264 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_inject_crash_flush_delay": "2", 2024-05-11T23:01:25.264 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_ioring": "false", 2024-05-11T23:01:25.264 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_ioring_hipri": "false", 2024-05-11T23:01:25.264 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_ioring_sqthread_poll": "false", 2024-05-11T23:01:25.264 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-05-11T23:01:25.264 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_read_buffer_alignment": "4096", 2024-05-11T23:01:25.264 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-05-11T23:01:25.264 INFO:teuthology.orchestra.run.smithi005.stdout: "bdev_type": "", 2024-05-11T23:01:25.265 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_alloc_size": "1048576", 2024-05-11T23:01:25.265 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_allocator": "hybrid", 2024-05-11T23:01:25.265 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_buffered_io": "true", 2024-05-11T23:01:25.265 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_check_for_zeros": "false", 2024-05-11T23:01:25.265 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_check_volume_selector_often": "false", 2024-05-11T23:01:25.265 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-05-11T23:01:25.265 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_compact_log_sync": "false", 2024-05-11T23:01:25.265 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-05-11T23:01:25.265 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-05-11T23:01:25.265 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_log_compact_min_size": "16777216", 2024-05-11T23:01:25.265 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_log_replay_check_allocations": "true", 2024-05-11T23:01:25.265 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_max_log_runway": "4194304", 2024-05-11T23:01:25.266 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_max_prefetch": "1048576", 2024-05-11T23:01:25.266 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_min_flush_size": "524288", 2024-05-11T23:01:25.266 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_min_log_runway": "1048576", 2024-05-11T23:01:25.266 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_replay_recovery": "false", 2024-05-11T23:01:25.266 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-05-11T23:01:25.266 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_shared_alloc_size": "65536", 2024-05-11T23:01:25.266 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_sync_write": "false", 2024-05-11T23:01:25.266 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-05-11T23:01:25.266 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-05-11T23:01:25.266 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-05-11T23:01:25.266 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_allocation_from_file": "true", 2024-05-11T23:01:25.266 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_allocator": "hybrid", 2024-05-11T23:01:25.266 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-05-11T23:01:25.266 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-05-11T23:01:25.267 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-05-11T23:01:25.267 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-05-11T23:01:25.267 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-05-11T23:01:25.267 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-05-11T23:01:25.267 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blobid_prealloc": "10240", 2024-05-11T23:01:25.267 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_block_create": "true", 2024-05-11T23:01:25.267 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_block_db_create": "false", 2024-05-11T23:01:25.267 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_block_db_path": "", 2024-05-11T23:01:25.267 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_block_db_size": "0", 2024-05-11T23:01:25.267 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_block_path": "", 2024-05-11T23:01:25.267 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_block_preallocate_file": "false", 2024-05-11T23:01:25.267 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_block_size": "107374182400", 2024-05-11T23:01:25.267 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_block_wal_create": "false", 2024-05-11T23:01:25.268 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_block_wal_path": "", 2024-05-11T23:01:25.268 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_block_wal_size": "100663296", 2024-05-11T23:01:25.268 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_bluefs": "true", 2024-05-11T23:01:25.268 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-05-11T23:01:25.268 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_bluefs_env_mirror": "false", 2024-05-11T23:01:25.268 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-05-11T23:01:25.268 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-05-11T23:01:25.269 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-05-11T23:01:25.269 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-05-11T23:01:25.269 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-05-11T23:01:25.269 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-05-11T23:01:25.269 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_autotune": "true", 2024-05-11T23:01:25.269 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-05-11T23:01:25.269 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-05-11T23:01:25.269 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-05-11T23:01:25.269 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-05-11T23:01:25.269 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_size": "0", 2024-05-11T23:01:25.269 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-05-11T23:01:25.269 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-05-11T23:01:25.269 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-05-11T23:01:25.269 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-05-11T23:01:25.270 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_type": "2q", 2024-05-11T23:01:25.270 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-05-11T23:01:25.270 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_clone_cow": "true", 2024-05-11T23:01:25.270 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_compression_algorithm": "snappy", 2024-05-11T23:01:25.270 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_compression_max_blob_size": "0", 2024-05-11T23:01:25.270 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-05-11T23:01:25.270 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-05-11T23:01:25.270 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_compression_min_blob_size": "0", 2024-05-11T23:01:25.270 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-05-11T23:01:25.270 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-05-11T23:01:25.270 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_compression_mode": "none", 2024-05-11T23:01:25.271 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-05-11T23:01:25.271 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_csum_type": "crc32c", 2024-05-11T23:01:25.271 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_enforce_settings": "default", 2024-05-11T23:01:25.271 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_freelist": "false", 2024-05-11T23:01:25.271 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_fsck_abort": "false", 2024-05-11T23:01:25.271 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-05-11T23:01:25.271 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-05-11T23:01:25.271 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_inject_read_err": "false", 2024-05-11T23:01:25.271 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_legacy_omap": "false", 2024-05-11T23:01:25.271 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-05-11T23:01:25.271 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-05-11T23:01:25.271 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-05-11T23:01:25.271 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-05-11T23:01:25.271 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_prefill": "0.000000", 2024-05-11T23:01:25.272 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-05-11T23:01:25.272 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-05-11T23:01:25.272 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-05-11T23:01:25.272 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_small_allocations": "0", 2024-05-11T23:01:25.272 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-05-11T23:01:25.272 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_default_buffered_read": "true", 2024-05-11T23:01:25.273 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_default_buffered_write": "false", 2024-05-11T23:01:25.273 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_deferred_batch_ops": "0", 2024-05-11T23:01:25.273 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-05-11T23:01:25.273 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-05-11T23:01:25.273 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_elastic_shared_blobs": "true", 2024-05-11T23:01:25.273 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-05-11T23:01:25.273 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-05-11T23:01:25.273 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-05-11T23:01:25.273 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-05-11T23:01:25.273 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-05-11T23:01:25.274 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fail_eio": "false", 2024-05-11T23:01:25.274 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-05-11T23:01:25.274 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-05-11T23:01:25.274 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-05-11T23:01:25.274 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-05-11T23:01:25.274 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_on_mkfs": "true", 2024-05-11T23:01:25.274 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-05-11T23:01:25.274 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_on_mount": "false", 2024-05-11T23:01:25.274 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-05-11T23:01:25.274 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_on_umount": "false", 2024-05-11T23:01:25.274 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-05-11T23:01:25.275 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-05-11T23:01:25.275 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-05-11T23:01:25.275 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-05-11T23:01:25.275 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-05-11T23:01:25.275 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-05-11T23:01:25.275 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-05-11T23:01:25.275 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-05-11T23:01:25.275 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_ignore_data_csum": "false", 2024-05-11T23:01:25.275 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-05-11T23:01:25.275 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_kvbackend": "rocksdb", 2024-05-11T23:01:25.275 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-05-11T23:01:25.275 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-05-11T23:01:25.275 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_log_op_age": "5.000000", 2024-05-11T23:01:25.275 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_max_alloc_size": "0", 2024-05-11T23:01:25.275 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_max_blob_size": "0", 2024-05-11T23:01:25.276 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-05-11T23:01:25.276 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-05-11T23:01:25.276 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_max_defer_interval": "3.000000", 2024-05-11T23:01:25.276 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_max_deferred_txc": "32", 2024-05-11T23:01:25.276 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_min_alloc_size": "0", 2024-05-11T23:01:25.276 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-05-11T23:01:25.276 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-05-11T23:01:25.276 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_nid_prealloc": "1024", 2024-05-11T23:01:25.276 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_prefer_deferred_size": "0", 2024-05-11T23:01:25.276 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-05-11T23:01:25.276 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-05-11T23:01:25.276 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_qfsck_on_mount": "true", 2024-05-11T23:01:25.276 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_retry_disk_reads": "3", 2024-05-11T23:01:25.276 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_rocksdb_cf": "true", 2024-05-11T23:01:25.276 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-05-11T23:01:25.277 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-05-11T23:01:25.277 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_rocksdb_options_annex": "", 2024-05-11T23:01:25.277 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_spdk_coremask": "0x1", 2024-05-11T23:01:25.277 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_spdk_io_sleep": "5", 2024-05-11T23:01:25.277 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_spdk_max_io_completion": "0", 2024-05-11T23:01:25.277 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_spdk_mem": "512", 2024-05-11T23:01:25.277 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_sync_submit_transaction": "false", 2024-05-11T23:01:25.277 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_throttle_bytes": "67108864", 2024-05-11T23:01:25.277 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_throttle_cost_per_io": "0", 2024-05-11T23:01:25.278 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-05-11T23:01:25.278 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-05-11T23:01:25.278 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-05-11T23:01:25.278 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-05-11T23:01:25.278 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_tracing": "false", 2024-05-11T23:01:25.278 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-05-11T23:01:25.278 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-05-11T23:01:25.278 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_volume_selection_reserved": "0", 2024-05-11T23:01:25.278 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-05-11T23:01:25.278 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-05-11T23:01:25.278 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-05-11T23:01:25.278 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-05-11T23:01:25.279 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-05-11T23:01:25.279 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-05-11T23:01:25.279 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_zero_block_detection": "false", 2024-05-11T23:01:25.279 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-05-11T23:01:25.279 INFO:teuthology.orchestra.run.smithi005.stdout: "cephfs_mirror_action_update_interval": "2", 2024-05-11T23:01:25.279 INFO:teuthology.orchestra.run.smithi005.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-05-11T23:01:25.279 INFO:teuthology.orchestra.run.smithi005.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-05-11T23:01:25.279 INFO:teuthology.orchestra.run.smithi005.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-05-11T23:01:25.279 INFO:teuthology.orchestra.run.smithi005.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-05-11T23:01:25.279 INFO:teuthology.orchestra.run.smithi005.stdout: "cephfs_mirror_mount_timeout": "10", 2024-05-11T23:01:25.279 INFO:teuthology.orchestra.run.smithi005.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-05-11T23:01:25.279 INFO:teuthology.orchestra.run.smithi005.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-05-11T23:01:25.279 INFO:teuthology.orchestra.run.smithi005.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-05-11T23:01:25.280 INFO:teuthology.orchestra.run.smithi005.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-05-11T23:01:25.280 INFO:teuthology.orchestra.run.smithi005.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-05-11T23:01:25.280 INFO:teuthology.orchestra.run.smithi005.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-05-11T23:01:25.280 INFO:teuthology.orchestra.run.smithi005.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-05-11T23:01:25.280 INFO:teuthology.orchestra.run.smithi005.stdout: "cephx_cluster_require_signatures": "false", 2024-05-11T23:01:25.280 INFO:teuthology.orchestra.run.smithi005.stdout: "cephx_cluster_require_version": "2", 2024-05-11T23:01:25.280 INFO:teuthology.orchestra.run.smithi005.stdout: "cephx_require_signatures": "false", 2024-05-11T23:01:25.280 INFO:teuthology.orchestra.run.smithi005.stdout: "cephx_require_version": "2", 2024-05-11T23:01:25.280 INFO:teuthology.orchestra.run.smithi005.stdout: "cephx_service_require_signatures": "false", 2024-05-11T23:01:25.280 INFO:teuthology.orchestra.run.smithi005.stdout: "cephx_service_require_version": "2", 2024-05-11T23:01:25.280 INFO:teuthology.orchestra.run.smithi005.stdout: "cephx_sign_messages": "true", 2024-05-11T23:01:25.280 INFO:teuthology.orchestra.run.smithi005.stdout: "chdir": "", 2024-05-11T23:01:25.280 INFO:teuthology.orchestra.run.smithi005.stdout: "client_acl_type": "", 2024-05-11T23:01:25.280 INFO:teuthology.orchestra.run.smithi005.stdout: "client_asio_thread_count": "2", 2024-05-11T23:01:25.280 INFO:teuthology.orchestra.run.smithi005.stdout: "client_cache_mid": "0.750000", 2024-05-11T23:01:25.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_cache_size": "16384", 2024-05-11T23:01:25.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_caps_release_delay": "5", 2024-05-11T23:01:25.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_check_pool_perm": "true", 2024-05-11T23:01:25.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_collect_and_send_global_metrics": "false", 2024-05-11T23:01:25.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_debug_force_sync_read": "false", 2024-05-11T23:01:25.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_debug_getattr_caps": "false", 2024-05-11T23:01:25.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_debug_inject_tick_delay": "0", 2024-05-11T23:01:25.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-05-11T23:01:25.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_die_on_failed_remount": "false", 2024-05-11T23:01:25.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_dirsize_rbytes": "true", 2024-05-11T23:01:25.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_force_lazyio": "false", 2024-05-11T23:01:25.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_fs": "", 2024-05-11T23:01:25.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_inject_fixed_oldest_tid": "false", 2024-05-11T23:01:25.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_inject_release_failure": "false", 2024-05-11T23:01:25.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client_max_inline_size": "4096", 2024-05-11T23:01:25.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client_max_retries_on_remount_failure": "5", 2024-05-11T23:01:25.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client_mds_namespace": "", 2024-05-11T23:01:25.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client_metadata": "", 2024-05-11T23:01:25.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client_mount_gid": "-1", 2024-05-11T23:01:25.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client_mount_timeout": "300", 2024-05-11T23:01:25.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client_mount_uid": "-1", 2024-05-11T23:01:25.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client_mountpoint": "/", 2024-05-11T23:01:25.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client_notify_timeout": "10", 2024-05-11T23:01:25.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client_oc": "true", 2024-05-11T23:01:25.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client_oc_max_dirty": "104857600", 2024-05-11T23:01:25.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client_oc_max_dirty_age": "5.000000", 2024-05-11T23:01:25.282 INFO:teuthology.orchestra.run.smithi005.stdout: "client_oc_max_objects": "1000", 2024-05-11T23:01:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "client_oc_size": "209715200", 2024-05-11T23:01:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "client_oc_target_dirty": "8388608", 2024-05-11T23:01:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "client_permissions": "true", 2024-05-11T23:01:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "client_quota": "true", 2024-05-11T23:01:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "client_quota_df": "true", 2024-05-11T23:01:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "client_readahead_max_bytes": "0", 2024-05-11T23:01:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "client_readahead_max_periods": "4", 2024-05-11T23:01:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "client_readahead_min": "131072", 2024-05-11T23:01:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "client_reconnect_stale": "false", 2024-05-11T23:01:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "client_shutdown_timeout": "30", 2024-05-11T23:01:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "client_snapdir": ".snap", 2024-05-11T23:01:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "client_tick_interval": "1", 2024-05-11T23:01:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "client_trace": "", 2024-05-11T23:01:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "client_try_dentry_invalidate": "false", 2024-05-11T23:01:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "client_use_faked_inos": "false", 2024-05-11T23:01:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "client_use_random_mds": "false", 2024-05-11T23:01:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "clog_to_graylog": "false", 2024-05-11T23:01:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-05-11T23:01:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "clog_to_graylog_port": "12201", 2024-05-11T23:01:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "clog_to_monitors": "default=true", 2024-05-11T23:01:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "clog_to_syslog": "false", 2024-05-11T23:01:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-05-11T23:01:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "clog_to_syslog_level": "info", 2024-05-11T23:01:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "-", 2024-05-11T23:01:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_network": "", 2024-05-11T23:01:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_network_interface": "", 2024-05-11T23:01:25.285 INFO:teuthology.orchestra.run.smithi005.stdout: "compressor_zlib_isal": "false", 2024-05-11T23:01:25.285 INFO:teuthology.orchestra.run.smithi005.stdout: "compressor_zlib_level": "5", 2024-05-11T23:01:25.285 INFO:teuthology.orchestra.run.smithi005.stdout: "compressor_zlib_winsize": "-15", 2024-05-11T23:01:25.285 INFO:teuthology.orchestra.run.smithi005.stdout: "compressor_zstd_level": "1", 2024-05-11T23:01:25.285 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-05-11T23:01:25.285 INFO:teuthology.orchestra.run.smithi005.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-05-11T23:01:25.285 INFO:teuthology.orchestra.run.smithi005.stdout: "crimson_alien_op_num_threads": "6", 2024-05-11T23:01:25.285 INFO:teuthology.orchestra.run.smithi005.stdout: "crimson_alien_thread_cpu_cores": "", 2024-05-11T23:01:25.285 INFO:teuthology.orchestra.run.smithi005.stdout: "crimson_osd_obc_lru_size": "512", 2024-05-11T23:01:25.285 INFO:teuthology.orchestra.run.smithi005.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-05-11T23:01:25.285 INFO:teuthology.orchestra.run.smithi005.stdout: "crimson_seastar_cpu_cores": "", 2024-05-11T23:01:25.285 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_location": "", 2024-05-11T23:01:25.285 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_location_hook": "", 2024-05-11T23:01:25.285 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_location_hook_timeout": "10", 2024-05-11T23:01:25.285 INFO:teuthology.orchestra.run.smithi005.stdout: "daemonize": "false", 2024-05-11T23:01:25.286 INFO:teuthology.orchestra.run.smithi005.stdout: "daos_pool": "tank", 2024-05-11T23:01:25.286 INFO:teuthology.orchestra.run.smithi005.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-05-11T23:01:25.286 INFO:teuthology.orchestra.run.smithi005.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-05-11T23:01:25.286 INFO:teuthology.orchestra.run.smithi005.stdout: "dbstore_db_name_prefix": "dbstore", 2024-05-11T23:01:25.286 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_alienstore": "0/5", 2024-05-11T23:01:25.286 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_allow_any_pool_priority": "false", 2024-05-11T23:01:25.286 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_asok": "1/5", 2024-05-11T23:01:25.286 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_asok_assert_abort": "false", 2024-05-11T23:01:25.286 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_asserts_on_shutdown": "true", 2024-05-11T23:01:25.286 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_auth": "20/20", 2024-05-11T23:01:25.286 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_bdev": "1/3", 2024-05-11T23:01:25.287 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_bluefs": "1/5", 2024-05-11T23:01:25.287 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_bluestore": "1/5", 2024-05-11T23:01:25.287 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_buffer": "0/1", 2024-05-11T23:01:25.287 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_ceph_exporter": "1/5", 2024-05-11T23:01:25.287 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_cephfs_mirror": "0/5", 2024-05-11T23:01:25.287 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_cephsqlite": "0/5", 2024-05-11T23:01:25.287 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_client": "0/5", 2024-05-11T23:01:25.287 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_compressor": "1/5", 2024-05-11T23:01:25.287 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_context": "0/1", 2024-05-11T23:01:25.287 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_crimson_interrupt": "0/5", 2024-05-11T23:01:25.287 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_crush": "1/1", 2024-05-11T23:01:25.287 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_crypto": "1/5", 2024-05-11T23:01:25.287 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_cyanstore": "0/5", 2024-05-11T23:01:25.287 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_deliberately_leak_memory": "false", 2024-05-11T23:01:25.287 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_disable_randomized_ping": "false", 2024-05-11T23:01:25.288 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_dpdk": "1/5", 2024-05-11T23:01:25.288 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_eventtrace": "1/5", 2024-05-11T23:01:25.288 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_filer": "0/1", 2024-05-11T23:01:25.288 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_filestore": "1/3", 2024-05-11T23:01:25.288 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_finisher": "1/1", 2024-05-11T23:01:25.288 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_fuse": "1/5", 2024-05-11T23:01:25.288 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_heartbeat_testing_span": "0", 2024-05-11T23:01:25.288 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_heartbeatmap": "1/5", 2024-05-11T23:01:25.288 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_immutable_obj_cache": "0/5", 2024-05-11T23:01:25.288 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_javaclient": "1/5", 2024-05-11T23:01:25.288 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_journal": "1/3", 2024-05-11T23:01:25.288 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_journaler": "0/5", 2024-05-11T23:01:25.288 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_kstore": "1/5", 2024-05-11T23:01:25.288 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_lockdep": "0/1", 2024-05-11T23:01:25.289 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mclock": "1/5", 2024-05-11T23:01:25.289 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mds": "1/5", 2024-05-11T23:01:25.289 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mds_balancer": "1/5", 2024-05-11T23:01:25.289 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mds_locker": "1/5", 2024-05-11T23:01:25.289 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mds_log": "1/5", 2024-05-11T23:01:25.289 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mds_log_expire": "1/5", 2024-05-11T23:01:25.289 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mds_migrator": "1/5", 2024-05-11T23:01:25.289 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mds_quiesce": "3/5", 2024-05-11T23:01:25.289 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_memstore": "1/5", 2024-05-11T23:01:25.289 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mgr": "2/5", 2024-05-11T23:01:25.289 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mgrc": "1/5", 2024-05-11T23:01:25.289 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_mon": "20/20", 2024-05-11T23:01:25.289 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_monc": "0/10", 2024-05-11T23:01:25.289 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_ms": "1/1", 2024-05-11T23:01:25.289 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_none": "0/5", 2024-05-11T23:01:25.290 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_objclass": "0/5", 2024-05-11T23:01:25.290 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_objectcacher": "0/5", 2024-05-11T23:01:25.290 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_objecter": "0/1", 2024-05-11T23:01:25.290 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_optracker": "0/5", 2024-05-11T23:01:25.290 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_osd": "1/5", 2024-05-11T23:01:25.290 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_paxos": "20/20", 2024-05-11T23:01:25.290 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_perfcounter": "1/5", 2024-05-11T23:01:25.290 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_prioritycache": "1/5", 2024-05-11T23:01:25.290 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rados": "0/5", 2024-05-11T23:01:25.290 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rbd": "0/5", 2024-05-11T23:01:25.290 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rbd_mirror": "0/5", 2024-05-11T23:01:25.290 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rbd_pwl": "0/5", 2024-05-11T23:01:25.290 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rbd_replay": "0/5", 2024-05-11T23:01:25.290 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_refs": "0/0", 2024-05-11T23:01:25.291 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_reserver": "1/1", 2024-05-11T23:01:25.291 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rgw": "1/5", 2024-05-11T23:01:25.291 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rgw_access": "1/5", 2024-05-11T23:01:25.291 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rgw_datacache": "1/5", 2024-05-11T23:01:25.291 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rgw_dbstore": "1/5", 2024-05-11T23:01:25.291 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rgw_flight": "1/5", 2024-05-11T23:01:25.291 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rgw_lifecycle": "1/5", 2024-05-11T23:01:25.291 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rgw_sync": "1/5", 2024-05-11T23:01:25.291 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_rocksdb": "4/5", 2024-05-11T23:01:25.291 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore": "0/5", 2024-05-11T23:01:25.291 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_backref": "0/5", 2024-05-11T23:01:25.291 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_cache": "0/5", 2024-05-11T23:01:25.291 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_cleaner": "0/5", 2024-05-11T23:01:25.291 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_device": "0/5", 2024-05-11T23:01:25.292 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_epm": "0/5", 2024-05-11T23:01:25.292 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-05-11T23:01:25.292 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_journal": "0/5", 2024-05-11T23:01:25.292 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_lba": "0/5", 2024-05-11T23:01:25.292 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_odata": "0/5", 2024-05-11T23:01:25.292 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_omap": "0/5", 2024-05-11T23:01:25.292 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_onode": "0/5", 2024-05-11T23:01:25.292 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_t": "0/5", 2024-05-11T23:01:25.292 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_seastore_tm": "0/5", 2024-05-11T23:01:25.292 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_striper": "0/1", 2024-05-11T23:01:25.292 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_test": "0/5", 2024-05-11T23:01:25.292 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_throttle": "1/1", 2024-05-11T23:01:25.292 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_timer": "0/1", 2024-05-11T23:01:25.292 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_tp": "0/5", 2024-05-11T23:01:25.292 INFO:teuthology.orchestra.run.smithi005.stdout: "debug_trace": "1/5", 2024-05-11T23:01:25.293 INFO:teuthology.orchestra.run.smithi005.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-05-11T23:01:25.293 INFO:teuthology.orchestra.run.smithi005.stdout: "device_failure_prediction_mode": "none", 2024-05-11T23:01:25.293 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-05-11T23:01:25.293 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-05-11T23:01:25.293 INFO:teuthology.orchestra.run.smithi005.stdout: "err_to_graylog": "false", 2024-05-11T23:01:25.293 INFO:teuthology.orchestra.run.smithi005.stdout: "err_to_journald": "false", 2024-05-11T23:01:25.293 INFO:teuthology.orchestra.run.smithi005.stdout: "err_to_stderr": "true", 2024-05-11T23:01:25.293 INFO:teuthology.orchestra.run.smithi005.stdout: "err_to_syslog": "false", 2024-05-11T23:01:25.293 INFO:teuthology.orchestra.run.smithi005.stdout: "event_tracing": "false", 2024-05-11T23:01:25.293 INFO:teuthology.orchestra.run.smithi005.stdout: "exporter_addr": "0.0.0.0", 2024-05-11T23:01:25.293 INFO:teuthology.orchestra.run.smithi005.stdout: "exporter_http_port": "9926", 2024-05-11T23:01:25.293 INFO:teuthology.orchestra.run.smithi005.stdout: "exporter_prio_limit": "5", 2024-05-11T23:01:25.293 INFO:teuthology.orchestra.run.smithi005.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-05-11T23:01:25.293 INFO:teuthology.orchestra.run.smithi005.stdout: "exporter_sort_metrics": "true", 2024-05-11T23:01:25.293 INFO:teuthology.orchestra.run.smithi005.stdout: "exporter_stats_period": "5", 2024-05-11T23:01:25.294 INFO:teuthology.orchestra.run.smithi005.stdout: "fake_statfs_for_testing": "0", 2024-05-11T23:01:25.294 INFO:teuthology.orchestra.run.smithi005.stdout: "fatal_signal_handlers": "true", 2024-05-11T23:01:25.294 INFO:teuthology.orchestra.run.smithi005.stdout: "filer_max_purge_ops": "10", 2024-05-11T23:01:25.294 INFO:teuthology.orchestra.run.smithi005.stdout: "filer_max_truncate_ops": "128", 2024-05-11T23:01:25.294 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_apply_finisher_threads": "1", 2024-05-11T23:01:25.294 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_blackhole": "false", 2024-05-11T23:01:25.294 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_btrfs_clone_range": "true", 2024-05-11T23:01:25.294 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_btrfs_snap": "true", 2024-05-11T23:01:25.294 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_caller_concurrency": "10", 2024-05-11T23:01:25.294 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_collect_device_partition_information": "true", 2024-05-11T23:01:25.294 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_commit_timeout": "600.000000", 2024-05-11T23:01:25.294 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_debug_inject_read_err": "false", 2024-05-11T23:01:25.294 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_debug_omap_check": "false", 2024-05-11T23:01:25.294 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_debug_random_read_err": "0.000000", 2024-05-11T23:01:25.295 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_debug_verify_split": "false", 2024-05-11T23:01:25.295 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_dump_file": "", 2024-05-11T23:01:25.295 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-05-11T23:01:25.295 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-05-11T23:01:25.295 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_fadvise": "true", 2024-05-11T23:01:25.295 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_fail_eio": "true", 2024-05-11T23:01:25.295 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_fd_cache_shards": "16", 2024-05-11T23:01:25.295 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_fd_cache_size": "128", 2024-05-11T23:01:25.295 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_fiemap": "false", 2024-05-11T23:01:25.295 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_fiemap_threshold": "4096", 2024-05-11T23:01:25.295 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-05-11T23:01:25.295 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_index_retry_probability": "0.000000", 2024-05-11T23:01:25.295 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_inject_stall": "0", 2024-05-11T23:01:25.295 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_journal_parallel": "false", 2024-05-11T23:01:25.295 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_journal_trailing": "false", 2024-05-11T23:01:25.296 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_journal_writeahead": "false", 2024-05-11T23:01:25.296 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_kill_at": "0", 2024-05-11T23:01:25.296 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-05-11T23:01:25.296 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_inline_xattr_size": "0", 2024-05-11T23:01:25.296 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-05-11T23:01:25.296 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-05-11T23:01:25.296 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-05-11T23:01:25.296 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_inline_xattrs": "0", 2024-05-11T23:01:25.296 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-05-11T23:01:25.296 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_inline_xattrs_other": "2", 2024-05-11T23:01:25.296 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-05-11T23:01:25.296 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_sync_interval": "5.000000", 2024-05-11T23:01:25.296 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_xattr_value_size": "0", 2024-05-11T23:01:25.296 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-05-11T23:01:25.296 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-05-11T23:01:25.297 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-05-11T23:01:25.297 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_merge_threshold": "-10", 2024-05-11T23:01:25.297 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_min_sync_interval": "0.010000", 2024-05-11T23:01:25.297 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_odsync_write": "false", 2024-05-11T23:01:25.297 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_omap_backend": "rocksdb", 2024-05-11T23:01:25.297 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_omap_backend_path": "", 2024-05-11T23:01:25.297 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_omap_header_cache_size": "1024", 2024-05-11T23:01:25.297 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_ondisk_finisher_threads": "1", 2024-05-11T23:01:25.297 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-05-11T23:01:25.297 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_op_thread_timeout": "60", 2024-05-11T23:01:25.297 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_op_threads": "2", 2024-05-11T23:01:25.297 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_punch_hole": "false", 2024-05-11T23:01:25.297 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-05-11T23:01:25.297 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-05-11T23:01:25.297 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-05-11T23:01:25.298 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-05-11T23:01:25.298 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-05-11T23:01:25.298 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_queue_max_bytes": "104857600", 2024-05-11T23:01:25.298 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-05-11T23:01:25.298 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-05-11T23:01:25.298 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-05-11T23:01:25.298 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_queue_max_ops": "50", 2024-05-11T23:01:25.298 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-05-11T23:01:25.298 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_seek_data_hole": "false", 2024-05-11T23:01:25.298 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_sloppy_crc": "false", 2024-05-11T23:01:25.298 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-05-11T23:01:25.298 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_splice": "false", 2024-05-11T23:01:25.298 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_split_multiple": "2", 2024-05-11T23:01:25.298 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_split_rand_factor": "20", 2024-05-11T23:01:25.299 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_update_to": "1000", 2024-05-11T23:01:25.299 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-05-11T23:01:25.299 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-05-11T23:01:25.299 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-05-11T23:01:25.299 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-05-11T23:01:25.299 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-05-11T23:01:25.299 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-05-11T23:01:25.299 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_enable": "true", 2024-05-11T23:01:25.299 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-05-11T23:01:25.299 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-05-11T23:01:25.299 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-05-11T23:01:25.299 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-05-11T23:01:25.299 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-05-11T23:01:25.299 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-05-11T23:01:25.299 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_xfs_extsize": "false", 2024-05-11T23:01:25.299 INFO:teuthology.orchestra.run.smithi005.stdout: "filestore_zfs_snap": "false", 2024-05-11T23:01:25.300 INFO:teuthology.orchestra.run.smithi005.stdout: "fio_dir": "/tmp/fio", 2024-05-11T23:01:25.300 INFO:teuthology.orchestra.run.smithi005.stdout: "fsid": "34b20022-f11b-4169-a5f6-6386270a2f8f", 2024-05-11T23:01:25.300 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_allow_other": "true", 2024-05-11T23:01:25.300 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_atomic_o_trunc": "true", 2024-05-11T23:01:25.300 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_big_writes": "true", 2024-05-11T23:01:25.300 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_debug": "false", 2024-05-11T23:01:25.300 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_default_permissions": "false", 2024-05-11T23:01:25.300 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_disable_pagecache": "false", 2024-05-11T23:01:25.300 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_max_write": "0", 2024-05-11T23:01:25.300 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_multithreaded": "true", 2024-05-11T23:01:25.300 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_require_active_mds": "true", 2024-05-11T23:01:25.300 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_set_user_groups": "true", 2024-05-11T23:01:25.300 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_splice_move": "true", 2024-05-11T23:01:25.300 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_splice_read": "true", 2024-05-11T23:01:25.300 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_splice_write": "true", 2024-05-11T23:01:25.301 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_syncfs_on_mksnap": "true", 2024-05-11T23:01:25.301 INFO:teuthology.orchestra.run.smithi005.stdout: "fuse_use_invalidate_cb": "true", 2024-05-11T23:01:25.301 INFO:teuthology.orchestra.run.smithi005.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-05-11T23:01:25.301 INFO:teuthology.orchestra.run.smithi005.stdout: "gss_target_name": "ceph", 2024-05-11T23:01:25.301 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_file": "", 2024-05-11T23:01:25.301 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_inject_failure": "0", 2024-05-11T23:01:25.301 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_interval": "5", 2024-05-11T23:01:25.301 INFO:teuthology.orchestra.run.smithi005.stdout: "host": "smithi005", 2024-05-11T23:01:25.301 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-05-11T23:01:25.301 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-05-11T23:01:25.301 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_max_size": "1073741824", 2024-05-11T23:01:25.301 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-05-11T23:01:25.301 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-05-11T23:01:25.301 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-05-11T23:01:25.302 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-05-11T23:01:25.302 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-05-11T23:01:25.302 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-05-11T23:01:25.302 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-05-11T23:01:25.302 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-05-11T23:01:25.302 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-05-11T23:01:25.302 INFO:teuthology.orchestra.run.smithi005.stdout: "immutable_object_cache_watermark": "0.900000", 2024-05-11T23:01:25.302 INFO:teuthology.orchestra.run.smithi005.stdout: "inject_early_sigterm": "false", 2024-05-11T23:01:25.302 INFO:teuthology.orchestra.run.smithi005.stdout: "jaeger_agent_port": "6799", 2024-05-11T23:01:25.302 INFO:teuthology.orchestra.run.smithi005.stdout: "jaeger_tracing_enable": "false", 2024-05-11T23:01:25.302 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_aio": "true", 2024-05-11T23:01:25.302 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_align_min_size": "65536", 2024-05-11T23:01:25.302 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_block_align": "true", 2024-05-11T23:01:25.302 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_block_size": "4096", 2024-05-11T23:01:25.303 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_dio": "true", 2024-05-11T23:01:25.303 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_discard": "false", 2024-05-11T23:01:25.303 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_force_aio": "false", 2024-05-11T23:01:25.303 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_ignore_corruption": "false", 2024-05-11T23:01:25.303 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_max_write_bytes": "10485760", 2024-05-11T23:01:25.303 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_max_write_entries": "100", 2024-05-11T23:01:25.303 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_replay_from": "0", 2024-05-11T23:01:25.303 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_throttle_high_multiple": "0.000000", 2024-05-11T23:01:25.303 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-05-11T23:01:25.303 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-05-11T23:01:25.303 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_throttle_max_multiple": "0.000000", 2024-05-11T23:01:25.303 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_write_header_frequency": "0", 2024-05-11T23:01:25.303 INFO:teuthology.orchestra.run.smithi005.stdout: "journal_zero_on_create": "false", 2024-05-11T23:01:25.303 INFO:teuthology.orchestra.run.smithi005.stdout: "journaler_prefetch_periods": "10", 2024-05-11T23:01:25.303 INFO:teuthology.orchestra.run.smithi005.stdout: "journaler_prezero_periods": "5", 2024-05-11T23:01:25.304 INFO:teuthology.orchestra.run.smithi005.stdout: "journaler_write_head_interval": "15", 2024-05-11T23:01:25.304 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "", 2024-05-11T23:01:25.304 INFO:teuthology.orchestra.run.smithi005.stdout: "keyfile": "", 2024-05-11T23:01:25.304 INFO:teuthology.orchestra.run.smithi005.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-05-11T23:01:25.304 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_backend": "rocksdb", 2024-05-11T23:01:25.304 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_default_stripe_size": "65536", 2024-05-11T23:01:25.304 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_fsck_on_mount": "false", 2024-05-11T23:01:25.304 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_fsck_on_mount_deep": "true", 2024-05-11T23:01:25.304 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_max_bytes": "67108864", 2024-05-11T23:01:25.304 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_max_ops": "512", 2024-05-11T23:01:25.305 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_nid_prealloc": "1024", 2024-05-11T23:01:25.305 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_onode_map_size": "1024", 2024-05-11T23:01:25.305 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-05-11T23:01:25.305 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_sync_submit_transaction": "false", 2024-05-11T23:01:25.305 INFO:teuthology.orchestra.run.smithi005.stdout: "kstore_sync_transaction": "false", 2024-05-11T23:01:25.305 INFO:teuthology.orchestra.run.smithi005.stdout: "librados_thread_count": "2", 2024-05-11T23:01:25.305 INFO:teuthology.orchestra.run.smithi005.stdout: "lockdep": "false", 2024-05-11T23:01:25.305 INFO:teuthology.orchestra.run.smithi005.stdout: "lockdep_force_backtrace": "false", 2024-05-11T23:01:25.305 INFO:teuthology.orchestra.run.smithi005.stdout: "log_coarse_timestamps": "true", 2024-05-11T23:01:25.305 INFO:teuthology.orchestra.run.smithi005.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-05-11T23:01:25.305 INFO:teuthology.orchestra.run.smithi005.stdout: "log_flush_on_exit": "false", 2024-05-11T23:01:25.305 INFO:teuthology.orchestra.run.smithi005.stdout: "log_graylog_host": "127.0.0.1", 2024-05-11T23:01:25.305 INFO:teuthology.orchestra.run.smithi005.stdout: "log_graylog_port": "12201", 2024-05-11T23:01:25.305 INFO:teuthology.orchestra.run.smithi005.stdout: "log_max_new": "1000", 2024-05-11T23:01:25.306 INFO:teuthology.orchestra.run.smithi005.stdout: "log_max_recent": "10000", 2024-05-11T23:01:25.306 INFO:teuthology.orchestra.run.smithi005.stdout: "log_stderr_prefix": "", 2024-05-11T23:01:25.306 INFO:teuthology.orchestra.run.smithi005.stdout: "log_stop_at_utilization": "0.970000", 2024-05-11T23:01:25.306 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": "true", 2024-05-11T23:01:25.306 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_graylog": "false", 2024-05-11T23:01:25.306 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_journald": "false", 2024-05-11T23:01:25.306 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_stderr": "false", 2024-05-11T23:01:25.306 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_syslog": "false", 2024-05-11T23:01:25.306 INFO:teuthology.orchestra.run.smithi005.stdout: "max_rotating_auth_attempts": "10", 2024-05-11T23:01:25.306 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-05-11T23:01:25.306 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_action_on_write_error": "1", 2024-05-11T23:01:25.306 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_alternate_name_max": "8192", 2024-05-11T23:01:25.306 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_asio_thread_count": "2", 2024-05-11T23:01:25.307 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_export_pin": "true", 2024-05-11T23:01:25.307 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_fragment_dirs": "true", 2024-05-11T23:01:25.307 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-05-11T23:01:25.307 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_fragment_interval": "5", 2024-05-11T23:01:25.307 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_fragment_size_max": "100000", 2024-05-11T23:01:25.307 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_idle_threshold": "0.000000", 2024-05-11T23:01:25.307 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_interval": "10", 2024-05-11T23:01:25.307 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_max": "-1", 2024-05-11T23:01:25.307 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_max_until": "-1", 2024-05-11T23:01:25.307 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_merge_size": "50", 2024-05-11T23:01:25.307 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_midchunk": "0.300000", 2024-05-11T23:01:25.307 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_min_rebalance": "0.100000", 2024-05-11T23:01:25.307 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_min_start": "0.200000", 2024-05-11T23:01:25.308 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_minchunk": "0.001000", 2024-05-11T23:01:25.308 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_mode": "0", 2024-05-11T23:01:25.308 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_need_max": "1.200000", 2024-05-11T23:01:25.308 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_need_min": "0.800000", 2024-05-11T23:01:25.308 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_overload_epochs": "2", 2024-05-11T23:01:25.308 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-05-11T23:01:25.308 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_sample_interval": "3.000000", 2024-05-11T23:01:25.308 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_split_bits": "3", 2024-05-11T23:01:25.308 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_split_rd": "25000.000000", 2024-05-11T23:01:25.308 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_split_size": "10000", 2024-05-11T23:01:25.308 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_split_wr": "10000.000000", 2024-05-11T23:01:25.308 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_target_decay": "10.000000", 2024-05-11T23:01:25.308 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-05-11T23:01:25.309 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_beacon_grace": "15.000000", 2024-05-11T23:01:25.309 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_beacon_interval": "4.000000", 2024-05-11T23:01:25.309 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_beacon_mon_down_grace": "60", 2024-05-11T23:01:25.309 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_memory_limit": "4294967296", 2024-05-11T23:01:25.309 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_mid": "0.700000", 2024-05-11T23:01:25.309 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-05-11T23:01:25.309 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_quiesce_delay": "0", 2024-05-11T23:01:25.309 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_quiesce_sleep": "200", 2024-05-11T23:01:25.309 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_quiesce_splitauth": "true", 2024-05-11T23:01:25.309 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_quiesce_threshold": "524288", 2024-05-11T23:01:25.309 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_release_free_interval": "10", 2024-05-11T23:01:25.309 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_reservation": "0.050000", 2024-05-11T23:01:25.310 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-05-11T23:01:25.310 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_trim_interval": "1", 2024-05-11T23:01:25.310 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache_trim_threshold": "262144", 2024-05-11T23:01:25.310 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-05-11T23:01:25.310 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-05-11T23:01:25.310 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_client_delegate_inos_pct": "50", 2024-05-11T23:01:25.310 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_client_prealloc_inos": "1000", 2024-05-11T23:01:25.310 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-05-11T23:01:25.310 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_connect_bootstrapping": "false", 2024-05-11T23:01:25.310 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_damage_table_max_entries": "10000", 2024-05-11T23:01:25.310 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-05-11T23:01:25.310 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_debug_auth_pins": "false", 2024-05-11T23:01:25.310 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_debug_frag": "false", 2024-05-11T23:01:25.311 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_debug_scatterstat": "false", 2024-05-11T23:01:25.311 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_debug_subtrees": "false", 2024-05-11T23:01:25.311 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_decay_halflife": "5.000000", 2024-05-11T23:01:25.311 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_default_dir_hash": "2", 2024-05-11T23:01:25.311 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_defer_session_stale": "true", 2024-05-11T23:01:25.311 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_deny_all_reconnect": "false", 2024-05-11T23:01:25.311 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_dir_keys_per_op": "16384", 2024-05-11T23:01:25.311 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_dir_max_commit_size": "10", 2024-05-11T23:01:25.311 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_dir_max_entries": "0", 2024-05-11T23:01:25.312 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_dir_prefetch": "true", 2024-05-11T23:01:25.312 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_dirstat_min_interval": "1.000000", 2024-05-11T23:01:25.312 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_dump_cache_after_rejoin": "false", 2024-05-11T23:01:25.312 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_dump_cache_on_map": "false", 2024-05-11T23:01:25.312 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_dump_cache_threshold_file": "0", 2024-05-11T23:01:25.312 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-05-11T23:01:25.312 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_early_reply": "true", 2024-05-11T23:01:25.312 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_enable_op_tracker": "true", 2024-05-11T23:01:25.312 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_enforce_unique_name": "true", 2024-05-11T23:01:25.312 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_export_ephemeral_distributed": "true", 2024-05-11T23:01:25.312 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-05-11T23:01:25.312 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_export_ephemeral_random": "true", 2024-05-11T23:01:25.312 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-05-11T23:01:25.312 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-05-11T23:01:25.313 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_forward_all_requests_to_auth": "false", 2024-05-11T23:01:25.313 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-05-11T23:01:25.313 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-05-11T23:01:25.313 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-05-11T23:01:25.313 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-05-11T23:01:25.313 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_health_cache_threshold": "1.500000", 2024-05-11T23:01:25.313 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_health_summarize_threshold": "10", 2024-05-11T23:01:25.313 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_heartbeat_grace": "15.000000", 2024-05-11T23:01:25.313 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_heartbeat_reset_grace": "1000", 2024-05-11T23:01:25.313 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_inject_health_dummy": "false", 2024-05-11T23:01:25.313 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-05-11T23:01:25.313 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_inject_migrator_session_race": "false", 2024-05-11T23:01:25.313 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-05-11T23:01:25.313 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-05-11T23:01:25.314 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-05-11T23:01:25.314 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_join_fs": "", 2024-05-11T23:01:25.314 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_journal_format": "1", 2024-05-11T23:01:25.314 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_create_at": "0", 2024-05-11T23:01:25.314 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_export_at": "0", 2024-05-11T23:01:25.314 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_import_at": "0", 2024-05-11T23:01:25.314 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_journal_at": "0", 2024-05-11T23:01:25.314 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_journal_expire_at": "0", 2024-05-11T23:01:25.314 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_journal_replay_at": "0", 2024-05-11T23:01:25.314 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_link_at": "0", 2024-05-11T23:01:25.314 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_mdstable_at": "0", 2024-05-11T23:01:25.314 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_openc_at": "0", 2024-05-11T23:01:25.314 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_rename_at": "0", 2024-05-11T23:01:25.314 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_shutdown_at": "0", 2024-05-11T23:01:25.315 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-05-11T23:01:25.315 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_event_large_threshold": "524288", 2024-05-11T23:01:25.315 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_events_per_segment": "1024", 2024-05-11T23:01:25.315 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_major_segment_event_ratio": "12", 2024-05-11T23:01:25.315 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_max_events": "-1", 2024-05-11T23:01:25.315 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_max_segments": "128", 2024-05-11T23:01:25.315 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_pause": "false", 2024-05-11T23:01:25.315 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_segment_size": "0", 2024-05-11T23:01:25.315 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_skip_corrupt_events": "false", 2024-05-11T23:01:25.315 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_skip_unbounded_events": "false", 2024-05-11T23:01:25.315 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-05-11T23:01:25.315 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_trim_threshold": "128", 2024-05-11T23:01:25.315 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-05-11T23:01:25.315 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log_warn_factor": "2.000000", 2024-05-11T23:01:25.315 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_max_caps_per_client": "1048576", 2024-05-11T23:01:25.315 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_max_completed_flushes": "100000", 2024-05-11T23:01:25.316 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_max_completed_requests": "100000", 2024-05-11T23:01:25.316 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_max_export_size": "20971520", 2024-05-11T23:01:25.316 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_max_file_recover": "32", 2024-05-11T23:01:25.316 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_max_purge_files": "64", 2024-05-11T23:01:25.316 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_max_purge_ops": "8192", 2024-05-11T23:01:25.316 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-05-11T23:01:25.316 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-05-11T23:01:25.316 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_max_snaps_per_dir": "100", 2024-05-11T23:01:25.316 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_metrics_update_interval": "2", 2024-05-11T23:01:25.316 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_min_caps_per_client": "100", 2024-05-11T23:01:25.316 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_min_caps_working_set": "10000", 2024-05-11T23:01:25.316 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-05-11T23:01:25.316 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_numa_node": "-1", 2024-05-11T23:01:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-05-11T23:01:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_op_complaint_time": "30.000000", 2024-05-11T23:01:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_op_history_duration": "600", 2024-05-11T23:01:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_op_history_size": "20", 2024-05-11T23:01:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_op_history_slow_op_size": "20", 2024-05-11T23:01:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_op_history_slow_op_threshold": "10", 2024-05-11T23:01:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_op_log_threshold": "5", 2024-05-11T23:01:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_ping_grace": "15", 2024-05-11T23:01:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_ping_interval": "5", 2024-05-11T23:01:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-05-11T23:01:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-05-11T23:01:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_recall_max_caps": "30000", 2024-05-11T23:01:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-05-11T23:01:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_recall_max_decay_threshold": "131072", 2024-05-11T23:01:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-05-11T23:01:25.317 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_recall_warning_threshold": "262144", 2024-05-11T23:01:25.318 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_reconnect_timeout": "45.000000", 2024-05-11T23:01:25.318 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_replay_interval": "1.000000", 2024-05-11T23:01:25.318 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-05-11T23:01:25.318 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-05-11T23:01:25.318 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_root_ino_gid": "0", 2024-05-11T23:01:25.318 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_root_ino_uid": "0", 2024-05-11T23:01:25.318 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-05-11T23:01:25.318 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_session_blocklist_on_evict": "true", 2024-05-11T23:01:25.318 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_session_blocklist_on_timeout": "true", 2024-05-11T23:01:25.318 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-05-11T23:01:25.318 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-05-11T23:01:25.318 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-05-11T23:01:25.318 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-05-11T23:01:25.318 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-05-11T23:01:25.319 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_session_metadata_threshold": "16777216", 2024-05-11T23:01:25.319 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-05-11T23:01:25.319 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_shutdown_check": "0", 2024-05-11T23:01:25.319 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_skip_ino": "0", 2024-05-11T23:01:25.319 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_sleep_rank_change": "0.000000", 2024-05-11T23:01:25.319 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_snap_max_uid": "4294967294", 2024-05-11T23:01:25.319 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_snap_min_uid": "0", 2024-05-11T23:01:25.319 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_snap_rstat": "false", 2024-05-11T23:01:25.319 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_standby_replay_damaged": "false", 2024-05-11T23:01:25.319 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_symlink_recovery": "true", 2024-05-11T23:01:25.319 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_task_status_update_interval": "2.000000", 2024-05-11T23:01:25.319 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_thrash_exports": "0", 2024-05-11T23:01:25.319 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_thrash_fragments": "0", 2024-05-11T23:01:25.319 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_tick_interval": "5.000000", 2024-05-11T23:01:25.320 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_valgrind_exit": "false", 2024-05-11T23:01:25.320 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_verify_backtrace": "1", 2024-05-11T23:01:25.320 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_verify_scatter": "false", 2024-05-11T23:01:25.320 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_wipe_ino_prealloc": "false", 2024-05-11T23:01:25.320 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_wipe_sessions": "false", 2024-05-11T23:01:25.320 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool_debug": "false", 2024-05-11T23:01:25.320 INFO:teuthology.orchestra.run.smithi005.stdout: "memstore_debug_omit_block_device_write": "false", 2024-05-11T23:01:25.320 INFO:teuthology.orchestra.run.smithi005.stdout: "memstore_device_bytes": "1073741824", 2024-05-11T23:01:25.320 INFO:teuthology.orchestra.run.smithi005.stdout: "memstore_page_set": "false", 2024-05-11T23:01:25.320 INFO:teuthology.orchestra.run.smithi005.stdout: "memstore_page_size": "65536", 2024-05-11T23:01:25.320 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_client_bytes": "134217728", 2024-05-11T23:01:25.320 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_client_messages": "512", 2024-05-11T23:01:25.320 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-05-11T23:01:25.320 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_connect_retry_interval": "1.000000", 2024-05-11T23:01:25.320 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-05-11T23:01:25.320 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-05-11T23:01:25.321 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_disabled_modules": "", 2024-05-11T23:01:25.321 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-05-11T23:01:25.321 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_max_pg_num_change": "128", 2024-05-11T23:01:25.321 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_mds_bytes": "134217728", 2024-05-11T23:01:25.321 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_mds_messages": "128", 2024-05-11T23:01:25.321 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-05-11T23:01:25.321 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_mon_bytes": "134217728", 2024-05-11T23:01:25.321 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_mon_messages": "128", 2024-05-11T23:01:25.321 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_osd_bytes": "536870912", 2024-05-11T23:01:25.321 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_osd_messages": "8192", 2024-05-11T23:01:25.321 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_pool": "true", 2024-05-11T23:01:25.321 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_service_beacon_grace": "60.000000", 2024-05-11T23:01:25.322 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_standby_modules": "true", 2024-05-11T23:01:25.322 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_stats_period": "5", 2024-05-11T23:01:25.322 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_stats_threshold": "5", 2024-05-11T23:01:25.322 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_tick_period": "2", 2024-05-11T23:01:25.322 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-05-11T23:01:25.322 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_accept_timeout_factor": "2.000000", 2024-05-11T23:01:25.322 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_allow_pool_delete": "true", 2024-05-11T23:01:25.322 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_allow_pool_size_one": "true", 2024-05-11T23:01:25.322 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_auth_validate_all_caps": "true", 2024-05-11T23:01:25.322 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-05-11T23:01:25.322 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-05-11T23:01:25.323 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_bytes": "104857600", 2024-05-11T23:01:25.323 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_directed_command_retry": "5", 2024-05-11T23:01:25.323 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_hunt_interval": "3.000000", 2024-05-11T23:01:25.323 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-05-11T23:01:25.323 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-05-11T23:01:25.323 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-05-11T23:01:25.323 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_hunt_parallel": "3", 2024-05-11T23:01:25.323 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_log_interval": "1.000000", 2024-05-11T23:01:25.323 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-05-11T23:01:25.323 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_ping_interval": "10.000000", 2024-05-11T23:01:25.323 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_ping_timeout": "30.000000", 2024-05-11T23:01:25.323 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_client_target_rank": "-1", 2024-05-11T23:01:25.323 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_clock_drift_allowed": "1.000000", 2024-05-11T23:01:25.323 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-05-11T23:01:25.324 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-05-11T23:01:25.324 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_file_level": "debug", 2024-05-11T23:01:25.324 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_to_file": "true", 2024-05-11T23:01:25.324 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_to_graylog": "false", 2024-05-11T23:01:25.324 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-05-11T23:01:25.324 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-05-11T23:01:25.324 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_to_journald": "false", 2024-05-11T23:01:25.324 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_to_stderr": "false", 2024-05-11T23:01:25.324 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-05-11T23:01:25.324 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-05-11T23:01:25.324 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-05-11T23:01:25.324 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_compact_on_bootstrap": "false", 2024-05-11T23:01:25.324 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_compact_on_start": "false", 2024-05-11T23:01:25.324 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_compact_on_trim": "true", 2024-05-11T23:01:25.324 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_con_tracker_persist_interval": "10", 2024-05-11T23:01:25.324 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_con_tracker_score_halflife": "43200", 2024-05-11T23:01:25.325 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_config_key_max_entry_size": "65536", 2024-05-11T23:01:25.325 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_cpu_threads": "4", 2024-05-11T23:01:25.325 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_crush_min_required_version": "hammer", 2024-05-11T23:01:25.325 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_daemon_bytes": "419430400", 2024-05-11T23:01:25.325 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-05-11T23:01:25.325 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_data_avail_crit": "5", 2024-05-11T23:01:25.325 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_data_avail_warn": "5", 2024-05-11T23:01:25.325 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_data_size_warn": "16106127360", 2024-05-11T23:01:25.325 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_block_osdmap_trim": "false", 2024-05-11T23:01:25.325 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-05-11T23:01:25.325 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_dump_json": "false", 2024-05-11T23:01:25.325 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-05-11T23:01:25.325 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_dump_transactions": "false", 2024-05-11T23:01:25.325 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_extra_checks": "false", 2024-05-11T23:01:25.325 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-05-11T23:01:25.325 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-05-11T23:01:25.326 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_no_require_reef": "false", 2024-05-11T23:01:25.326 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_no_require_squid": "false", 2024-05-11T23:01:25.326 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-05-11T23:01:25.326 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_delta_reset_interval": "10.000000", 2024-05-11T23:01:25.326 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_dns_srv_name": "ceph-mon", 2024-05-11T23:01:25.326 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_down_mkfs_grace": "120", 2024-05-11T23:01:25.326 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_down_uptime_grace": "60", 2024-05-11T23:01:25.326 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_election_default_strategy": "1", 2024-05-11T23:01:25.326 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_election_timeout": "5.000000", 2024-05-11T23:01:25.326 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-05-11T23:01:25.327 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_elector_ping_divisor": "2", 2024-05-11T23:01:25.327 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_elector_ping_timeout": "2.000000", 2024-05-11T23:01:25.327 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_enable_op_tracker": "true", 2024-05-11T23:01:25.327 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_fake_pool_delete": "false", 2024-05-11T23:01:25.327 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_force_quorum_join": "false", 2024-05-11T23:01:25.327 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_globalid_prealloc": "10000", 2024-05-11T23:01:25.327 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_health_detail_to_clog": "false", 2024-05-11T23:01:25.327 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_health_log_update_period": "5", 2024-05-11T23:01:25.327 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_health_max_detail": "50", 2024-05-11T23:01:25.327 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_health_to_clog": "true", 2024-05-11T23:01:25.327 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_health_to_clog_interval": "600", 2024-05-11T23:01:25.327 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-05-11T23:01:25.328 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_host": "172.21.15.5,[v2:172.21.15.5:3301,v1:172.21.15.5:6790],172.21.15.115", 2024-05-11T23:01:25.328 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_host_override": "", 2024-05-11T23:01:25.328 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_initial_members": "", 2024-05-11T23:01:25.328 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-05-11T23:01:25.328 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-05-11T23:01:25.328 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-05-11T23:01:25.328 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-05-11T23:01:25.328 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_keyvaluedb": "rocksdb", 2024-05-11T23:01:25.328 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_lease": "5.000000", 2024-05-11T23:01:25.328 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-05-11T23:01:25.328 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-05-11T23:01:25.328 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_log_full_interval": "50", 2024-05-11T23:01:25.328 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_log_max": "10000", 2024-05-11T23:01:25.328 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_log_max_summary": "50", 2024-05-11T23:01:25.329 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_max_log_entries_per_event": "4096", 2024-05-11T23:01:25.329 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_max_log_epochs": "500", 2024-05-11T23:01:25.329 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_max_mdsmap_epochs": "500", 2024-05-11T23:01:25.329 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_max_mgrmap_epochs": "500", 2024-05-11T23:01:25.329 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_max_osd": "10000", 2024-05-11T23:01:25.329 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_max_pg_per_osd": "10000", 2024-05-11T23:01:25.329 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_max_pool_pg_num": "65536", 2024-05-11T23:01:25.329 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-05-11T23:01:25.329 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-05-11T23:01:25.329 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_mds_force_trim_to": "0", 2024-05-11T23:01:25.329 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_mds_skip_sanity": "false", 2024-05-11T23:01:25.329 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_memory_autotune": "true", 2024-05-11T23:01:25.329 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_memory_target": "2147483648", 2024-05-11T23:01:25.329 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_mgr_beacon_grace": "30", 2024-05-11T23:01:25.329 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-05-11T23:01:25.329 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_mgr_digest_period": "5", 2024-05-11T23:01:25.330 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_mgr_inactive_grace": "60", 2024-05-11T23:01:25.330 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_mgr_mkfs_grace": "240", 2024-05-11T23:01:25.330 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-05-11T23:01:25.330 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_min_osdmap_epochs": "500", 2024-05-11T23:01:25.330 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_op_complaint_time": "30", 2024-05-11T23:01:25.330 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_op_history_duration": "600", 2024-05-11T23:01:25.330 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_op_history_size": "20", 2024-05-11T23:01:25.330 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_op_history_slow_op_size": "20", 2024-05-11T23:01:25.330 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_op_history_slow_op_threshold": "10", 2024-05-11T23:01:25.330 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_op_log_threshold": "5", 2024-05-11T23:01:25.330 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-05-11T23:01:25.330 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-05-11T23:01:25.330 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-05-11T23:01:25.330 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_auto_mark_in": "false", 2024-05-11T23:01:25.330 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_auto_mark_new_in": "true", 2024-05-11T23:01:25.331 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-05-11T23:01:25.331 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-05-11T23:01:25.331 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_cache_size": "500", 2024-05-11T23:01:25.331 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_cache_size_min": "134217728", 2024-05-11T23:01:25.331 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_crush_smoke_test": "true", 2024-05-11T23:01:25.331 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_destroyed_out_interval": "600", 2024-05-11T23:01:25.331 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_down_out_interval": "600", 2024-05-11T23:01:25.331 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-05-11T23:01:25.331 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-05-11T23:01:25.331 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_force_trim_to": "0", 2024-05-11T23:01:25.331 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_full_ratio": "0.950000", 2024-05-11T23:01:25.331 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-05-11T23:01:25.331 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_laggy_halflife": "3600", 2024-05-11T23:01:25.331 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_laggy_max_interval": "300", 2024-05-11T23:01:25.331 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_laggy_weight": "0.300000", 2024-05-11T23:01:25.332 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-05-11T23:01:25.332 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_max_creating_pgs": "1024", 2024-05-11T23:01:25.332 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_max_initial_pgs": "1024", 2024-05-11T23:01:25.332 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_min_down_reporters": "2", 2024-05-11T23:01:25.332 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-05-11T23:01:25.332 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-05-11T23:01:25.332 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-05-11T23:01:25.332 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_prime_pg_temp": "true", 2024-05-11T23:01:25.332 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-05-11T23:01:25.332 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-05-11T23:01:25.333 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_report_timeout": "900", 2024-05-11T23:01:25.333 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-05-11T23:01:25.333 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-05-11T23:01:25.333 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_warn_num_repaired": "10", 2024-05-11T23:01:25.333 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osd_warn_op_age": "32.000000", 2024-05-11T23:01:25.333 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-05-11T23:01:25.333 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osdmap_full_prune_interval": "10", 2024-05-11T23:01:25.333 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osdmap_full_prune_min": "10000", 2024-05-11T23:01:25.333 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-05-11T23:01:25.333 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-05-11T23:01:25.333 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_pg_stuck_threshold": "60", 2024-05-11T23:01:25.333 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-05-11T23:01:25.333 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_pg_warn_min_objects": "10000", 2024-05-11T23:01:25.333 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_pg_warn_min_per_osd": "0", 2024-05-11T23:01:25.333 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-05-11T23:01:25.334 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_pool_quota_crit_threshold": "0", 2024-05-11T23:01:25.334 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_pool_quota_warn_threshold": "0", 2024-05-11T23:01:25.334 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_probe_timeout": "2.000000", 2024-05-11T23:01:25.334 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_reweight_max_change": "0.050000", 2024-05-11T23:01:25.334 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_reweight_max_osds": "4", 2024-05-11T23:01:25.334 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-05-11T23:01:25.334 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-05-11T23:01:25.334 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-05-11T23:01:25.334 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-05-11T23:01:25.334 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-05-11T23:01:25.334 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_scrub_interval": "86400", 2024-05-11T23:01:25.334 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_scrub_max_keys": "100", 2024-05-11T23:01:25.334 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_scrub_timeout": "300", 2024-05-11T23:01:25.334 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_session_timeout": "300", 2024-05-11T23:01:25.335 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_smart_report_timeout": "5", 2024-05-11T23:01:25.335 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_stat_smooth_intervals": "6", 2024-05-11T23:01:25.335 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-05-11T23:01:25.335 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_stretch_pool_min_size": "2", 2024-05-11T23:01:25.335 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_stretch_pool_size": "4", 2024-05-11T23:01:25.335 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-05-11T23:01:25.335 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_subscribe_interval": "86400.000000", 2024-05-11T23:01:25.335 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_sync_debug": "false", 2024-05-11T23:01:25.335 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_sync_max_payload_keys": "2000", 2024-05-11T23:01:25.335 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_sync_max_payload_size": "1048576", 2024-05-11T23:01:25.335 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_sync_provider_kill_at": "0", 2024-05-11T23:01:25.335 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_sync_requester_kill_at": "0", 2024-05-11T23:01:25.335 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_sync_timeout": "60.000000", 2024-05-11T23:01:25.335 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_target_pg_per_osd": "100", 2024-05-11T23:01:25.336 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_tick_interval": "5", 2024-05-11T23:01:25.336 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_timecheck_interval": "300.000000", 2024-05-11T23:01:25.336 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-05-11T23:01:25.336 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_use_min_delay_socket": "false", 2024-05-11T23:01:25.336 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_older_version_delay": "604800", 2024-05-11T23:01:25.336 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-05-11T23:01:25.336 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-05-11T23:01:25.336 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-05-11T23:01:25.336 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_filestore_osds": "false", 2024-05-11T23:01:25.336 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-05-11T23:01:25.336 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-05-11T23:01:25.336 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-05-11T23:01:25.336 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_misplaced": "false", 2024-05-11T23:01:25.336 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-05-11T23:01:25.337 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_older_version": "true", 2024-05-11T23:01:25.337 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-05-11T23:01:25.337 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_pool_no_app": "false", 2024-05-11T23:01:25.337 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-05-11T23:01:25.337 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-05-11T23:01:25.337 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-05-11T23:01:25.337 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-05-11T23:01:25.337 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_on_too_few_osds": "false", 2024-05-11T23:01:25.337 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-05-11T23:01:25.337 INFO:teuthology.orchestra.run.smithi005.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-05-11T23:01:25.337 INFO:teuthology.orchestra.run.smithi005.stdout: "monmap": "", 2024-05-11T23:01:25.337 INFO:teuthology.orchestra.run.smithi005.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-05-11T23:01:25.337 INFO:teuthology.orchestra.run.smithi005.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-05-11T23:01:25.337 INFO:teuthology.orchestra.run.smithi005.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-05-11T23:01:25.337 INFO:teuthology.orchestra.run.smithi005.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-05-11T23:01:25.338 INFO:teuthology.orchestra.run.smithi005.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-05-11T23:01:25.338 INFO:teuthology.orchestra.run.smithi005.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-05-11T23:01:25.338 INFO:teuthology.orchestra.run.smithi005.stdout: "motr_tracing_enabled": "false", 2024-05-11T23:01:25.338 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_op_threads": "3", 2024-05-11T23:01:25.338 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_buffer_size": "131072", 2024-05-11T23:01:25.338 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_cm": "false", 2024-05-11T23:01:25.338 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_device_name": "", 2024-05-11T23:01:25.338 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_dscp": "96", 2024-05-11T23:01:25.338 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-05-11T23:01:25.338 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_gid_idx": "0", 2024-05-11T23:01:25.338 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_local_gid": "", 2024-05-11T23:01:25.338 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_polling_us": "1000", 2024-05-11T23:01:25.338 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_port_num": "1", 2024-05-11T23:01:25.339 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-05-11T23:01:25.339 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-05-11T23:01:25.339 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_roce_ver": "1", 2024-05-11T23:01:25.339 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_send_buffers": "1024", 2024-05-11T23:01:25.339 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_sl": "3", 2024-05-11T23:01:25.339 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_support_srq": "true", 2024-05-11T23:01:25.339 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_rdma_type": "ib", 2024-05-11T23:01:25.339 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_async_reap_threshold": "5", 2024-05-11T23:01:25.339 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_bind_before_connect": "false", 2024-05-11T23:01:25.339 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_bind_ipv4": "true", 2024-05-11T23:01:25.339 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_bind_ipv6": "false", 2024-05-11T23:01:25.340 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_bind_msgr1": "true", 2024-05-11T23:01:25.340 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_bind_msgr2": "true", 2024-05-11T23:01:25.340 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_bind_port_max": "7568", 2024-05-11T23:01:25.340 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_bind_port_min": "6800", 2024-05-11T23:01:25.340 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_bind_prefer_ipv4": "false", 2024-05-11T23:01:25.340 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_bind_retry_count": "3", 2024-05-11T23:01:25.340 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_bind_retry_delay": "5", 2024-05-11T23:01:25.340 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_blackhole_client": "false", 2024-05-11T23:01:25.340 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_blackhole_mds": "false", 2024-05-11T23:01:25.340 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_blackhole_mgr": "false", 2024-05-11T23:01:25.340 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_blackhole_mon": "false", 2024-05-11T23:01:25.340 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_blackhole_osd": "false", 2024-05-11T23:01:25.340 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_client_mode": "crc secure", 2024-05-11T23:01:25.340 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_cluster_mode": "crc secure", 2024-05-11T23:01:25.341 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_cluster_type": "", 2024-05-11T23:01:25.341 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_compress_secure": "false", 2024-05-11T23:01:25.341 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_connection_idle_timeout": "900", 2024-05-11T23:01:25.341 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_connection_ready_timeout": "10", 2024-05-11T23:01:25.341 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_crc_data": "true", 2024-05-11T23:01:25.341 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_crc_header": "true", 2024-05-11T23:01:25.341 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_die_on_bad_msg": "false", 2024-05-11T23:01:25.341 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_die_on_bug": "true", 2024-05-11T23:01:25.341 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_die_on_old_message": "true", 2024-05-11T23:01:25.341 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_die_on_skipped_message": "false", 2024-05-11T23:01:25.341 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_die_on_unhandled_msg": "false", 2024-05-11T23:01:25.342 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-05-11T23:01:25.342 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_coremask": "0xF", 2024-05-11T23:01:25.342 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-05-11T23:01:25.342 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_devs_allowlist": "", 2024-05-11T23:01:25.342 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_enable_tso": "true", 2024-05-11T23:01:25.342 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-05-11T23:01:25.342 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-05-11T23:01:25.342 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_hugepages": "", 2024-05-11T23:01:25.342 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_hw_flow_control": "true", 2024-05-11T23:01:25.343 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-05-11T23:01:25.343 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_lro": "true", 2024-05-11T23:01:25.343 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_memory_channel": "4", 2024-05-11T23:01:25.343 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-05-11T23:01:25.343 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_pmd": "", 2024-05-11T23:01:25.343 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_port_id": "0", 2024-05-11T23:01:25.343 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-05-11T23:01:25.343 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dump_corrupt_message_level": "1", 2024-05-11T23:01:25.343 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_dump_on_send": "false", 2024-05-11T23:01:25.343 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_initial_backoff": "0.200000", 2024-05-11T23:01:25.343 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_inject_delay_max": "1.000000", 2024-05-11T23:01:25.343 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_inject_delay_probability": "0.000000", 2024-05-11T23:01:25.344 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_inject_delay_type": "", 2024-05-11T23:01:25.344 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_inject_internal_delays": "0.000000", 2024-05-11T23:01:25.344 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_inject_network_congestion": "0", 2024-05-11T23:01:25.344 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_inject_socket_failures": "5000", 2024-05-11T23:01:25.344 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_learn_addr_from_peer": "true", 2024-05-11T23:01:25.344 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_max_accept_failures": "4", 2024-05-11T23:01:25.344 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_max_backoff": "15.000000", 2024-05-11T23:01:25.344 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_mon_client_mode": "secure crc", 2024-05-11T23:01:25.344 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_mon_cluster_mode": "secure crc", 2024-05-11T23:01:25.344 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_mon_service_mode": "secure crc", 2024-05-11T23:01:25.344 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_osd_compress_min_size": "1024", 2024-05-11T23:01:25.344 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_osd_compress_mode": "none", 2024-05-11T23:01:25.344 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_osd_compression_algorithm": "snappy", 2024-05-11T23:01:25.344 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-05-11T23:01:25.344 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_pq_min_cost": "65536", 2024-05-11T23:01:25.345 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_public_type": "", 2024-05-11T23:01:25.345 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_service_mode": "crc secure", 2024-05-11T23:01:25.345 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_tcp_listen_backlog": "512", 2024-05-11T23:01:25.345 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_tcp_nodelay": "true", 2024-05-11T23:01:25.345 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-05-11T23:01:25.345 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_tcp_rcvbuf": "0", 2024-05-11T23:01:25.345 INFO:teuthology.orchestra.run.smithi005.stdout: "ms_type": "async+posix", 2024-05-11T23:01:25.345 INFO:teuthology.orchestra.run.smithi005.stdout: "no_config_file": "false", 2024-05-11T23:01:25.345 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter_completion_locks_per_session": "32", 2024-05-11T23:01:25.345 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter_debug_inject_relock_delay": "false", 2024-05-11T23:01:25.345 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter_inflight_op_bytes": "104857600", 2024-05-11T23:01:25.345 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter_inflight_ops": "1024", 2024-05-11T23:01:25.345 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter_inject_no_watch_ping": "false", 2024-05-11T23:01:25.345 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-05-11T23:01:25.345 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter_tick_interval": "5.000000", 2024-05-11T23:01:25.346 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter_timeout": "10.000000", 2024-05-11T23:01:25.346 INFO:teuthology.orchestra.run.smithi005.stdout: "objectstore_blackhole": "false", 2024-05-11T23:01:25.346 INFO:teuthology.orchestra.run.smithi005.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-05-11T23:01:25.346 INFO:teuthology.orchestra.run.smithi005.stdout: "openssl_engine_opts": "", 2024-05-11T23:01:25.346 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_agent_delay_time": "5.000000", 2024-05-11T23:01:25.346 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_agent_hist_halflife": "1000", 2024-05-11T23:01:25.346 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_agent_max_low_ops": "2", 2024-05-11T23:01:25.346 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_agent_max_ops": "4", 2024-05-11T23:01:25.346 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-05-11T23:01:25.346 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_agent_quantize_effort": "0.100000", 2024-05-11T23:01:25.346 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_agent_slop": "0.020000", 2024-05-11T23:01:25.346 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-05-11T23:01:25.346 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_allow_recovery_below_min_size": "true", 2024-05-11T23:01:25.346 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_asio_thread_count": "2", 2024-05-11T23:01:25.346 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_async_recovery_min_cost": "100", 2024-05-11T23:01:25.347 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_auto_mark_unfound_lost": "false", 2024-05-11T23:01:25.347 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_backfill_retry_interval": "30.000000", 2024-05-11T23:01:25.347 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_backfill_scan_max": "512", 2024-05-11T23:01:25.347 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_backfill_scan_min": "64", 2024-05-11T23:01:25.347 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_backoff_on_degraded": "false", 2024-05-11T23:01:25.347 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_backoff_on_peering": "false", 2024-05-11T23:01:25.347 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_backoff_on_unfound": "true", 2024-05-11T23:01:25.347 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_beacon_report_interval": "300", 2024-05-11T23:01:25.347 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bench_duration": "30", 2024-05-11T23:01:25.347 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-05-11T23:01:25.347 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bench_max_block_size": "67108864", 2024-05-11T23:01:25.347 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bench_small_size_max_iops": "100", 2024-05-11T23:01:25.347 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_blkin_trace_all": "false", 2024-05-11T23:01:25.347 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_blocked_scrub_grace_period": "120", 2024-05-11T23:01:25.347 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-05-11T23:01:25.348 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-05-11T23:01:25.348 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-05-11T23:01:25.348 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_check_for_log_corruption": "false", 2024-05-11T23:01:25.348 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-05-11T23:01:25.348 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-05-11T23:01:25.348 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-05-11T23:01:25.348 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-05-11T23:01:25.348 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_class_update_on_start": "true", 2024-05-11T23:01:25.348 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_client_message_cap": "256", 2024-05-11T23:01:25.348 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_client_message_size_cap": "524288000", 2024-05-11T23:01:25.348 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_client_op_priority": "63", 2024-05-11T23:01:25.348 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_client_watch_timeout": "30", 2024-05-11T23:01:25.348 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_command_max_records": "256", 2024-05-11T23:01:25.348 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_compact_on_start": "false", 2024-05-11T23:01:25.348 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-05-11T23:01:25.349 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_crush_chooseleaf_type": "0", 2024-05-11T23:01:25.349 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_crush_initial_weight": "-1.000000", 2024-05-11T23:01:25.349 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_crush_update_on_start": "true", 2024-05-11T23:01:25.349 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_crush_update_weight_set": "true", 2024-05-11T23:01:25.349 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-05-11T23:01:25.349 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-05-11T23:01:25.349 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-05-11T23:01:25.349 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_drop_ping_duration": "0", 2024-05-11T23:01:25.349 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-05-11T23:01:25.349 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_feed_pullee": "-1", 2024-05-11T23:01:25.349 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-05-11T23:01:25.350 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-05-11T23:01:25.350 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-05-11T23:01:25.350 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_misdirected_ops": "false", 2024-05-11T23:01:25.350 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_no_acting_change": "false", 2024-05-11T23:01:25.350 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_no_purge_strays": "false", 2024-05-11T23:01:25.350 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_op_order": "false", 2024-05-11T23:01:25.350 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_pg_log_writeout": "false", 2024-05-11T23:01:25.350 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_pretend_recovery_active": "false", 2024-05-11T23:01:25.350 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-05-11T23:01:25.350 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-05-11T23:01:25.350 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_shutdown": "false", 2024-05-11T23:01:25.350 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-05-11T23:01:25.350 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-05-11T23:01:25.351 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_trim_objects": "false", 2024-05-11T23:01:25.351 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_verify_cached_snaps": "false", 2024-05-11T23:01:25.351 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_verify_missing_on_start": "false", 2024-05-11T23:01:25.351 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_verify_snaps": "false", 2024-05-11T23:01:25.351 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-05-11T23:01:25.351 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-05-11T23:01:25.351 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_deep_scrub_keys": "1024", 2024-05-11T23:01:25.351 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-05-11T23:01:25.351 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-05-11T23:01:25.351 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-05-11T23:01:25.351 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_deep_scrub_stride": "524288", 2024-05-11T23:01:25.351 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-05-11T23:01:25.351 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_default_data_pool_replay_window": "5", 2024-05-11T23:01:25.351 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_default_notify_timeout": "30", 2024-05-11T23:01:25.351 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_delete_sleep": "0.000000", 2024-05-11T23:01:25.352 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-05-11T23:01:25.352 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-05-11T23:01:25.352 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-05-11T23:01:25.352 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_discard_disconnected_ops": "true", 2024-05-11T23:01:25.352 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_enable_op_tracker": "true", 2024-05-11T23:01:25.352 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-05-11T23:01:25.352 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_extblkdev_plugins": "vdo", 2024-05-11T23:01:25.352 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-05-11T23:01:25.352 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-05-11T23:01:25.352 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_fast_info": "true", 2024-05-11T23:01:25.352 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_fast_shutdown": "true", 2024-05-11T23:01:25.352 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-05-11T23:01:25.352 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_fast_shutdown_timeout": "15", 2024-05-11T23:01:25.352 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-05-11T23:01:25.352 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-05-11T23:01:25.353 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-05-11T23:01:25.353 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_function_tracing": "false", 2024-05-11T23:01:25.353 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_heartbeat_grace": "20", 2024-05-11T23:01:25.353 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_heartbeat_interval": "6", 2024-05-11T23:01:25.353 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-05-11T23:01:25.353 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_heartbeat_min_peers": "10", 2024-05-11T23:01:25.353 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_heartbeat_min_size": "2000", 2024-05-11T23:01:25.353 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_heartbeat_stale": "600", 2024-05-11T23:01:25.354 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-05-11T23:01:25.354 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_hit_set_max_size": "100000", 2024-05-11T23:01:25.354 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_hit_set_min_size": "1000", 2024-05-11T23:01:25.354 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-05-11T23:01:25.354 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-05-11T23:01:25.354 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-05-11T23:01:25.354 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-05-11T23:01:25.354 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-05-11T23:01:25.354 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_journal_flush_on_shutdown": "true", 2024-05-11T23:01:25.354 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_journal_size": "5120", 2024-05-11T23:01:25.355 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_kill_backfill_at": "0", 2024-05-11T23:01:25.355 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_loop_before_reset_tphandle": "64", 2024-05-11T23:01:25.355 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_map_cache_size": "50", 2024-05-11T23:01:25.355 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_map_dedup": "true", 2024-05-11T23:01:25.355 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_map_message_max": "40", 2024-05-11T23:01:25.355 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_map_message_max_bytes": "10485760", 2024-05-11T23:01:25.355 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_map_share_max_epochs": "40", 2024-05-11T23:01:25.355 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_attr_name_len": "100", 2024-05-11T23:01:25.356 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_attr_size": "0", 2024-05-11T23:01:25.356 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_backfills": "1", 2024-05-11T23:01:25.356 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_markdown_count": "5", 2024-05-11T23:01:25.356 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_markdown_period": "600", 2024-05-11T23:01:25.356 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_object_name_len": "2048", 2024-05-11T23:01:25.356 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_object_namespace_len": "256", 2024-05-11T23:01:25.356 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_object_size": "134217728", 2024-05-11T23:01:25.356 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-05-11T23:01:25.356 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_omap_entries_per_request": "1024", 2024-05-11T23:01:25.356 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_pg_blocked_by": "16", 2024-05-11T23:01:25.356 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_pg_log_entries": "10000", 2024-05-11T23:01:25.356 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-05-11T23:01:25.356 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_pgls": "1024", 2024-05-11T23:01:25.356 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_push_cost": "8388608", 2024-05-11T23:01:25.357 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_push_objects": "10", 2024-05-11T23:01:25.357 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_scrubs": "3", 2024-05-11T23:01:25.357 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-05-11T23:01:25.357 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_trimming_pgs": "2", 2024-05-11T23:01:25.357 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_write_op_reply_len": "64", 2024-05-11T23:01:25.357 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_max_write_size": "90", 2024-05-11T23:01:25.357 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-05-11T23:01:25.357 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-05-11T23:01:25.357 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-05-11T23:01:25.357 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-05-11T23:01:25.357 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-05-11T23:01:25.357 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-05-11T23:01:25.357 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-05-11T23:01:25.357 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_override_recovery_settings": "false", 2024-05-11T23:01:25.358 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_profile": "balanced", 2024-05-11T23:01:25.358 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-05-11T23:01:25.358 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-05-11T23:01:25.358 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-05-11T23:01:25.358 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-05-11T23:01:25.358 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-05-11T23:01:25.358 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-05-11T23:01:25.358 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-05-11T23:01:25.358 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-05-11T23:01:25.358 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-05-11T23:01:25.358 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-05-11T23:01:25.358 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mclock_skip_benchmark": "false", 2024-05-11T23:01:25.358 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_memory_base": "805306368", 2024-05-11T23:01:25.359 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_memory_cache_min": "134217728", 2024-05-11T23:01:25.359 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-05-11T23:01:25.359 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-05-11T23:01:25.359 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_memory_target": "4294967296", 2024-05-11T23:01:25.359 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_memory_target_autotune": "false", 2024-05-11T23:01:25.359 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-05-11T23:01:25.359 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_min_pg_log_entries": "250", 2024-05-11T23:01:25.359 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_min_recovery_priority": "0", 2024-05-11T23:01:25.359 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mon_heartbeat_interval": "30", 2024-05-11T23:01:25.359 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-05-11T23:01:25.359 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mon_report_interval": "5", 2024-05-11T23:01:25.359 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mon_report_max_in_flight": "2", 2024-05-11T23:01:25.359 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-05-11T23:01:25.360 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_num_cache_shards": "32", 2024-05-11T23:01:25.360 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_num_op_tracker_shard": "32", 2024-05-11T23:01:25.360 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_numa_auto_affinity": "true", 2024-05-11T23:01:25.360 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_numa_node": "-1", 2024-05-11T23:01:25.360 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_numa_prefer_iface": "true", 2024-05-11T23:01:25.360 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-05-11T23:01:25.360 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_objecter_finishers": "1", 2024-05-11T23:01:25.360 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_objectstore": "bluestore", 2024-05-11T23:01:25.360 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_objectstore_fuse": "false", 2024-05-11T23:01:25.360 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_objectstore_tracing": "false", 2024-05-11T23:01:25.360 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_complaint_time": "30.000000", 2024-05-11T23:01:25.360 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_history_duration": "600", 2024-05-11T23:01:25.360 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_history_size": "20", 2024-05-11T23:01:25.360 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_history_slow_op_size": "20", 2024-05-11T23:01:25.360 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-05-11T23:01:25.361 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_log_threshold": "5", 2024-05-11T23:01:25.361 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_num_shards": "0", 2024-05-11T23:01:25.361 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_num_shards_hdd": "5", 2024-05-11T23:01:25.361 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_num_shards_ssd": "8", 2024-05-11T23:01:25.361 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_num_threads_per_shard": "0", 2024-05-11T23:01:25.361 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-05-11T23:01:25.361 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-05-11T23:01:25.361 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-05-11T23:01:25.361 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_pq_min_cost": "65536", 2024-05-11T23:01:25.361 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_queue": "mclock_scheduler", 2024-05-11T23:01:25.361 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_queue_cut_off": "high", 2024-05-11T23:01:25.361 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_thread_suicide_timeout": "150", 2024-05-11T23:01:25.361 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_op_thread_timeout": "15", 2024-05-11T23:01:25.361 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_open_classes_on_start": "true", 2024-05-11T23:01:25.361 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_os_flags": "0", 2024-05-11T23:01:25.362 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_peering_op_priority": "255", 2024-05-11T23:01:25.362 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_delete_cost": "1048576", 2024-05-11T23:01:25.362 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_delete_priority": "5", 2024-05-11T23:01:25.362 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-05-11T23:01:25.362 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-05-11T23:01:25.362 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_log_dups_tracked": "3000", 2024-05-11T23:01:25.362 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_log_trim_max": "10000", 2024-05-11T23:01:25.362 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_log_trim_min": "100", 2024-05-11T23:01:25.362 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-05-11T23:01:25.362 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_object_context_cache_count": "64", 2024-05-11T23:01:25.362 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-05-11T23:01:25.362 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-05-11T23:01:25.362 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-05-11T23:01:25.362 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-05-11T23:01:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-05-11T23:01:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-05-11T23:01:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-05-11T23:01:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-05-11T23:01:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_crimson": "false", 2024-05-11T23:01:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_crush_rule": "-1", 2024-05-11T23:01:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_ec_fast_read": "false", 2024-05-11T23:01:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-05-11T23:01:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_flag_bulk": "false", 2024-05-11T23:01:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-05-11T23:01:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_flag_nodelete": "false", 2024-05-11T23:01:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-05-11T23:01:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-05-11T23:01:25.364 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_flags": "0", 2024-05-11T23:01:25.364 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-05-11T23:01:25.364 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_min_size": "0", 2024-05-11T23:01:25.364 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-05-11T23:01:25.364 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_pg_num": "32", 2024-05-11T23:01:25.364 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_pgp_num": "0", 2024-05-11T23:01:25.364 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-05-11T23:01:25.364 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_read_ratio": "70", 2024-05-11T23:01:25.364 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_size": "2", 2024-05-11T23:01:25.364 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_default_type": "replicated", 2024-05-11T23:01:25.364 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-05-11T23:01:25.364 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pool_use_gmt_hitset": "true", 2024-05-11T23:01:25.365 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_push_per_object_cost": "1000", 2024-05-11T23:01:25.365 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_read_ec_check_for_errors": "false", 2024-05-11T23:01:25.365 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recover_clone_overlap": "true", 2024-05-11T23:01:25.365 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recover_clone_overlap_limit": "10", 2024-05-11T23:01:25.365 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_cost": "20971520", 2024-05-11T23:01:25.365 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_delay_start": "0.000000", 2024-05-11T23:01:25.365 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_max_active": "0", 2024-05-11T23:01:25.365 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_max_active_hdd": "3", 2024-05-11T23:01:25.365 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_max_active_ssd": "10", 2024-05-11T23:01:25.365 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_max_chunk": "8388608", 2024-05-11T23:01:25.365 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-05-11T23:01:25.365 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_max_single_start": "1", 2024-05-11T23:01:25.365 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_op_priority": "3", 2024-05-11T23:01:25.365 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_op_warn_multiple": "16", 2024-05-11T23:01:25.365 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_priority": "5", 2024-05-11T23:01:25.365 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_retry_interval": "30.000000", 2024-05-11T23:01:25.366 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_sleep": "0.000000", 2024-05-11T23:01:25.366 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-05-11T23:01:25.366 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-05-11T23:01:25.366 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-05-11T23:01:25.366 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_repair_during_recovery": "false", 2024-05-11T23:01:25.366 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_requested_scrub_priority": "120", 2024-05-11T23:01:25.366 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-05-11T23:01:25.366 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_rollback_to_cluster_snap": "", 2024-05-11T23:01:25.366 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_auto_repair": "false", 2024-05-11T23:01:25.367 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-05-11T23:01:25.367 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-05-11T23:01:25.367 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_begin_hour": "0", 2024-05-11T23:01:25.367 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_begin_week_day": "0", 2024-05-11T23:01:25.367 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_chunk_max": "25", 2024-05-11T23:01:25.367 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_chunk_min": "5", 2024-05-11T23:01:25.367 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_cost": "52428800", 2024-05-11T23:01:25.367 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-05-11T23:01:25.367 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_during_recovery": "false", 2024-05-11T23:01:25.367 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_end_hour": "0", 2024-05-11T23:01:25.367 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_end_week_day": "0", 2024-05-11T23:01:25.367 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_event_cost": "4096", 2024-05-11T23:01:25.367 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-05-11T23:01:25.367 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-05-11T23:01:25.367 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_invalid_stats": "true", 2024-05-11T23:01:25.368 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_load_threshold": "0.500000", 2024-05-11T23:01:25.368 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_max_interval": "604800.000000", 2024-05-11T23:01:25.368 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_max_preemptions": "5", 2024-05-11T23:01:25.368 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_min_interval": "86400.000000", 2024-05-11T23:01:25.368 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_priority": "5", 2024-05-11T23:01:25.368 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_scrub_sleep": "0.000000", 2024-05-11T23:01:25.368 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-05-11T23:01:25.368 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-05-11T23:01:25.368 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_shutdown_pgref_assert": "false", 2024-05-11T23:01:25.368 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_skip_data_digest": "false", 2024-05-11T23:01:25.368 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_smart_report_timeout": "5", 2024-05-11T23:01:25.368 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_snap_trim_cost": "1048576", 2024-05-11T23:01:25.368 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_snap_trim_priority": "5", 2024-05-11T23:01:25.368 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_snap_trim_sleep": "0.000000", 2024-05-11T23:01:25.369 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-05-11T23:01:25.369 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-05-11T23:01:25.369 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-05-11T23:01:25.369 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-05-11T23:01:25.369 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_stats_update_period_scrubbing": "15", 2024-05-11T23:01:25.369 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-05-11T23:01:25.369 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_target_transaction_size": "30", 2024-05-11T23:01:25.369 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-05-11T23:01:25.369 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-05-11T23:01:25.369 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-05-11T23:01:25.369 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-05-11T23:01:25.369 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-05-11T23:01:25.369 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-05-11T23:01:25.369 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-05-11T23:01:25.370 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tier_default_cache_mode": "writeback", 2024-05-11T23:01:25.370 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-05-11T23:01:25.370 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-05-11T23:01:25.370 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_tracing": "false", 2024-05-11T23:01:25.370 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_use_stale_snap": "false", 2024-05-11T23:01:25.370 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-05-11T23:01:25.370 INFO:teuthology.orchestra.run.smithi005.stdout: "osdc_blkin_trace_all": "false", 2024-05-11T23:01:25.370 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_kill_at": "0", 2024-05-11T23:01:25.370 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_max_join_drift": "10", 2024-05-11T23:01:25.370 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_min": "500", 2024-05-11T23:01:25.370 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_min_wait": "0.050000", 2024-05-11T23:01:25.370 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_propose_interval": "1.000000", 2024-05-11T23:01:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_service_trim_max": "500", 2024-05-11T23:01:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_service_trim_max_multiplier": "20", 2024-05-11T23:01:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_service_trim_min": "250", 2024-05-11T23:01:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_stash_full_interval": "25", 2024-05-11T23:01:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_trim_max": "500", 2024-05-11T23:01:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: "paxos_trim_min": "250", 2024-05-11T23:01:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: "perf": "true", 2024-05-11T23:01:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-05-11T23:01:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-05-11T23:01:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-05-11T23:01:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "-", 2024-05-11T23:01:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrv": "", 2024-05-11T23:01:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: "public_bind_addr": "-", 2024-05-11T23:01:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: "public_network": "", 2024-05-11T23:01:25.371 INFO:teuthology.orchestra.run.smithi005.stdout: "public_network_interface": "", 2024-05-11T23:01:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: "qat_compressor_enabled": "false", 2024-05-11T23:01:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: "qat_compressor_session_max_number": "256", 2024-05-11T23:01:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: "rados_mon_op_timeout": "0", 2024-05-11T23:01:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: "rados_osd_op_timeout": "0", 2024-05-11T23:01:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: "rados_tracing": "false", 2024-05-11T23:01:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_atime_update_interval": "60", 2024-05-11T23:01:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-05-11T23:01:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_balance_parent_reads": "false", 2024-05-11T23:01:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_balance_snap_reads": "false", 2024-05-11T23:01:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_blkin_trace_all": "false", 2024-05-11T23:01:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_blocklist_expire_seconds": "0", 2024-05-11T23:01:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_blocklist_on_break_lock": "true", 2024-05-11T23:01:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_cache": "true", 2024-05-11T23:01:25.372 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_cache_block_writes_upfront": "false", 2024-05-11T23:01:25.373 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_cache_max_dirty": "25165824", 2024-05-11T23:01:25.373 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-05-11T23:01:25.373 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_cache_max_dirty_object": "0", 2024-05-11T23:01:25.373 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_cache_policy": "writearound", 2024-05-11T23:01:25.373 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_cache_size": "33554432", 2024-05-11T23:01:25.373 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_cache_target_dirty": "16777216", 2024-05-11T23:01:25.373 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-05-11T23:01:25.373 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_clone_copy_on_read": "false", 2024-05-11T23:01:25.373 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_compression_hint": "none", 2024-05-11T23:01:25.373 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_concurrent_management_ops": "10", 2024-05-11T23:01:25.373 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-05-11T23:01:25.373 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_default_clone_format": "auto", 2024-05-11T23:01:25.373 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_default_data_pool": "", 2024-05-11T23:01:25.373 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_default_features": "61", 2024-05-11T23:01:25.374 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_default_format": "2", 2024-05-11T23:01:25.374 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_default_map_options": "", 2024-05-11T23:01:25.374 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_default_order": "22", 2024-05-11T23:01:25.374 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_default_pool": "rbd", 2024-05-11T23:01:25.374 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-05-11T23:01:25.374 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_default_stripe_count": "0", 2024-05-11T23:01:25.374 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_default_stripe_unit": "0", 2024-05-11T23:01:25.374 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_disable_zero_copy_writes": "true", 2024-05-11T23:01:25.374 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_discard_granularity_bytes": "65536", 2024-05-11T23:01:25.375 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-05-11T23:01:25.375 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_enable_alloc_hint": "true", 2024-05-11T23:01:25.375 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-05-11T23:01:25.375 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_io_scheduler": "simple", 2024-05-11T23:01:25.375 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-05-11T23:01:25.375 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_commit_age": "5.000000", 2024-05-11T23:01:25.375 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-05-11T23:01:25.375 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-05-11T23:01:25.375 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-05-11T23:01:25.375 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-05-11T23:01:25.375 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_object_flush_interval": "0", 2024-05-11T23:01:25.375 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-05-11T23:01:25.375 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-05-11T23:01:25.375 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_order": "24", 2024-05-11T23:01:25.375 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_pool": "", 2024-05-11T23:01:25.376 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_journal_splay_width": "4", 2024-05-11T23:01:25.376 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_localize_parent_reads": "false", 2024-05-11T23:01:25.376 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_localize_snap_reads": "false", 2024-05-11T23:01:25.376 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-05-11T23:01:25.376 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-05-11T23:01:25.376 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-05-11T23:01:25.376 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-05-11T23:01:25.376 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-05-11T23:01:25.376 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-05-11T23:01:25.376 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_image_policy_type": "simple", 2024-05-11T23:01:25.376 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-05-11T23:01:25.376 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-05-11T23:01:25.376 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-05-11T23:01:25.376 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-05-11T23:01:25.377 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-05-11T23:01:25.377 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-05-11T23:01:25.377 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-05-11T23:01:25.377 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_memory_autotune": "true", 2024-05-11T23:01:25.377 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_memory_base": "805306368", 2024-05-11T23:01:25.377 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-05-11T23:01:25.377 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-05-11T23:01:25.377 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-05-11T23:01:25.377 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-05-11T23:01:25.377 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_memory_target": "4294967296", 2024-05-11T23:01:25.377 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-05-11T23:01:25.377 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-05-11T23:01:25.377 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-05-11T23:01:25.377 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirroring_delete_delay": "0", 2024-05-11T23:01:25.378 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-05-11T23:01:25.378 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirroring_replay_delay": "0", 2024-05-11T23:01:25.378 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-05-11T23:01:25.378 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-05-11T23:01:25.378 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_move_to_trash_on_remove": "false", 2024-05-11T23:01:25.378 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-05-11T23:01:25.378 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_mtime_update_interval": "60", 2024-05-11T23:01:25.378 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_non_blocking_aio": "true", 2024-05-11T23:01:25.378 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_op_thread_timeout": "60", 2024-05-11T23:01:25.378 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_op_threads": "1", 2024-05-11T23:01:25.378 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_parent_cache_enabled": "false", 2024-05-11T23:01:25.378 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_persistent_cache_mode": "disabled", 2024-05-11T23:01:25.378 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_persistent_cache_path": "/tmp", 2024-05-11T23:01:25.379 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_persistent_cache_size": "1073741824", 2024-05-11T23:01:25.379 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_plugins": "", 2024-05-11T23:01:25.379 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_bps_burst": "0", 2024-05-11T23:01:25.379 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-05-11T23:01:25.379 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_bps_limit": "0", 2024-05-11T23:01:25.379 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_exclude_ops": "0", 2024-05-11T23:01:25.379 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_iops_burst": "0", 2024-05-11T23:01:25.379 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-05-11T23:01:25.379 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_iops_limit": "0", 2024-05-11T23:01:25.379 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_read_bps_burst": "0", 2024-05-11T23:01:25.379 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-05-11T23:01:25.379 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_read_bps_limit": "0", 2024-05-11T23:01:25.379 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_read_iops_burst": "0", 2024-05-11T23:01:25.379 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-05-11T23:01:25.380 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_read_iops_limit": "0", 2024-05-11T23:01:25.380 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_schedule_tick_min": "50", 2024-05-11T23:01:25.380 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_write_bps_burst": "0", 2024-05-11T23:01:25.380 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-05-11T23:01:25.380 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_write_bps_limit": "0", 2024-05-11T23:01:25.380 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_write_iops_burst": "0", 2024-05-11T23:01:25.380 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-05-11T23:01:25.380 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_qos_write_iops_limit": "0", 2024-05-11T23:01:25.380 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_quiesce_notification_attempts": "10", 2024-05-11T23:01:25.380 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_read_from_replica_policy": "default", 2024-05-11T23:01:25.380 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-05-11T23:01:25.380 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_readahead_max_bytes": "524288", 2024-05-11T23:01:25.380 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_readahead_trigger_requests": "10", 2024-05-11T23:01:25.380 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_request_timed_out_seconds": "30", 2024-05-11T23:01:25.380 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_skip_partial_discard": "true", 2024-05-11T23:01:25.380 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-05-11T23:01:25.381 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_tracing": "false", 2024-05-11T23:01:25.381 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_validate_names": "true", 2024-05-11T23:01:25.381 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_validate_pool": "true", 2024-05-11T23:01:25.381 INFO:teuthology.orchestra.run.smithi005.stdout: "restapi_base_url": "", 2024-05-11T23:01:25.381 INFO:teuthology.orchestra.run.smithi005.stdout: "restapi_log_level": "", 2024-05-11T23:01:25.381 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-05-11T23:01:25.381 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_account_default_quota_max_size": "-1", 2024-05-11T23:01:25.381 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_acl_grants_max_num": "100", 2024-05-11T23:01:25.381 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_admin_entry": "admin", 2024-05-11T23:01:25.381 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-05-11T23:01:25.381 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_backend_store": "rados", 2024-05-11T23:01:25.381 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_barbican_url": "", 2024-05-11T23:01:25.381 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_beast_enable_async": "true", 2024-05-11T23:01:25.381 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_bucket_counters_cache": "false", 2024-05-11T23:01:25.381 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-05-11T23:01:25.382 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-05-11T23:01:25.382 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-05-11T23:01:25.382 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_bucket_index_max_aio": "128", 2024-05-11T23:01:25.382 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-05-11T23:01:25.382 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-05-11T23:01:25.382 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_bucket_quota_ttl": "600", 2024-05-11T23:01:25.382 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-05-11T23:01:25.382 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_cache_enabled": "true", 2024-05-11T23:01:25.382 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_cache_expiry_interval": "900", 2024-05-11T23:01:25.382 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_cache_lru_size": "25000", 2024-05-11T23:01:25.382 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_config_store": "rados", 2024-05-11T23:01:25.382 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_content_length_compat": "false", 2024-05-11T23:01:25.382 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_copy_obj_progress": "true", 2024-05-11T23:01:25.383 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-05-11T23:01:25.383 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_cors_rules_max_num": "100", 2024-05-11T23:01:25.383 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_cross_domain_policy": "", 2024-05-11T23:01:25.383 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_default_encryption_key": "", 2024-05-11T23:01:25.383 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_kmip_addr": "", 2024-05-11T23:01:25.383 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_kmip_ca_path": "", 2024-05-11T23:01:25.383 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_kmip_client_cert": "", 2024-05-11T23:01:25.383 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_kmip_client_key": "", 2024-05-11T23:01:25.383 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-05-11T23:01:25.383 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_kmip_password": "", 2024-05-11T23:01:25.383 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-05-11T23:01:25.383 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_kmip_username": "", 2024-05-11T23:01:25.383 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_require_ssl": "true", 2024-05-11T23:01:25.384 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-05-11T23:01:25.384 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-05-11T23:01:25.384 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-05-11T23:01:25.384 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-05-11T23:01:25.384 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-05-11T23:01:25.384 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-05-11T23:01:25.384 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-05-11T23:01:25.384 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-05-11T23:01:25.384 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-05-11T23:01:25.384 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-05-11T23:01:25.384 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-05-11T23:01:25.384 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-05-11T23:01:25.385 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-05-11T23:01:25.385 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-05-11T23:01:25.385 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_suppress_logs": "true", 2024-05-11T23:01:25.385 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_vault_addr": "", 2024-05-11T23:01:25.385 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_vault_auth": "token", 2024-05-11T23:01:25.385 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_vault_namespace": "", 2024-05-11T23:01:25.385 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_vault_prefix": "", 2024-05-11T23:01:25.385 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-05-11T23:01:25.385 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-05-11T23:01:25.385 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-05-11T23:01:25.385 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-05-11T23:01:25.385 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_vault_token_file": "", 2024-05-11T23:01:25.385 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-05-11T23:01:25.386 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_curl_buffersize": "524288", 2024-05-11T23:01:25.386 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_curl_low_speed_limit": "1024", 2024-05-11T23:01:25.386 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_curl_low_speed_time": "300", 2024-05-11T23:01:25.386 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_curl_tcp_keepalive": "0", 2024-05-11T23:01:25.386 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-05-11T23:01:25.386 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-05-11T23:01:25.386 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-05-11T23:01:25.386 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-05-11T23:01:25.386 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-05-11T23:01:25.386 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d3n_l1_fadvise": "4", 2024-05-11T23:01:25.386 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-05-11T23:01:25.386 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-05-11T23:01:25.387 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-05-11T23:01:25.387 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d4n_host": "127.0.0.1", 2024-05-11T23:01:25.387 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_d4n_port": "6379", 2024-05-11T23:01:25.387 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-05-11T23:01:25.387 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_data_log_changes_size": "1000", 2024-05-11T23:01:25.387 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_data_log_num_shards": "128", 2024-05-11T23:01:25.387 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-05-11T23:01:25.387 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_data_log_window": "30", 2024-05-11T23:01:25.387 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_data_notify_interval_msec": "0", 2024-05-11T23:01:25.387 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_data_sync_poll_interval": "20", 2024-05-11T23:01:25.387 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_data_sync_spawn_window": "20", 2024-05-11T23:01:25.387 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-05-11T23:01:25.387 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-05-11T23:01:25.387 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-05-11T23:01:25.387 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_default_data_log_backing": "fifo", 2024-05-11T23:01:25.387 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-05-11T23:01:25.388 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_default_region_info_oid": "default.region", 2024-05-11T23:01:25.388 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-05-11T23:01:25.388 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-05-11T23:01:25.388 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_defer_to_bucket_acls": "", 2024-05-11T23:01:25.388 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-05-11T23:01:25.388 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-05-11T23:01:25.388 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-05-11T23:01:25.388 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-05-11T23:01:25.388 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-05-11T23:01:25.388 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-05-11T23:01:25.388 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-05-11T23:01:25.388 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-05-11T23:01:25.388 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_data_res": "500.000000", 2024-05-11T23:01:25.388 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-05-11T23:01:25.388 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-05-11T23:01:25.388 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-05-11T23:01:25.388 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-05-11T23:01:25.389 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dns_name": "", 2024-05-11T23:01:25.389 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dns_s3website_name": "", 2024-05-11T23:01:25.389 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_dynamic_resharding": "true", 2024-05-11T23:01:25.389 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-05-11T23:01:25.389 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_enable_gc_threads": "true", 2024-05-11T23:01:25.389 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_enable_lc_threads": "true", 2024-05-11T23:01:25.389 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_enable_ops_log": "false", 2024-05-11T23:01:25.389 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_enable_quota_threads": "true", 2024-05-11T23:01:25.389 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_enable_static_website": "false", 2024-05-11T23:01:25.389 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_enable_usage_log": "false", 2024-05-11T23:01:25.389 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_enforce_swift_acls": "true", 2024-05-11T23:01:25.389 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_exit_timeout_secs": "120", 2024-05-11T23:01:25.389 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_expose_bucket": "false", 2024-05-11T23:01:25.389 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_extended_http_attrs": "", 2024-05-11T23:01:25.390 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_filter": "none", 2024-05-11T23:01:25.390 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-05-11T23:01:25.390 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_frontends": "beast port=7480", 2024-05-11T23:01:25.390 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_gc_max_concurrent_io": "10", 2024-05-11T23:01:25.390 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_gc_max_deferred": "50", 2024-05-11T23:01:25.390 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-05-11T23:01:25.390 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_gc_max_objs": "32", 2024-05-11T23:01:25.390 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_gc_max_queue_size": "134213632", 2024-05-11T23:01:25.390 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_gc_max_trim_chunk": "16", 2024-05-11T23:01:25.390 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_gc_obj_min_wait": "7200", 2024-05-11T23:01:25.390 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_gc_processor_max_time": "3600", 2024-05-11T23:01:25.390 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_gc_processor_period": "3600", 2024-05-11T23:01:25.390 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-05-11T23:01:25.391 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_get_obj_window_size": "16777216", 2024-05-11T23:01:25.391 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_healthcheck_disabling_path": "", 2024-05-11T23:01:25.391 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ignore_get_invalid_range": "false", 2024-05-11T23:01:25.391 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_init_timeout": "300", 2024-05-11T23:01:25.391 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-05-11T23:01:25.391 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-05-11T23:01:25.391 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_kafka_connection_idle": "300", 2024-05-11T23:01:25.391 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_kafka_message_timeout": "5000", 2024-05-11T23:01:25.391 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_kafka_sleep_timeout": "10", 2024-05-11T23:01:25.391 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-05-11T23:01:25.391 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-05-11T23:01:25.391 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-05-11T23:01:25.391 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_admin_domain": "", 2024-05-11T23:01:25.391 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_admin_password": "", 2024-05-11T23:01:25.391 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_admin_password_path": "", 2024-05-11T23:01:25.392 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_admin_project": "", 2024-05-11T23:01:25.392 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_admin_tenant": "", 2024-05-11T23:01:25.392 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_admin_token": "", 2024-05-11T23:01:25.392 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_admin_token_path": "", 2024-05-11T23:01:25.392 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_admin_user": "", 2024-05-11T23:01:25.392 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_api_version": "2", 2024-05-11T23:01:25.392 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_barbican_domain": "", 2024-05-11T23:01:25.392 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_barbican_password": "", 2024-05-11T23:01:25.392 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_barbican_project": "", 2024-05-11T23:01:25.392 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_barbican_tenant": "", 2024-05-11T23:01:25.392 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_barbican_user": "", 2024-05-11T23:01:25.392 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-05-11T23:01:25.392 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_implicit_tenants": "false", 2024-05-11T23:01:25.392 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-05-11T23:01:25.393 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_service_token_enabled": "false", 2024-05-11T23:01:25.393 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_token_cache_size": "10000", 2024-05-11T23:01:25.393 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_url": "", 2024-05-11T23:01:25.393 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_keystone_verify_ssl": "true", 2024-05-11T23:01:25.393 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_lc_debug_interval": "-1", 2024-05-11T23:01:25.393 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_lc_lock_max_time": "90", 2024-05-11T23:01:25.393 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_lc_max_objs": "32", 2024-05-11T23:01:25.394 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_lc_max_rules": "1000", 2024-05-11T23:01:25.394 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_lc_max_worker": "3", 2024-05-11T23:01:25.394 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_lc_max_wp_worker": "3", 2024-05-11T23:01:25.394 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_lc_thread_delay": "0", 2024-05-11T23:01:25.394 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-05-11T23:01:25.394 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ldap_dnattr": "uid", 2024-05-11T23:01:25.394 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-05-11T23:01:25.395 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ldap_searchfilter": "", 2024-05-11T23:01:25.395 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-05-11T23:01:25.395 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ldap_uri": "", 2024-05-11T23:01:25.395 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-05-11T23:01:25.395 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-05-11T23:01:25.395 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-05-11T23:01:25.395 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_log_http_headers": "", 2024-05-11T23:01:25.395 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_log_nonexistent_bucket": "false", 2024-05-11T23:01:25.395 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-05-11T23:01:25.395 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_log_object_name_utc": "false", 2024-05-11T23:01:25.396 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-05-11T23:01:25.396 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-05-11T23:01:25.396 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_attr_name_len": "0", 2024-05-11T23:01:25.396 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_attr_size": "0", 2024-05-11T23:01:25.396 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_attrs_num_in_req": "0", 2024-05-11T23:01:25.396 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_chunk_size": "4194304", 2024-05-11T23:01:25.396 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_concurrent_requests": "1024", 2024-05-11T23:01:25.396 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-05-11T23:01:25.396 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_dynamic_shards": "1999", 2024-05-11T23:01:25.396 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_listing_results": "1000", 2024-05-11T23:01:25.396 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_notify_retries": "10", 2024-05-11T23:01:25.396 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_objs_per_shard": "100000", 2024-05-11T23:01:25.396 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_put_param_size": "1048576", 2024-05-11T23:01:25.396 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_put_size": "5368709120", 2024-05-11T23:01:25.396 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_max_slo_entries": "1000", 2024-05-11T23:01:25.397 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_md_log_max_shards": "64", 2024-05-11T23:01:25.397 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_md_notify_interval_msec": "200", 2024-05-11T23:01:25.397 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_meta_sync_poll_interval": "20", 2024-05-11T23:01:25.397 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_meta_sync_spawn_window": "20", 2024-05-11T23:01:25.397 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-05-11T23:01:25.397 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_mp_lock_max_time": "600", 2024-05-11T23:01:25.397 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-05-11T23:01:25.397 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_multipart_min_part_size": "5242880", 2024-05-11T23:01:25.397 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-05-11T23:01:25.397 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-05-11T23:01:25.397 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_fhcache_size": "2017", 2024-05-11T23:01:25.397 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-05-11T23:01:25.397 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-05-11T23:01:25.397 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_lru_lanes": "5", 2024-05-11T23:01:25.397 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_max_gc": "300", 2024-05-11T23:01:25.398 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-05-11T23:01:25.398 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_run_gc_threads": "false", 2024-05-11T23:01:25.398 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_run_lc_threads": "false", 2024-05-11T23:01:25.398 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_run_quota_threads": "false", 2024-05-11T23:01:25.398 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_run_sync_thread": "false", 2024-05-11T23:01:25.398 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-05-11T23:01:25.398 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-05-11T23:01:25.398 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_num_async_rados_threads": "32", 2024-05-11T23:01:25.398 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_num_control_oids": "8", 2024-05-11T23:01:25.398 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_numa_node": "-1", 2024-05-11T23:01:25.398 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_obj_stripe_size": "4194304", 2024-05-11T23:01:25.398 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-05-11T23:01:25.398 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_objexp_chunk_size": "100", 2024-05-11T23:01:25.398 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_objexp_gc_interval": "600", 2024-05-11T23:01:25.398 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_objexp_hints_num_shards": "127", 2024-05-11T23:01:25.399 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-05-11T23:01:25.399 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-05-11T23:01:25.399 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_op_thread_timeout": "600", 2024-05-11T23:01:25.399 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_op_tracing": "false", 2024-05-11T23:01:25.399 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_opa_token": "", 2024-05-11T23:01:25.399 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_opa_url": "", 2024-05-11T23:01:25.399 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_opa_verify_ssl": "true", 2024-05-11T23:01:25.399 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-05-11T23:01:25.399 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-05-11T23:01:25.399 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ops_log_rados": "false", 2024-05-11T23:01:25.399 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_ops_log_socket_path": "", 2024-05-11T23:01:25.399 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-05-11T23:01:25.399 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-05-11T23:01:25.399 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-05-11T23:01:25.399 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_period_push_interval": "2.000000", 2024-05-11T23:01:25.401 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_period_push_interval_max": "30.000000", 2024-05-11T23:01:25.401 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_period_root_pool": ".rgw.root", 2024-05-11T23:01:25.401 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-05-11T23:01:25.401 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-05-11T23:01:25.401 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_posix_cache_lanes": "3", 2024-05-11T23:01:25.402 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-05-11T23:01:25.402 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_posix_cache_max_buckets": "100", 2024-05-11T23:01:25.402 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_posix_cache_partitions": "3", 2024-05-11T23:01:25.402 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-05-11T23:01:25.402 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_print_continue": "true", 2024-05-11T23:01:25.402 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_print_prohibited_content_length": "false", 2024-05-11T23:01:25.402 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-05-11T23:01:25.402 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-05-11T23:01:25.402 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-05-11T23:01:25.402 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-05-11T23:01:25.403 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_rados_tracing": "false", 2024-05-11T23:01:25.403 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_realm": "", 2024-05-11T23:01:25.403 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_realm_id": "", 2024-05-11T23:01:25.403 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-05-11T23:01:25.403 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_region": "", 2024-05-11T23:01:25.403 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_region_root_pool": ".rgw.root", 2024-05-11T23:01:25.403 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_relaxed_region_enforcement": "false", 2024-05-11T23:01:25.403 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-05-11T23:01:25.403 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-05-11T23:01:25.403 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_request_uri": "", 2024-05-11T23:01:25.403 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_reshard_batch_size": "64", 2024-05-11T23:01:25.403 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-05-11T23:01:25.403 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_reshard_max_aio": "128", 2024-05-11T23:01:25.403 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_reshard_num_logs": "16", 2024-05-11T23:01:25.404 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_reshard_thread_interval": "600", 2024-05-11T23:01:25.404 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_resolve_cname": "false", 2024-05-11T23:01:25.404 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_rest_getusage_op_compat": "false", 2024-05-11T23:01:25.404 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_run_sync_thread": "true", 2024-05-11T23:01:25.404 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-05-11T23:01:25.404 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-05-11T23:01:25.404 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_s3_auth_use_keystone": "false", 2024-05-11T23:01:25.404 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_s3_auth_use_ldap": "false", 2024-05-11T23:01:25.404 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_s3_auth_use_rados": "true", 2024-05-11T23:01:25.404 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_s3_auth_use_sts": "false", 2024-05-11T23:01:25.404 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-05-11T23:01:25.404 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_s3_success_create_obj_status": "0", 2024-05-11T23:01:25.404 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-05-11T23:01:25.404 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_scheduler_type": "throttler", 2024-05-11T23:01:25.404 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_script_uri": "", 2024-05-11T23:01:25.405 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_service_provider_name": "", 2024-05-11T23:01:25.405 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-05-11T23:01:25.405 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sts_client_id": "", 2024-05-11T23:01:25.405 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sts_client_secret": "", 2024-05-11T23:01:25.405 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sts_entry": "sts", 2024-05-11T23:01:25.405 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sts_key": "sts", 2024-05-11T23:01:25.405 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sts_max_session_duration": "43200", 2024-05-11T23:01:25.405 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sts_min_session_duration": "900", 2024-05-11T23:01:25.405 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sts_token_introspection_url": "", 2024-05-11T23:01:25.405 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_account_in_url": "false", 2024-05-11T23:01:25.405 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_auth_entry": "auth", 2024-05-11T23:01:25.405 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_auth_url": "", 2024-05-11T23:01:25.406 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_custom_header": "", 2024-05-11T23:01:25.406 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_enforce_content_length": "false", 2024-05-11T23:01:25.406 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_need_stats": "true", 2024-05-11T23:01:25.406 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_tenant_name": "", 2024-05-11T23:01:25.406 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_token_expiration": "86400", 2024-05-11T23:01:25.406 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_url": "", 2024-05-11T23:01:25.406 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_url_prefix": "swift", 2024-05-11T23:01:25.406 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_swift_versioning_enabled": "false", 2024-05-11T23:01:25.407 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-05-11T23:01:25.407 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-05-11T23:01:25.407 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_lease_period": "120", 2024-05-11T23:01:25.407 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-05-11T23:01:25.407 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_log_trim_interval": "1200", 2024-05-11T23:01:25.407 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-05-11T23:01:25.408 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-05-11T23:01:25.408 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-05-11T23:01:25.408 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_obj_etag_verify": "false", 2024-05-11T23:01:25.408 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_trace_history_size": "4096", 2024-05-11T23:01:25.408 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-05-11T23:01:25.408 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-05-11T23:01:25.408 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_thread_pool_size": "512", 2024-05-11T23:01:25.409 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_topic_persistency_max_retries": "0", 2024-05-11T23:01:25.409 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-05-11T23:01:25.409 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-05-11T23:01:25.409 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_topic_require_publish_policy": "false", 2024-05-11T23:01:25.409 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_torrent_comment": "", 2024-05-11T23:01:25.409 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_torrent_createby": "", 2024-05-11T23:01:25.409 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_torrent_encoding": "", 2024-05-11T23:01:25.409 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_torrent_flag": "false", 2024-05-11T23:01:25.409 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_torrent_max_size": "5368709120", 2024-05-11T23:01:25.410 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_torrent_origin": "", 2024-05-11T23:01:25.410 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_torrent_sha_unit": "524288", 2024-05-11T23:01:25.410 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_torrent_tracker": "", 2024-05-11T23:01:25.410 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_trust_forwarded_https": "false", 2024-05-11T23:01:25.410 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-05-11T23:01:25.410 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_usage_log_tick_interval": "30", 2024-05-11T23:01:25.410 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_usage_max_shards": "32", 2024-05-11T23:01:25.410 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_usage_max_user_shards": "1", 2024-05-11T23:01:25.410 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_use_opa_authz": "false", 2024-05-11T23:01:25.410 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_counters_cache": "false", 2024-05-11T23:01:25.410 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_counters_cache_size": "10000", 2024-05-11T23:01:25.411 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-05-11T23:01:25.411 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_default_quota_max_size": "-1", 2024-05-11T23:01:25.411 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_max_buckets": "1000", 2024-05-11T23:01:25.411 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_policies_max_num": "100", 2024-05-11T23:01:25.411 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-05-11T23:01:25.411 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-05-11T23:01:25.411 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_quota_sync_interval": "86400", 2024-05-11T23:01:25.411 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-05-11T23:01:25.411 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_user_unique_email": "true", 2024-05-11T23:01:25.411 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_verify_ssl": "true", 2024-05-11T23:01:25.411 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_website_routing_rules_max_num": "50", 2024-05-11T23:01:25.412 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_zone": "", 2024-05-11T23:01:25.412 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_zone_id": "", 2024-05-11T23:01:25.412 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-05-11T23:01:25.412 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_zonegroup": "", 2024-05-11T23:01:25.412 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_zonegroup_id": "", 2024-05-11T23:01:25.412 INFO:teuthology.orchestra.run.smithi005.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-05-11T23:01:25.412 INFO:teuthology.orchestra.run.smithi005.stdout: "rgwlc_auto_session_clear": "true", 2024-05-11T23:01:25.412 INFO:teuthology.orchestra.run.smithi005.stdout: "rgwlc_skip_bucket_step": "false", 2024-05-11T23:01:25.412 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_block_size": "4096", 2024-05-11T23:01:25.412 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-05-11T23:01:25.412 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-05-11T23:01:25.412 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-05-11T23:01:25.413 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-05-11T23:01:25.413 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_cache_shard_bits": "4", 2024-05-11T23:01:25.413 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_cache_size": "536870912", 2024-05-11T23:01:25.413 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_cache_type": "binned_lru", 2024-05-11T23:01:25.413 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-05-11T23:01:25.413 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-05-11T23:01:25.413 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-05-11T23:01:25.414 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_collect_compaction_stats": "false", 2024-05-11T23:01:25.414 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_collect_extended_stats": "false", 2024-05-11T23:01:25.414 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_collect_memory_stats": "false", 2024-05-11T23:01:25.414 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-05-11T23:01:25.414 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_index_type": "binary_search", 2024-05-11T23:01:25.414 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_log_to_ceph_log": "true", 2024-05-11T23:01:25.414 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_metadata_block_size": "4096", 2024-05-11T23:01:25.414 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_partition_filters": "false", 2024-05-11T23:01:25.414 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_perf": "false", 2024-05-11T23:01:25.414 INFO:teuthology.orchestra.run.smithi005.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-05-11T23:01:25.414 INFO:teuthology.orchestra.run.smithi005.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-05-11T23:01:25.414 INFO:teuthology.orchestra.run.smithi005.stdout: "rotating_keys_renewal_timeout": "10", 2024-05-11T23:01:25.414 INFO:teuthology.orchestra.run.smithi005.stdout: "run_dir": "/var/run/ceph", 2024-05-11T23:01:25.414 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_block_create": "true", 2024-05-11T23:01:25.415 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_cache_lru_size": "67108864", 2024-05-11T23:01:25.415 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_cbjournal_size": "5368709120", 2024-05-11T23:01:25.415 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_data_delta_based_overwrite": "0", 2024-05-11T23:01:25.415 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_default_max_object_size": "16777216", 2024-05-11T23:01:25.415 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-05-11T23:01:25.415 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_device_size": "53687091200", 2024-05-11T23:01:25.415 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_journal_batch_capacity": "16", 2024-05-11T23:01:25.415 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-05-11T23:01:25.415 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-05-11T23:01:25.415 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_journal_iodepth_limit": "5", 2024-05-11T23:01:25.415 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_main_device_type": "SSD", 2024-05-11T23:01:25.415 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_max_concurrent_transactions": "8", 2024-05-11T23:01:25.415 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-05-11T23:01:25.415 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-05-11T23:01:25.415 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-05-11T23:01:25.416 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-05-11T23:01:25.416 INFO:teuthology.orchestra.run.smithi005.stdout: "seastore_segment_size": "67108864", 2024-05-11T23:01:25.416 INFO:teuthology.orchestra.run.smithi005.stdout: "set_keepcaps": "false", 2024-05-11T23:01:25.416 INFO:teuthology.orchestra.run.smithi005.stdout: "setgroup": "", 2024-05-11T23:01:25.416 INFO:teuthology.orchestra.run.smithi005.stdout: "setuser": "", 2024-05-11T23:01:25.416 INFO:teuthology.orchestra.run.smithi005.stdout: "setuser_match_path": "", 2024-05-11T23:01:25.416 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_misplaced_ratio": "0.050000", 2024-05-11T23:01:25.416 INFO:teuthology.orchestra.run.smithi005.stdout: "thp": "false", 2024-05-11T23:01:25.416 INFO:teuthology.orchestra.run.smithi005.stdout: "threadpool_default_timeout": "60", 2024-05-11T23:01:25.416 INFO:teuthology.orchestra.run.smithi005.stdout: "threadpool_empty_queue_max_wait": "2", 2024-05-11T23:01:25.416 INFO:teuthology.orchestra.run.smithi005.stdout: "throttler_perf_counter": "true" 2024-05-11T23:01:25.416 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:25.418 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-05-11T23:01:25.539 INFO:teuthology.orchestra.run.smithi005.stdout:mon.a: {} 2024-05-11T23:01:25.539 INFO:teuthology.orchestra.run.smithi005.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-11T23:01:25.549 INFO:teuthology.orchestra.run.smithi005.stdout:mon.b: {} 2024-05-11T23:01:25.549 INFO:teuthology.orchestra.run.smithi005.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-11T23:01:25.558 INFO:teuthology.orchestra.run.smithi005.stdout:mon.c: {} 2024-05-11T23:01:25.558 INFO:teuthology.orchestra.run.smithi005.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-11T23:01:25.569 DEBUG:teuthology.orchestra.run.smithi115:> type ceph-objectstore-tool 2024-05-11T23:01:25.588 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-05-11T23:01:25.588 DEBUG:teuthology.orchestra.run.smithi005:> type ceph-objectstore-tool 2024-05-11T23:01:25.603 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-05-11T23:01:25.603 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-05-11T23:01:25.615 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-05-11T23:01:25.616 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-05-11T23:01:25.616 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:01:25.616 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:01:25.616 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-05-11T23:01:25.616 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-05-11T23:01:25.616 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-05-11T23:01:25.617 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-05-11T23:01:25.617 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:01:25.617 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-05-11T23:01:25.617 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:01:25.617 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-05-11T23:01:25.618 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-11T23:01:25.618 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-05-11T23:01:25.629 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:25.629+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:25.681 INFO:teuthology.orchestra.run.smithi005.stdout:mapped writes DISABLED 2024-05-11T23:01:25.682 INFO:teuthology.orchestra.run.smithi005.stdout:mapped reads DISABLED 2024-05-11T23:01:25.682 INFO:teuthology.orchestra.run.smithi005.stdout:Seed set to 4014 2024-05-11T23:01:25.718 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:25.718+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:25.771 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:25.771 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:25.771 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:01:25.771 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:25.781 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:01:25.798 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:01:25.798 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:01:25.819 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:25.819+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:25.826 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-05-11T23:01:25.829 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:01:25.830 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:01:25.849 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:01:25.849 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:01:25.866 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:01:25.866 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:01:25.885 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:01:25.885 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:01:25.895 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:25.895 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:25.903 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:01:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:01:25.905 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:01:25.921 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:01:25.922 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:01:25.922 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:25.922+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:25.941 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:01:25.941 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:01:25.978 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-05-11T23:01:26.021 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:26.021+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:26.036 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:26.036 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:26.046 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:01:26.130 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:26.130+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:26.161 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-05-11T23:01:26.199 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:26.199 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:26.208 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:01:26.249 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:26.249+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:26.324 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:26.323+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:26.341 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-05-11T23:01:26.362 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:26.362 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:26.371 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:01:26.450 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-05-11T23:01:26.452 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:26.451+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:26.493 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-05-11T23:01:26.526 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:26.526+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:26.529 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:26.529 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:26.539 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:01:26.627 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:26.627+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:26.675 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-05-11T23:01:26.695 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:26.696 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:26.705 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:01:26.728 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:26.728+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:26.818 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-05-11T23:01:26.829 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:26.829+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:26.857 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:26.857 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:26.866 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:01:26.930 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:26.929+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:27.020 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:27.020 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:27.029 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:01:27.033 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:27.031+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:27.132 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:27.132+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:27.139 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:27.139 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:27.149 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:01:27.233 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:27.232+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:27.299 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:27.299 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:27.308 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:01:27.333 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:27.333+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:27.418 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:27.418 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:27.427 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:01:27.472 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:27.472+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:27.535 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:27.535+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:27.552 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:27.552 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:27.561 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:01:27.645 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:27.645+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:27.707 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:27.707 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:27.715 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:01:27.757 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:27.756+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:27.838 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:27.838+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:27.861 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:27.861 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:27.869 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:01:27.939 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:27.938+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:27.955 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:01:27.975 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:27.975 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:27.984 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:01:28.040 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:28.040+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:28.091 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:01:28.092 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:01:28.109 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:01:28.110 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:01:28.126 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:01:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:01:28.129 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:28.129 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:28.138 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:01:28.141 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:28.140+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:28.143 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:01:28.143 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:01:28.161 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:01:28.161 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:01:28.178 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:01:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:01:28.197 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:01:28.197 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:01:28.215 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:01:28.215 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:01:28.241 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:28.241+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:28.247 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:28.247 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:01:28.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:01:28.247 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [] 2024-05-11T23:01:28.247 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:28.256 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:01:28.342 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:28.342+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:28.402 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:28.403 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:28.403 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:01:28.403 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:28.411 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:01:28.453 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:28.453+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:28.470 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-11T23:01:28.545 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:28.545+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:28.557 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:28.557 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:28.557 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:01:28.557 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:01:28.557 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:28.566 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:01:28.645 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:28.645+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:28.671 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:28.671 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:01:28.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:01:28.671 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [] 2024-05-11T23:01:28.671 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:28.679 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:01:28.763 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:28.763+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:28.826 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:28.826 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:28.826 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:01:28.826 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:28.834 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:01:28.875 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:28.875+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:28.948 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:28.948+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:28.980 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:28.981 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:28.981 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:01:28.981 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:01:28.981 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:28.989 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:01:29.049 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:29.049+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:29.097 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:29.097 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:01:29.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:01:29.098 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:01:29.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.098 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:01:29.098 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:01:29.098 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.932628481, 2024-05-11T23:01:29.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:01:29.098 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:29.098 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:29.098 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.098 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:29.098 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:29.098 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:29.099 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:01:29.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.099 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:29.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:29.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:01:29.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:29.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:01:29.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:29.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:01:29.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:01:29.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:29.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:01:29.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:01:29.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:29.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:01:29.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:01:29.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:01:29.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:01:29.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:01:29.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:01:29.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:01:29.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:01:29.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:01:29.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:01:29.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:01:29.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:29.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:01:29.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:01:29.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:29.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:01:29.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:01:29.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:29.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:01:29.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:01:29.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:29.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:01:29.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:01:29.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:29.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:01:29.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:01:29.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:29.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:01:29.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:01:29.104 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.104 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.104 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.104 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:29.104 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.384920+0000", 2024-05-11T23:01:29.104 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.710960396000001, 2024-05-11T23:01:29.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0064708179999999997, 2024-05-11T23:01:29.104 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:29.104 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:29.104 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.105 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:29.105 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:29.105 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:29.105 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 107 2024-05-11T23:01:29.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.105 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:29.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:29.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.384920+0000", 2024-05-11T23:01:29.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:29.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.384920+0000", 2024-05-11T23:01:29.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:29.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.384921+0000", 2024-05-11T23:01:29.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2399999999999998e-07 2024-05-11T23:01:29.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:29.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.384925+0000", 2024-05-11T23:01:29.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1339999999999997e-06 2024-05-11T23:01:29.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:29.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.384931+0000", 2024-05-11T23:01:29.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4679999999999997e-06 2024-05-11T23:01:29.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.384937+0000", 2024-05-11T23:01:29.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7069999999999997e-06 2024-05-11T23:01:29.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.386080+0000", 2024-05-11T23:01:29.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011428639999999999 2024-05-11T23:01:29.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:29.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.386125+0000", 2024-05-11T23:01:29.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5426000000000002e-05 2024-05-11T23:01:29.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:29.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.386197+0000", 2024-05-11T23:01:29.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1532000000000006e-05 2024-05-11T23:01:29.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:29.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.387442+0000", 2024-05-11T23:01:29.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001245187 2024-05-11T23:01:29.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:29.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391342+0000", 2024-05-11T23:01:29.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039000860000000001 2024-05-11T23:01:29.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:29.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391370+0000", 2024-05-11T23:01:29.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7708e-05 2024-05-11T23:01:29.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:29.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391391+0000", 2024-05-11T23:01:29.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1081999999999999e-05 2024-05-11T23:01:29.109 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.109 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.109 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.109 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:29.109 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.385087+0000", 2024-05-11T23:01:29.109 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.710793150000001, 2024-05-11T23:01:29.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497600000000003, 2024-05-11T23:01:29.109 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:29.109 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:29.110 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.110 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:29.110 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:29.110 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:29.110 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 108 2024-05-11T23:01:29.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.110 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:29.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:29.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385087+0000", 2024-05-11T23:01:29.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:29.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385087+0000", 2024-05-11T23:01:29.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:29.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385088+0000", 2024-05-11T23:01:29.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7899999999999998e-07 2024-05-11T23:01:29.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:29.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385093+0000", 2024-05-11T23:01:29.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5149999999999997e-06 2024-05-11T23:01:29.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:29.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385102+0000", 2024-05-11T23:01:29.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3969999999999994e-06 2024-05-11T23:01:29.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385107+0000", 2024-05-11T23:01:29.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7049999999999996e-06 2024-05-11T23:01:29.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.386616+0000", 2024-05-11T23:01:29.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001508038 2024-05-11T23:01:29.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:29.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.386661+0000", 2024-05-11T23:01:29.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5043999999999998e-05 2024-05-11T23:01:29.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:29.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.386731+0000", 2024-05-11T23:01:29.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0479000000000001e-05 2024-05-11T23:01:29.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:29.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.388118+0000", 2024-05-11T23:01:29.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013864979999999999 2024-05-11T23:01:29.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:29.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391397+0000", 2024-05-11T23:01:29.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032796510000000002 2024-05-11T23:01:29.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:29.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391420+0000", 2024-05-11T23:01:29.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2793999999999999e-05 2024-05-11T23:01:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391437+0000", 2024-05-11T23:01:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.696e-05 2024-05-11T23:01:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.385150+0000", 2024-05-11T23:01:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.710730801, 2024-05-11T23:01:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063253179999999999, 2024-05-11T23:01:29.114 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 109 2024-05-11T23:01:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385150+0000", 2024-05-11T23:01:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385150+0000", 2024-05-11T23:01:29.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385150+0000", 2024-05-11T23:01:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7599999999999997e-07 2024-05-11T23:01:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385239+0000", 2024-05-11T23:01:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8583000000000002e-05 2024-05-11T23:01:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385247+0000", 2024-05-11T23:01:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999993e-06 2024-05-11T23:01:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385255+0000", 2024-05-11T23:01:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6890000000000004e-06 2024-05-11T23:01:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.386954+0000", 2024-05-11T23:01:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016993819999999999 2024-05-11T23:01:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.386999+0000", 2024-05-11T23:01:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4625e-05 2024-05-11T23:01:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:29.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.387069+0000", 2024-05-11T23:01:29.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0051999999999999e-05 2024-05-11T23:01:29.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:29.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.388593+0000", 2024-05-11T23:01:29.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001524192 2024-05-11T23:01:29.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:29.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391440+0000", 2024-05-11T23:01:29.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028466009999999998 2024-05-11T23:01:29.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:29.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391460+0000", 2024-05-11T23:01:29.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0032999999999999e-05 2024-05-11T23:01:29.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:29.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391475+0000", 2024-05-11T23:01:29.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5014999999999999e-05 2024-05-11T23:01:29.119 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.119 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.119 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.119 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:29.119 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.385267+0000", 2024-05-11T23:01:29.120 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.710613534, 2024-05-11T23:01:29.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062465000000000003, 2024-05-11T23:01:29.120 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:29.120 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:29.120 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.120 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:29.120 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:29.120 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:29.120 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 110 2024-05-11T23:01:29.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.120 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:29.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:29.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385267+0000", 2024-05-11T23:01:29.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:29.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385267+0000", 2024-05-11T23:01:29.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:29.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385267+0000", 2024-05-11T23:01:29.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6199999999999997e-07 2024-05-11T23:01:29.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:29.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385272+0000", 2024-05-11T23:01:29.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1189999999999999e-06 2024-05-11T23:01:29.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:29.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385278+0000", 2024-05-11T23:01:29.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:01:29.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385283+0000", 2024-05-11T23:01:29.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.887e-06 2024-05-11T23:01:29.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.387468+0000", 2024-05-11T23:01:29.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00218535 2024-05-11T23:01:29.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:29.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.387514+0000", 2024-05-11T23:01:29.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5318999999999998e-05 2024-05-11T23:01:29.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:29.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.387584+0000", 2024-05-11T23:01:29.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0501999999999997e-05 2024-05-11T23:01:29.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:29.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.388637+0000", 2024-05-11T23:01:29.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00105237 2024-05-11T23:01:29.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:29.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391478+0000", 2024-05-11T23:01:29.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028413259999999999 2024-05-11T23:01:29.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:29.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391497+0000", 2024-05-11T23:01:29.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9225999999999999e-05 2024-05-11T23:01:29.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:29.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391513+0000", 2024-05-11T23:01:29.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6229e-05 2024-05-11T23:01:29.125 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.125 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.125 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.125 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:29.125 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.385291+0000", 2024-05-11T23:01:29.125 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.710589165, 2024-05-11T23:01:29.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062594599999999997, 2024-05-11T23:01:29.125 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:29.125 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:29.125 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:29.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:29.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:29.125 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 111 2024-05-11T23:01:29.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.126 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:29.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:29.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385291+0000", 2024-05-11T23:01:29.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:29.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385291+0000", 2024-05-11T23:01:29.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:29.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385292+0000", 2024-05-11T23:01:29.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4299999999999998e-07 2024-05-11T23:01:29.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:29.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385295+0000", 2024-05-11T23:01:29.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5999999999999998e-06 2024-05-11T23:01:29.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:29.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385302+0000", 2024-05-11T23:01:29.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9630000000000001e-06 2024-05-11T23:01:29.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385307+0000", 2024-05-11T23:01:29.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0900000000000004e-06 2024-05-11T23:01:29.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.387785+0000", 2024-05-11T23:01:29.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024774340000000001 2024-05-11T23:01:29.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:29.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.387829+0000", 2024-05-11T23:01:29.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3998000000000001e-05 2024-05-11T23:01:29.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:29.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.387899+0000", 2024-05-11T23:01:29.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0549000000000005e-05 2024-05-11T23:01:29.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:29.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.389405+0000", 2024-05-11T23:01:29.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015058490000000001 2024-05-11T23:01:29.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:29.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391516+0000", 2024-05-11T23:01:29.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021109330000000002 2024-05-11T23:01:29.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:29.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391536+0000", 2024-05-11T23:01:29.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.969e-05 2024-05-11T23:01:29.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:29.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391551+0000", 2024-05-11T23:01:29.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4911e-05 2024-05-11T23:01:29.130 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.130 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.130 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.130 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:29.130 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.385389+0000", 2024-05-11T23:01:29.130 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.710490906, 2024-05-11T23:01:29.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063276280000000001, 2024-05-11T23:01:29.131 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:29.131 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:29.131 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:29.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:29.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:29.131 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 112 2024-05-11T23:01:29.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.131 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:29.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:29.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385389+0000", 2024-05-11T23:01:29.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:29.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385389+0000", 2024-05-11T23:01:29.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:29.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385390+0000", 2024-05-11T23:01:29.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3799999999999997e-07 2024-05-11T23:01:29.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:29.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385487+0000", 2024-05-11T23:01:29.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7330999999999994e-05 2024-05-11T23:01:29.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:29.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385496+0000", 2024-05-11T23:01:29.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1899999999999995e-06 2024-05-11T23:01:29.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385501+0000", 2024-05-11T23:01:29.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.434e-06 2024-05-11T23:01:29.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.388145+0000", 2024-05-11T23:01:29.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026436789999999999 2024-05-11T23:01:29.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:29.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.388188+0000", 2024-05-11T23:01:29.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3711999999999998e-05 2024-05-11T23:01:29.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:29.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.388264+0000", 2024-05-11T23:01:29.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5405000000000001e-05 2024-05-11T23:01:29.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:29.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.389441+0000", 2024-05-11T23:01:29.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011768569999999999 2024-05-11T23:01:29.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:29.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391676+0000", 2024-05-11T23:01:29.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0022357570000000001 2024-05-11T23:01:29.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:29.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391700+0000", 2024-05-11T23:01:29.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3540000000000002e-05 2024-05-11T23:01:29.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:29.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391717+0000", 2024-05-11T23:01:29.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7085e-05 2024-05-11T23:01:29.135 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.135 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.136 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.136 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:29.136 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.385513+0000", 2024-05-11T23:01:29.136 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.710367722000001, 2024-05-11T23:01:29.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006245273, 2024-05-11T23:01:29.136 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:29.136 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:29.136 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.136 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:29.136 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:29.136 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:29.136 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113 2024-05-11T23:01:29.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.137 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:29.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:29.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385513+0000", 2024-05-11T23:01:29.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:29.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385513+0000", 2024-05-11T23:01:29.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:29.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385513+0000", 2024-05-11T23:01:29.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-05-11T23:01:29.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:29.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385519+0000", 2024-05-11T23:01:29.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.93e-06 2024-05-11T23:01:29.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:29.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385529+0000", 2024-05-11T23:01:29.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9599999999999995e-06 2024-05-11T23:01:29.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385535+0000", 2024-05-11T23:01:29.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0120000000000002e-06 2024-05-11T23:01:29.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.388657+0000", 2024-05-11T23:01:29.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003121651 2024-05-11T23:01:29.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:29.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.388708+0000", 2024-05-11T23:01:29.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1598999999999998e-05 2024-05-11T23:01:29.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:29.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.388800+0000", 2024-05-11T23:01:29.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1688999999999996e-05 2024-05-11T23:01:29.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:29.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.390238+0000", 2024-05-11T23:01:29.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001438384 2024-05-11T23:01:29.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:29.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391720+0000", 2024-05-11T23:01:29.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00148175 2024-05-11T23:01:29.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:29.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391742+0000", 2024-05-11T23:01:29.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1676999999999999e-05 2024-05-11T23:01:29.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:29.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391758+0000", 2024-05-11T23:01:29.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5998000000000002e-05 2024-05-11T23:01:29.141 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.141 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.141 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.141 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:29.141 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.385548+0000", 2024-05-11T23:01:29.141 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.710332320999999, 2024-05-11T23:01:29.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062520400000000004, 2024-05-11T23:01:29.141 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:29.141 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:29.142 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.142 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:29.142 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:29.142 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:29.142 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 114 2024-05-11T23:01:29.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.142 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:29.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:29.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385548+0000", 2024-05-11T23:01:29.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:29.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385548+0000", 2024-05-11T23:01:29.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:29.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385548+0000", 2024-05-11T23:01:29.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5999999999999999e-07 2024-05-11T23:01:29.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:29.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385553+0000", 2024-05-11T23:01:29.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0640000000000004e-06 2024-05-11T23:01:29.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:29.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385559+0000", 2024-05-11T23:01:29.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7039999999999997e-06 2024-05-11T23:01:29.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385564+0000", 2024-05-11T23:01:29.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0640000000000001e-06 2024-05-11T23:01:29.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.389003+0000", 2024-05-11T23:01:29.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0034383590000000002 2024-05-11T23:01:29.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:29.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.389051+0000", 2024-05-11T23:01:29.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8167000000000001e-05 2024-05-11T23:01:29.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:29.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.389125+0000", 2024-05-11T23:01:29.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4270000000000006e-05 2024-05-11T23:01:29.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:29.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.390278+0000", 2024-05-11T23:01:29.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011527740000000001 2024-05-11T23:01:29.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:29.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391761+0000", 2024-05-11T23:01:29.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0014835759999999999 2024-05-11T23:01:29.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:29.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391783+0000", 2024-05-11T23:01:29.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1203999999999999e-05 2024-05-11T23:01:29.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:29.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391800+0000", 2024-05-11T23:01:29.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7397999999999998e-05 2024-05-11T23:01:29.146 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.146 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.146 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.147 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:29.147 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.385598+0000", 2024-05-11T23:01:29.147 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.710282563, 2024-05-11T23:01:29.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062412780000000003, 2024-05-11T23:01:29.147 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:29.147 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:29.147 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.147 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:29.147 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:29.147 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:29.147 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 115 2024-05-11T23:01:29.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.147 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:29.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:29.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385598+0000", 2024-05-11T23:01:29.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:29.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385598+0000", 2024-05-11T23:01:29.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:29.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385598+0000", 2024-05-11T23:01:29.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.82e-07 2024-05-11T23:01:29.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:29.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385702+0000", 2024-05-11T23:01:29.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103829 2024-05-11T23:01:29.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:29.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385711+0000", 2024-05-11T23:01:29.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2690000000000005e-06 2024-05-11T23:01:29.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385717+0000", 2024-05-11T23:01:29.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6169999999999998e-06 2024-05-11T23:01:29.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.389467+0000", 2024-05-11T23:01:29.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037499220000000002 2024-05-11T23:01:29.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:29.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.389510+0000", 2024-05-11T23:01:29.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2858999999999997e-05 2024-05-11T23:01:29.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:29.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.389583+0000", 2024-05-11T23:01:29.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3164999999999995e-05 2024-05-11T23:01:29.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:29.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.390591+0000", 2024-05-11T23:01:29.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010080930000000001 2024-05-11T23:01:29.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:29.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391803+0000", 2024-05-11T23:01:29.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001211869 2024-05-11T23:01:29.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:29.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391824+0000", 2024-05-11T23:01:29.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0817000000000001e-05 2024-05-11T23:01:29.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:29.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391839+0000", 2024-05-11T23:01:29.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5356e-05 2024-05-11T23:01:29.152 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.152 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.152 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.152 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:29.152 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.385730+0000", 2024-05-11T23:01:29.152 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.710150745, 2024-05-11T23:01:29.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061485799999999998, 2024-05-11T23:01:29.152 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:29.152 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:29.152 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.153 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:29.153 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:29.153 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:29.153 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116 2024-05-11T23:01:29.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.153 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:29.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:29.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385730+0000", 2024-05-11T23:01:29.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:29.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385730+0000", 2024-05-11T23:01:29.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:29.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385730+0000", 2024-05-11T23:01:29.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.44e-07 2024-05-11T23:01:29.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:29.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385734+0000", 2024-05-11T23:01:29.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1969999999999998e-06 2024-05-11T23:01:29.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:29.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385741+0000", 2024-05-11T23:01:29.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0319999999999998e-06 2024-05-11T23:01:29.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385747+0000", 2024-05-11T23:01:29.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.079e-06 2024-05-11T23:01:29.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.389914+0000", 2024-05-11T23:01:29.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041674980000000004 2024-05-11T23:01:29.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:29.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.389958+0000", 2024-05-11T23:01:29.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4372000000000001e-05 2024-05-11T23:01:29.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:29.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.390028+0000", 2024-05-11T23:01:29.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9573000000000006e-05 2024-05-11T23:01:29.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:29.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.390855+0000", 2024-05-11T23:01:29.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00082668599999999998 2024-05-11T23:01:29.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:29.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391842+0000", 2024-05-11T23:01:29.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00098736700000000006 2024-05-11T23:01:29.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:29.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391863+0000", 2024-05-11T23:01:29.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1109000000000001e-05 2024-05-11T23:01:29.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:29.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391878+0000", 2024-05-11T23:01:29.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5023e-05 2024-05-11T23:01:29.157 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.157 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.157 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.157 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:29.157 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.385755+0000", 2024-05-11T23:01:29.157 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.71012507, 2024-05-11T23:01:29.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061623590000000001, 2024-05-11T23:01:29.158 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:29.158 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:29.158 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:29.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:29.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:29.158 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 117 2024-05-11T23:01:29.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.158 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:29.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:29.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385755+0000", 2024-05-11T23:01:29.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:29.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385755+0000", 2024-05-11T23:01:29.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:29.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385756+0000", 2024-05-11T23:01:29.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2300000000000001e-07 2024-05-11T23:01:29.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:29.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385760+0000", 2024-05-11T23:01:29.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9169999999999999e-06 2024-05-11T23:01:29.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:29.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385766+0000", 2024-05-11T23:01:29.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0440000000000001e-06 2024-05-11T23:01:29.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.385770+0000", 2024-05-11T23:01:29.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6630000000000001e-06 2024-05-11T23:01:29.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.390296+0000", 2024-05-11T23:01:29.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045260609999999996 2024-05-11T23:01:29.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:29.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.390343+0000", 2024-05-11T23:01:29.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6700999999999999e-05 2024-05-11T23:01:29.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:29.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.390416+0000", 2024-05-11T23:01:29.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3281000000000005e-05 2024-05-11T23:01:29.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:29.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391096+0000", 2024-05-11T23:01:29.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00067949500000000001 2024-05-11T23:01:29.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:29.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391881+0000", 2024-05-11T23:01:29.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00078490899999999998 2024-05-11T23:01:29.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:29.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391902+0000", 2024-05-11T23:01:29.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1114000000000001e-05 2024-05-11T23:01:29.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:29.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.391918+0000", 2024-05-11T23:01:29.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5651e-05 2024-05-11T23:01:29.162 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.162 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.162 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.163 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:29.163 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.796498+0000", 2024-05-11T23:01:29.163 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.299382681999999, 2024-05-11T23:01:29.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0074976929999999997, 2024-05-11T23:01:29.163 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:29.163 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:29.163 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.163 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:29.163 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:29.163 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:29.163 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 134 2024-05-11T23:01:29.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.163 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:29.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:29.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796498+0000", 2024-05-11T23:01:29.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:29.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796498+0000", 2024-05-11T23:01:29.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:29.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796498+0000", 2024-05-11T23:01:29.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.58e-07 2024-05-11T23:01:29.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:29.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796596+0000", 2024-05-11T23:01:29.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7763000000000006e-05 2024-05-11T23:01:29.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:29.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796602+0000", 2024-05-11T23:01:29.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2899999999999999e-06 2024-05-11T23:01:29.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796608+0000", 2024-05-11T23:01:29.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4299999999999997e-06 2024-05-11T23:01:29.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.798525+0000", 2024-05-11T23:01:29.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019173549999999999 2024-05-11T23:01:29.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:29.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.798569+0000", 2024-05-11T23:01:29.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4489e-05 2024-05-11T23:01:29.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:29.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.798638+0000", 2024-05-11T23:01:29.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8337999999999995e-05 2024-05-11T23:01:29.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:29.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.799863+0000", 2024-05-11T23:01:29.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0012252630000000001 2024-05-11T23:01:29.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:29.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.803941+0000", 2024-05-11T23:01:29.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040776659999999998 2024-05-11T23:01:29.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:29.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.803973+0000", 2024-05-11T23:01:29.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1990999999999999e-05 2024-05-11T23:01:29.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:29.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.803995+0000", 2024-05-11T23:01:29.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2650000000000002e-05 2024-05-11T23:01:29.168 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.168 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.168 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.168 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:29.168 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.796618+0000", 2024-05-11T23:01:29.168 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.29926283, 2024-05-11T23:01:29.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0074346610000000004, 2024-05-11T23:01:29.168 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:29.168 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:29.168 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.168 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:29.169 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:29.169 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:29.169 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 135 2024-05-11T23:01:29.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.169 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:29.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:29.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796618+0000", 2024-05-11T23:01:29.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:29.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796618+0000", 2024-05-11T23:01:29.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:29.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796618+0000", 2024-05-11T23:01:29.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5400000000000001e-07 2024-05-11T23:01:29.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:29.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796622+0000", 2024-05-11T23:01:29.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6049999999999998e-06 2024-05-11T23:01:29.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:29.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796629+0000", 2024-05-11T23:01:29.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-05-11T23:01:29.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796633+0000", 2024-05-11T23:01:29.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4020000000000004e-06 2024-05-11T23:01:29.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.799136+0000", 2024-05-11T23:01:29.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0025028369999999999 2024-05-11T23:01:29.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:29.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.799186+0000", 2024-05-11T23:01:29.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9619e-05 2024-05-11T23:01:29.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:29.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.799267+0000", 2024-05-11T23:01:29.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1274999999999997e-05 2024-05-11T23:01:29.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:29.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.800703+0000", 2024-05-11T23:01:29.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00143651 2024-05-11T23:01:29.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:29.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.804002+0000", 2024-05-11T23:01:29.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032983230000000001 2024-05-11T23:01:29.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:29.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.804034+0000", 2024-05-11T23:01:29.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2348999999999998e-05 2024-05-11T23:01:29.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:29.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.804052+0000", 2024-05-11T23:01:29.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8159e-05 2024-05-11T23:01:29.173 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.173 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.173 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.174 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:29.174 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.796641+0000", 2024-05-11T23:01:29.174 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.299239484999999, 2024-05-11T23:01:29.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.007459032, 2024-05-11T23:01:29.174 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:29.174 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:29.174 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.174 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:29.174 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:29.174 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:29.174 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 136 2024-05-11T23:01:29.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.174 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:29.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:29.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796641+0000", 2024-05-11T23:01:29.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:29.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796641+0000", 2024-05-11T23:01:29.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:29.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796641+0000", 2024-05-11T23:01:29.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1199999999999998e-07 2024-05-11T23:01:29.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:29.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796645+0000", 2024-05-11T23:01:29.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3730000000000001e-06 2024-05-11T23:01:29.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:29.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796651+0000", 2024-05-11T23:01:29.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8360000000000003e-06 2024-05-11T23:01:29.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796656+0000", 2024-05-11T23:01:29.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.352e-06 2024-05-11T23:01:29.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.799494+0000", 2024-05-11T23:01:29.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028383789999999998 2024-05-11T23:01:29.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:29.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.799539+0000", 2024-05-11T23:01:29.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4441999999999999e-05 2024-05-11T23:01:29.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:29.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.799608+0000", 2024-05-11T23:01:29.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8986999999999995e-05 2024-05-11T23:01:29.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:29.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.800738+0000", 2024-05-11T23:01:29.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011300279999999999 2024-05-11T23:01:29.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:29.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.804057+0000", 2024-05-11T23:01:29.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033197259999999998 2024-05-11T23:01:29.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:29.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.804083+0000", 2024-05-11T23:01:29.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5905999999999999e-05 2024-05-11T23:01:29.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:29.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.804100+0000", 2024-05-11T23:01:29.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6591000000000001e-05 2024-05-11T23:01:29.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.179 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.179 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:29.179 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.796663+0000", 2024-05-11T23:01:29.179 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.29921689, 2024-05-11T23:01:29.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0074824360000000003, 2024-05-11T23:01:29.179 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:29.179 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:29.179 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:29.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:29.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:29.180 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 137 2024-05-11T23:01:29.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.180 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:29.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:29.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796663+0000", 2024-05-11T23:01:29.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:29.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796663+0000", 2024-05-11T23:01:29.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:29.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796664+0000", 2024-05-11T23:01:29.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.89e-07 2024-05-11T23:01:29.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:29.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796708+0000", 2024-05-11T23:01:29.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4481000000000003e-05 2024-05-11T23:01:29.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:29.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796714+0000", 2024-05-11T23:01:29.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5260000000000001e-06 2024-05-11T23:01:29.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796719+0000", 2024-05-11T23:01:29.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6759999999999998e-06 2024-05-11T23:01:29.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.799888+0000", 2024-05-11T23:01:29.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0031689959999999999 2024-05-11T23:01:29.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:29.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.799931+0000", 2024-05-11T23:01:29.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3219000000000001e-05 2024-05-11T23:01:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.800007+0000", 2024-05-11T23:01:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6552999999999997e-05 2024-05-11T23:01:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801639+0000", 2024-05-11T23:01:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001631867 2024-05-11T23:01:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.804103+0000", 2024-05-11T23:01:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024634779999999998 2024-05-11T23:01:29.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.804126+0000", 2024-05-11T23:01:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2993000000000002e-05 2024-05-11T23:01:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.804146+0000", 2024-05-11T23:01:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0157999999999999e-05 2024-05-11T23:01:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.184 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.796745+0000", 2024-05-11T23:01:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.299135464000001, 2024-05-11T23:01:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0074424540000000003, 2024-05-11T23:01:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 138 2024-05-11T23:01:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796745+0000", 2024-05-11T23:01:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796745+0000", 2024-05-11T23:01:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796745+0000", 2024-05-11T23:01:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6100000000000001e-07 2024-05-11T23:01:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796750+0000", 2024-05-11T23:01:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1880000000000004e-06 2024-05-11T23:01:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796755+0000", 2024-05-11T23:01:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7509999999999997e-06 2024-05-11T23:01:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796762+0000", 2024-05-11T23:01:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0280000000000003e-06 2024-05-11T23:01:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.800360+0000", 2024-05-11T23:01:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035973099999999998 2024-05-11T23:01:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.800402+0000", 2024-05-11T23:01:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2651000000000002e-05 2024-05-11T23:01:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.800471+0000", 2024-05-11T23:01:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8627999999999994e-05 2024-05-11T23:01:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801672+0000", 2024-05-11T23:01:29.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001201014 2024-05-11T23:01:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.804149+0000", 2024-05-11T23:01:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024770920000000002 2024-05-11T23:01:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.804171+0000", 2024-05-11T23:01:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2390000000000001e-05 2024-05-11T23:01:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:29.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.804187+0000", 2024-05-11T23:01:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5940999999999999e-05 2024-05-11T23:01:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.796771+0000", 2024-05-11T23:01:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.299109243, 2024-05-11T23:01:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010000207000000001, 2024-05-11T23:01:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:29.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 139 2024-05-11T23:01:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796771+0000", 2024-05-11T23:01:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796771+0000", 2024-05-11T23:01:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796772+0000", 2024-05-11T23:01:29.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.46e-07 2024-05-11T23:01:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796777+0000", 2024-05-11T23:01:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0200000000000002e-06 2024-05-11T23:01:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796783+0000", 2024-05-11T23:01:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9630000000000003e-06 2024-05-11T23:01:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796787+0000", 2024-05-11T23:01:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9309999999999999e-06 2024-05-11T23:01:29.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.800763+0000", 2024-05-11T23:01:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039755750000000003 2024-05-11T23:01:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.800807+0000", 2024-05-11T23:01:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4070000000000003e-05 2024-05-11T23:01:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.800892+0000", 2024-05-11T23:01:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5402000000000007e-05 2024-05-11T23:01:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801760+0000", 2024-05-11T23:01:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00086746299999999996 2024-05-11T23:01:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806715+0000", 2024-05-11T23:01:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049554760000000003 2024-05-11T23:01:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:29.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806747+0000", 2024-05-11T23:01:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1606999999999997e-05 2024-05-11T23:01:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806771+0000", 2024-05-11T23:01:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4253999999999999e-05 2024-05-11T23:01:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.796795+0000", 2024-05-11T23:01:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.299085094, 2024-05-11T23:01:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010021855, 2024-05-11T23:01:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:29.195 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 140 2024-05-11T23:01:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796795+0000", 2024-05-11T23:01:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796795+0000", 2024-05-11T23:01:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:29.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796796+0000", 2024-05-11T23:01:29.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3499999999999996e-07 2024-05-11T23:01:29.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:29.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796852+0000", 2024-05-11T23:01:29.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5785000000000002e-05 2024-05-11T23:01:29.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:29.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796859+0000", 2024-05-11T23:01:29.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5560000000000002e-06 2024-05-11T23:01:29.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796864+0000", 2024-05-11T23:01:29.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0000000000000004e-06 2024-05-11T23:01:29.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801142+0000", 2024-05-11T23:01:29.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042779740000000004 2024-05-11T23:01:29.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:29.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801190+0000", 2024-05-11T23:01:29.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7466000000000003e-05 2024-05-11T23:01:29.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:29.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801272+0000", 2024-05-11T23:01:29.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2534999999999995e-05 2024-05-11T23:01:29.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:29.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.802308+0000", 2024-05-11T23:01:29.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010360129999999999 2024-05-11T23:01:29.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:29.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806778+0000", 2024-05-11T23:01:29.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044694699999999997 2024-05-11T23:01:29.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:29.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806801+0000", 2024-05-11T23:01:29.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2972999999999998e-05 2024-05-11T23:01:29.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:29.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806817+0000", 2024-05-11T23:01:29.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6548000000000001e-05 2024-05-11T23:01:29.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.200 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.200 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:29.200 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.796875+0000", 2024-05-11T23:01:29.200 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.299005561, 2024-05-11T23:01:29.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0099879219999999998, 2024-05-11T23:01:29.200 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:29.200 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:29.200 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:29.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:29.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:29.201 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 141 2024-05-11T23:01:29.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.201 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:29.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:29.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796875+0000", 2024-05-11T23:01:29.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:29.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796875+0000", 2024-05-11T23:01:29.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:29.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:29.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796875+0000", 2024-05-11T23:01:29.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7299999999999996e-07 2024-05-11T23:01:29.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:29.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796879+0000", 2024-05-11T23:01:29.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9929999999999997e-06 2024-05-11T23:01:29.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:29.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796885+0000", 2024-05-11T23:01:29.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0279999999999997e-06 2024-05-11T23:01:29.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796890+0000", 2024-05-11T23:01:29.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3279999999999999e-06 2024-05-11T23:01:29.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:29.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801781+0000", 2024-05-11T23:01:29.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048916599999999999 2024-05-11T23:01:29.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:29.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801824+0000", 2024-05-11T23:01:29.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3006999999999997e-05 2024-05-11T23:01:29.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:29.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801893+0000", 2024-05-11T23:01:29.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8940000000000001e-05 2024-05-11T23:01:29.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:29.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.802554+0000", 2024-05-11T23:01:29.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00066069900000000005 2024-05-11T23:01:29.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:29.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806820+0000", 2024-05-11T23:01:29.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004265975 2024-05-11T23:01:29.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:29.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806843+0000", 2024-05-11T23:01:29.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3467000000000001e-05 2024-05-11T23:01:29.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:29.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:29.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:29.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806863+0000", 2024-05-11T23:01:29.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9252000000000001e-05 2024-05-11T23:01:29.205 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.205 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.205 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.205 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:29.205 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:29.205 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:29.207 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:01:29.207 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:29.206+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:29.307 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:29.307+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:29.342 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:29.343 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:29.343 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:01:29.343 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:29.343 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:01:29.408 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:29.408+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:29.452 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:29.452 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:29.452 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:01:29.452 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:01:29.452 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:29.458 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-05-11T23:01:29.464 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:01:29.517 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:29.517+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:29.578 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:29.578 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:01:29.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:01:29.578 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:01:29.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.578 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:17 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'7, mlcod=16'5)", 2024-05-11T23:01:29.578 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.178546+0000", 2024-05-11T23:01:29.578 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 11.398390302999999, 2024-05-11T23:01:29.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010843890000000001, 2024-05-11T23:01:29.578 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:29.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:29.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.579 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:29.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:29.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.178546+0000", 2024-05-11T23:01:29.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:29.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.178546+0000", 2024-05-11T23:01:29.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:29.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.178547+0000", 2024-05-11T23:01:29.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.553e-06 2024-05-11T23:01:29.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:29.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.178560+0000", 2024-05-11T23:01:29.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2407e-05 2024-05-11T23:01:29.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:29.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.178584+0000", 2024-05-11T23:01:29.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4606000000000001e-05 2024-05-11T23:01:29.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.178596+0000", 2024-05-11T23:01:29.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2249000000000001e-05 2024-05-11T23:01:29.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:29.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.178727+0000", 2024-05-11T23:01:29.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013057800000000001 2024-05-11T23:01:29.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:29.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.178763+0000", 2024-05-11T23:01:29.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5482999999999999e-05 2024-05-11T23:01:29.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:29.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.179567+0000", 2024-05-11T23:01:29.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00080404499999999998 2024-05-11T23:01:29.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:29.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.179630+0000", 2024-05-11T23:01:29.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3467999999999996e-05 2024-05-11T23:01:29.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.582 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.582 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:20 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-05-11T23:01:29.582 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.182141+0000", 2024-05-11T23:01:29.582 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 11.394795287999999, 2024-05-11T23:01:29.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001126089, 2024-05-11T23:01:29.583 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:29.583 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:29.583 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.583 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:29.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:29.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182141+0000", 2024-05-11T23:01:29.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:29.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182141+0000", 2024-05-11T23:01:29.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:29.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182141+0000", 2024-05-11T23:01:29.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3099999999999997e-07 2024-05-11T23:01:29.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:29.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182311+0000", 2024-05-11T23:01:29.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000169958 2024-05-11T23:01:29.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:29.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182323+0000", 2024-05-11T23:01:29.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2089999999999999e-05 2024-05-11T23:01:29.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182338+0000", 2024-05-11T23:01:29.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4083000000000001e-05 2024-05-11T23:01:29.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:29.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182383+0000", 2024-05-11T23:01:29.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5014e-05 2024-05-11T23:01:29.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:29.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182397+0000", 2024-05-11T23:01:29.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.432e-05 2024-05-11T23:01:29.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:29.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.183231+0000", 2024-05-11T23:01:29.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00083457099999999999 2024-05-11T23:01:29.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:29.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.183267+0000", 2024-05-11T23:01:29.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5321999999999998e-05 2024-05-11T23:01:29.586 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.586 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.586 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.587 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:21 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'11, mlcod=16'8)", 2024-05-11T23:01:29.587 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.182698+0000", 2024-05-11T23:01:29.587 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 11.394238447999999, 2024-05-11T23:01:29.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010004790000000001, 2024-05-11T23:01:29.587 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:29.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:29.587 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.587 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:29.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:29.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182698+0000", 2024-05-11T23:01:29.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:29.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182698+0000", 2024-05-11T23:01:29.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:29.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182698+0000", 2024-05-11T23:01:29.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9999999999999997e-07 2024-05-11T23:01:29.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:29.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182802+0000", 2024-05-11T23:01:29.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000103529 2024-05-11T23:01:29.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:29.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182815+0000", 2024-05-11T23:01:29.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3489000000000001e-05 2024-05-11T23:01:29.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182822+0000", 2024-05-11T23:01:29.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3949999999999998e-06 2024-05-11T23:01:29.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:29.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182886+0000", 2024-05-11T23:01:29.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3962999999999997e-05 2024-05-11T23:01:29.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:29.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182931+0000", 2024-05-11T23:01:29.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5018999999999997e-05 2024-05-11T23:01:29.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:29.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.183659+0000", 2024-05-11T23:01:29.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000728708 2024-05-11T23:01:29.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:29.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.183698+0000", 2024-05-11T23:01:29.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8776000000000003e-05 2024-05-11T23:01:29.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.590 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.591 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.591 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:31 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-05-11T23:01:29.591 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.190484+0000", 2024-05-11T23:01:29.591 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 11.386451556000001, 2024-05-11T23:01:29.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00095951200000000004, 2024-05-11T23:01:29.591 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:29.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:29.591 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.591 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:29.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:29.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.190484+0000", 2024-05-11T23:01:29.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:29.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.190484+0000", 2024-05-11T23:01:29.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:29.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.190485+0000", 2024-05-11T23:01:29.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.92e-07 2024-05-11T23:01:29.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:29.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.190491+0000", 2024-05-11T23:01:29.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5840000000000002e-06 2024-05-11T23:01:29.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:29.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.190499+0000", 2024-05-11T23:01:29.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6510000000000007e-06 2024-05-11T23:01:29.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.190507+0000", 2024-05-11T23:01:29.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.481e-06 2024-05-11T23:01:29.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:29.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.190563+0000", 2024-05-11T23:01:29.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6045999999999998e-05 2024-05-11T23:01:29.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:29.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.190587+0000", 2024-05-11T23:01:29.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4031000000000001e-05 2024-05-11T23:01:29.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:29.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.191381+0000", 2024-05-11T23:01:29.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00079456600000000002 2024-05-11T23:01:29.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:29.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.191444+0000", 2024-05-11T23:01:29.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2460999999999997e-05 2024-05-11T23:01:29.595 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.595 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.595 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.595 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:32 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-05-11T23:01:29.595 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.190931+0000", 2024-05-11T23:01:29.595 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 11.386005003999999, 2024-05-11T23:01:29.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00091497200000000001, 2024-05-11T23:01:29.595 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:29.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:29.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.595 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:29.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:29.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.190931+0000", 2024-05-11T23:01:29.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:29.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.190931+0000", 2024-05-11T23:01:29.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:29.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.190932+0000", 2024-05-11T23:01:29.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7400000000000002e-07 2024-05-11T23:01:29.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:29.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.190993+0000", 2024-05-11T23:01:29.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1608999999999999e-05 2024-05-11T23:01:29.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:29.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.191006+0000", 2024-05-11T23:01:29.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2965e-05 2024-05-11T23:01:29.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.191014+0000", 2024-05-11T23:01:29.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0790000000000001e-06 2024-05-11T23:01:29.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:29.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.191069+0000", 2024-05-11T23:01:29.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4252000000000002e-05 2024-05-11T23:01:29.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:29.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.191084+0000", 2024-05-11T23:01:29.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5277e-05 2024-05-11T23:01:29.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:29.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.191814+0000", 2024-05-11T23:01:29.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00073040699999999997 2024-05-11T23:01:29.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:29.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.191846+0000", 2024-05-11T23:01:29.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1609000000000002e-05 2024-05-11T23:01:29.599 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.599 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.599 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.599 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:35 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'17)", 2024-05-11T23:01:29.599 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.192947+0000", 2024-05-11T23:01:29.599 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 11.383989014999999, 2024-05-11T23:01:29.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010417689999999999, 2024-05-11T23:01:29.599 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:29.599 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:29.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.600 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:29.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:29.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.192947+0000", 2024-05-11T23:01:29.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:29.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.192947+0000", 2024-05-11T23:01:29.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:29.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.192947+0000", 2024-05-11T23:01:29.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5200000000000002e-07 2024-05-11T23:01:29.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:29.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.193019+0000", 2024-05-11T23:01:29.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1215000000000002e-05 2024-05-11T23:01:29.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:29.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.193029+0000", 2024-05-11T23:01:29.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0268000000000001e-05 2024-05-11T23:01:29.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.193035+0000", 2024-05-11T23:01:29.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6250000000000004e-06 2024-05-11T23:01:29.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:29.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.193094+0000", 2024-05-11T23:01:29.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9891e-05 2024-05-11T23:01:29.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:29.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.193109+0000", 2024-05-11T23:01:29.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4225e-05 2024-05-11T23:01:29.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:29.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.193952+0000", 2024-05-11T23:01:29.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00084333800000000001 2024-05-11T23:01:29.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:29.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.193989+0000", 2024-05-11T23:01:29.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6755000000000003e-05 2024-05-11T23:01:29.603 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.603 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.603 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.603 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:55 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'33, mlcod=16'29)", 2024-05-11T23:01:29.603 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.209819+0000", 2024-05-11T23:01:29.603 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 11.367116696, 2024-05-11T23:01:29.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001022427, 2024-05-11T23:01:29.603 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:29.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:29.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.603 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:29.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:29.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.209819+0000", 2024-05-11T23:01:29.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:29.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.209819+0000", 2024-05-11T23:01:29.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:29.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.209820+0000", 2024-05-11T23:01:29.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1800000000000001e-07 2024-05-11T23:01:29.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:29.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.209897+0000", 2024-05-11T23:01:29.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7195999999999997e-05 2024-05-11T23:01:29.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:29.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.209907+0000", 2024-05-11T23:01:29.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0054000000000001e-05 2024-05-11T23:01:29.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.209919+0000", 2024-05-11T23:01:29.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2038000000000001e-05 2024-05-11T23:01:29.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:29.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.209968+0000", 2024-05-11T23:01:29.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9197999999999999e-05 2024-05-11T23:01:29.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:29.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.209989+0000", 2024-05-11T23:01:29.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0905000000000001e-05 2024-05-11T23:01:29.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:29.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.210795+0000", 2024-05-11T23:01:29.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00080568399999999998 2024-05-11T23:01:29.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:29.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.210842+0000", 2024-05-11T23:01:29.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6934000000000002e-05 2024-05-11T23:01:29.607 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.607 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.607 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.607 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:57 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'35, mlcod=16'33)", 2024-05-11T23:01:29.607 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.211853+0000", 2024-05-11T23:01:29.607 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 11.365083046000001, 2024-05-11T23:01:29.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00097442099999999995, 2024-05-11T23:01:29.607 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:29.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:29.607 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.608 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:29.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:29.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.211853+0000", 2024-05-11T23:01:29.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:29.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.211853+0000", 2024-05-11T23:01:29.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:29.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.211853+0000", 2024-05-11T23:01:29.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2399999999999998e-07 2024-05-11T23:01:29.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:29.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.211928+0000", 2024-05-11T23:01:29.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4481999999999996e-05 2024-05-11T23:01:29.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:29.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.211940+0000", 2024-05-11T23:01:29.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2441e-05 2024-05-11T23:01:29.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.211957+0000", 2024-05-11T23:01:29.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6922000000000002e-05 2024-05-11T23:01:29.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:29.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.212001+0000", 2024-05-11T23:01:29.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.371e-05 2024-05-11T23:01:29.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:29.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.212015+0000", 2024-05-11T23:01:29.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3509000000000001e-05 2024-05-11T23:01:29.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:29.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.212778+0000", 2024-05-11T23:01:29.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00076356299999999998 2024-05-11T23:01:29.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:29.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.212827+0000", 2024-05-11T23:01:29.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9270000000000001e-05 2024-05-11T23:01:29.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.611 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.611 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:68 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-05-11T23:01:29.611 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.219998+0000", 2024-05-11T23:01:29.611 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 11.356938271000001, 2024-05-11T23:01:29.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001022579, 2024-05-11T23:01:29.612 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:29.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:29.612 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.612 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:29.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:29.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.219998+0000", 2024-05-11T23:01:29.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:29.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.219998+0000", 2024-05-11T23:01:29.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:29.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.219998+0000", 2024-05-11T23:01:29.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1199999999999998e-07 2024-05-11T23:01:29.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:29.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.220004+0000", 2024-05-11T23:01:29.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4090000000000002e-06 2024-05-11T23:01:29.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:29.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.220012+0000", 2024-05-11T23:01:29.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4759999999999998e-06 2024-05-11T23:01:29.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.220016+0000", 2024-05-11T23:01:29.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2030000000000002e-06 2024-05-11T23:01:29.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:29.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.220049+0000", 2024-05-11T23:01:29.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3152999999999999e-05 2024-05-11T23:01:29.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:29.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.220061+0000", 2024-05-11T23:01:29.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1443e-05 2024-05-11T23:01:29.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:29.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.220970+0000", 2024-05-11T23:01:29.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00090934200000000005 2024-05-11T23:01:29.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:29.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.221020+0000", 2024-05-11T23:01:29.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0141e-05 2024-05-11T23:01:29.615 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.615 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.615 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.615 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:71 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'45, mlcod=16'43)", 2024-05-11T23:01:29.615 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.223440+0000", 2024-05-11T23:01:29.616 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 11.353496104, 2024-05-11T23:01:29.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000933791, 2024-05-11T23:01:29.616 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:29.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:29.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.616 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:29.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:29.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.223440+0000", 2024-05-11T23:01:29.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:29.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.223440+0000", 2024-05-11T23:01:29.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:29.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.223441+0000", 2024-05-11T23:01:29.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2399999999999998e-07 2024-05-11T23:01:29.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:29.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.223495+0000", 2024-05-11T23:01:29.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4602999999999999e-05 2024-05-11T23:01:29.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:29.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.223506+0000", 2024-05-11T23:01:29.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0688e-05 2024-05-11T23:01:29.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.223511+0000", 2024-05-11T23:01:29.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0520000000000001e-06 2024-05-11T23:01:29.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:29.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.223551+0000", 2024-05-11T23:01:29.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0528999999999998e-05 2024-05-11T23:01:29.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:29.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.223566+0000", 2024-05-11T23:01:29.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4381e-05 2024-05-11T23:01:29.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:29.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.224330+0000", 2024-05-11T23:01:29.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00076462299999999997 2024-05-11T23:01:29.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:29.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.224374+0000", 2024-05-11T23:01:29.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3291000000000003e-05 2024-05-11T23:01:29.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.620 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.620 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:83 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'53, mlcod=16'50)", 2024-05-11T23:01:29.620 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.232478+0000", 2024-05-11T23:01:29.620 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 11.344457651000001, 2024-05-11T23:01:29.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00097954599999999998, 2024-05-11T23:01:29.620 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:29.620 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:29.620 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.620 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:29.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:29.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.232478+0000", 2024-05-11T23:01:29.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:29.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.232478+0000", 2024-05-11T23:01:29.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:29.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.232479+0000", 2024-05-11T23:01:29.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4999999999999998e-07 2024-05-11T23:01:29.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:29.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.232482+0000", 2024-05-11T23:01:29.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4450000000000001e-06 2024-05-11T23:01:29.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:29.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.232488+0000", 2024-05-11T23:01:29.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.964e-06 2024-05-11T23:01:29.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.232492+0000", 2024-05-11T23:01:29.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5810000000000001e-06 2024-05-11T23:01:29.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:29.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.232535+0000", 2024-05-11T23:01:29.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2988999999999998e-05 2024-05-11T23:01:29.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:29.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.232559+0000", 2024-05-11T23:01:29.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4791999999999999e-05 2024-05-11T23:01:29.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:29.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.233414+0000", 2024-05-11T23:01:29.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00085448999999999996 2024-05-11T23:01:29.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:29.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.233458+0000", 2024-05-11T23:01:29.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3934999999999999e-05 2024-05-11T23:01:29.624 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.624 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.624 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.624 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:84 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-05-11T23:01:29.624 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.232954+0000", 2024-05-11T23:01:29.624 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 11.343982345000001, 2024-05-11T23:01:29.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00089954599999999998, 2024-05-11T23:01:29.624 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:29.624 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:29.625 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.625 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:29.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:29.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.232954+0000", 2024-05-11T23:01:29.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:29.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.232954+0000", 2024-05-11T23:01:29.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:29.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.232954+0000", 2024-05-11T23:01:29.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8700000000000001e-07 2024-05-11T23:01:29.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:29.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.233012+0000", 2024-05-11T23:01:29.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7658000000000003e-05 2024-05-11T23:01:29.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:29.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.233020+0000", 2024-05-11T23:01:29.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4889999999999995e-06 2024-05-11T23:01:29.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.233024+0000", 2024-05-11T23:01:29.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1999999999999996e-06 2024-05-11T23:01:29.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:29.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.233063+0000", 2024-05-11T23:01:29.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8507999999999998e-05 2024-05-11T23:01:29.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:29.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.233085+0000", 2024-05-11T23:01:29.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2435000000000001e-05 2024-05-11T23:01:29.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:29.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.233815+0000", 2024-05-11T23:01:29.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072962299999999999 2024-05-11T23:01:29.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:29.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.233853+0000", 2024-05-11T23:01:29.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8245999999999999e-05 2024-05-11T23:01:29.628 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.628 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.628 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.628 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:86 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'56, mlcod=16'52)", 2024-05-11T23:01:29.628 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.234135+0000", 2024-05-11T23:01:29.628 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 11.342801333000001, 2024-05-11T23:01:29.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00095073800000000002, 2024-05-11T23:01:29.628 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:29.628 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:29.628 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.629 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:29.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:29.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.234135+0000", 2024-05-11T23:01:29.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:29.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.234135+0000", 2024-05-11T23:01:29.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:29.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.234135+0000", 2024-05-11T23:01:29.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8400000000000004e-07 2024-05-11T23:01:29.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:29.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.234140+0000", 2024-05-11T23:01:29.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1359999999999996e-06 2024-05-11T23:01:29.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:29.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.234150+0000", 2024-05-11T23:01:29.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7489999999999993e-06 2024-05-11T23:01:29.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.234162+0000", 2024-05-11T23:01:29.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1299e-05 2024-05-11T23:01:29.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:29.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.234215+0000", 2024-05-11T23:01:29.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3764999999999998e-05 2024-05-11T23:01:29.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:29.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.234230+0000", 2024-05-11T23:01:29.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.434e-05 2024-05-11T23:01:29.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:29.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.235058+0000", 2024-05-11T23:01:29.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00082790900000000005 2024-05-11T23:01:29.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:29.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.235085+0000", 2024-05-11T23:01:29.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7855999999999999e-05 2024-05-11T23:01:29.632 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.632 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.632 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.632 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:105 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'67, mlcod=16'64)", 2024-05-11T23:01:29.632 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.385395+0000", 2024-05-11T23:01:29.632 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 11.191540693, 2024-05-11T23:01:29.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00090768700000000001, 2024-05-11T23:01:29.632 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:29.632 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:29.633 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.633 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:29.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:29.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.385395+0000", 2024-05-11T23:01:29.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:29.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.385395+0000", 2024-05-11T23:01:29.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:29.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.385396+0000", 2024-05-11T23:01:29.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.06e-07 2024-05-11T23:01:29.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:29.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.385402+0000", 2024-05-11T23:01:29.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5370000000000003e-06 2024-05-11T23:01:29.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:29.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.385410+0000", 2024-05-11T23:01:29.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.695e-06 2024-05-11T23:01:29.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.385416+0000", 2024-05-11T23:01:29.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8560000000000003e-06 2024-05-11T23:01:29.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:29.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.385464+0000", 2024-05-11T23:01:29.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8072999999999999e-05 2024-05-11T23:01:29.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:29.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.385487+0000", 2024-05-11T23:01:29.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3051e-05 2024-05-11T23:01:29.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:29.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.386271+0000", 2024-05-11T23:01:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078372000000000001 2024-05-11T23:01:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.386303+0000", 2024-05-11T23:01:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2249000000000002e-05 2024-05-11T23:01:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:106 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-05-11T23:01:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.385836+0000", 2024-05-11T23:01:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 11.191100083, 2024-05-11T23:01:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000905703, 2024-05-11T23:01:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.385836+0000", 2024-05-11T23:01:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.385836+0000", 2024-05-11T23:01:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.385836+0000", 2024-05-11T23:01:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1200000000000003e-07 2024-05-11T23:01:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.385898+0000", 2024-05-11T23:01:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1711999999999995e-05 2024-05-11T23:01:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.385906+0000", 2024-05-11T23:01:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6240000000000002e-06 2024-05-11T23:01:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.385916+0000", 2024-05-11T23:01:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0088999999999999e-05 2024-05-11T23:01:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.385960+0000", 2024-05-11T23:01:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3989999999999997e-05 2024-05-11T23:01:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.385973+0000", 2024-05-11T23:01:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2809000000000001e-05 2024-05-11T23:01:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.386705+0000", 2024-05-11T23:01:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00073231399999999997 2024-05-11T23:01:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.386742+0000", 2024-05-11T23:01:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6653000000000003e-05 2024-05-11T23:01:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:124 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'86, mlcod=16'84)", 2024-05-11T23:01:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.793456+0000", 2024-05-11T23:01:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.783479917999999, 2024-05-11T23:01:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00099138099999999995, 2024-05-11T23:01:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.793456+0000", 2024-05-11T23:01:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.793456+0000", 2024-05-11T23:01:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.793458+0000", 2024-05-11T23:01:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1500000000000002e-06 2024-05-11T23:01:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.793480+0000", 2024-05-11T23:01:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1413e-05 2024-05-11T23:01:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.793510+0000", 2024-05-11T23:01:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9980000000000001e-05 2024-05-11T23:01:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.793525+0000", 2024-05-11T23:01:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5182e-05 2024-05-11T23:01:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.793693+0000", 2024-05-11T23:01:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000168474 2024-05-11T23:01:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.793759+0000", 2024-05-11T23:01:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5754999999999996e-05 2024-05-11T23:01:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.794411+0000", 2024-05-11T23:01:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00065239200000000001 2024-05-11T23:01:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.794447+0000", 2024-05-11T23:01:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6035000000000003e-05 2024-05-11T23:01:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:134 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'92, mlcod=16'87)", 2024-05-11T23:01:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.798709+0000", 2024-05-11T23:01:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.778227342999999, 2024-05-11T23:01:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00090358500000000004, 2024-05-11T23:01:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.798709+0000", 2024-05-11T23:01:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.798709+0000", 2024-05-11T23:01:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.798709+0000", 2024-05-11T23:01:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3099999999999998e-07 2024-05-11T23:01:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.798767+0000", 2024-05-11T23:01:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7479999999999999e-05 2024-05-11T23:01:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.798776+0000", 2024-05-11T23:01:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5599999999999999e-06 2024-05-11T23:01:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.798782+0000", 2024-05-11T23:01:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:01:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.798812+0000", 2024-05-11T23:01:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9847000000000001e-05 2024-05-11T23:01:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.798823+0000", 2024-05-11T23:01:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1097999999999999e-05 2024-05-11T23:01:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.799587+0000", 2024-05-11T23:01:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00076371700000000002 2024-05-11T23:01:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.799612+0000", 2024-05-11T23:01:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5639999999999998e-05 2024-05-11T23:01:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:153 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'107, mlcod=16'105)", 2024-05-11T23:01:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:19.320031+0000", 2024-05-11T23:01:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.256904487, 2024-05-11T23:01:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00092248199999999997, 2024-05-11T23:01:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320031+0000", 2024-05-11T23:01:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320031+0000", 2024-05-11T23:01:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320032+0000", 2024-05-11T23:01:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8899999999999999e-07 2024-05-11T23:01:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320038+0000", 2024-05-11T23:01:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.733e-06 2024-05-11T23:01:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320048+0000", 2024-05-11T23:01:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8810000000000008e-06 2024-05-11T23:01:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320054+0000", 2024-05-11T23:01:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0950000000000001e-06 2024-05-11T23:01:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320100+0000", 2024-05-11T23:01:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6421000000000002e-05 2024-05-11T23:01:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320116+0000", 2024-05-11T23:01:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5268999999999999e-05 2024-05-11T23:01:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320902+0000", 2024-05-11T23:01:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078689600000000004 2024-05-11T23:01:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320954+0000", 2024-05-11T23:01:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1498000000000001e-05 2024-05-11T23:01:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:154 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'108, mlcod=16'105)", 2024-05-11T23:01:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:19.320542+0000", 2024-05-11T23:01:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.256394467, 2024-05-11T23:01:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001033349, 2024-05-11T23:01:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320542+0000", 2024-05-11T23:01:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:29.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320542+0000", 2024-05-11T23:01:29.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:29.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320542+0000", 2024-05-11T23:01:29.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7300000000000002e-07 2024-05-11T23:01:29.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:29.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320546+0000", 2024-05-11T23:01:29.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8770000000000003e-06 2024-05-11T23:01:29.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:29.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320552+0000", 2024-05-11T23:01:29.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:01:29.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320556+0000", 2024-05-11T23:01:29.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7910000000000002e-06 2024-05-11T23:01:29.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:29.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320604+0000", 2024-05-11T23:01:29.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7713999999999998e-05 2024-05-11T23:01:29.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:29.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320626+0000", 2024-05-11T23:01:29.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2296000000000001e-05 2024-05-11T23:01:29.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:29.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.321530+0000", 2024-05-11T23:01:29.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00090343499999999996 2024-05-11T23:01:29.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:29.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.321575+0000", 2024-05-11T23:01:29.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5306999999999998e-05 2024-05-11T23:01:29.657 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.657 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.657 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.657 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:155 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'109, mlcod=16'105)", 2024-05-11T23:01:29.657 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:19.321082+0000", 2024-05-11T23:01:29.657 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.255854119, 2024-05-11T23:01:29.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00097425599999999995, 2024-05-11T23:01:29.657 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:29.657 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:29.657 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:29.657 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:29.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:29.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.321082+0000", 2024-05-11T23:01:29.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:29.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.321082+0000", 2024-05-11T23:01:29.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:29.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:29.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.321082+0000", 2024-05-11T23:01:29.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.89e-07 2024-05-11T23:01:29.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:29.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.321152+0000", 2024-05-11T23:01:29.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9622000000000005e-05 2024-05-11T23:01:29.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:29.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.321160+0000", 2024-05-11T23:01:29.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3359999999999995e-06 2024-05-11T23:01:29.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:29.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.321166+0000", 2024-05-11T23:01:29.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3020000000000002e-06 2024-05-11T23:01:29.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:29.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.321199+0000", 2024-05-11T23:01:29.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3827e-05 2024-05-11T23:01:29.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:29.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.321217+0000", 2024-05-11T23:01:29.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7316000000000002e-05 2024-05-11T23:01:29.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:29.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.322011+0000", 2024-05-11T23:01:29.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00079464800000000001 2024-05-11T23:01:29.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:29.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:29.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:29.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.322056+0000", 2024-05-11T23:01:29.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4815999999999999e-05 2024-05-11T23:01:29.661 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.661 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.661 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.661 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:29.661 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:29.661 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:29.663 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:01:29.663 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:29.662+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:29.763 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:29.763+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:29.814 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:29.814 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:29.814 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:01:29.815 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:29.823 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:01:29.865 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:29.865+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:29.965 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:29.965+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:29.973 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:29.973 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:29.973 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:01:29.973 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:01:29.973 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:29.983 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:01:30.066 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:30.066+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:30.133 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:01:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:01:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:01:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.133 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:7 3.1a e21/19 3:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-05-11T23:01:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:28.459598+0000", 2024-05-11T23:01:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.6741648599999999, 2024-05-11T23:01:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00082882099999999998, 2024-05-11T23:01:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:28.459598+0000", 2024-05-11T23:01:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:28.459598+0000", 2024-05-11T23:01:30.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:28.459598+0000", 2024-05-11T23:01:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6999999999999999e-07 2024-05-11T23:01:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:28.459603+0000", 2024-05-11T23:01:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7609999999999998e-06 2024-05-11T23:01:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:30.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:28.459617+0000", 2024-05-11T23:01:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.454e-05 2024-05-11T23:01:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:28.459626+0000", 2024-05-11T23:01:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8789999999999992e-06 2024-05-11T23:01:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:28.459680+0000", 2024-05-11T23:01:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3829999999999998e-05 2024-05-11T23:01:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:30.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:28.459698+0000", 2024-05-11T23:01:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8377999999999999e-05 2024-05-11T23:01:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:28.460388+0000", 2024-05-11T23:01:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00068997800000000005 2024-05-11T23:01:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:28.460426+0000", 2024-05-11T23:01:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8284999999999997e-05 2024-05-11T23:01:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:30.138 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:9 3.1a e22/19 3:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-05-11T23:01:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:29.465985+0000", 2024-05-11T23:01:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.66777796300000003, 2024-05-11T23:01:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00088430099999999999, 2024-05-11T23:01:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:30.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.465985+0000", 2024-05-11T23:01:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.465985+0000", 2024-05-11T23:01:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:30.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.465985+0000", 2024-05-11T23:01:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8299999999999998e-07 2024-05-11T23:01:30.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.465993+0000", 2024-05-11T23:01:30.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3240000000000003e-06 2024-05-11T23:01:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.466017+0000", 2024-05-11T23:01:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3595e-05 2024-05-11T23:01:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.466028+0000", 2024-05-11T23:01:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0964999999999999e-05 2024-05-11T23:01:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.466096+0000", 2024-05-11T23:01:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7935000000000006e-05 2024-05-11T23:01:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.466117+0000", 2024-05-11T23:01:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.175e-05 2024-05-11T23:01:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.466833+0000", 2024-05-11T23:01:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00071593199999999996 2024-05-11T23:01:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.466869+0000", 2024-05-11T23:01:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5516999999999999e-05 2024-05-11T23:01:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:30.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:12 3.6 e22/19 3:6451e15f:::rbd_header.111e479bff4d:head v 22'1, mlcod=0'0)", 2024-05-11T23:01:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:29.481445+0000", 2024-05-11T23:01:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.65231761499999996, 2024-05-11T23:01:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00077657099999999999, 2024-05-11T23:01:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.481445+0000", 2024-05-11T23:01:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.481445+0000", 2024-05-11T23:01:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.481445+0000", 2024-05-11T23:01:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.17e-07 2024-05-11T23:01:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.481450+0000", 2024-05-11T23:01:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5519999999999998e-06 2024-05-11T23:01:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.481470+0000", 2024-05-11T23:01:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.012e-05 2024-05-11T23:01:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.481479+0000", 2024-05-11T23:01:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5009999999999993e-06 2024-05-11T23:01:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:30.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.481536+0000", 2024-05-11T23:01:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6949e-05 2024-05-11T23:01:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.481560+0000", 2024-05-11T23:01:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3495000000000001e-05 2024-05-11T23:01:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.482188+0000", 2024-05-11T23:01:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00062834299999999996 2024-05-11T23:01:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:30.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.482221+0000", 2024-05-11T23:01:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3393999999999999e-05 2024-05-11T23:01:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:25 3.6 e22/19 3:6451e15f:::rbd_header.111e479bff4d:head v 22'2, mlcod=0'0)", 2024-05-11T23:01:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:29.491576+0000", 2024-05-11T23:01:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.64218630200000004, 2024-05-11T23:01:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00055113899999999997, 2024-05-11T23:01:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:30.150 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.491576+0000", 2024-05-11T23:01:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.491576+0000", 2024-05-11T23:01:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.491577+0000", 2024-05-11T23:01:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3209999999999999e-06 2024-05-11T23:01:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.491588+0000", 2024-05-11T23:01:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0550999999999999e-05 2024-05-11T23:01:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.491608+0000", 2024-05-11T23:01:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9868999999999999e-05 2024-05-11T23:01:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.491618+0000", 2024-05-11T23:01:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8830000000000001e-06 2024-05-11T23:01:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.491690+0000", 2024-05-11T23:01:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2595999999999994e-05 2024-05-11T23:01:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.491708+0000", 2024-05-11T23:01:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7717999999999999e-05 2024-05-11T23:01:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.492095+0000", 2024-05-11T23:01:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00038727100000000001 2024-05-11T23:01:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.492127+0000", 2024-05-11T23:01:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1930000000000001e-05 2024-05-11T23:01:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:30.154 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:30.154 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:30.156 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:01:30.200 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:30.200+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:30.227 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:01:30.267 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:30.267+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:30.268 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:30.268 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:30.268 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:01:30.268 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:30.277 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:01:30.357 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:01:30.358 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:01:30.368 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:30.368+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:30.376 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:01:30.376 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:01:30.393 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:01:30.393 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:01:30.410 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:01:30.411 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:01:30.429 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:01:30.429 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:01:30.432 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:30.432 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:30.442 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:01:30.447 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:01:30.448 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:01:30.466 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:01:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:01:30.469 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:30.468+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:30.485 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:01:30.485 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:01:30.561 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:30.561 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:30.570 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:01:30.570 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:30.569+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:30.670 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:30.669+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:30.722 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:30.723 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:30.731 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:01:30.771 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:30.771+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:30.842 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:30.842 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:30.851 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:01:30.872 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:30.872+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:30.962 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:30.962 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:30.972 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:01:30.973 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:30.973+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:31.074 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:31.073+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:31.089 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:31.089 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:31.099 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:01:31.183 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:31.183+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:31.208 INFO:teuthology.orchestra.run.smithi005.stdout:skipping zero length punch hole 2024-05-11T23:01:31.218 INFO:teuthology.orchestra.run.smithi005.stdout:5 write 0x3eb310f thru 0x3ebbf49 (0x8e3b bytes) 2024-05-11T23:01:31.227 INFO:teuthology.orchestra.run.smithi005.stdout:6 write 0x50363b9 thru 0x50381a7 (0x1def bytes) 2024-05-11T23:01:31.231 INFO:teuthology.orchestra.run.smithi005.stdout:7 write 0x4de2830 thru 0x4de8dea (0x65bb bytes) 2024-05-11T23:01:31.241 INFO:teuthology.orchestra.run.smithi005.stdout:8 write 0x609a1de thru 0x609c006 (0x1e29 bytes) 2024-05-11T23:01:31.244 INFO:teuthology.orchestra.run.smithi005.stdout:9 read 0x4324c67 thru 0x4327b73 (0x2f0d bytes) 2024-05-11T23:01:31.244 INFO:teuthology.orchestra.run.smithi005.stdout:10 read 0x4e6fb64 thru 0x4e74ff0 (0x548d bytes) 2024-05-11T23:01:31.249 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:31.249 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:31.258 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:01:31.259 INFO:teuthology.orchestra.run.smithi005.stdout:12 write 0xd04b346 thru 0xd04bb34 (0x7ef bytes) 2024-05-11T23:01:31.262 INFO:teuthology.orchestra.run.smithi005.stdout:13 punch from 0x986abee to 0x9870248, (0x565a bytes) 2024-05-11T23:01:31.262 INFO:teuthology.orchestra.run.smithi005.stdout:14 read 0x1770fc8 thru 0x177195e (0x997 bytes) 2024-05-11T23:01:31.262 INFO:teuthology.orchestra.run.smithi005.stdout:15 writesame 0x9a95e44 thru 0x9a974e7 data_size 0x17(0x16a4 bytes) 2024-05-11T23:01:31.270 INFO:teuthology.orchestra.run.smithi005.stdout:16 writesame 0xd242868 thru 0xd24ad5c data_size 0x1(0x84f5 bytes) 2024-05-11T23:01:31.276 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:31.276+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:31.277 INFO:teuthology.orchestra.run.smithi005.stdout:17 write 0xdf91015 thru 0xdf9f258 (0xe244 bytes) 2024-05-11T23:01:31.280 INFO:teuthology.orchestra.run.smithi005.stdout:20 read 0xa621f73 thru 0xa6238e8 (0x1976 bytes) 2024-05-11T23:01:31.280 INFO:teuthology.orchestra.run.smithi005.stdout:21 punch from 0x3e8263d to 0x3e87db1, (0x5774 bytes) 2024-05-11T23:01:31.280 INFO:teuthology.orchestra.run.smithi005.stdout:22 write 0x1c858d2 thru 0x1c90273 (0xa9a2 bytes) 2024-05-11T23:01:31.283 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:31.283 INFO:teuthology.orchestra.run.smithi005.stdout:truncating to largest ever: 0xea1843f 2024-05-11T23:01:31.284 INFO:teuthology.orchestra.run.smithi005.stdout:27 trunc from 0xdf9f259 to 0xea1843f 2024-05-11T23:01:31.289 INFO:teuthology.orchestra.run.smithi005.stdout:28 read 0xc385e25 thru 0xc388c88 (0x2e64 bytes) 2024-05-11T23:01:31.289 INFO:teuthology.orchestra.run.smithi005.stdout:29 clone 1 order 24 su 65536 sc 12 2024-05-11T23:01:31.377 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:31.377+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:31.404 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:31.404 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:31.412 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:01:31.476 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-11T23:01:31.496 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:31.496+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:31.561 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:31.562 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:31.570 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:01:31.578 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:31.578+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:31.653 INFO:teuthology.orchestra.run.smithi005.stdout:31 write 0x7c156e9 thru 0x7c15a52 (0x36a bytes) 2024-05-11T23:01:31.662 INFO:teuthology.orchestra.run.smithi005.stdout:32 write 0x6c1c2da thru 0x6c1cc84 (0x9ab bytes) 2024-05-11T23:01:31.667 INFO:teuthology.orchestra.run.smithi005.stdout:35 write 0xab7e635 thru 0xab8e254 (0xfc20 bytes) 2024-05-11T23:01:31.675 INFO:teuthology.orchestra.run.smithi005.stdout:36 write 0x124d4f0 thru 0x125b74f (0xe260 bytes) 2024-05-11T23:01:31.679 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:31.679+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:31.680 INFO:teuthology.orchestra.run.smithi005.stdout:37 read 0x26051cb thru 0x2611f03 (0xcd39 bytes) 2024-05-11T23:01:31.680 INFO:teuthology.orchestra.run.smithi005.stdout:39 read 0x8478f0 thru 0x84a6fe (0x2e0f bytes) 2024-05-11T23:01:31.681 INFO:teuthology.orchestra.run.smithi005.stdout:44 write 0xb0b89b0 thru 0xb0c00c9 (0x771a bytes) 2024-05-11T23:01:31.682 INFO:teuthology.orchestra.run.smithi005.stdout:45 write 0x45053c7 thru 0x450b364 (0x5f9e bytes) 2024-05-11T23:01:31.688 INFO:teuthology.orchestra.run.smithi005.stdout:47 read 0x3886971 thru 0x388e9e3 (0x8073 bytes) 2024-05-11T23:01:31.702 INFO:teuthology.orchestra.run.smithi005.stdout:48 trunc from 0xea1843f to 0x2a78da8 2024-05-11T23:01:31.725 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:31.726 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:31.730 INFO:teuthology.orchestra.run.smithi005.stdout:49 writesame 0x3af2259 thru 0x3aff602 data_size 0x21(0xd3aa bytes) 2024-05-11T23:01:31.733 INFO:teuthology.orchestra.run.smithi005.stdout:50 trunc from 0x3aff603 to 0x1cc750c 2024-05-11T23:01:31.735 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:01:31.756 INFO:teuthology.orchestra.run.smithi005.stdout:51 write 0x9ca7573 thru 0x9cb47ea (0xd278 bytes) 2024-05-11T23:01:31.767 INFO:teuthology.orchestra.run.smithi005.stdout:53 writesame 0xce90a26 thru 0xce9aca0 data_size 0x8d(0xa27b bytes) 2024-05-11T23:01:31.767 INFO:teuthology.orchestra.run.smithi005.stdout:55 write 0x2851332 thru 0x2857a59 (0x6728 bytes) 2024-05-11T23:01:31.776 INFO:teuthology.orchestra.run.smithi005.stdout:57 trunc from 0xce9aca1 to 0x5e4afd8 2024-05-11T23:01:31.779 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:31.779+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:31.785 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:31.785 INFO:teuthology.orchestra.run.smithi005.stdout:59 write 0x502f15a thru 0x5039528 (0xa3cf bytes) 2024-05-11T23:01:31.787 INFO:teuthology.orchestra.run.smithi005.stdout:61 punch from 0x510c2b7 to 0x510cfcc, (0xd15 bytes) 2024-05-11T23:01:31.791 INFO:teuthology.orchestra.run.smithi005.stdout:62 trunc from 0x5e4afd8 to 0x25559fa 2024-05-11T23:01:31.803 INFO:teuthology.orchestra.run.smithi005.stdout:65 read 0x154c6e8 thru 0x155a536 (0xde4f bytes) 2024-05-11T23:01:31.804 INFO:teuthology.orchestra.run.smithi005.stdout:66 read 0x1e359e5 thru 0x1e437e1 (0xddfd bytes) 2024-05-11T23:01:31.809 INFO:teuthology.orchestra.run.smithi005.stdout:67 trunc from 0x25559fa to 0x70839a0 2024-05-11T23:01:31.814 INFO:teuthology.orchestra.run.smithi005.stdout:69 write 0x1404b1e thru 0x140c366 (0x7849 bytes) 2024-05-11T23:01:31.816 INFO:teuthology.orchestra.run.smithi005.stdout:70 read 0x66e9165 thru 0x66f7aee (0xe98a bytes) 2024-05-11T23:01:31.817 INFO:teuthology.orchestra.run.smithi005.stdout:71 read 0x11a79aa thru 0x11b564d (0xdca4 bytes) 2024-05-11T23:01:31.825 INFO:teuthology.orchestra.run.smithi005.stdout:73 write 0x9e7e40a thru 0x9e80b30 (0x2727 bytes) 2024-05-11T23:01:31.827 INFO:teuthology.orchestra.run.smithi005.stdout:75 punch from 0x1a713a5 to 0x1a76f62, (0x5bbd bytes) 2024-05-11T23:01:31.827 INFO:teuthology.orchestra.run.smithi005.stdout:76 write 0x33f6dda thru 0x33f89f5 (0x1c1c bytes) 2024-05-11T23:01:31.828 INFO:teuthology.orchestra.run.smithi005.stdout:77 read 0x39ac3cb thru 0x39b0c0e (0x4844 bytes) 2024-05-11T23:01:31.828 INFO:teuthology.orchestra.run.smithi005.stdout:78 read 0x49d4e20 thru 0x49dd8c9 (0x8aaa bytes) 2024-05-11T23:01:31.836 INFO:teuthology.orchestra.run.smithi005.stdout:79 write 0xc14b95d thru 0xc158d9a (0xd43e bytes) 2024-05-11T23:01:31.840 INFO:teuthology.orchestra.run.smithi005.stdout:80 writesame 0x8b9f444 thru 0x8b9ff1d data_size 0x3(0xada bytes) 2024-05-11T23:01:31.841 INFO:teuthology.orchestra.run.smithi005.stdout:81 read 0x599483e thru 0x599777b (0x2f3e bytes) 2024-05-11T23:01:31.841 INFO:teuthology.orchestra.run.smithi005.stdout:82 write 0x7922d3f thru 0x792fb4c (0xce0e bytes) 2024-05-11T23:01:31.843 INFO:teuthology.orchestra.run.smithi005.stdout:83 read 0x5331c38 thru 0x533a391 (0x875a bytes) 2024-05-11T23:01:31.843 INFO:teuthology.orchestra.run.smithi005.stdout:84 read 0x5f439f6 thru 0x5f510eb (0xd6f6 bytes) 2024-05-11T23:01:31.844 INFO:teuthology.orchestra.run.smithi005.stdout:85 writesame 0x290258d thru 0x2906ca1 data_size 0x1(0x4715 bytes) 2024-05-11T23:01:31.846 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:31.846 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:31.851 INFO:teuthology.orchestra.run.smithi005.stdout:86 read 0x58248c0 thru 0x58293ee (0x4b2f bytes) 2024-05-11T23:01:31.852 INFO:teuthology.orchestra.run.smithi005.stdout:87 punch from 0xb62ae68 to 0xb6393de, (0xe576 bytes) 2024-05-11T23:01:31.852 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:31.852 INFO:teuthology.orchestra.run.smithi005.stdout:89 punch from 0xa04ad88 to 0xa051b77, (0x6def bytes) 2024-05-11T23:01:31.852 INFO:teuthology.orchestra.run.smithi005.stdout:91 write 0x442b728 thru 0x44341ce (0x8aa7 bytes) 2024-05-11T23:01:31.854 INFO:teuthology.orchestra.run.smithi005.stdout:92 read 0x81b3a9d thru 0x81b5014 (0x1578 bytes) 2024-05-11T23:01:31.854 INFO:teuthology.orchestra.run.smithi005.stdout:93 punch from 0x352bcdd to 0x352fda1, (0x40c4 bytes) 2024-05-11T23:01:31.856 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:01:31.862 INFO:teuthology.orchestra.run.smithi005.stdout:95 trunc from 0xc158d9b to 0x453ec26 2024-05-11T23:01:31.875 INFO:teuthology.orchestra.run.smithi005.stdout:96 read 0x2a23967 thru 0x2a2a843 (0x6edd bytes) 2024-05-11T23:01:31.875 INFO:teuthology.orchestra.run.smithi005.stdout:97 read 0x18ea32 thru 0x19814e (0x971d bytes) 2024-05-11T23:01:31.876 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:31.876 INFO:teuthology.orchestra.run.smithi005.stdout:99 read 0x2bec06c thru 0x2bec305 (0x29a bytes) 2024-05-11T23:01:31.879 INFO:teuthology.orchestra.run.smithi005.stdout:100 trunc from 0x453ec26 to 0x79dfd8f 2024-05-11T23:01:31.880 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:31.880+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:31.885 INFO:teuthology.orchestra.run.smithi005.stdout:101 write 0x24d1be9 thru 0x24df46f (0xd887 bytes) 2024-05-11T23:01:31.886 INFO:teuthology.orchestra.run.smithi005.stdout:103 write 0x5d42a61 thru 0x5d46dcf (0x436f bytes) 2024-05-11T23:01:31.888 INFO:teuthology.orchestra.run.smithi005.stdout:104 punch from 0x48e36b to 0x496ab0, (0x8745 bytes) 2024-05-11T23:01:31.888 INFO:teuthology.orchestra.run.smithi005.stdout:105 read 0x37e3738 thru 0x37e7a13 (0x42dc bytes) 2024-05-11T23:01:31.898 INFO:teuthology.orchestra.run.smithi005.stdout:106 writesame 0xb71eecf thru 0xb7241a5 data_size 0x1(0x52d7 bytes) 2024-05-11T23:01:31.899 INFO:teuthology.orchestra.run.smithi005.stdout:107 punch from 0x97b8378 to 0x97b8d25, (0x9ad bytes) 2024-05-11T23:01:31.899 INFO:teuthology.orchestra.run.smithi005.stdout:108 read 0x3c6da4b thru 0x3c74c42 (0x71f8 bytes) 2024-05-11T23:01:31.899 INFO:teuthology.orchestra.run.smithi005.stdout:111 writesame 0xa081bec thru 0xa085c30 data_size 0x1(0x4045 bytes) 2024-05-11T23:01:31.916 INFO:teuthology.orchestra.run.smithi005.stdout:112 trunc from 0xb7241a6 to 0x334351a 2024-05-11T23:01:31.934 INFO:teuthology.orchestra.run.smithi005.stdout:113 writesame 0x40486b0 thru 0x4058518 data_size 0x1(0xfe69 bytes) 2024-05-11T23:01:31.945 INFO:teuthology.orchestra.run.smithi005.stdout:114 write 0x7d3c135 thru 0x7d48b26 (0xc9f2 bytes) 2024-05-11T23:01:31.946 INFO:teuthology.orchestra.run.smithi005.stdout:115 read 0x70135c6 thru 0x701d5f5 (0xa030 bytes) 2024-05-11T23:01:31.947 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:31.947 INFO:teuthology.orchestra.run.smithi005.stdout:117 write 0x610a4b1 thru 0x6111a72 (0x75c2 bytes) 2024-05-11T23:01:31.948 INFO:teuthology.orchestra.run.smithi005.stdout:119 writesame 0x244a0a2 thru 0x2459d11 data_size 0x38(0xfc70 bytes) 2024-05-11T23:01:31.950 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:31.950 INFO:teuthology.orchestra.run.smithi005.stdout:122 write 0x5e63638 thru 0x5e6447f (0xe48 bytes) 2024-05-11T23:01:31.963 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:01:31.963 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:01:31.963 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-05-11T23:01:31.963 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-11T23:01:31.963 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-11T23:01:31.963 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-05-11T23:01:31.964 INFO:teuthology.orchestra.run.smithi005.stdout:123 write 0xd988562 thru 0xd98d2ca (0x4d69 bytes) 2024-05-11T23:01:31.978 INFO:teuthology.orchestra.run.smithi005.stdout:124 trunc from 0xd98d2cb to 0x3121a8a 2024-05-11T23:01:31.980 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:31.980+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:31.995 INFO:teuthology.orchestra.run.smithi005.stdout:127 punch from 0x1568fe2 to 0x157440c, (0xb42a bytes) 2024-05-11T23:01:32.001 INFO:teuthology.orchestra.run.smithi005.stdout:128 write 0x4418648 thru 0x441ba66 (0x341f bytes) 2024-05-11T23:01:32.002 INFO:teuthology.orchestra.run.smithi005.stdout:132 read 0x41ae460 thru 0x41b516f (0x6d10 bytes) 2024-05-11T23:01:32.003 INFO:teuthology.orchestra.run.smithi005.stdout:133 read 0x343413 thru 0x343ed7 (0xac5 bytes) 2024-05-11T23:01:32.003 INFO:teuthology.orchestra.run.smithi005.stdout:135 write 0x2420442 thru 0x2424170 (0x3d2f bytes) 2024-05-11T23:01:32.005 INFO:teuthology.orchestra.run.smithi005.stdout:137 punch from 0x12675b4 to 0x12718c5, (0xa311 bytes) 2024-05-11T23:01:32.005 INFO:teuthology.orchestra.run.smithi005.stdout:138 read 0x1f8997d thru 0x1f8eb28 (0x51ac bytes) 2024-05-11T23:01:32.008 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:32.008 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:32.015 INFO:teuthology.orchestra.run.smithi005.stdout:139 write 0x877386d thru 0x8779ed8 (0x666c bytes) 2024-05-11T23:01:32.017 INFO:teuthology.orchestra.run.smithi005.stdout:140 read 0x4da6d55 thru 0x4da9287 (0x2533 bytes) 2024-05-11T23:01:32.017 INFO:teuthology.orchestra.run.smithi005.stdout:141 read 0x2b71b08 thru 0x2b7239f (0x898 bytes) 2024-05-11T23:01:32.018 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:32.018 INFO:teuthology.orchestra.run.smithi005.stdout:143 write 0x2d48e78 thru 0x2d4aa22 (0x1bab bytes) 2024-05-11T23:01:32.019 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:01:32.019 INFO:teuthology.orchestra.run.smithi005.stdout:144 write 0x49c541c thru 0x49d380d (0xe3f2 bytes) 2024-05-11T23:01:32.020 INFO:teuthology.orchestra.run.smithi005.stdout:145 write 0x6e776b7 thru 0x6e814d6 (0x9e20 bytes) 2024-05-11T23:01:32.033 INFO:teuthology.orchestra.run.smithi005.stdout:146 write 0xbb78735 thru 0xbb7e55a (0x5e26 bytes) 2024-05-11T23:01:32.034 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:32.034 INFO:teuthology.orchestra.run.smithi005.stdout:149 read 0x82251e7 thru 0x8231248 (0xc062 bytes) 2024-05-11T23:01:32.035 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:32.039 INFO:teuthology.orchestra.run.smithi005.stdout:152 trunc from 0xbb7e55b to 0x845072f 2024-05-11T23:01:32.049 INFO:teuthology.orchestra.run.smithi005.stdout:153 punch from 0x6e14373 to 0x6e1f75c, (0xb3e9 bytes) 2024-05-11T23:01:32.049 INFO:teuthology.orchestra.run.smithi005.stdout:154 read 0x1427e22 thru 0x142a139 (0x2318 bytes) 2024-05-11T23:01:32.050 INFO:teuthology.orchestra.run.smithi005.stdout:155 write 0x532128f thru 0x5324c96 (0x3a08 bytes) 2024-05-11T23:01:32.057 INFO:teuthology.orchestra.run.smithi005.stdout:156 write 0x877f7d5 thru 0x878ec05 (0xf431 bytes) 2024-05-11T23:01:32.059 INFO:teuthology.orchestra.run.smithi005.stdout:157 punch from 0x574538f to 0x574a5ef, (0x5260 bytes) 2024-05-11T23:01:32.059 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:32.059 INFO:teuthology.orchestra.run.smithi005.stdout:159 punch from 0x19fb08a to 0x1a0479e, (0x9714 bytes) 2024-05-11T23:01:32.070 INFO:teuthology.orchestra.run.smithi005.stdout:160 write 0xc588689 thru 0xc588b7e (0x4f6 bytes) 2024-05-11T23:01:32.072 INFO:teuthology.orchestra.run.smithi005.stdout:161 read 0xa6b6450 thru 0xa6bd353 (0x6f04 bytes) 2024-05-11T23:01:32.073 INFO:teuthology.orchestra.run.smithi005.stdout:162 write 0xb976be3 thru 0xb978f07 (0x2325 bytes) 2024-05-11T23:01:32.074 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:32.075 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:32.075 INFO:teuthology.orchestra.run.smithi005.stdout:165 punch from 0x21dc4e6 to 0x21de8ba, (0x23d4 bytes) 2024-05-11T23:01:32.075 INFO:teuthology.orchestra.run.smithi005.stdout:166 write 0x9c183dd thru 0x9c1dd10 (0x5934 bytes) 2024-05-11T23:01:32.076 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:32.076 INFO:teuthology.orchestra.run.smithi005.stdout:168 clone 2 order 18 su 16384 sc 2 2024-05-11T23:01:32.081 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:32.081+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:32.092 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-11T23:01:32.092 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:32.126 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:32.126 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:32.135 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:01:32.182 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:32.182+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:32.281 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:32.281 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:32.283 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:32.283+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:32.289 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:01:32.384 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:32.384+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:32.394 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:32.395 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:32.404 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:01:32.469 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-05-11T23:01:32.485 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:32.485+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:32.499 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:01:32.511 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:32.511 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:32.521 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:01:32.538 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone1 from 0xc588b7f (overlap 0x1cc750c) to 0x0 2024-05-11T23:01:32.591 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-05-11T23:01:32.605 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:32.605+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:32.667 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:32.667 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:32.676 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:01:32.699 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:01:32.699 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:01:32.700 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:32.699+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:32.729 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:01:32.729 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:01:32.749 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:01:32.749 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:01:32.766 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:01:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:01:32.785 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:01:32.785 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:01:32.787 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:32.787+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:32.792 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:32.792 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:32.792 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:01:32.792 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:32.802 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:01:32.803 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:01:32.803 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:01:32.821 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:01:32.821 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:01:32.839 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:01:32.840 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:01:32.865 INFO:teuthology.orchestra.run.smithi005.stdout:169 write 0xc3fac79 thru 0xc3fb014 (0x39c bytes) 2024-05-11T23:01:32.886 INFO:teuthology.orchestra.run.smithi005.stdout:170 trunc from 0xc588b7f to 0x1a67a2a 2024-05-11T23:01:32.888 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:32.888+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:32.900 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:32.918 INFO:teuthology.orchestra.run.smithi005.stdout:173 write 0xd2268fe thru 0xd235895 (0xef98 bytes) 2024-05-11T23:01:32.923 INFO:teuthology.orchestra.run.smithi005.stdout:174 write 0x12c7e16 thru 0x12d0120 (0x830b bytes) 2024-05-11T23:01:32.930 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:32.930 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:32.930 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:01:32.930 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:01:32.930 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:32.930 INFO:teuthology.orchestra.run.smithi005.stdout:175 clone 3 order 20 su 8192 sc 8 2024-05-11T23:01:32.939 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:01:33.024 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:33.024+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:33.090 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:33.090+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:33.091 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:01:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:01:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:01:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:17 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'7, mlcod=16'5)", 2024-05-11T23:01:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.178546+0000", 2024-05-11T23:01:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.911512433, 2024-05-11T23:01:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010843890000000001, 2024-05-11T23:01:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.178546+0000", 2024-05-11T23:01:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.178546+0000", 2024-05-11T23:01:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.178547+0000", 2024-05-11T23:01:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.553e-06 2024-05-11T23:01:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.178560+0000", 2024-05-11T23:01:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2407e-05 2024-05-11T23:01:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.178584+0000", 2024-05-11T23:01:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4606000000000001e-05 2024-05-11T23:01:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.178596+0000", 2024-05-11T23:01:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2249000000000001e-05 2024-05-11T23:01:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.178727+0000", 2024-05-11T23:01:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013057800000000001 2024-05-11T23:01:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.178763+0000", 2024-05-11T23:01:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5482999999999999e-05 2024-05-11T23:01:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.179567+0000", 2024-05-11T23:01:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00080404499999999998 2024-05-11T23:01:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.179630+0000", 2024-05-11T23:01:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3467999999999996e-05 2024-05-11T23:01:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:20 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-05-11T23:01:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.182141+0000", 2024-05-11T23:01:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.907917418, 2024-05-11T23:01:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001126089, 2024-05-11T23:01:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182141+0000", 2024-05-11T23:01:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182141+0000", 2024-05-11T23:01:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182141+0000", 2024-05-11T23:01:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3099999999999997e-07 2024-05-11T23:01:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182311+0000", 2024-05-11T23:01:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000169958 2024-05-11T23:01:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182323+0000", 2024-05-11T23:01:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2089999999999999e-05 2024-05-11T23:01:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182338+0000", 2024-05-11T23:01:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4083000000000001e-05 2024-05-11T23:01:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182383+0000", 2024-05-11T23:01:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5014e-05 2024-05-11T23:01:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182397+0000", 2024-05-11T23:01:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.432e-05 2024-05-11T23:01:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.183231+0000", 2024-05-11T23:01:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00083457099999999999 2024-05-11T23:01:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.183267+0000", 2024-05-11T23:01:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5321999999999998e-05 2024-05-11T23:01:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:21 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'11, mlcod=16'8)", 2024-05-11T23:01:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.182698+0000", 2024-05-11T23:01:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.907360578, 2024-05-11T23:01:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010004790000000001, 2024-05-11T23:01:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182698+0000", 2024-05-11T23:01:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182698+0000", 2024-05-11T23:01:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182698+0000", 2024-05-11T23:01:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9999999999999997e-07 2024-05-11T23:01:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182802+0000", 2024-05-11T23:01:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000103529 2024-05-11T23:01:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182815+0000", 2024-05-11T23:01:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3489000000000001e-05 2024-05-11T23:01:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182822+0000", 2024-05-11T23:01:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3949999999999998e-06 2024-05-11T23:01:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182886+0000", 2024-05-11T23:01:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3962999999999997e-05 2024-05-11T23:01:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.182931+0000", 2024-05-11T23:01:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5018999999999997e-05 2024-05-11T23:01:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.183659+0000", 2024-05-11T23:01:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000728708 2024-05-11T23:01:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.183698+0000", 2024-05-11T23:01:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8776000000000003e-05 2024-05-11T23:01:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:31 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-05-11T23:01:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.190484+0000", 2024-05-11T23:01:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.899573686, 2024-05-11T23:01:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00095951200000000004, 2024-05-11T23:01:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.190484+0000", 2024-05-11T23:01:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.190484+0000", 2024-05-11T23:01:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.190485+0000", 2024-05-11T23:01:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.92e-07 2024-05-11T23:01:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.190491+0000", 2024-05-11T23:01:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5840000000000002e-06 2024-05-11T23:01:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.190499+0000", 2024-05-11T23:01:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6510000000000007e-06 2024-05-11T23:01:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.190507+0000", 2024-05-11T23:01:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.481e-06 2024-05-11T23:01:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.190563+0000", 2024-05-11T23:01:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6045999999999998e-05 2024-05-11T23:01:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.190587+0000", 2024-05-11T23:01:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4031000000000001e-05 2024-05-11T23:01:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.191381+0000", 2024-05-11T23:01:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00079456600000000002 2024-05-11T23:01:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.191444+0000", 2024-05-11T23:01:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2460999999999997e-05 2024-05-11T23:01:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:35 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'17)", 2024-05-11T23:01:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.192947+0000", 2024-05-11T23:01:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.897111145, 2024-05-11T23:01:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010417689999999999, 2024-05-11T23:01:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.192947+0000", 2024-05-11T23:01:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.192947+0000", 2024-05-11T23:01:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.192947+0000", 2024-05-11T23:01:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5200000000000002e-07 2024-05-11T23:01:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.193019+0000", 2024-05-11T23:01:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1215000000000002e-05 2024-05-11T23:01:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.193029+0000", 2024-05-11T23:01:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0268000000000001e-05 2024-05-11T23:01:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.193035+0000", 2024-05-11T23:01:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6250000000000004e-06 2024-05-11T23:01:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.193094+0000", 2024-05-11T23:01:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9891e-05 2024-05-11T23:01:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.193109+0000", 2024-05-11T23:01:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4225e-05 2024-05-11T23:01:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.193952+0000", 2024-05-11T23:01:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00084333800000000001 2024-05-11T23:01:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.193989+0000", 2024-05-11T23:01:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6755000000000003e-05 2024-05-11T23:01:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:55 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'33, mlcod=16'29)", 2024-05-11T23:01:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.209819+0000", 2024-05-11T23:01:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.880238825999999, 2024-05-11T23:01:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001022427, 2024-05-11T23:01:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.209819+0000", 2024-05-11T23:01:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.209819+0000", 2024-05-11T23:01:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.209820+0000", 2024-05-11T23:01:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1800000000000001e-07 2024-05-11T23:01:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.209897+0000", 2024-05-11T23:01:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7195999999999997e-05 2024-05-11T23:01:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.209907+0000", 2024-05-11T23:01:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0054000000000001e-05 2024-05-11T23:01:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.209919+0000", 2024-05-11T23:01:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2038000000000001e-05 2024-05-11T23:01:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.209968+0000", 2024-05-11T23:01:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9197999999999999e-05 2024-05-11T23:01:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.209989+0000", 2024-05-11T23:01:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0905000000000001e-05 2024-05-11T23:01:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.210795+0000", 2024-05-11T23:01:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00080568399999999998 2024-05-11T23:01:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.210842+0000", 2024-05-11T23:01:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6934000000000002e-05 2024-05-11T23:01:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:57 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'35, mlcod=16'33)", 2024-05-11T23:01:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.211853+0000", 2024-05-11T23:01:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.878205176, 2024-05-11T23:01:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00097442099999999995, 2024-05-11T23:01:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.211853+0000", 2024-05-11T23:01:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.211853+0000", 2024-05-11T23:01:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.211853+0000", 2024-05-11T23:01:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2399999999999998e-07 2024-05-11T23:01:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.211928+0000", 2024-05-11T23:01:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4481999999999996e-05 2024-05-11T23:01:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.211940+0000", 2024-05-11T23:01:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2441e-05 2024-05-11T23:01:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.211957+0000", 2024-05-11T23:01:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6922000000000002e-05 2024-05-11T23:01:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.212001+0000", 2024-05-11T23:01:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.371e-05 2024-05-11T23:01:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.212015+0000", 2024-05-11T23:01:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3509000000000001e-05 2024-05-11T23:01:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.212778+0000", 2024-05-11T23:01:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00076356299999999998 2024-05-11T23:01:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.212827+0000", 2024-05-11T23:01:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9270000000000001e-05 2024-05-11T23:01:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:68 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-05-11T23:01:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.219998+0000", 2024-05-11T23:01:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.870060401, 2024-05-11T23:01:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001022579, 2024-05-11T23:01:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.219998+0000", 2024-05-11T23:01:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.219998+0000", 2024-05-11T23:01:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.219998+0000", 2024-05-11T23:01:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1199999999999998e-07 2024-05-11T23:01:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.220004+0000", 2024-05-11T23:01:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4090000000000002e-06 2024-05-11T23:01:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.220012+0000", 2024-05-11T23:01:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4759999999999998e-06 2024-05-11T23:01:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.220016+0000", 2024-05-11T23:01:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2030000000000002e-06 2024-05-11T23:01:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.220049+0000", 2024-05-11T23:01:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3152999999999999e-05 2024-05-11T23:01:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.220061+0000", 2024-05-11T23:01:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1443e-05 2024-05-11T23:01:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.220970+0000", 2024-05-11T23:01:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00090934200000000005 2024-05-11T23:01:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.221020+0000", 2024-05-11T23:01:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0141e-05 2024-05-11T23:01:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:71 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'45, mlcod=16'43)", 2024-05-11T23:01:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.223440+0000", 2024-05-11T23:01:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.866618234000001, 2024-05-11T23:01:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000933791, 2024-05-11T23:01:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.223440+0000", 2024-05-11T23:01:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.223440+0000", 2024-05-11T23:01:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.223441+0000", 2024-05-11T23:01:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2399999999999998e-07 2024-05-11T23:01:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.223495+0000", 2024-05-11T23:01:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4602999999999999e-05 2024-05-11T23:01:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.223506+0000", 2024-05-11T23:01:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0688e-05 2024-05-11T23:01:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.223511+0000", 2024-05-11T23:01:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0520000000000001e-06 2024-05-11T23:01:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.223551+0000", 2024-05-11T23:01:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0528999999999998e-05 2024-05-11T23:01:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.223566+0000", 2024-05-11T23:01:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4381e-05 2024-05-11T23:01:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.224330+0000", 2024-05-11T23:01:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00076462299999999997 2024-05-11T23:01:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.224374+0000", 2024-05-11T23:01:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3291000000000003e-05 2024-05-11T23:01:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:83 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'53, mlcod=16'50)", 2024-05-11T23:01:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.232478+0000", 2024-05-11T23:01:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.857579781, 2024-05-11T23:01:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00097954599999999998, 2024-05-11T23:01:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.232478+0000", 2024-05-11T23:01:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.232478+0000", 2024-05-11T23:01:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.232479+0000", 2024-05-11T23:01:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4999999999999998e-07 2024-05-11T23:01:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.232482+0000", 2024-05-11T23:01:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4450000000000001e-06 2024-05-11T23:01:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.232488+0000", 2024-05-11T23:01:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.964e-06 2024-05-11T23:01:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.232492+0000", 2024-05-11T23:01:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5810000000000001e-06 2024-05-11T23:01:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.232535+0000", 2024-05-11T23:01:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2988999999999998e-05 2024-05-11T23:01:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.232559+0000", 2024-05-11T23:01:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4791999999999999e-05 2024-05-11T23:01:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.233414+0000", 2024-05-11T23:01:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00085448999999999996 2024-05-11T23:01:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.233458+0000", 2024-05-11T23:01:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3934999999999999e-05 2024-05-11T23:01:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.133 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.133 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:86 1.0 e16/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'56, mlcod=16'52)", 2024-05-11T23:01:33.133 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.234135+0000", 2024-05-11T23:01:33.133 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.855923463, 2024-05-11T23:01:33.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00095073800000000002, 2024-05-11T23:01:33.133 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.133 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.234135+0000", 2024-05-11T23:01:33.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.234135+0000", 2024-05-11T23:01:33.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.234135+0000", 2024-05-11T23:01:33.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8400000000000004e-07 2024-05-11T23:01:33.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.234140+0000", 2024-05-11T23:01:33.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1359999999999996e-06 2024-05-11T23:01:33.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.234150+0000", 2024-05-11T23:01:33.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7489999999999993e-06 2024-05-11T23:01:33.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.234162+0000", 2024-05-11T23:01:33.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1299e-05 2024-05-11T23:01:33.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.234215+0000", 2024-05-11T23:01:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3764999999999998e-05 2024-05-11T23:01:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.234230+0000", 2024-05-11T23:01:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.434e-05 2024-05-11T23:01:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.235058+0000", 2024-05-11T23:01:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00082790900000000005 2024-05-11T23:01:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.235085+0000", 2024-05-11T23:01:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7855999999999999e-05 2024-05-11T23:01:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:124 1.0 e16/15 1:30306672:devicehealth::main.db.0000000000000000:head v 16'86, mlcod=16'84)", 2024-05-11T23:01:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:18.793456+0000", 2024-05-11T23:01:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.296602048, 2024-05-11T23:01:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00099138099999999995, 2024-05-11T23:01:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.793456+0000", 2024-05-11T23:01:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.793456+0000", 2024-05-11T23:01:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.793458+0000", 2024-05-11T23:01:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1500000000000002e-06 2024-05-11T23:01:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.793480+0000", 2024-05-11T23:01:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1413e-05 2024-05-11T23:01:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.793510+0000", 2024-05-11T23:01:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9980000000000001e-05 2024-05-11T23:01:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.793525+0000", 2024-05-11T23:01:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5182e-05 2024-05-11T23:01:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.793693+0000", 2024-05-11T23:01:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000168474 2024-05-11T23:01:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.793759+0000", 2024-05-11T23:01:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5754999999999996e-05 2024-05-11T23:01:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.794411+0000", 2024-05-11T23:01:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00065239200000000001 2024-05-11T23:01:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:18.794447+0000", 2024-05-11T23:01:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6035000000000003e-05 2024-05-11T23:01:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:153 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'107, mlcod=16'105)", 2024-05-11T23:01:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:19.320031+0000", 2024-05-11T23:01:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.770026616999999, 2024-05-11T23:01:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00092248199999999997, 2024-05-11T23:01:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320031+0000", 2024-05-11T23:01:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320031+0000", 2024-05-11T23:01:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320032+0000", 2024-05-11T23:01:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8899999999999999e-07 2024-05-11T23:01:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320038+0000", 2024-05-11T23:01:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.733e-06 2024-05-11T23:01:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320048+0000", 2024-05-11T23:01:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8810000000000008e-06 2024-05-11T23:01:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320054+0000", 2024-05-11T23:01:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0950000000000001e-06 2024-05-11T23:01:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320100+0000", 2024-05-11T23:01:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6421000000000002e-05 2024-05-11T23:01:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320116+0000", 2024-05-11T23:01:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5268999999999999e-05 2024-05-11T23:01:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320902+0000", 2024-05-11T23:01:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078689600000000004 2024-05-11T23:01:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320954+0000", 2024-05-11T23:01:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1498000000000001e-05 2024-05-11T23:01:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:154 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'108, mlcod=16'105)", 2024-05-11T23:01:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:19.320542+0000", 2024-05-11T23:01:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.769516596999999, 2024-05-11T23:01:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001033349, 2024-05-11T23:01:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320542+0000", 2024-05-11T23:01:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320542+0000", 2024-05-11T23:01:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320542+0000", 2024-05-11T23:01:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7300000000000002e-07 2024-05-11T23:01:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320546+0000", 2024-05-11T23:01:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8770000000000003e-06 2024-05-11T23:01:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320552+0000", 2024-05-11T23:01:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:01:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320556+0000", 2024-05-11T23:01:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7910000000000002e-06 2024-05-11T23:01:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320604+0000", 2024-05-11T23:01:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7713999999999998e-05 2024-05-11T23:01:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.320626+0000", 2024-05-11T23:01:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2296000000000001e-05 2024-05-11T23:01:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.321530+0000", 2024-05-11T23:01:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00090343499999999996 2024-05-11T23:01:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.321575+0000", 2024-05-11T23:01:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5306999999999998e-05 2024-05-11T23:01:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4215.0:155 1.0 e17/15 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'109, mlcod=16'105)", 2024-05-11T23:01:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:19.321082+0000", 2024-05-11T23:01:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.768976249, 2024-05-11T23:01:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00097425599999999995, 2024-05-11T23:01:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.321082+0000", 2024-05-11T23:01:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.321082+0000", 2024-05-11T23:01:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.321082+0000", 2024-05-11T23:01:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.89e-07 2024-05-11T23:01:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.321152+0000", 2024-05-11T23:01:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9622000000000005e-05 2024-05-11T23:01:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.321160+0000", 2024-05-11T23:01:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3359999999999995e-06 2024-05-11T23:01:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.321166+0000", 2024-05-11T23:01:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3020000000000002e-06 2024-05-11T23:01:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.321199+0000", 2024-05-11T23:01:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3827e-05 2024-05-11T23:01:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.321217+0000", 2024-05-11T23:01:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7316000000000002e-05 2024-05-11T23:01:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.322011+0000", 2024-05-11T23:01:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00079464800000000001 2024-05-11T23:01:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:19.322056+0000", 2024-05-11T23:01:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4815999999999999e-05 2024-05-11T23:01:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-11T23:01:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:29.475475+0000", 2024-05-11T23:01:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.6145829090000001, 2024-05-11T23:01:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0054919060000000004, 2024-05-11T23:01:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 11 2024-05-11T23:01:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.475475+0000", 2024-05-11T23:01:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.475475+0000", 2024-05-11T23:01:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.475477+0000", 2024-05-11T23:01:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0719999999999998e-06 2024-05-11T23:01:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.475486+0000", 2024-05-11T23:01:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7819999999999999e-06 2024-05-11T23:01:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.475506+0000", 2024-05-11T23:01:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0293999999999999e-05 2024-05-11T23:01:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.475519+0000", 2024-05-11T23:01:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2298e-05 2024-05-11T23:01:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.475592+0000", 2024-05-11T23:01:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3609000000000006e-05 2024-05-11T23:01:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.475758+0000", 2024-05-11T23:01:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016588700000000001 2024-05-11T23:01:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 5", 2024-05-11T23:01:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.476005+0000", 2024-05-11T23:01:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00024689200000000002 2024-05-11T23:01:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.476778+0000", 2024-05-11T23:01:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00077338299999999999 2024-05-11T23:01:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.480891+0000", 2024-05-11T23:01:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.004112101 2024-05-11T23:01:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.480935+0000", 2024-05-11T23:01:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4351000000000003e-05 2024-05-11T23:01:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.480967+0000", 2024-05-11T23:01:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2237000000000003e-05 2024-05-11T23:01:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:12 3.6 3:6451e15f:::rbd_header.111e479bff4d:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-11T23:01:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:29.481063+0000", 2024-05-11T23:01:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.6089955160000002, 2024-05-11T23:01:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001330003, 2024-05-11T23:01:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 12 2024-05-11T23:01:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.481063+0000", 2024-05-11T23:01:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.481063+0000", 2024-05-11T23:01:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.481063+0000", 2024-05-11T23:01:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.23e-07 2024-05-11T23:01:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.481067+0000", 2024-05-11T23:01:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8920000000000002e-06 2024-05-11T23:01:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.481076+0000", 2024-05-11T23:01:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5450000000000003e-06 2024-05-11T23:01:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.481081+0000", 2024-05-11T23:01:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8389999999999997e-06 2024-05-11T23:01:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.481124+0000", 2024-05-11T23:01:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3770999999999998e-05 2024-05-11T23:01:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.481205+0000", 2024-05-11T23:01:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0790000000000004e-05 2024-05-11T23:01:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-05-11T23:01:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.481320+0000", 2024-05-11T23:01:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011520500000000001 2024-05-11T23:01:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.481768+0000", 2024-05-11T23:01:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00044742499999999999 2024-05-11T23:01:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.482339+0000", 2024-05-11T23:01:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00057135300000000003 2024-05-11T23:01:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.482368+0000", 2024-05-11T23:01:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8411e-05 2024-05-11T23:01:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.482393+0000", 2024-05-11T23:01:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4949000000000001e-05 2024-05-11T23:01:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:13 3.1 3:8452a023:::rbd_object_map.111e479bff4d:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-11T23:01:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:29.482491+0000", 2024-05-11T23:01:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.6075668309999998, 2024-05-11T23:01:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001237649, 2024-05-11T23:01:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 13 2024-05-11T23:01:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.482491+0000", 2024-05-11T23:01:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.482491+0000", 2024-05-11T23:01:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.482493+0000", 2024-05-11T23:01:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.282e-06 2024-05-11T23:01:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.482499+0000", 2024-05-11T23:01:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3749999999999999e-06 2024-05-11T23:01:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.482512+0000", 2024-05-11T23:01:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3239e-05 2024-05-11T23:01:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.482521+0000", 2024-05-11T23:01:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9930000000000001e-06 2024-05-11T23:01:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.482570+0000", 2024-05-11T23:01:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8359999999999998e-05 2024-05-11T23:01:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.482652+0000", 2024-05-11T23:01:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1987e-05 2024-05-11T23:01:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.482761+0000", 2024-05-11T23:01:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010920399999999999 2024-05-11T23:01:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.483316+0000", 2024-05-11T23:01:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00055569099999999998 2024-05-11T23:01:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.483671+0000", 2024-05-11T23:01:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00035459599999999999 2024-05-11T23:01:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.483702+0000", 2024-05-11T23:01:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1222e-05 2024-05-11T23:01:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.483729+0000", 2024-05-11T23:01:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6699999999999998e-05 2024-05-11T23:01:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25 3.6 3:6451e15f:::rbd_header.111e479bff4d:head [watch watch cookie 139809507083296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-11T23:01:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:29.491303+0000", 2024-05-11T23:01:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.5987551079999998, 2024-05-11T23:01:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010794450000000001, 2024-05-11T23:01:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25 2024-05-11T23:01:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.491303+0000", 2024-05-11T23:01:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.491303+0000", 2024-05-11T23:01:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.491304+0000", 2024-05-11T23:01:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0399999999999996e-07 2024-05-11T23:01:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.491307+0000", 2024-05-11T23:01:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5310000000000002e-06 2024-05-11T23:01:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.491315+0000", 2024-05-11T23:01:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0330000000000008e-06 2024-05-11T23:01:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.491320+0000", 2024-05-11T23:01:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0209999999999999e-06 2024-05-11T23:01:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.491354+0000", 2024-05-11T23:01:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4236999999999998e-05 2024-05-11T23:01:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.491403+0000", 2024-05-11T23:01:33.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8702000000000002e-05 2024-05-11T23:01:33.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-05-11T23:01:33.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.491498+0000", 2024-05-11T23:01:33.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4647000000000004e-05 2024-05-11T23:01:33.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:33.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.491980+0000", 2024-05-11T23:01:33.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00048232100000000002 2024-05-11T23:01:33.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:33.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.492241+0000", 2024-05-11T23:01:33.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00026103200000000002 2024-05-11T23:01:33.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.492300+0000", 2024-05-11T23:01:33.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8888999999999998e-05 2024-05-11T23:01:33.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:29.492382+0000", 2024-05-11T23:01:33.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2528000000000007e-05 2024-05-11T23:01:33.176 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.176 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.177 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.177 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:1332 3.6 3:67604ae5:::rbd_data.111e50384849.0000000000000069:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 211498] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-11T23:01:33.177 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:32.895447+0000", 2024-05-11T23:01:33.177 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.19461094800000001, 2024-05-11T23:01:33.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0012554840000000001, 2024-05-11T23:01:33.177 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.177 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.178 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.178 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:33.178 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:33.178 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:33.178 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1332 2024-05-11T23:01:33.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.178 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.895447+0000", 2024-05-11T23:01:33.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.895447+0000", 2024-05-11T23:01:33.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.895449+0000", 2024-05-11T23:01:33.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5099999999999999e-06 2024-05-11T23:01:33.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.895457+0000", 2024-05-11T23:01:33.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3699999999999995e-06 2024-05-11T23:01:33.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.895473+0000", 2024-05-11T23:01:33.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6240999999999999e-05 2024-05-11T23:01:33.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.895480+0000", 2024-05-11T23:01:33.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2119999999999997e-06 2024-05-11T23:01:33.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.895540+0000", 2024-05-11T23:01:33.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9274000000000002e-05 2024-05-11T23:01:33.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.895658+0000", 2024-05-11T23:01:33.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011805400000000001 2024-05-11T23:01:33.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-05-11T23:01:33.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.895776+0000", 2024-05-11T23:01:33.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117867 2024-05-11T23:01:33.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:33.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.896391+0000", 2024-05-11T23:01:33.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00061534899999999995 2024-05-11T23:01:33.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:33.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.896634+0000", 2024-05-11T23:01:33.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000243212 2024-05-11T23:01:33.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.896668+0000", 2024-05-11T23:01:33.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3596000000000002e-05 2024-05-11T23:01:33.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.896703+0000", 2024-05-11T23:01:33.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4798999999999997e-05 2024-05-11T23:01:33.183 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.183 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.183 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.183 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.183 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.183 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:33.185 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:01:33.226 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:33.226+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:33.291 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:33.291+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:33.334 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:33.335 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:33.335 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:01:33.335 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:33.343 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:01:33.392 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:33.392+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:33.492 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:33.492+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:33.497 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:33.497 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:33.497 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:01:33.497 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:01:33.497 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:33.507 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:01:33.540 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone2 from 0xd235896 (overlap 0x1a67a2a) to 0xfd7c7f 2024-05-11T23:01:33.594 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:33.593+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:33.649 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:33.649 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:01:33.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:01:33.649 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:01:33.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.650 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:1386 3.4 3:20d95d3e:::rbd_data.111e7a935520.000000000000000b:4 [read 1622016~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-05-11T23:01:33.650 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:32.925105+0000", 2024-05-11T23:01:33.650 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.72325908999999999, 2024-05-11T23:01:33.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00099423400000000009, 2024-05-11T23:01:33.650 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.650 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:33.650 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:33.650 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:33.650 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:33.650 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1386 2024-05-11T23:01:33.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.650 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.925105+0000", 2024-05-11T23:01:33.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.925105+0000", 2024-05-11T23:01:33.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.925106+0000", 2024-05-11T23:01:33.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2219999999999999e-06 2024-05-11T23:01:33.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.925112+0000", 2024-05-11T23:01:33.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6840000000000001e-06 2024-05-11T23:01:33.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.925125+0000", 2024-05-11T23:01:33.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3586e-05 2024-05-11T23:01:33.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.925134+0000", 2024-05-11T23:01:33.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5480000000000002e-06 2024-05-11T23:01:33.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.925924+0000", 2024-05-11T23:01:33.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078968500000000004 2024-05-11T23:01:33.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.926001+0000", 2024-05-11T23:01:33.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7245999999999999e-05 2024-05-11T23:01:33.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.926099+0000", 2024-05-11T23:01:33.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8263000000000004e-05 2024-05-11T23:01:33.653 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.653 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.654 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.654 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:1390 3.0 3:03c4b8c1:::rbd_data.111e50384849.000000000000004b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 146966~16874] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-11T23:01:33.654 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:32.929862+0000", 2024-05-11T23:01:33.654 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.71850208400000004, 2024-05-11T23:01:33.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001632243, 2024-05-11T23:01:33.654 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.654 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.654 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.654 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:33.654 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:33.654 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:33.654 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1390 2024-05-11T23:01:33.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.655 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.929862+0000", 2024-05-11T23:01:33.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.929862+0000", 2024-05-11T23:01:33.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.929863+0000", 2024-05-11T23:01:33.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6400000000000001e-07 2024-05-11T23:01:33.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.929873+0000", 2024-05-11T23:01:33.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0039e-05 2024-05-11T23:01:33.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.929884+0000", 2024-05-11T23:01:33.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0801999999999999e-05 2024-05-11T23:01:33.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.929890+0000", 2024-05-11T23:01:33.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.224e-06 2024-05-11T23:01:33.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.929948+0000", 2024-05-11T23:01:33.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7794000000000003e-05 2024-05-11T23:01:33.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.930094+0000", 2024-05-11T23:01:33.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014629500000000001 2024-05-11T23:01:33.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-05-11T23:01:33.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.930326+0000", 2024-05-11T23:01:33.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000232494 2024-05-11T23:01:33.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:33.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.931070+0000", 2024-05-11T23:01:33.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00074400200000000001 2024-05-11T23:01:33.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:33.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.931387+0000", 2024-05-11T23:01:33.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00031613399999999998 2024-05-11T23:01:33.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.931449+0000", 2024-05-11T23:01:33.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1971999999999996e-05 2024-05-11T23:01:33.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.931494+0000", 2024-05-11T23:01:33.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5623e-05 2024-05-11T23:01:33.659 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.659 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.659 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.659 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:1427 3.19 3:98c091ca:::rbd_header.111e7a935520:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:33.659 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.500853+0000", 2024-05-11T23:01:33.659 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.147510735, 2024-05-11T23:01:33.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019582919999999999, 2024-05-11T23:01:33.659 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.660 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.660 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:33.660 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:33.660 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:33.660 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:33.660 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1427 2024-05-11T23:01:33.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.660 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.500853+0000", 2024-05-11T23:01:33.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.500853+0000", 2024-05-11T23:01:33.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.500856+0000", 2024-05-11T23:01:33.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2409999999999998e-06 2024-05-11T23:01:33.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.500865+0000", 2024-05-11T23:01:33.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8789999999999998e-06 2024-05-11T23:01:33.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.500888+0000", 2024-05-11T23:01:33.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2297999999999999e-05 2024-05-11T23:01:33.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.662 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.500902+0000", 2024-05-11T23:01:33.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4671999999999999e-05 2024-05-11T23:01:33.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.662 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.500972+0000", 2024-05-11T23:01:33.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9839000000000007e-05 2024-05-11T23:01:33.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op must wait for map", 2024-05-11T23:01:33.662 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.500977+0000", 2024-05-11T23:01:33.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3320000000000002e-06 2024-05-11T23:01:33.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.502613+0000", 2024-05-11T23:01:33.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0016361329999999999 2024-05-11T23:01:33.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.502679+0000", 2024-05-11T23:01:33.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6223999999999995e-05 2024-05-11T23:01:33.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.502812+0000", 2024-05-11T23:01:33.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000132674 2024-05-11T23:01:33.664 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.664 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.664 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.664 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:1497 3.19 3:9a9a8d6f:::rbd_data.111e50384849.0000000000000047:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:33.664 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.543359+0000", 2024-05-11T23:01:33.664 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.10500538399999999, 2024-05-11T23:01:33.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002046431, 2024-05-11T23:01:33.664 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.664 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.664 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:33.664 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:33.664 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:33.664 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:33.665 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1497 2024-05-11T23:01:33.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.665 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.543359+0000", 2024-05-11T23:01:33.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.543359+0000", 2024-05-11T23:01:33.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.543360+0000", 2024-05-11T23:01:33.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.457e-06 2024-05-11T23:01:33.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.543374+0000", 2024-05-11T23:01:33.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4239e-05 2024-05-11T23:01:33.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.543391+0000", 2024-05-11T23:01:33.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6790999999999999e-05 2024-05-11T23:01:33.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.543399+0000", 2024-05-11T23:01:33.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5299999999999999e-06 2024-05-11T23:01:33.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.543464+0000", 2024-05-11T23:01:33.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5569999999999997e-05 2024-05-11T23:01:33.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.543613+0000", 2024-05-11T23:01:33.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000148868 2024-05-11T23:01:33.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.545405+0000", 2024-05-11T23:01:33.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001791976 2024-05-11T23:01:33.668 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.668 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.668 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.668 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:1515 3.4 3:20d95d3e:::rbd_data.111e7a935520.000000000000000b:4 [read 1409024~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:33.668 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.545364+0000", 2024-05-11T23:01:33.668 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.102999696, 2024-05-11T23:01:33.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001012184, 2024-05-11T23:01:33.669 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.669 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.669 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:33.669 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:33.669 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:33.669 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:33.669 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1515 2024-05-11T23:01:33.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.669 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.545364+0000", 2024-05-11T23:01:33.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.545364+0000", 2024-05-11T23:01:33.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.545365+0000", 2024-05-11T23:01:33.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2200000000000004e-07 2024-05-11T23:01:33.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.545375+0000", 2024-05-11T23:01:33.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0183000000000001e-05 2024-05-11T23:01:33.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.545385+0000", 2024-05-11T23:01:33.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4150000000000005e-06 2024-05-11T23:01:33.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.545392+0000", 2024-05-11T23:01:33.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0049999999999998e-06 2024-05-11T23:01:33.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.545688+0000", 2024-05-11T23:01:33.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00029685399999999999 2024-05-11T23:01:33.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.545788+0000", 2024-05-11T23:01:33.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9544999999999996e-05 2024-05-11T23:01:33.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.546377+0000", 2024-05-11T23:01:33.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00058856000000000002 2024-05-11T23:01:33.672 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.672 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.672 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.672 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:1595 3.19 3:9a7df64e:::rbd_data.111e7a935520.0000000000000007:4 [read 1474560~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:33.673 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.547753+0000", 2024-05-11T23:01:33.673 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.100611349, 2024-05-11T23:01:33.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0011394420000000001, 2024-05-11T23:01:33.673 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.673 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.673 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:33.673 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:33.673 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:33.673 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:33.673 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1595 2024-05-11T23:01:33.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.673 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.547753+0000", 2024-05-11T23:01:33.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.547753+0000", 2024-05-11T23:01:33.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.547753+0000", 2024-05-11T23:01:33.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0899999999999997e-07 2024-05-11T23:01:33.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.547755+0000", 2024-05-11T23:01:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.968e-06 2024-05-11T23:01:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.547761+0000", 2024-05-11T23:01:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1890000000000004e-06 2024-05-11T23:01:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.547766+0000", 2024-05-11T23:01:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3369999999999999e-06 2024-05-11T23:01:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.675 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.548698+0000", 2024-05-11T23:01:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00093180699999999999 2024-05-11T23:01:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.548781+0000", 2024-05-11T23:01:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2903000000000002e-05 2024-05-11T23:01:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.548892+0000", 2024-05-11T23:01:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011092899999999999 2024-05-11T23:01:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:1669 3.19 3:9a9a8d6f:::rbd_data.111e50384849.0000000000000047:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.553509+0000", 2024-05-11T23:01:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.094854780999999999, 2024-05-11T23:01:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0023001509999999998, 2024-05-11T23:01:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1669 2024-05-11T23:01:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.553509+0000", 2024-05-11T23:01:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.553509+0000", 2024-05-11T23:01:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.553511+0000", 2024-05-11T23:01:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8419999999999999e-06 2024-05-11T23:01:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.553518+0000", 2024-05-11T23:01:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5910000000000001e-06 2024-05-11T23:01:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.553534+0000", 2024-05-11T23:01:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6495e-05 2024-05-11T23:01:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.553543+0000", 2024-05-11T23:01:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3629999999999994e-06 2024-05-11T23:01:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.553613+0000", 2024-05-11T23:01:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0116000000000004e-05 2024-05-11T23:01:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.553673+0000", 2024-05-11T23:01:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0012000000000001e-05 2024-05-11T23:01:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-05-11T23:01:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.553858+0000", 2024-05-11T23:01:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018535000000000001 2024-05-11T23:01:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.554681+0000", 2024-05-11T23:01:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00082336300000000002 2024-05-11T23:01:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.555730+0000", 2024-05-11T23:01:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010490460000000001 2024-05-11T23:01:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.555766+0000", 2024-05-11T23:01:33.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6047999999999998e-05 2024-05-11T23:01:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.555809+0000", 2024-05-11T23:01:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2925e-05 2024-05-11T23:01:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:1670 3.0 3:03c4b8c1:::rbd_data.111e50384849.000000000000004b:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.553563+0000", 2024-05-11T23:01:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.094800572, 2024-05-11T23:01:33.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001906048, 2024-05-11T23:01:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1670 2024-05-11T23:01:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.553563+0000", 2024-05-11T23:01:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.553563+0000", 2024-05-11T23:01:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.553564+0000", 2024-05-11T23:01:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2500000000000005e-07 2024-05-11T23:01:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.553568+0000", 2024-05-11T23:01:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.455e-06 2024-05-11T23:01:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.553578+0000", 2024-05-11T23:01:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0242e-05 2024-05-11T23:01:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.553584+0000", 2024-05-11T23:01:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2600000000000002e-06 2024-05-11T23:01:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.553639+0000", 2024-05-11T23:01:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4410999999999998e-05 2024-05-11T23:01:33.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.553692+0000", 2024-05-11T23:01:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3038000000000003e-05 2024-05-11T23:01:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-05-11T23:01:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.553908+0000", 2024-05-11T23:01:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000216145 2024-05-11T23:01:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.555252+0000", 2024-05-11T23:01:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001344661 2024-05-11T23:01:33.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.555365+0000", 2024-05-11T23:01:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000113013 2024-05-11T23:01:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.555420+0000", 2024-05-11T23:01:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4178999999999997e-05 2024-05-11T23:01:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.555470+0000", 2024-05-11T23:01:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9919000000000001e-05 2024-05-11T23:01:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "pg_update_log_missing(3.6 epoch 25/19 rep_tid 131 entries 25'21 (0'0) error 3:6048c4f2:::rbd_data.111e50384849.0000000000000052:head by client.4382.0:1684 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 24'19)", 2024-05-11T23:01:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.558427+0000", 2024-05-11T23:01:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.089937013999999996, 2024-05-11T23:01:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0022494709999999998, 2024-05-11T23:01:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.558427+0000", 2024-05-11T23:01:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.558427+0000", 2024-05-11T23:01:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.558427+0000", 2024-05-11T23:01:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7500000000000001e-07 2024-05-11T23:01:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.558434+0000", 2024-05-11T23:01:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7039999999999997e-06 2024-05-11T23:01:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.558455+0000", 2024-05-11T23:01:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0961000000000001e-05 2024-05-11T23:01:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.558472+0000", 2024-05-11T23:01:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7181e-05 2024-05-11T23:01:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.558615+0000", 2024-05-11T23:01:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014246899999999999 2024-05-11T23:01:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.560676+0000", 2024-05-11T23:01:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0020618809999999998 2024-05-11T23:01:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:1859 3.4 3:21f195d1:::rbd_data.111e50384849.0000000000000059:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.568951+0000", 2024-05-11T23:01:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.079413281000000002, 2024-05-11T23:01:33.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015317620000000001, 2024-05-11T23:01:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1859 2024-05-11T23:01:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.568951+0000", 2024-05-11T23:01:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.568951+0000", 2024-05-11T23:01:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.568952+0000", 2024-05-11T23:01:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8700000000000004e-07 2024-05-11T23:01:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.568956+0000", 2024-05-11T23:01:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7729999999999999e-06 2024-05-11T23:01:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.568969+0000", 2024-05-11T23:01:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.277e-05 2024-05-11T23:01:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.568977+0000", 2024-05-11T23:01:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5739999999999998e-06 2024-05-11T23:01:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569036+0000", 2024-05-11T23:01:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9169999999999998e-05 2024-05-11T23:01:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569184+0000", 2024-05-11T23:01:33.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014824200000000001 2024-05-11T23:01:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.570483+0000", 2024-05-11T23:01:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0012983459999999999 2024-05-11T23:01:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:1857 3.8 e25/19 3:13f935e7:::rbd_data.111e50384849.0000000000000054:head v 25'19, mlcod=24'17)", 2024-05-11T23:01:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.569228+0000", 2024-05-11T23:01:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.079135664999999994, 2024-05-11T23:01:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0011254419999999999, 2024-05-11T23:01:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.695 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569228+0000", 2024-05-11T23:01:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569228+0000", 2024-05-11T23:01:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569229+0000", 2024-05-11T23:01:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8000000000000005e-07 2024-05-11T23:01:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569234+0000", 2024-05-11T23:01:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.339e-06 2024-05-11T23:01:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569245+0000", 2024-05-11T23:01:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0858e-05 2024-05-11T23:01:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569252+0000", 2024-05-11T23:01:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4570000000000002e-06 2024-05-11T23:01:33.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569322+0000", 2024-05-11T23:01:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9931999999999994e-05 2024-05-11T23:01:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569342+0000", 2024-05-11T23:01:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9996999999999998e-05 2024-05-11T23:01:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.570302+0000", 2024-05-11T23:01:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00096048300000000004 2024-05-11T23:01:33.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.570354+0000", 2024-05-11T23:01:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1595999999999999e-05 2024-05-11T23:01:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:1913 3.4 3:21f195d1:::rbd_data.111e50384849.0000000000000059:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.573022+0000", 2024-05-11T23:01:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.075341722, 2024-05-11T23:01:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015164950000000001, 2024-05-11T23:01:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.699 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1913 2024-05-11T23:01:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.573022+0000", 2024-05-11T23:01:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.573022+0000", 2024-05-11T23:01:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.573023+0000", 2024-05-11T23:01:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.001e-06 2024-05-11T23:01:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.573032+0000", 2024-05-11T23:01:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.901e-06 2024-05-11T23:01:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.573048+0000", 2024-05-11T23:01:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5342e-05 2024-05-11T23:01:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.573056+0000", 2024-05-11T23:01:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0669999999999992e-06 2024-05-11T23:01:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.573126+0000", 2024-05-11T23:01:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0695e-05 2024-05-11T23:01:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.573205+0000", 2024-05-11T23:01:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8588e-05 2024-05-11T23:01:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-05-11T23:01:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.573498+0000", 2024-05-11T23:01:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00029291099999999997 2024-05-11T23:01:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.574173+0000", 2024-05-11T23:01:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00067565699999999997 2024-05-11T23:01:33.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.574428+0000", 2024-05-11T23:01:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000255009 2024-05-11T23:01:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.574494+0000", 2024-05-11T23:01:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5767999999999998e-05 2024-05-11T23:01:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.574539+0000", 2024-05-11T23:01:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4555999999999998e-05 2024-05-11T23:01:33.704 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:1967 3.4 3:25486f9d:::rbd_data.111e50384849.000000000000005e:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.577782+0000", 2024-05-11T23:01:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.070582275, 2024-05-11T23:01:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0013115360000000001, 2024-05-11T23:01:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:33.705 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1967 2024-05-11T23:01:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.577782+0000", 2024-05-11T23:01:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.577782+0000", 2024-05-11T23:01:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.577783+0000", 2024-05-11T23:01:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1299999999999999e-07 2024-05-11T23:01:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.577787+0000", 2024-05-11T23:01:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9330000000000003e-06 2024-05-11T23:01:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.577797+0000", 2024-05-11T23:01:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0698e-05 2024-05-11T23:01:33.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.577805+0000", 2024-05-11T23:01:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5170000000000002e-06 2024-05-11T23:01:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.577863+0000", 2024-05-11T23:01:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8019999999999997e-05 2024-05-11T23:01:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578010+0000", 2024-05-11T23:01:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014760500000000001 2024-05-11T23:01:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.579093+0000", 2024-05-11T23:01:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010829500000000001 2024-05-11T23:01:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "pg_update_log_missing(3.6 epoch 25/19 rep_tid 143 entries 25'23 (0'0) error 3:639efcc8:::rbd_data.111e50384849.000000000000005f:head by client.4382.0:1969 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'21)", 2024-05-11T23:01:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.578170+0000", 2024-05-11T23:01:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.070194090000000001, 2024-05-11T23:01:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010232139999999999, 2024-05-11T23:01:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578170+0000", 2024-05-11T23:01:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578170+0000", 2024-05-11T23:01:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578171+0000", 2024-05-11T23:01:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1100000000000004e-07 2024-05-11T23:01:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578181+0000", 2024-05-11T23:01:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0166000000000001e-05 2024-05-11T23:01:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578203+0000", 2024-05-11T23:01:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2323e-05 2024-05-11T23:01:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578222+0000", 2024-05-11T23:01:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8471999999999999e-05 2024-05-11T23:01:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578339+0000", 2024-05-11T23:01:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000117625 2024-05-11T23:01:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.579193+0000", 2024-05-11T23:01:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00085371700000000004 2024-05-11T23:01:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:2022 3.0 3:0097cebb:::rbd_data.111e50384849.0000000000000062:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.582325+0000", 2024-05-11T23:01:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.066039064999999994, 2024-05-11T23:01:33.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0023856010000000002, 2024-05-11T23:01:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2022 2024-05-11T23:01:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.582325+0000", 2024-05-11T23:01:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.582325+0000", 2024-05-11T23:01:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.582326+0000", 2024-05-11T23:01:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.02e-06 2024-05-11T23:01:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.582331+0000", 2024-05-11T23:01:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9559999999999996e-06 2024-05-11T23:01:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.582347+0000", 2024-05-11T23:01:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6271999999999999e-05 2024-05-11T23:01:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.582356+0000", 2024-05-11T23:01:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3310000000000004e-06 2024-05-11T23:01:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.582412+0000", 2024-05-11T23:01:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6736e-05 2024-05-11T23:01:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.582542+0000", 2024-05-11T23:01:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012922500000000001 2024-05-11T23:01:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.584711+0000", 2024-05-11T23:01:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021690609999999999 2024-05-11T23:01:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:2023 3.4 3:25486f9d:::rbd_data.111e50384849.000000000000005e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.582374+0000", 2024-05-11T23:01:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.065989609000000005, 2024-05-11T23:01:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0024117629999999999, 2024-05-11T23:01:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.717 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2023 2024-05-11T23:01:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.582374+0000", 2024-05-11T23:01:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.582374+0000", 2024-05-11T23:01:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.582375+0000", 2024-05-11T23:01:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0100000000000005e-07 2024-05-11T23:01:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.582380+0000", 2024-05-11T23:01:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6020000000000002e-06 2024-05-11T23:01:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.582389+0000", 2024-05-11T23:01:33.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9479999999999997e-06 2024-05-11T23:01:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.582395+0000", 2024-05-11T23:01:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1099999999999999e-06 2024-05-11T23:01:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.582451+0000", 2024-05-11T23:01:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6487000000000003e-05 2024-05-11T23:01:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.582533+0000", 2024-05-11T23:01:33.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1434999999999995e-05 2024-05-11T23:01:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-05-11T23:01:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.582742+0000", 2024-05-11T23:01:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020987600000000001 2024-05-11T23:01:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.584642+0000", 2024-05-11T23:01:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018992 2024-05-11T23:01:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.584701+0000", 2024-05-11T23:01:33.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8893e-05 2024-05-11T23:01:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.584748+0000", 2024-05-11T23:01:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.71e-05 2024-05-11T23:01:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.584786+0000", 2024-05-11T23:01:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8510999999999998e-05 2024-05-11T23:01:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:2025 3.6 e25/19 3:639efcc8:::rbd_data.111e50384849.000000000000005f:head v 25'24, mlcod=25'22)", 2024-05-11T23:01:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.583300+0000", 2024-05-11T23:01:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.065064171000000004, 2024-05-11T23:01:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0013822890000000001, 2024-05-11T23:01:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583300+0000", 2024-05-11T23:01:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583300+0000", 2024-05-11T23:01:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583301+0000", 2024-05-11T23:01:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7000000000000003e-07 2024-05-11T23:01:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583309+0000", 2024-05-11T23:01:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6500000000000002e-06 2024-05-11T23:01:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583319+0000", 2024-05-11T23:01:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4360000000000008e-06 2024-05-11T23:01:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583326+0000", 2024-05-11T23:01:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6939999999999998e-06 2024-05-11T23:01:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583402+0000", 2024-05-11T23:01:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6939000000000003e-05 2024-05-11T23:01:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583423+0000", 2024-05-11T23:01:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0630999999999999e-05 2024-05-11T23:01:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.584639+0000", 2024-05-11T23:01:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0012156000000000001 2024-05-11T23:01:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.584682+0000", 2024-05-11T23:01:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3469e-05 2024-05-11T23:01:33.726 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:2112 3.0 3:0097cebb:::rbd_data.111e50384849.0000000000000062:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.587759+0000", 2024-05-11T23:01:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.060605353000000001, 2024-05-11T23:01:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017102479999999999, 2024-05-11T23:01:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2112 2024-05-11T23:01:33.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.587759+0000", 2024-05-11T23:01:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.587759+0000", 2024-05-11T23:01:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.587759+0000", 2024-05-11T23:01:33.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2200000000000004e-07 2024-05-11T23:01:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.587766+0000", 2024-05-11T23:01:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7719999999999997e-06 2024-05-11T23:01:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.587773+0000", 2024-05-11T23:01:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4959999999999999e-06 2024-05-11T23:01:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.587779+0000", 2024-05-11T23:01:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.48e-06 2024-05-11T23:01:33.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.587832+0000", 2024-05-11T23:01:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2772999999999998e-05 2024-05-11T23:01:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.587920+0000", 2024-05-11T23:01:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8441000000000002e-05 2024-05-11T23:01:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-05-11T23:01:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.588130+0000", 2024-05-11T23:01:33.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020935699999999999 2024-05-11T23:01:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.589102+0000", 2024-05-11T23:01:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00097261199999999998 2024-05-11T23:01:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.589373+0000", 2024-05-11T23:01:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00027058900000000001 2024-05-11T23:01:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.589429+0000", 2024-05-11T23:01:33.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6749999999999997e-05 2024-05-11T23:01:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.589469+0000", 2024-05-11T23:01:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9456000000000003e-05 2024-05-11T23:01:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:2137 3.6 e25/19 3:67604ae5:::rbd_data.111e50384849.0000000000000069:head v 25'26, mlcod=25'23)", 2024-05-11T23:01:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.592629+0000", 2024-05-11T23:01:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.055735492999999997, 2024-05-11T23:01:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010406969999999999, 2024-05-11T23:01:33.732 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.592629+0000", 2024-05-11T23:01:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.592629+0000", 2024-05-11T23:01:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.592629+0000", 2024-05-11T23:01:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5099999999999998e-07 2024-05-11T23:01:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.592637+0000", 2024-05-11T23:01:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0749999999999998e-06 2024-05-11T23:01:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.592651+0000", 2024-05-11T23:01:33.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3155e-05 2024-05-11T23:01:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.592659+0000", 2024-05-11T23:01:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9349999999999994e-06 2024-05-11T23:01:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.592735+0000", 2024-05-11T23:01:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6296000000000003e-05 2024-05-11T23:01:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.735 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.592756+0000", 2024-05-11T23:01:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.103e-05 2024-05-11T23:01:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.593627+0000", 2024-05-11T23:01:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000870826 2024-05-11T23:01:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.593669+0000", 2024-05-11T23:01:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2528999999999999e-05 2024-05-11T23:01:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.736 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:2224 3.19 3:99f4f060:::rbd_data.111e50384849.000000000000003f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 179327] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.611921+0000", 2024-05-11T23:01:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.036442883000000002, 2024-05-11T23:01:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0012068000000000001, 2024-05-11T23:01:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2224 2024-05-11T23:01:33.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.611921+0000", 2024-05-11T23:01:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.611921+0000", 2024-05-11T23:01:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.611922+0000", 2024-05-11T23:01:33.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.088e-06 2024-05-11T23:01:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.611927+0000", 2024-05-11T23:01:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0749999999999997e-06 2024-05-11T23:01:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.611938+0000", 2024-05-11T23:01:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0859999999999999e-05 2024-05-11T23:01:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.739 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.611946+0000", 2024-05-11T23:01:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1079999999999993e-06 2024-05-11T23:01:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.611998+0000", 2024-05-11T23:01:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1490999999999999e-05 2024-05-11T23:01:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.612107+0000", 2024-05-11T23:01:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010897599999999999 2024-05-11T23:01:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 3", 2024-05-11T23:01:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.612236+0000", 2024-05-11T23:01:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012892399999999999 2024-05-11T23:01:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.612755+0000", 2024-05-11T23:01:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000519624 2024-05-11T23:01:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.613035+0000", 2024-05-11T23:01:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000279616 2024-05-11T23:01:33.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.613091+0000", 2024-05-11T23:01:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5896000000000002e-05 2024-05-11T23:01:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.613128+0000", 2024-05-11T23:01:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7141999999999998e-05 2024-05-11T23:01:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.742 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.742 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:33.744 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:01:33.745 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-05-11T23:01:33.745 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:33.745 INFO:teuthology.orchestra.run.smithi005.stdout:177 trunc from 0xd235896 to 0x2e3c8c 2024-05-11T23:01:33.745 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:33.744+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:33.777 INFO:teuthology.orchestra.run.smithi005.stdout:178 punch from 0xf1c2 to 0x15638, (0x6476 bytes) 2024-05-11T23:01:33.785 INFO:teuthology.orchestra.run.smithi005.stdout:179 trunc from 0x2e3c8c to 0x667d991 2024-05-11T23:01:33.790 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:33.790 INFO:teuthology.orchestra.run.smithi005.stdout:182 read 0x57bcc76 thru 0x57c7230 (0xa5bb bytes) 2024-05-11T23:01:33.804 INFO:teuthology.orchestra.run.smithi005.stdout:183 write 0xdbdfb49 thru 0xdbe4cda (0x5192 bytes) 2024-05-11T23:01:33.811 INFO:teuthology.orchestra.run.smithi005.stdout:184 write 0x38847ce thru 0x38899b0 (0x51e3 bytes) 2024-05-11T23:01:33.816 INFO:teuthology.orchestra.run.smithi005.stdout:185 read 0x11c131b thru 0x11c3002 (0x1ce8 bytes) 2024-05-11T23:01:33.816 INFO:teuthology.orchestra.run.smithi005.stdout:186 read 0x960e6ec thru 0x961b74a (0xd05f bytes) 2024-05-11T23:01:33.816 INFO:teuthology.orchestra.run.smithi005.stdout:187 writesame 0xaf0df8f thru 0xaf0efac data_size 0x1(0x101e bytes) 2024-05-11T23:01:33.821 INFO:teuthology.orchestra.run.smithi005.stdout:188 read 0x5c29979 thru 0x5c2ab48 (0x11d0 bytes) 2024-05-11T23:01:33.821 INFO:teuthology.orchestra.run.smithi005.stdout:189 read 0x937392c thru 0x937e31d (0xa9f2 bytes) 2024-05-11T23:01:33.821 INFO:teuthology.orchestra.run.smithi005.stdout:190 read 0x2c416fe thru 0x2c43d17 (0x261a bytes) 2024-05-11T23:01:33.821 INFO:teuthology.orchestra.run.smithi005.stdout:191 writesame 0xaa32877 thru 0xaa36330 data_size 0x1(0x3aba bytes) 2024-05-11T23:01:33.829 INFO:teuthology.orchestra.run.smithi005.stdout:193 write 0x58954bb thru 0x589b61b (0x6161 bytes) 2024-05-11T23:01:33.836 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:33.836 INFO:teuthology.orchestra.run.smithi005.stdout:195 read 0x48f00f9 thru 0x48f39be (0x38c6 bytes) 2024-05-11T23:01:33.836 INFO:teuthology.orchestra.run.smithi005.stdout:197 writesame 0xeeb835 thru 0xef7015 data_size 0x47(0xb7e1 bytes) 2024-05-11T23:01:33.845 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:33.844+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:33.848 INFO:teuthology.orchestra.run.smithi005.stdout:198 read 0xa5cf1be thru 0xa5dcf7f (0xddc2 bytes) 2024-05-11T23:01:33.848 INFO:teuthology.orchestra.run.smithi005.stdout:200 writesame 0xb342ab3 thru 0xb344d45 data_size 0x1(0x2293 bytes) 2024-05-11T23:01:33.857 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:33.857 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:33.857 INFO:teuthology.orchestra.run.smithi005.stdout:203 write 0x55d1bd6 thru 0x55d61a0 (0x45cb bytes) 2024-05-11T23:01:33.862 INFO:teuthology.orchestra.run.smithi005.stdout:204 write 0x273b937 thru 0x273d10d (0x17d7 bytes) 2024-05-11T23:01:33.867 INFO:teuthology.orchestra.run.smithi005.stdout:205 read 0x309ae5d thru 0x30a35f9 (0x879d bytes) 2024-05-11T23:01:33.883 INFO:teuthology.orchestra.run.smithi005.stdout:206 trunc from 0xdbe4cdb to 0x1f559 2024-05-11T23:01:33.903 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:33.904 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:01:33.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.904 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:3482 3.0 e25/19 3:05412a4a:::rbd_data.111e89cdde7d.0000000000000005:head v 25'10, mlcod=25'8)", 2024-05-11T23:01:33.904 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.903716+0000", 2024-05-11T23:01:33.904 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.00036173299999999999, 2024-05-11T23:01:33.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00038574699999999998, 2024-05-11T23:01:33.904 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:33.904 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:33.904 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:33.904 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:33.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:33.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.903716+0000", 2024-05-11T23:01:33.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:33.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.903716+0000", 2024-05-11T23:01:33.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:33.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:33.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.903717+0000", 2024-05-11T23:01:33.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6599999999999994e-07 2024-05-11T23:01:33.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:33.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.903723+0000", 2024-05-11T23:01:33.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0170000000000002e-06 2024-05-11T23:01:33.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:33.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.903737+0000", 2024-05-11T23:01:33.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4051e-05 2024-05-11T23:01:33.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:33.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.903744+0000", 2024-05-11T23:01:33.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9999999999999999e-06 2024-05-11T23:01:33.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:33.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.903798+0000", 2024-05-11T23:01:33.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3989000000000001e-05 2024-05-11T23:01:33.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:33.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:33.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:33.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.903820+0000", 2024-05-11T23:01:33.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1793999999999998e-05 2024-05-11T23:01:33.908 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.908 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:33.908 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.908 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:33.908 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:01:33.908 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 1 2024-05-11T23:01:33.908 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:33.912 INFO:teuthology.orchestra.run.smithi005.stdout:207 punch from 0xe948 to 0x1b7f3, (0xceab bytes) 2024-05-11T23:01:33.912 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:33.913 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:01:33.937 INFO:teuthology.orchestra.run.smithi005.stdout:210 writesame 0xedf5226 thru 0xedfbc64 data_size 0x1(0x6a3f bytes) 2024-05-11T23:01:33.938 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:33.938 INFO:teuthology.orchestra.run.smithi005.stdout:212 write 0xa3e835c thru 0xa3f2ec6 (0xab6b bytes) 2024-05-11T23:01:33.945 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:33.945+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:33.947 INFO:teuthology.orchestra.run.smithi005.stdout:213 writesame 0x631e62c thru 0x632d608 data_size 0x1(0xefdd bytes) 2024-05-11T23:01:33.962 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:33.978 INFO:teuthology.orchestra.run.smithi005.stdout:217 trunc from 0xedfbc65 to 0xa3318b 2024-05-11T23:01:33.995 INFO:teuthology.orchestra.run.smithi005.stdout:219 read 0x8e4a2f thru 0x8e5dff (0x13d1 bytes) 2024-05-11T23:01:33.995 INFO:teuthology.orchestra.run.smithi005.stdout:221 read 0x749ed2 thru 0x74bb70 (0x1c9f bytes) 2024-05-11T23:01:33.995 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:33.995 INFO:teuthology.orchestra.run.smithi005.stdout:225 punch from 0x836902 to 0x83aa78, (0x4176 bytes) 2024-05-11T23:01:33.995 INFO:teuthology.orchestra.run.smithi005.stdout:226 punch from 0x8b1858 to 0x8bb79a, (0x9f42 bytes) 2024-05-11T23:01:33.995 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:33.995 INFO:teuthology.orchestra.run.smithi005.stdout:228 punch from 0x69d19a to 0x6a526f, (0x80d5 bytes) 2024-05-11T23:01:33.995 INFO:teuthology.orchestra.run.smithi005.stdout:229 read 0x8bd19 thru 0x9a21b (0xe503 bytes) 2024-05-11T23:01:33.996 INFO:teuthology.orchestra.run.smithi005.stdout:230 punch from 0x862f33 to 0x865ddb, (0x2ea8 bytes) 2024-05-11T23:01:33.996 INFO:teuthology.orchestra.run.smithi005.stdout:231 read 0x236249 thru 0x23749a (0x1252 bytes) 2024-05-11T23:01:33.996 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:33.996 INFO:teuthology.orchestra.run.smithi005.stdout:236 punch from 0x651529 to 0x6519a6, (0x47d bytes) 2024-05-11T23:01:33.996 INFO:teuthology.orchestra.run.smithi005.stdout:237 read 0x36ed98 thru 0x374749 (0x59b2 bytes) 2024-05-11T23:01:33.997 INFO:teuthology.orchestra.run.smithi005.stdout:238 write 0x6432d5 thru 0x645e5e (0x2b8a bytes) 2024-05-11T23:01:34.015 INFO:teuthology.orchestra.run.smithi005.stdout:239 write 0x983b0fe thru 0x9843ced (0x8bf0 bytes) 2024-05-11T23:01:34.026 INFO:teuthology.orchestra.run.smithi005.stdout:240 writesame 0x990a090 thru 0x990a642 data_size 0x1(0x5b3 bytes) 2024-05-11T23:01:34.029 INFO:teuthology.orchestra.run.smithi005.stdout:241 trunc from 0x990a643 to 0xb2a35eb 2024-05-11T23:01:34.037 INFO:teuthology.orchestra.run.smithi005.stdout:242 trunc from 0xb2a35eb to 0x97091dc 2024-05-11T23:01:34.047 INFO:teuthology.orchestra.run.smithi005.stdout:244 write 0x31a74a thru 0x31c317 (0x1bce bytes) 2024-05-11T23:01:34.047 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:34.047+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:34.048 INFO:teuthology.orchestra.run.smithi005.stdout:245 read 0x69248af thru 0x692a28a (0x59dc bytes) 2024-05-11T23:01:34.048 INFO:teuthology.orchestra.run.smithi005.stdout:246 punch from 0x751a130 to 0x752201e, (0x7eee bytes) 2024-05-11T23:01:34.048 INFO:teuthology.orchestra.run.smithi005.stdout:247 read 0x94268a6 thru 0x942c871 (0x5fcc bytes) 2024-05-11T23:01:34.048 INFO:teuthology.orchestra.run.smithi005.stdout:249 write 0x5466bb9 thru 0x546ea50 (0x7e98 bytes) 2024-05-11T23:01:34.056 INFO:teuthology.orchestra.run.smithi005.stdout:250 punch from 0x78a21dd to 0x78a5b57, (0x397a bytes) 2024-05-11T23:01:34.056 INFO:teuthology.orchestra.run.smithi005.stdout:251 write 0xdf9a88 thru 0xe094e0 (0xfa59 bytes) 2024-05-11T23:01:34.071 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:34.071 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:34.071 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:01:34.071 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:01:34.072 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:34.076 INFO:teuthology.orchestra.run.smithi005.stdout:252 trunc from 0x97091dc to 0xe532eb8 2024-05-11T23:01:34.082 INFO:teuthology.orchestra.run.smithi005.stdout:255 punch from 0xab213be to 0xab21bbf, (0x801 bytes) 2024-05-11T23:01:34.082 INFO:teuthology.orchestra.run.smithi005.stdout:258 writesame 0x80f8c0e thru 0x80f9494 data_size 0x1(0x887 bytes) 2024-05-11T23:01:34.084 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:01:34.087 INFO:teuthology.orchestra.run.smithi005.stdout:259 read 0x452fb70 thru 0x453f914 (0xfda5 bytes) 2024-05-11T23:01:34.087 INFO:teuthology.orchestra.run.smithi005.stdout:261 write 0xb96f779 thru 0xb9772b5 (0x7b3d bytes) 2024-05-11T23:01:34.096 INFO:teuthology.orchestra.run.smithi005.stdout:262 punch from 0x131d557 to 0x132d1b5, (0xfc5e bytes) 2024-05-11T23:01:34.096 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:34.097 INFO:teuthology.orchestra.run.smithi005.stdout:267 writesame 0x5ce7e5e thru 0x5cec1c8 data_size 0xb(0x436b bytes) 2024-05-11T23:01:34.107 INFO:teuthology.orchestra.run.smithi005.stdout:268 trunc from 0xe532eb8 to 0xcb9fed0 2024-05-11T23:01:34.115 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:34.116 INFO:teuthology.orchestra.run.smithi005.stdout:272 read 0x2c6545e thru 0x2c690a8 (0x3c4b bytes) 2024-05-11T23:01:34.116 INFO:teuthology.orchestra.run.smithi005.stdout:274 write 0xb46268a thru 0xb466b95 (0x450c bytes) 2024-05-11T23:01:34.128 INFO:teuthology.orchestra.run.smithi005.stdout:275 punch from 0x266355c to 0x2664a38, (0x14dc bytes) 2024-05-11T23:01:34.128 INFO:teuthology.orchestra.run.smithi005.stdout:276 write 0x2a4f482 thru 0x2a54e5d (0x59dc bytes) 2024-05-11T23:01:34.135 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:34.135 INFO:teuthology.orchestra.run.smithi005.stdout:279 trunc from 0xcb9fed0 to 0xc9e5bde 2024-05-11T23:01:34.140 INFO:teuthology.orchestra.run.smithi005.stdout:280 write 0x6e53659 thru 0x6e53ccf (0x677 bytes) 2024-05-11T23:01:34.144 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:34.144 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:34.144 INFO:teuthology.orchestra.run.smithi005.stdout:284 writesame 0xcb0a3c thru 0xcb23ff data_size 0x11(0x19c4 bytes) 2024-05-11T23:01:34.148 INFO:teuthology.orchestra.run.smithi005.stdout:286 write 0xb896455 thru 0xb89b926 (0x54d2 bytes) 2024-05-11T23:01:34.148 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:34.148+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:34.156 INFO:teuthology.orchestra.run.smithi005.stdout:287 write 0x1297a92 thru 0x12a2c5f (0xb1ce bytes) 2024-05-11T23:01:34.168 INFO:teuthology.orchestra.run.smithi005.stdout:288 read 0x9de7ad5 thru 0x9dee84e (0x6d7a bytes) 2024-05-11T23:01:34.168 INFO:teuthology.orchestra.run.smithi005.stdout:289 read 0x50a4d60 thru 0x50ab061 (0x6302 bytes) 2024-05-11T23:01:34.168 INFO:teuthology.orchestra.run.smithi005.stdout:290 write 0xbbe11e7 thru 0xbbefb89 (0xe9a3 bytes) 2024-05-11T23:01:34.177 INFO:teuthology.orchestra.run.smithi005.stdout:291 punch from 0x46e0bd9 to 0x46ee46e, (0xd895 bytes) 2024-05-11T23:01:34.177 INFO:teuthology.orchestra.run.smithi005.stdout:292 read 0x27eeac thru 0x288eec (0xa041 bytes) 2024-05-11T23:01:34.177 INFO:teuthology.orchestra.run.smithi005.stdout:293 read 0x9fb516 thru 0xa03270 (0x7d5b bytes) 2024-05-11T23:01:34.178 INFO:teuthology.orchestra.run.smithi005.stdout:294 write 0x3616923 thru 0x3625575 (0xec53 bytes) 2024-05-11T23:01:34.188 INFO:teuthology.orchestra.run.smithi005.stdout:296 read 0xb9745e0 thru 0xb97f125 (0xab46 bytes) 2024-05-11T23:01:34.189 INFO:teuthology.orchestra.run.smithi005.stdout:299 read 0x8e29084 thru 0x8e32002 (0x8f7f bytes) 2024-05-11T23:01:34.189 INFO:teuthology.orchestra.run.smithi005.stdout:300 writesame 0x12da6e thru 0x136f7f data_size 0x1(0x9512 bytes) 2024-05-11T23:01:34.200 INFO:teuthology.orchestra.run.smithi005.stdout:303 write 0xbd94435 thru 0xbda2a58 (0xe624 bytes) 2024-05-11T23:01:34.203 INFO:teuthology.orchestra.run.smithi005.stdout:305 write 0x485cbfd thru 0x486b2f3 (0xe6f7 bytes) 2024-05-11T23:01:34.209 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:34.209 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:01:34.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:01:34.210 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:01:34.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.210 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:1390 3.0 e24/19 3:03c4b8c1:::rbd_data.111e50384849.000000000000004b:head v 24'2, mlcod=0'0)", 2024-05-11T23:01:34.210 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:32.930428+0000", 2024-05-11T23:01:34.210 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.2789176040000001, 2024-05-11T23:01:34.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00084374099999999996, 2024-05-11T23:01:34.210 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.210 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.210 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:34.210 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.930428+0000", 2024-05-11T23:01:34.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.930428+0000", 2024-05-11T23:01:34.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.930429+0000", 2024-05-11T23:01:34.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2780000000000001e-06 2024-05-11T23:01:34.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.930447+0000", 2024-05-11T23:01:34.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7592e-05 2024-05-11T23:01:34.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.930464+0000", 2024-05-11T23:01:34.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6747000000000001e-05 2024-05-11T23:01:34.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.930473+0000", 2024-05-11T23:01:34.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2909999999999996e-06 2024-05-11T23:01:34.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.930530+0000", 2024-05-11T23:01:34.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7401000000000002e-05 2024-05-11T23:01:34.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:34.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.930549+0000", 2024-05-11T23:01:34.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8785000000000001e-05 2024-05-11T23:01:34.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:34.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.931236+0000", 2024-05-11T23:01:34.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00068625900000000004 2024-05-11T23:01:34.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.931272+0000", 2024-05-11T23:01:34.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6387999999999998e-05 2024-05-11T23:01:34.216 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.216 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.216 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.216 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:1670 3.0 e25/19 3:03c4b8c1:::rbd_data.111e50384849.000000000000004b:head v 25'4, mlcod=24'1)", 2024-05-11T23:01:34.216 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.554035+0000", 2024-05-11T23:01:34.216 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.65531094099999998, 2024-05-11T23:01:34.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0011790030000000001, 2024-05-11T23:01:34.216 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:34.217 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.554035+0000", 2024-05-11T23:01:34.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.554035+0000", 2024-05-11T23:01:34.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.554035+0000", 2024-05-11T23:01:34.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9900000000000001e-07 2024-05-11T23:01:34.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.554042+0000", 2024-05-11T23:01:34.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.939e-06 2024-05-11T23:01:34.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.554062+0000", 2024-05-11T23:01:34.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0256000000000001e-05 2024-05-11T23:01:34.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.554072+0000", 2024-05-11T23:01:34.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4760000000000004e-06 2024-05-11T23:01:34.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.554148+0000", 2024-05-11T23:01:34.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5865999999999995e-05 2024-05-11T23:01:34.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:34.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.554173+0000", 2024-05-11T23:01:34.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4839000000000001e-05 2024-05-11T23:01:34.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:34.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.555166+0000", 2024-05-11T23:01:34.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00099379299999999998 2024-05-11T23:01:34.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.555214+0000", 2024-05-11T23:01:34.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7435000000000002e-05 2024-05-11T23:01:34.220 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.220 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.220 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.220 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "pg_update_log_missing(3.4 epoch 25/19 rep_tid 71 entries 25'17 (0'0) error 3:21f195d1:::rbd_data.111e50384849.0000000000000059:head by client.4382.0:1859 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 23'14)", 2024-05-11T23:01:34.221 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.569432+0000", 2024-05-11T23:01:34.221 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.63991424699999999, 2024-05-11T23:01:34.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00080634299999999995, 2024-05-11T23:01:34.221 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.221 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.221 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:34.221 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569432+0000", 2024-05-11T23:01:34.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569432+0000", 2024-05-11T23:01:34.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569433+0000", 2024-05-11T23:01:34.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.658e-06 2024-05-11T23:01:34.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569446+0000", 2024-05-11T23:01:34.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2679e-05 2024-05-11T23:01:34.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569478+0000", 2024-05-11T23:01:34.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2008000000000003e-05 2024-05-11T23:01:34.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569496+0000", 2024-05-11T23:01:34.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8479999999999999e-05 2024-05-11T23:01:34.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569602+0000", 2024-05-11T23:01:34.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010609 2024-05-11T23:01:34.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.570238+0000", 2024-05-11T23:01:34.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00063542799999999995 2024-05-11T23:01:34.224 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.224 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.224 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.224 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:1913 3.4 e25/19 3:21f195d1:::rbd_data.111e50384849.0000000000000059:head v 25'18, mlcod=24'16)", 2024-05-11T23:01:34.224 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.573582+0000", 2024-05-11T23:01:34.224 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.63576332000000002, 2024-05-11T23:01:34.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00070258699999999996, 2024-05-11T23:01:34.224 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.224 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.224 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:34.224 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.573582+0000", 2024-05-11T23:01:34.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.573582+0000", 2024-05-11T23:01:34.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.573584+0000", 2024-05-11T23:01:34.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1629999999999999e-06 2024-05-11T23:01:34.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.573592+0000", 2024-05-11T23:01:34.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2009999999999999e-06 2024-05-11T23:01:34.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.573610+0000", 2024-05-11T23:01:34.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8250999999999999e-05 2024-05-11T23:01:34.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.573620+0000", 2024-05-11T23:01:34.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7680000000000003e-06 2024-05-11T23:01:34.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.573689+0000", 2024-05-11T23:01:34.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9547000000000003e-05 2024-05-11T23:01:34.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:34.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.573710+0000", 2024-05-11T23:01:34.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.101e-05 2024-05-11T23:01:34.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:34.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.574246+0000", 2024-05-11T23:01:34.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00053544900000000002 2024-05-11T23:01:34.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.574285+0000", 2024-05-11T23:01:34.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9198e-05 2024-05-11T23:01:34.228 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.228 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.228 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.228 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "pg_update_log_missing(3.4 epoch 25/19 rep_tid 73 entries 25'19 (0'0) error 3:25486f9d:::rbd_data.111e50384849.000000000000005e:head by client.4382.0:1967 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'17)", 2024-05-11T23:01:34.228 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.578248+0000", 2024-05-11T23:01:34.228 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.63109795499999999, 2024-05-11T23:01:34.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00063199100000000004, 2024-05-11T23:01:34.229 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.229 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.229 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:34.229 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578248+0000", 2024-05-11T23:01:34.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578248+0000", 2024-05-11T23:01:34.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578249+0000", 2024-05-11T23:01:34.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.353e-06 2024-05-11T23:01:34.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578263+0000", 2024-05-11T23:01:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3668e-05 2024-05-11T23:01:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578293+0000", 2024-05-11T23:01:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9991000000000001e-05 2024-05-11T23:01:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578312+0000", 2024-05-11T23:01:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8838000000000002e-05 2024-05-11T23:01:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578408+0000", 2024-05-11T23:01:34.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6804000000000004e-05 2024-05-11T23:01:34.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578880+0000", 2024-05-11T23:01:34.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000471337 2024-05-11T23:01:34.232 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.232 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.232 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.232 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "pg_update_log_missing(3.0 epoch 25/19 rep_tid 74 entries 25'5 (0'0) error 3:0097cebb:::rbd_data.111e50384849.0000000000000062:head by client.4382.0:2022 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 24'2)", 2024-05-11T23:01:34.232 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.583075+0000", 2024-05-11T23:01:34.233 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.62627107000000004, 2024-05-11T23:01:34.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001200989, 2024-05-11T23:01:34.233 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.233 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.233 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:34.233 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583075+0000", 2024-05-11T23:01:34.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583075+0000", 2024-05-11T23:01:34.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583076+0000", 2024-05-11T23:01:34.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.215e-06 2024-05-11T23:01:34.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583090+0000", 2024-05-11T23:01:34.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3798000000000001e-05 2024-05-11T23:01:34.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583119+0000", 2024-05-11T23:01:34.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9669999999999999e-05 2024-05-11T23:01:34.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583139+0000", 2024-05-11T23:01:34.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.98e-05 2024-05-11T23:01:34.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583225+0000", 2024-05-11T23:01:34.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6113999999999996e-05 2024-05-11T23:01:34.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.584276+0000", 2024-05-11T23:01:34.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001050392 2024-05-11T23:01:34.236 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.236 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.236 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.236 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:2023 3.4 e25/19 3:25486f9d:::rbd_data.111e50384849.000000000000005e:head v 25'20, mlcod=25'18)", 2024-05-11T23:01:34.236 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.583356+0000", 2024-05-11T23:01:34.236 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.62598950600000003, 2024-05-11T23:01:34.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00095195800000000004, 2024-05-11T23:01:34.236 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.237 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.237 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:34.237 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583356+0000", 2024-05-11T23:01:34.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583356+0000", 2024-05-11T23:01:34.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583357+0000", 2024-05-11T23:01:34.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5499999999999998e-07 2024-05-11T23:01:34.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583362+0000", 2024-05-11T23:01:34.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4890000000000003e-06 2024-05-11T23:01:34.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583371+0000", 2024-05-11T23:01:34.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7960000000000001e-06 2024-05-11T23:01:34.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583377+0000", 2024-05-11T23:01:34.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9150000000000001e-06 2024-05-11T23:01:34.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583429+0000", 2024-05-11T23:01:34.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1795000000000001e-05 2024-05-11T23:01:34.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:34.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583448+0000", 2024-05-11T23:01:34.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9511999999999999e-05 2024-05-11T23:01:34.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:34.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.584273+0000", 2024-05-11T23:01:34.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00082449499999999996 2024-05-11T23:01:34.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.584308+0000", 2024-05-11T23:01:34.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5500999999999998e-05 2024-05-11T23:01:34.240 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.241 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.241 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.241 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:2112 3.0 e25/19 3:0097cebb:::rbd_data.111e50384849.0000000000000062:head v 25'6, mlcod=25'4)", 2024-05-11T23:01:34.241 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.588426+0000", 2024-05-11T23:01:34.241 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.62091991999999996, 2024-05-11T23:01:34.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00079788199999999997, 2024-05-11T23:01:34.242 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.242 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.242 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:34.242 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.588426+0000", 2024-05-11T23:01:34.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.588426+0000", 2024-05-11T23:01:34.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.588427+0000", 2024-05-11T23:01:34.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.432e-06 2024-05-11T23:01:34.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.588435+0000", 2024-05-11T23:01:34.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.746e-06 2024-05-11T23:01:34.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.588455+0000", 2024-05-11T23:01:34.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9573999999999999e-05 2024-05-11T23:01:34.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.588464+0000", 2024-05-11T23:01:34.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1169999999999993e-06 2024-05-11T23:01:34.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.588525+0000", 2024-05-11T23:01:34.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0781e-05 2024-05-11T23:01:34.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:34.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.588548+0000", 2024-05-11T23:01:34.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3110000000000001e-05 2024-05-11T23:01:34.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:34.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.589179+0000", 2024-05-11T23:01:34.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00063125200000000001 2024-05-11T23:01:34.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.589224+0000", 2024-05-11T23:01:34.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4870000000000002e-05 2024-05-11T23:01:34.246 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.246 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.246 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:3350 3.0 e25/19 3:05412a4a:::rbd_data.111e89cdde7d.0000000000000005:head v 25'7, mlcod=25'5)", 2024-05-11T23:01:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.762424+0000", 2024-05-11T23:01:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.446922029, 2024-05-11T23:01:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00071265599999999999, 2024-05-11T23:01:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762424+0000", 2024-05-11T23:01:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762424+0000", 2024-05-11T23:01:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762425+0000", 2024-05-11T23:01:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.499e-06 2024-05-11T23:01:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762437+0000", 2024-05-11T23:01:34.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1307e-05 2024-05-11T23:01:34.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762456+0000", 2024-05-11T23:01:34.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9398999999999998e-05 2024-05-11T23:01:34.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762466+0000", 2024-05-11T23:01:34.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0003999999999999e-05 2024-05-11T23:01:34.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762532+0000", 2024-05-11T23:01:34.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6458999999999995e-05 2024-05-11T23:01:34.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:34.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762557+0000", 2024-05-11T23:01:34.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4380999999999999e-05 2024-05-11T23:01:34.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.763092+0000", 2024-05-11T23:01:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00053478599999999999 2024-05-11T23:01:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.763136+0000", 2024-05-11T23:01:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4821000000000003e-05 2024-05-11T23:01:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:3399 3.0 e25/19 3:04edcdf9:::rbd_data.111e89cdde7d.00000000000000af:head v 25'8, mlcod=25'6)", 2024-05-11T23:01:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.821079+0000", 2024-05-11T23:01:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.38826652699999997, 2024-05-11T23:01:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00071352700000000004, 2024-05-11T23:01:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.821079+0000", 2024-05-11T23:01:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.821079+0000", 2024-05-11T23:01:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.821081+0000", 2024-05-11T23:01:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7069999999999999e-06 2024-05-11T23:01:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.821094+0000", 2024-05-11T23:01:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2785e-05 2024-05-11T23:01:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.821113+0000", 2024-05-11T23:01:34.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9377000000000001e-05 2024-05-11T23:01:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.821122+0000", 2024-05-11T23:01:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4270000000000008e-06 2024-05-11T23:01:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.821184+0000", 2024-05-11T23:01:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2791000000000002e-05 2024-05-11T23:01:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.821206+0000", 2024-05-11T23:01:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1772000000000001e-05 2024-05-11T23:01:34.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.821766+0000", 2024-05-11T23:01:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00055980799999999996 2024-05-11T23:01:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.821793+0000", 2024-05-11T23:01:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.686e-05 2024-05-11T23:01:34.257 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:3469 3.0 e25/19 3:04edcdf9:::rbd_data.111e89cdde7d.00000000000000af:head v 25'9, mlcod=25'7)", 2024-05-11T23:01:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.894813+0000", 2024-05-11T23:01:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.31453302500000002, 2024-05-11T23:01:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078683099999999999, 2024-05-11T23:01:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.894813+0000", 2024-05-11T23:01:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.894813+0000", 2024-05-11T23:01:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.894814+0000", 2024-05-11T23:01:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.688e-06 2024-05-11T23:01:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.894825+0000", 2024-05-11T23:01:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0499999999999999e-05 2024-05-11T23:01:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.894847+0000", 2024-05-11T23:01:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1861e-05 2024-05-11T23:01:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.894856+0000", 2024-05-11T23:01:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.329e-06 2024-05-11T23:01:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.894937+0000", 2024-05-11T23:01:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0589999999999999e-05 2024-05-11T23:01:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.894975+0000", 2024-05-11T23:01:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7827000000000002e-05 2024-05-11T23:01:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.895570+0000", 2024-05-11T23:01:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00059529899999999998 2024-05-11T23:01:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.895600+0000", 2024-05-11T23:01:34.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9737e-05 2024-05-11T23:01:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:3482 3.0 e25/19 3:05412a4a:::rbd_data.111e89cdde7d.0000000000000005:head v 25'10, mlcod=25'8)", 2024-05-11T23:01:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.903716+0000", 2024-05-11T23:01:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.305629812, 2024-05-11T23:01:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000634715, 2024-05-11T23:01:34.262 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.903716+0000", 2024-05-11T23:01:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.903716+0000", 2024-05-11T23:01:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.903717+0000", 2024-05-11T23:01:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6599999999999994e-07 2024-05-11T23:01:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.903723+0000", 2024-05-11T23:01:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0170000000000002e-06 2024-05-11T23:01:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.903737+0000", 2024-05-11T23:01:34.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4051e-05 2024-05-11T23:01:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.903744+0000", 2024-05-11T23:01:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9999999999999999e-06 2024-05-11T23:01:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.903798+0000", 2024-05-11T23:01:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3989000000000001e-05 2024-05-11T23:01:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.903820+0000", 2024-05-11T23:01:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1793999999999998e-05 2024-05-11T23:01:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.904311+0000", 2024-05-11T23:01:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000491557 2024-05-11T23:01:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.904351+0000", 2024-05-11T23:01:34.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9341000000000002e-05 2024-05-11T23:01:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:3627 3.0 e25/19 3:05412a4a:::rbd_data.111e89cdde7d.0000000000000005:head v 25'11, mlcod=25'9)", 2024-05-11T23:01:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:34.048484+0000", 2024-05-11T23:01:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.16086138, 2024-05-11T23:01:34.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00070438200000000003, 2024-05-11T23:01:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.048484+0000", 2024-05-11T23:01:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.048484+0000", 2024-05-11T23:01:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.048486+0000", 2024-05-11T23:01:34.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.362e-06 2024-05-11T23:01:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.048503+0000", 2024-05-11T23:01:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7062e-05 2024-05-11T23:01:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.048520+0000", 2024-05-11T23:01:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7156e-05 2024-05-11T23:01:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.048541+0000", 2024-05-11T23:01:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1239000000000001e-05 2024-05-11T23:01:34.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.048598+0000", 2024-05-11T23:01:34.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6783000000000002e-05 2024-05-11T23:01:34.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.274 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:34.274 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.048618+0000", 2024-05-11T23:01:34.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0356999999999998e-05 2024-05-11T23:01:34.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:34.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.049144+0000", 2024-05-11T23:01:34.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00052583399999999998 2024-05-11T23:01:34.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.049189+0000", 2024-05-11T23:01:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4589000000000003e-05 2024-05-11T23:01:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.276 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.276 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:34.278 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:34.278 INFO:teuthology.orchestra.run.smithi005.stdout:307 write 0x537f7d6 thru 0x53889ea (0x9215 bytes) 2024-05-11T23:01:34.278 INFO:teuthology.orchestra.run.smithi005.stdout:308 punch from 0xc8cf84a to 0xc8dcef2, (0xd6a8 bytes) 2024-05-11T23:01:34.278 INFO:teuthology.orchestra.run.smithi005.stdout:309 write 0x4ff9f6 thru 0x50e92b (0xef36 bytes) 2024-05-11T23:01:34.278 INFO:teuthology.orchestra.run.smithi005.stdout:310 read 0x4202443 thru 0x4208207 (0x5dc5 bytes) 2024-05-11T23:01:34.279 INFO:teuthology.orchestra.run.smithi005.stdout:311 writesame 0x33c2431 thru 0x33c644c data_size 0x4(0x401c bytes) 2024-05-11T23:01:34.280 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:01:34.281 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:34.281 INFO:teuthology.orchestra.run.smithi005.stdout:313 punch from 0xb4f08f9 to 0xb4f0b98, (0x29f bytes) 2024-05-11T23:01:34.281 INFO:teuthology.orchestra.run.smithi005.stdout:314 flatten 2024-05-11T23:01:34.281 INFO:teuthology.orchestra.run.smithi005.stdout:315 read 0x5ee4783 thru 0x5ee7cc6 (0x3544 bytes) 2024-05-11T23:01:34.281 INFO:teuthology.orchestra.run.smithi005.stdout:317 trunc from 0xc9e5bde to 0x5be68f8 2024-05-11T23:01:34.282 INFO:teuthology.orchestra.run.smithi005.stdout:318 writesame 0x16ae156 thru 0x16bd7bd data_size 0x53(0xf668 bytes) 2024-05-11T23:01:34.282 INFO:teuthology.orchestra.run.smithi005.stdout:319 write 0xe3bc8a4 thru 0xe3c87ea (0xbf47 bytes) 2024-05-11T23:01:34.282 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:34.277+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:34.292 INFO:teuthology.orchestra.run.smithi005.stdout:320 write 0x54ade34 thru 0x54af361 (0x152e bytes) 2024-05-11T23:01:34.295 INFO:teuthology.orchestra.run.smithi005.stdout:321 read 0x15b4ca6 thru 0x15c0685 (0xb9e0 bytes) 2024-05-11T23:01:34.296 INFO:teuthology.orchestra.run.smithi005.stdout:322 read 0x75f4ef9 thru 0x75fa7ec (0x58f4 bytes) 2024-05-11T23:01:34.296 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:34.296 INFO:teuthology.orchestra.run.smithi005.stdout:325 punch from 0xd4188a9 to 0xd423dfd, (0xb554 bytes) 2024-05-11T23:01:34.303 INFO:teuthology.orchestra.run.smithi005.stdout:326 trunc from 0xe3c87eb to 0x78df9e2 2024-05-11T23:01:34.331 INFO:teuthology.orchestra.run.smithi005.stdout:327 write 0xae330c0 thru 0xae429c5 (0xf906 bytes) 2024-05-11T23:01:34.342 INFO:teuthology.orchestra.run.smithi005.stdout:328 writesame 0x2a23bd6 thru 0x2a3372d data_size 0x54(0xfb58 bytes) 2024-05-11T23:01:34.351 INFO:teuthology.orchestra.run.smithi005.stdout:330 write 0xa5cf332 thru 0xa5d6b72 (0x7841 bytes) 2024-05-11T23:01:34.357 INFO:teuthology.orchestra.run.smithi005.stdout:332 read 0x9051a10 thru 0x905ae84 (0x9475 bytes) 2024-05-11T23:01:34.364 INFO:teuthology.orchestra.run.smithi005.stdout:333 trunc from 0xae429c6 to 0x4eb01fe 2024-05-11T23:01:34.378 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:34.378+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:34.384 INFO:teuthology.orchestra.run.smithi005.stdout:334 writesame 0x35f2ec8 thru 0x35fa76b data_size 0xe(0x78a4 bytes) 2024-05-11T23:01:34.387 INFO:teuthology.orchestra.run.smithi005.stdout:335 punch from 0x888524 to 0x89184b, (0x9327 bytes) 2024-05-11T23:01:34.402 INFO:teuthology.orchestra.run.smithi005.stdout:336 write 0xe41a989 thru 0xe41c010 (0x1688 bytes) 2024-05-11T23:01:34.407 INFO:teuthology.orchestra.run.smithi005.stdout:337 write 0x799e117 thru 0x79a36ec (0x55d6 bytes) 2024-05-11T23:01:34.415 INFO:teuthology.orchestra.run.smithi005.stdout:339 write 0xaab1f83 thru 0xaabc5ea (0xa668 bytes) 2024-05-11T23:01:34.419 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:34.419 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:34.419 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:01:34.419 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:34.423 INFO:teuthology.orchestra.run.smithi005.stdout:342 punch from 0x60a2dd7 to 0x60af00f, (0xc238 bytes) 2024-05-11T23:01:34.423 INFO:teuthology.orchestra.run.smithi005.stdout:343 punch from 0xa978b80 to 0xa988a4c, (0xfecc bytes) 2024-05-11T23:01:34.423 INFO:teuthology.orchestra.run.smithi005.stdout:345 write 0x87119fb thru 0x871c33d (0xa943 bytes) 2024-05-11T23:01:34.430 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:01:34.432 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:34.432 INFO:teuthology.orchestra.run.smithi005.stdout:349 read 0x96b1ab3 thru 0x96be124 (0xc672 bytes) 2024-05-11T23:01:34.432 INFO:teuthology.orchestra.run.smithi005.stdout:350 read 0x1a222b0 thru 0x1a314e8 (0xf239 bytes) 2024-05-11T23:01:34.432 INFO:teuthology.orchestra.run.smithi005.stdout:351 read 0xc6d1b29 thru 0xc6d763d (0x5b15 bytes) 2024-05-11T23:01:34.433 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:34.433 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:34.433 INFO:teuthology.orchestra.run.smithi005.stdout:354 punch from 0xe007b9 to 0xe0412d, (0x3974 bytes) 2024-05-11T23:01:34.433 INFO:teuthology.orchestra.run.smithi005.stdout:355 read 0x59b4ded thru 0x59c3340 (0xe554 bytes) 2024-05-11T23:01:34.433 INFO:teuthology.orchestra.run.smithi005.stdout:356 punch from 0xa5270f0 to 0xa532c0c, (0xbb1c bytes) 2024-05-11T23:01:34.446 INFO:teuthology.orchestra.run.smithi005.stdout:358 trunc from 0xe41c011 to 0x161261f 2024-05-11T23:01:34.479 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:34.479+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:34.486 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-11T23:01:34.489 INFO:teuthology.orchestra.run.smithi005.stdout:360 write 0x84bcf70 thru 0x84cc64b (0xf6dc bytes) 2024-05-11T23:01:34.503 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:34.515 INFO:teuthology.orchestra.run.smithi005.stdout:362 write 0xe097305 thru 0xe09e934 (0x7630 bytes) 2024-05-11T23:01:34.525 INFO:teuthology.orchestra.run.smithi005.stdout:363 write 0x43a7d4b thru 0x43afadc (0x7d92 bytes) 2024-05-11T23:01:34.534 INFO:teuthology.orchestra.run.smithi005.stdout:365 punch from 0x83490cc to 0x8352037, (0x8f6b bytes) 2024-05-11T23:01:34.534 INFO:teuthology.orchestra.run.smithi005.stdout:367 write 0x2bb64ff thru 0x2bbae68 (0x496a bytes) 2024-05-11T23:01:34.549 INFO:teuthology.orchestra.run.smithi005.stdout:368 write 0xe81a6a3 thru 0xe81ed24 (0x4682 bytes) 2024-05-11T23:01:34.561 INFO:teuthology.orchestra.run.smithi005.stdout:369 writesame 0x3a6252f thru 0x3a67db7 data_size 0x5(0x5889 bytes) 2024-05-11T23:01:34.563 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:34.563 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:34.563 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:01:34.563 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:01:34.563 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:34.570 INFO:teuthology.orchestra.run.smithi005.stdout:371 writesame 0x6f09337 thru 0x6f13f20 data_size 0x87(0xabea bytes) 2024-05-11T23:01:34.573 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:01:34.580 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:34.580+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:34.591 INFO:teuthology.orchestra.run.smithi005.stdout:372 trunc from 0xe81ed25 to 0x3031048 2024-05-11T23:01:34.618 INFO:teuthology.orchestra.run.smithi005.stdout:374 read 0x26c28d1 thru 0x26cd5f3 (0xad23 bytes) 2024-05-11T23:01:34.628 INFO:teuthology.orchestra.run.smithi005.stdout:375 writesame 0x65c8d27 thru 0x65d1595 data_size 0x1(0x886f bytes) 2024-05-11T23:01:34.628 INFO:teuthology.orchestra.run.smithi005.stdout:376 clone 4 order 24 su 8388608 sc 14 2024-05-11T23:01:34.681 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:34.681+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:34.708 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:34.708 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:01:34.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:01:34.708 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:01:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:1966 3.7 3:e5c0e1bc:::rbd_data.111e50384849.0000000000000051:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.578077+0000", 2024-05-11T23:01:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.129472558, 2024-05-11T23:01:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012474607, 2024-05-11T23:01:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1966 2024-05-11T23:01:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578077+0000", 2024-05-11T23:01:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578077+0000", 2024-05-11T23:01:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578078+0000", 2024-05-11T23:01:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5600000000000005e-07 2024-05-11T23:01:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578081+0000", 2024-05-11T23:01:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6169999999999998e-06 2024-05-11T23:01:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578091+0000", 2024-05-11T23:01:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7070000000000004e-06 2024-05-11T23:01:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578097+0000", 2024-05-11T23:01:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2929999999999997e-06 2024-05-11T23:01:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.581103+0000", 2024-05-11T23:01:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0030059240000000001 2024-05-11T23:01:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.581179+0000", 2024-05-11T23:01:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6038000000000006e-05 2024-05-11T23:01:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 7", 2024-05-11T23:01:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.581434+0000", 2024-05-11T23:01:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025452 2024-05-11T23:01:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.584458+0000", 2024-05-11T23:01:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0030242889999999999 2024-05-11T23:01:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.590448+0000", 2024-05-11T23:01:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0059902200000000001 2024-05-11T23:01:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.590509+0000", 2024-05-11T23:01:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0673e-05 2024-05-11T23:01:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.590552+0000", 2024-05-11T23:01:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2670000000000003e-05 2024-05-11T23:01:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3119 3.1e 3:78cd6643:::rbd_data.111e7a935520.0000000000000004:4 [read 245760~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749195+0000", 2024-05-11T23:01:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.95835429699999997, 2024-05-11T23:01:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012256286999999999, 2024-05-11T23:01:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3119 2024-05-11T23:01:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749195+0000", 2024-05-11T23:01:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749195+0000", 2024-05-11T23:01:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749196+0000", 2024-05-11T23:01:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4099999999999999e-07 2024-05-11T23:01:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749198+0000", 2024-05-11T23:01:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1409999999999999e-06 2024-05-11T23:01:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749203+0000", 2024-05-11T23:01:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.366e-06 2024-05-11T23:01:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749207+0000", 2024-05-11T23:01:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9349999999999996e-06 2024-05-11T23:01:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761351+0000", 2024-05-11T23:01:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012143426000000001 2024-05-11T23:01:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761396+0000", 2024-05-11T23:01:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5361999999999997e-05 2024-05-11T23:01:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761452+0000", 2024-05-11T23:01:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5516000000000002e-05 2024-05-11T23:01:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3129 3.1e 3:7877a10d:::rbd_data.111e7a935520.0000000000000001:4 [read 311296~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749226+0000", 2024-05-11T23:01:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.95832357999999995, 2024-05-11T23:01:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012371007, 2024-05-11T23:01:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3129 2024-05-11T23:01:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749226+0000", 2024-05-11T23:01:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749226+0000", 2024-05-11T23:01:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749227+0000", 2024-05-11T23:01:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8500000000000002e-07 2024-05-11T23:01:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749229+0000", 2024-05-11T23:01:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2970000000000002e-06 2024-05-11T23:01:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749235+0000", 2024-05-11T23:01:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5810000000000002e-06 2024-05-11T23:01:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749240+0000", 2024-05-11T23:01:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6829999999999999e-06 2024-05-11T23:01:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761485+0000", 2024-05-11T23:01:34.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012245056000000001 2024-05-11T23:01:34.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:34.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761535+0000", 2024-05-11T23:01:34.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9589000000000002e-05 2024-05-11T23:01:34.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761597+0000", 2024-05-11T23:01:34.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2316000000000004e-05 2024-05-11T23:01:34.722 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.722 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.722 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.723 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3133 3.1e 3:78cd6643:::rbd_data.111e7a935520.0000000000000004:4 [read 311296~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:34.723 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749291+0000", 2024-05-11T23:01:34.723 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.95825904100000003, 2024-05-11T23:01:34.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012427446, 2024-05-11T23:01:34.723 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.723 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:34.723 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:34.723 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:34.723 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:34.723 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3133 2024-05-11T23:01:34.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.724 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749291+0000", 2024-05-11T23:01:34.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749291+0000", 2024-05-11T23:01:34.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749291+0000", 2024-05-11T23:01:34.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4600000000000005e-07 2024-05-11T23:01:34.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749293+0000", 2024-05-11T23:01:34.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2180000000000002e-06 2024-05-11T23:01:34.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749299+0000", 2024-05-11T23:01:34.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.93e-06 2024-05-11T23:01:34.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749305+0000", 2024-05-11T23:01:34.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4199999999999998e-06 2024-05-11T23:01:34.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761614+0000", 2024-05-11T23:01:34.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012309621999999999 2024-05-11T23:01:34.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:34.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761663+0000", 2024-05-11T23:01:34.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.884e-05 2024-05-11T23:01:34.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761718+0000", 2024-05-11T23:01:34.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4870000000000002e-05 2024-05-11T23:01:34.726 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.727 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.727 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.727 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3141 3.1e 3:7877a10d:::rbd_data.111e7a935520.0000000000000001:4 [read 376832~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:34.727 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749311+0000", 2024-05-11T23:01:34.727 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.95823835000000002, 2024-05-11T23:01:34.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012575493, 2024-05-11T23:01:34.727 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.727 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:34.727 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:34.727 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:34.727 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:34.727 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3141 2024-05-11T23:01:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749311+0000", 2024-05-11T23:01:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749311+0000", 2024-05-11T23:01:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749312+0000", 2024-05-11T23:01:34.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:01:34.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749314+0000", 2024-05-11T23:01:34.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0549999999999998e-06 2024-05-11T23:01:34.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749321+0000", 2024-05-11T23:01:34.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5340000000000002e-06 2024-05-11T23:01:34.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749325+0000", 2024-05-11T23:01:34.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8990000000000003e-06 2024-05-11T23:01:34.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761757+0000", 2024-05-11T23:01:34.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012431413000000001 2024-05-11T23:01:34.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:34.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761828+0000", 2024-05-11T23:01:34.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1741000000000003e-05 2024-05-11T23:01:34.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761887+0000", 2024-05-11T23:01:34.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8468000000000003e-05 2024-05-11T23:01:34.731 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.731 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.731 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.731 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3144 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:34.731 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749351+0000", 2024-05-11T23:01:34.731 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.95819886099999996, 2024-05-11T23:01:34.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012554991, 2024-05-11T23:01:34.731 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.731 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:34.731 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:34.731 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:34.732 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:34.732 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3144 2024-05-11T23:01:34.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.732 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749351+0000", 2024-05-11T23:01:34.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749351+0000", 2024-05-11T23:01:34.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749351+0000", 2024-05-11T23:01:34.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0299999999999999e-07 2024-05-11T23:01:34.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749353+0000", 2024-05-11T23:01:34.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9609999999999999e-06 2024-05-11T23:01:34.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749359+0000", 2024-05-11T23:01:34.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9359999999999996e-06 2024-05-11T23:01:34.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749364+0000", 2024-05-11T23:01:34.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4379999999999997e-06 2024-05-11T23:01:34.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761901+0000", 2024-05-11T23:01:34.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012537074 2024-05-11T23:01:34.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761906+0000", 2024-05-11T23:01:34.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.079e-06 2024-05-11T23:01:34.734 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.734 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.735 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.735 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3152 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:34.735 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749388+0000", 2024-05-11T23:01:34.735 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.95816158200000001, 2024-05-11T23:01:34.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012551034000000001, 2024-05-11T23:01:34.735 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:34.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:34.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:34.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:34.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3152 2024-05-11T23:01:34.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.736 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749388+0000", 2024-05-11T23:01:34.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749388+0000", 2024-05-11T23:01:34.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749389+0000", 2024-05-11T23:01:34.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4300000000000002e-07 2024-05-11T23:01:34.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749393+0000", 2024-05-11T23:01:34.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8619999999999996e-06 2024-05-11T23:01:34.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749402+0000", 2024-05-11T23:01:34.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1050000000000001e-06 2024-05-11T23:01:34.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749430+0000", 2024-05-11T23:01:34.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8039e-05 2024-05-11T23:01:34.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761929+0000", 2024-05-11T23:01:34.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012498746999999999 2024-05-11T23:01:34.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761939+0000", 2024-05-11T23:01:34.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0538e-05 2024-05-11T23:01:34.738 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.738 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.738 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.739 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3155 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:34.739 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749503+0000", 2024-05-11T23:01:34.739 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.95804641400000001, 2024-05-11T23:01:34.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012482586, 2024-05-11T23:01:34.739 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.739 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:34.739 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:34.739 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:34.739 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:34.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3155 2024-05-11T23:01:34.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.739 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749503+0000", 2024-05-11T23:01:34.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749503+0000", 2024-05-11T23:01:34.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749504+0000", 2024-05-11T23:01:34.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0299999999999999e-07 2024-05-11T23:01:34.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749511+0000", 2024-05-11T23:01:34.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4560000000000003e-06 2024-05-11T23:01:34.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749519+0000", 2024-05-11T23:01:34.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6009999999999997e-06 2024-05-11T23:01:34.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749525+0000", 2024-05-11T23:01:34.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8699999999999997e-06 2024-05-11T23:01:34.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761976+0000", 2024-05-11T23:01:34.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012451234 2024-05-11T23:01:34.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761986+0000", 2024-05-11T23:01:34.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8220000000000002e-06 2024-05-11T23:01:34.742 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.742 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.742 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.742 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3164 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:34.742 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749534+0000", 2024-05-11T23:01:34.742 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.95801619100000002, 2024-05-11T23:01:34.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012486199, 2024-05-11T23:01:34.743 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.743 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.743 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:34.743 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:34.743 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:34.743 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:34.743 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3164 2024-05-11T23:01:34.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.743 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749534+0000", 2024-05-11T23:01:34.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749534+0000", 2024-05-11T23:01:34.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749534+0000", 2024-05-11T23:01:34.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7700000000000004e-07 2024-05-11T23:01:34.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749536+0000", 2024-05-11T23:01:34.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0789999999999999e-06 2024-05-11T23:01:34.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749542+0000", 2024-05-11T23:01:34.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:01:34.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749548+0000", 2024-05-11T23:01:34.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0090000000000004e-06 2024-05-11T23:01:34.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762014+0000", 2024-05-11T23:01:34.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012466090000000001 2024-05-11T23:01:34.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762020+0000", 2024-05-11T23:01:34.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4099999999999999e-06 2024-05-11T23:01:34.746 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.746 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.746 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.746 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3167 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:34.746 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749577+0000", 2024-05-11T23:01:34.746 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.95797241, 2024-05-11T23:01:34.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012471350000000001, 2024-05-11T23:01:34.746 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.746 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.747 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:34.747 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:34.747 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:34.747 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:34.747 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3167 2024-05-11T23:01:34.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.747 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749577+0000", 2024-05-11T23:01:34.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749577+0000", 2024-05-11T23:01:34.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749578+0000", 2024-05-11T23:01:34.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2800000000000002e-07 2024-05-11T23:01:34.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749580+0000", 2024-05-11T23:01:34.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.058e-06 2024-05-11T23:01:34.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749587+0000", 2024-05-11T23:01:34.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4089999999999997e-06 2024-05-11T23:01:34.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749591+0000", 2024-05-11T23:01:34.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3050000000000003e-06 2024-05-11T23:01:34.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762040+0000", 2024-05-11T23:01:34.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012448489 2024-05-11T23:01:34.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762049+0000", 2024-05-11T23:01:34.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6610000000000006e-06 2024-05-11T23:01:34.750 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.750 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.750 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.750 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3176 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:34.750 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749637+0000", 2024-05-11T23:01:34.750 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.95791274199999998, 2024-05-11T23:01:34.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012425937999999999, 2024-05-11T23:01:34.750 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.750 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.751 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:34.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:34.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:34.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:34.751 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3176 2024-05-11T23:01:34.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.751 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749637+0000", 2024-05-11T23:01:34.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749637+0000", 2024-05-11T23:01:34.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749637+0000", 2024-05-11T23:01:34.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0900000000000002e-07 2024-05-11T23:01:34.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749640+0000", 2024-05-11T23:01:34.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6070000000000002e-06 2024-05-11T23:01:34.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749649+0000", 2024-05-11T23:01:34.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5629999999999992e-06 2024-05-11T23:01:34.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749654+0000", 2024-05-11T23:01:34.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5570000000000003e-06 2024-05-11T23:01:34.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762059+0000", 2024-05-11T23:01:34.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012404567 2024-05-11T23:01:34.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762063+0000", 2024-05-11T23:01:34.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1350000000000002e-06 2024-05-11T23:01:34.754 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.754 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.754 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.754 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3179 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:34.754 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749695+0000", 2024-05-11T23:01:34.754 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.95785491700000003, 2024-05-11T23:01:34.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012391855, 2024-05-11T23:01:34.754 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.754 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.754 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:34.754 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:34.754 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:34.755 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:34.755 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3179 2024-05-11T23:01:34.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.755 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749695+0000", 2024-05-11T23:01:34.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749695+0000", 2024-05-11T23:01:34.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749695+0000", 2024-05-11T23:01:34.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.63e-07 2024-05-11T23:01:34.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749698+0000", 2024-05-11T23:01:34.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.83e-06 2024-05-11T23:01:34.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749707+0000", 2024-05-11T23:01:34.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4810000000000006e-06 2024-05-11T23:01:34.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749711+0000", 2024-05-11T23:01:34.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6859999999999997e-06 2024-05-11T23:01:34.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762079+0000", 2024-05-11T23:01:34.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012367515000000001 2024-05-11T23:01:34.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762087+0000", 2024-05-11T23:01:34.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8800000000000008e-06 2024-05-11T23:01:34.757 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.757 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.758 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.758 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3188 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:34.758 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749747+0000", 2024-05-11T23:01:34.758 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.95780286800000003, 2024-05-11T23:01:34.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012374299, 2024-05-11T23:01:34.758 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.758 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.758 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:34.758 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:34.758 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:34.758 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:34.758 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3188 2024-05-11T23:01:34.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.759 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749747+0000", 2024-05-11T23:01:34.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749747+0000", 2024-05-11T23:01:34.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749747+0000", 2024-05-11T23:01:34.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1199999999999998e-07 2024-05-11T23:01:34.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749750+0000", 2024-05-11T23:01:34.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2869999999999999e-06 2024-05-11T23:01:34.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749756+0000", 2024-05-11T23:01:34.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8199999999999999e-06 2024-05-11T23:01:34.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749763+0000", 2024-05-11T23:01:34.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3149999999999997e-06 2024-05-11T23:01:34.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762116+0000", 2024-05-11T23:01:34.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012353207999999999 2024-05-11T23:01:34.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762121+0000", 2024-05-11T23:01:34.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2569999999999998e-06 2024-05-11T23:01:34.761 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.761 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.761 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.762 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3191 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:34.762 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749794+0000", 2024-05-11T23:01:34.762 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.95775524000000001, 2024-05-11T23:01:34.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012357628000000001, 2024-05-11T23:01:34.762 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.762 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.762 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:34.762 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:34.762 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:34.762 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:34.762 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3191 2024-05-11T23:01:34.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.762 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749794+0000", 2024-05-11T23:01:34.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749794+0000", 2024-05-11T23:01:34.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749795+0000", 2024-05-11T23:01:34.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:01:34.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749800+0000", 2024-05-11T23:01:34.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7129999999999996e-06 2024-05-11T23:01:34.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749807+0000", 2024-05-11T23:01:34.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3039999999999999e-06 2024-05-11T23:01:34.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749812+0000", 2024-05-11T23:01:34.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3569999999999997e-06 2024-05-11T23:01:34.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762147+0000", 2024-05-11T23:01:34.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012335084 2024-05-11T23:01:34.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762152+0000", 2024-05-11T23:01:34.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6909999999999996e-06 2024-05-11T23:01:34.765 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.765 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.765 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.765 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3201 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:34.765 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749845+0000", 2024-05-11T23:01:34.766 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.95770451899999998, 2024-05-11T23:01:34.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012333903, 2024-05-11T23:01:34.766 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.766 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:34.766 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:34.766 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:34.766 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:34.766 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3201 2024-05-11T23:01:34.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.766 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749845+0000", 2024-05-11T23:01:34.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749845+0000", 2024-05-11T23:01:34.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749846+0000", 2024-05-11T23:01:34.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4999999999999998e-07 2024-05-11T23:01:34.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749852+0000", 2024-05-11T23:01:34.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5729999999999996e-06 2024-05-11T23:01:34.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749861+0000", 2024-05-11T23:01:34.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0080000000000008e-06 2024-05-11T23:01:34.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749866+0000", 2024-05-11T23:01:34.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6720000000000003e-06 2024-05-11T23:01:34.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762175+0000", 2024-05-11T23:01:34.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012308852 2024-05-11T23:01:34.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762179+0000", 2024-05-11T23:01:34.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3479999999999997e-06 2024-05-11T23:01:34.769 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.769 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.769 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.769 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3204 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:34.769 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749889+0000", 2024-05-11T23:01:34.769 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.95766026900000001, 2024-05-11T23:01:34.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012314227000000001, 2024-05-11T23:01:34.769 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.770 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.770 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:34.770 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:34.770 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:34.770 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:34.770 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3204 2024-05-11T23:01:34.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.770 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749889+0000", 2024-05-11T23:01:34.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749889+0000", 2024-05-11T23:01:34.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749890+0000", 2024-05-11T23:01:34.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:01:34.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749892+0000", 2024-05-11T23:01:34.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9970000000000001e-06 2024-05-11T23:01:34.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749897+0000", 2024-05-11T23:01:34.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5500000000000002e-06 2024-05-11T23:01:34.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749902+0000", 2024-05-11T23:01:34.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3409999999999996e-06 2024-05-11T23:01:34.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762199+0000", 2024-05-11T23:01:34.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012297577000000001 2024-05-11T23:01:34.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762204+0000", 2024-05-11T23:01:34.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5310000000000003e-06 2024-05-11T23:01:34.773 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.773 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.773 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.773 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3209 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:34.773 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749907+0000", 2024-05-11T23:01:34.773 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.95764234999999998, 2024-05-11T23:01:34.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012319788999999999, 2024-05-11T23:01:34.773 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.773 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:34.773 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:34.773 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:34.774 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:34.774 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3209 2024-05-11T23:01:34.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.774 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749907+0000", 2024-05-11T23:01:34.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749907+0000", 2024-05-11T23:01:34.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749908+0000", 2024-05-11T23:01:34.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-05-11T23:01:34.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749916+0000", 2024-05-11T23:01:34.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8550000000000008e-06 2024-05-11T23:01:34.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749923+0000", 2024-05-11T23:01:34.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7850000000000003e-06 2024-05-11T23:01:34.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749928+0000", 2024-05-11T23:01:34.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8609999999999997e-06 2024-05-11T23:01:34.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762223+0000", 2024-05-11T23:01:34.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012294589999999999 2024-05-11T23:01:34.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762227+0000", 2024-05-11T23:01:34.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4510000000000002e-06 2024-05-11T23:01:34.777 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.777 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.777 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.777 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3212 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:34.777 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749956+0000", 2024-05-11T23:01:34.777 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.95759396399999996, 2024-05-11T23:01:34.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012295808, 2024-05-11T23:01:34.777 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.777 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:34.777 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:34.777 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:34.778 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:34.778 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3212 2024-05-11T23:01:34.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.778 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749956+0000", 2024-05-11T23:01:34.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749956+0000", 2024-05-11T23:01:34.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749956+0000", 2024-05-11T23:01:34.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:01:34.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749959+0000", 2024-05-11T23:01:34.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7879999999999998e-06 2024-05-11T23:01:34.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749965+0000", 2024-05-11T23:01:34.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2269999999999998e-06 2024-05-11T23:01:34.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749970+0000", 2024-05-11T23:01:34.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0849999999999996e-06 2024-05-11T23:01:34.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762247+0000", 2024-05-11T23:01:34.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012276989 2024-05-11T23:01:34.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762252+0000", 2024-05-11T23:01:34.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3669999999999999e-06 2024-05-11T23:01:34.780 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.780 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.781 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.781 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3221 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:34.781 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749998+0000", 2024-05-11T23:01:34.781 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.95755139499999997, 2024-05-11T23:01:34.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012277692999999999, 2024-05-11T23:01:34.781 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.781 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.781 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:34.781 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:34.781 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:34.781 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:34.781 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3221 2024-05-11T23:01:34.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.782 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749998+0000", 2024-05-11T23:01:34.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749998+0000", 2024-05-11T23:01:34.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749999+0000", 2024-05-11T23:01:34.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3300000000000002e-07 2024-05-11T23:01:34.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750001+0000", 2024-05-11T23:01:34.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.007e-06 2024-05-11T23:01:34.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750007+0000", 2024-05-11T23:01:34.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2160000000000003e-06 2024-05-11T23:01:34.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750012+0000", 2024-05-11T23:01:34.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0200000000000002e-06 2024-05-11T23:01:34.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762271+0000", 2024-05-11T23:01:34.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012258626999999999 2024-05-11T23:01:34.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762276+0000", 2024-05-11T23:01:34.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2900000000000002e-06 2024-05-11T23:01:34.784 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.784 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.784 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.785 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3225 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:34.785 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.750040+0000", 2024-05-11T23:01:34.785 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.95750958399999997, 2024-05-11T23:01:34.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012260699999999999, 2024-05-11T23:01:34.785 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:34.785 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:34.785 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:34.785 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:34.785 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:34.785 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:34.785 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3225 2024-05-11T23:01:34.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.785 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:34.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:34.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750040+0000", 2024-05-11T23:01:34.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:34.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750040+0000", 2024-05-11T23:01:34.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:34.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:34.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750040+0000", 2024-05-11T23:01:34.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-05-11T23:01:34.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:34.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750043+0000", 2024-05-11T23:01:34.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.672e-06 2024-05-11T23:01:34.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:34.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750049+0000", 2024-05-11T23:01:34.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3230000000000003e-06 2024-05-11T23:01:34.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:34.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750055+0000", 2024-05-11T23:01:34.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.609e-06 2024-05-11T23:01:34.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:34.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762297+0000", 2024-05-11T23:01:34.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012241432 2024-05-11T23:01:34.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:34.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:34.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:34.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762301+0000", 2024-05-11T23:01:34.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2939999999999999e-06 2024-05-11T23:01:34.788 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.788 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.788 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.788 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:34.788 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:34.788 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:34.790 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:01:34.790 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:34.790+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:34.852 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:01:34.891 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:34.891+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:34.896 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:01:34.896 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:34.905 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:01:34.978 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:01:34.978 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:01:34.992 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:34.991+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:34.996 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:01:34.996 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:01:35.013 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:01:35.013 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:01:35.029 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:01:35.030 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:01:35.047 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:01:35.047 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:01:35.049 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:35.050 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:35.059 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:01:35.064 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:01:35.064 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:01:35.082 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:01:35.082 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:01:35.099 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:35.099+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:35.100 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:01:35.100 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:01:35.193 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:35.193+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:35.203 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:35.203 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:35.213 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:01:35.297 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:35.297+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:35.359 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:35.359 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:35.367 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:01:35.408 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:35.408+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:35.479 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-05-11T23:01:35.495 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:35.495+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:35.524 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:35.524 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:35.530 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:01:35.596 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:35.595+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:35.620 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-05-11T23:01:35.647 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:35.647 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:35.657 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:01:35.684 INFO:teuthology.orchestra.run.smithi005.stdout:377 punch from 0x60c8516 to 0x60cd5bc, (0x50a6 bytes) 2024-05-11T23:01:35.694 INFO:teuthology.orchestra.run.smithi005.stdout:378 trunc from 0x65d1596 to 0x137f890 2024-05-11T23:01:35.697 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:35.697+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:35.714 INFO:teuthology.orchestra.run.smithi005.stdout:379 write 0x5976676 thru 0x597b569 (0x4ef4 bytes) 2024-05-11T23:01:35.730 INFO:teuthology.orchestra.run.smithi005.stdout:380 write 0xb63e064 thru 0xb648c87 (0xac24 bytes) 2024-05-11T23:01:35.733 INFO:teuthology.orchestra.run.smithi005.stdout:382 write 0x112d1cf thru 0x11353ec (0x821e bytes) 2024-05-11T23:01:35.734 INFO:teuthology.orchestra.run.smithi005.stdout:384 read 0x273217 thru 0x27dc3e (0xaa28 bytes) 2024-05-11T23:01:35.735 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:35.735 INFO:teuthology.orchestra.run.smithi005.stdout:386 write 0x49d0a07 thru 0x49dfdf3 (0xf3ed bytes) 2024-05-11T23:01:35.738 INFO:teuthology.orchestra.run.smithi005.stdout:387 read 0x4078c91 thru 0x4084d26 (0xc096 bytes) 2024-05-11T23:01:35.738 INFO:teuthology.orchestra.run.smithi005.stdout:390 read 0x3c449c1 thru 0x3c48eb1 (0x44f1 bytes) 2024-05-11T23:01:35.739 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:35.739 INFO:teuthology.orchestra.run.smithi005.stdout:392 writesame 0x5504ece thru 0x5513f8e data_size 0xd(0xf0c1 bytes) 2024-05-11T23:01:35.742 INFO:teuthology.orchestra.run.smithi005.stdout:394 punch from 0x5bfad5a to 0x5bffecb, (0x5171 bytes) 2024-05-11T23:01:35.742 INFO:teuthology.orchestra.run.smithi005.stdout:395 writesame 0x75a1a3d thru 0x75a4873 data_size 0x1(0x2e37 bytes) 2024-05-11T23:01:35.750 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:35.750 INFO:teuthology.orchestra.run.smithi005.stdout:397 read 0x5acdb51 thru 0x5ada024 (0xc4d4 bytes) 2024-05-11T23:01:35.751 INFO:teuthology.orchestra.run.smithi005.stdout:399 read 0x65a16cb thru 0x65a3ccd (0x2603 bytes) 2024-05-11T23:01:35.751 INFO:teuthology.orchestra.run.smithi005.stdout:400 punch from 0x75cce70 to 0x75d0b7d, (0x3d0d bytes) 2024-05-11T23:01:35.751 INFO:teuthology.orchestra.run.smithi005.stdout:402 read 0x1c4e0b4 thru 0x1c5b8eb (0xd838 bytes) 2024-05-11T23:01:35.751 INFO:teuthology.orchestra.run.smithi005.stdout:403 write 0x4d64bb5 thru 0x4d7443a (0xf886 bytes) 2024-05-11T23:01:35.765 INFO:teuthology.orchestra.run.smithi005.stdout:404 trunc from 0xb648c88 to 0x8f5928 2024-05-11T23:01:35.795 INFO:teuthology.orchestra.run.smithi005.stdout:405 read 0x4cfef thru 0x51f05 (0x4f17 bytes) 2024-05-11T23:01:35.799 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:35.798+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:35.811 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:35.812 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:35.814 INFO:teuthology.orchestra.run.smithi005.stdout:406 writesame 0xc4adcc6 thru 0xc4b6bff data_size 0x7e(0x8f3a bytes) 2024-05-11T23:01:35.815 INFO:teuthology.orchestra.run.smithi005.stdout:407 write 0x3c57abd thru 0x3c62b05 (0xb049 bytes) 2024-05-11T23:01:35.817 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:35.819 INFO:teuthology.orchestra.run.smithi005.stdout:409 trunc from 0xc4b6c00 to 0xaf677f6 2024-05-11T23:01:35.821 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:01:35.826 INFO:teuthology.orchestra.run.smithi005.stdout:410 trunc from 0xaf677f6 to 0xa456ff9 2024-05-11T23:01:35.832 INFO:teuthology.orchestra.run.smithi005.stdout:411 write 0x26264ee thru 0x263625f (0xfd72 bytes) 2024-05-11T23:01:35.838 INFO:teuthology.orchestra.run.smithi005.stdout:412 write 0x9e400dd thru 0x9e4cad5 (0xc9f9 bytes) 2024-05-11T23:01:35.838 INFO:teuthology.orchestra.run.smithi005.stdout:413 write 0x99b440d thru 0x99b5aed (0x16e1 bytes) 2024-05-11T23:01:35.850 INFO:teuthology.orchestra.run.smithi005.stdout:414 trunc from 0xa456ff9 to 0x2d9780 2024-05-11T23:01:35.865 INFO:teuthology.orchestra.run.smithi005.stdout:416 read 0x1c8b31 thru 0x1c90be (0x58e bytes) 2024-05-11T23:01:35.876 INFO:teuthology.orchestra.run.smithi005.stdout:417 writesame 0x582f1ef thru 0x5838338 data_size 0x3(0x914a bytes) 2024-05-11T23:01:35.882 INFO:teuthology.orchestra.run.smithi005.stdout:418 trunc from 0x5838339 to 0xad315f4 2024-05-11T23:01:35.887 INFO:teuthology.orchestra.run.smithi005.stdout:419 read 0x8184629 thru 0x8189752 (0x512a bytes) 2024-05-11T23:01:35.893 INFO:teuthology.orchestra.run.smithi005.stdout:420 write 0xb9f2ca4 thru 0xba01d16 (0xf073 bytes) 2024-05-11T23:01:35.896 INFO:teuthology.orchestra.run.smithi005.stdout:421 write 0x7193766 thru 0x71959a3 (0x223e bytes) 2024-05-11T23:01:35.900 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:35.899+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:35.903 INFO:teuthology.orchestra.run.smithi005.stdout:422 writesame 0xc8afebb thru 0xc8b35d1 data_size 0x3(0x3717 bytes) 2024-05-11T23:01:35.915 INFO:teuthology.orchestra.run.smithi005.stdout:423 trunc from 0xc8b35d2 to 0x21e4f94 2024-05-11T23:01:35.925 INFO:teuthology.orchestra.run.smithi005.stdout:424 read 0x1d95305 thru 0x1da0614 (0xb310 bytes) 2024-05-11T23:01:35.926 INFO:teuthology.orchestra.run.smithi005.stdout:426 read 0x1a5ab89 thru 0x1a5cbb4 (0x202c bytes) 2024-05-11T23:01:35.926 INFO:teuthology.orchestra.run.smithi005.stdout:427 punch from 0xccff10 to 0xcd24ab, (0x259b bytes) 2024-05-11T23:01:35.928 INFO:teuthology.orchestra.run.smithi005.stdout:429 trunc from 0x21e4f94 to 0x48e1e9f 2024-05-11T23:01:35.933 INFO:teuthology.orchestra.run.smithi005.stdout:430 clone 5 order 23 su 524288 sc 15 2024-05-11T23:01:35.935 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:35.935 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:35.944 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:01:36.000 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:36.000+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:36.090 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:36.090 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:36.098 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:01:36.101 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:36.101+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:36.201 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:36.201+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:36.204 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:36.204 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:36.213 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:01:36.302 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:36.302+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:36.359 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:36.359 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:36.367 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:01:36.408 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:36.408+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:36.504 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:36.504+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:36.514 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:36.514 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:36.523 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:01:36.605 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:36.605+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:36.645 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:36.645 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:36.654 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:01:36.706 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:36.706+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:36.770 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:36.771 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:36.779 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:01:36.806 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:36.806+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:36.838 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone4 from 0x48e1e9f (overlap 0x2d9780) to 0x91e4c 2024-05-11T23:01:36.884 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-05-11T23:01:36.907 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:36.907+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:36.947 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:36.947 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:36.957 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:01:37.003 INFO:teuthology.orchestra.run.smithi005.stdout:432 read 0x2bb1aab thru 0x2bb57d4 (0x3d2a bytes) 2024-05-11T23:01:37.003 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:37.003 INFO:teuthology.orchestra.run.smithi005.stdout:435 punch from 0x344ef0 to 0x34f469, (0xa579 bytes) 2024-05-11T23:01:37.007 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:37.008 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:37.008+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:37.020 INFO:teuthology.orchestra.run.smithi005.stdout:437 write 0xb0ef44f thru 0xb0f44ca (0x507c bytes) 2024-05-11T23:01:37.023 INFO:teuthology.orchestra.run.smithi005.stdout:440 read 0x3ca2ebd thru 0x3caeaf6 (0xbc3a bytes) 2024-05-11T23:01:37.025 INFO:teuthology.orchestra.run.smithi005.stdout:441 trunc from 0xb0f44cb to 0x9ce4145 2024-05-11T23:01:37.031 INFO:teuthology.orchestra.run.smithi005.stdout:442 writesame 0x33fd471 thru 0x3401852 data_size 0x1(0x43e2 bytes) 2024-05-11T23:01:37.038 INFO:teuthology.orchestra.run.smithi005.stdout:443 read 0x8f25a27 thru 0x8f271fd (0x17d7 bytes) 2024-05-11T23:01:37.039 INFO:teuthology.orchestra.run.smithi005.stdout:445 write 0x58cfef4 thru 0x58d5bae (0x5cbb bytes) 2024-05-11T23:01:37.052 INFO:teuthology.orchestra.run.smithi005.stdout:446 write 0xd2e2621 thru 0xd2e68b7 (0x4297 bytes) 2024-05-11T23:01:37.055 INFO:teuthology.orchestra.run.smithi005.stdout:447 read 0x7cb62c6 thru 0x7cbb709 (0x5444 bytes) 2024-05-11T23:01:37.055 INFO:teuthology.orchestra.run.smithi005.stdout:448 read 0xd097d06 thru 0xd097d3c (0x37 bytes) 2024-05-11T23:01:37.055 INFO:teuthology.orchestra.run.smithi005.stdout:450 punch from 0x4995518 to 0x49a51cc, (0xfcb4 bytes) 2024-05-11T23:01:37.055 INFO:teuthology.orchestra.run.smithi005.stdout:451 punch from 0x5d62d65 to 0x5d72737, (0xf9d2 bytes) 2024-05-11T23:01:37.055 INFO:teuthology.orchestra.run.smithi005.stdout:452 write 0x28b9461 thru 0x28bf09b (0x5c3b bytes) 2024-05-11T23:01:37.058 INFO:teuthology.orchestra.run.smithi005.stdout:453 punch from 0xbbe88f2 to 0xbbf2059, (0x9767 bytes) 2024-05-11T23:01:37.059 INFO:teuthology.orchestra.run.smithi005.stdout:454 read 0x57b8d79 thru 0x57b92e1 (0x569 bytes) 2024-05-11T23:01:37.072 INFO:teuthology.orchestra.run.smithi005.stdout:455 trunc from 0xd2e68b8 to 0x2789231 2024-05-11T23:01:37.097 INFO:teuthology.orchestra.run.smithi005.stdout:456 trunc from 0x2789231 to 0x27e7b8b 2024-05-11T23:01:37.102 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:01:37.102 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:01:37.102 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-05-11T23:01:37.102 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2024-05-11T23:01:37.103 INFO:teuthology.orchestra.run.smithi005.stdout:458 read 0x85610 thru 0x93bf3 (0xe5e4 bytes) 2024-05-11T23:01:37.108 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:37.108+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:37.111 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:01:37.116 INFO:teuthology.orchestra.run.smithi005.stdout:459 write 0x9319707 thru 0x931b7ae (0x20a8 bytes) 2024-05-11T23:01:37.124 INFO:teuthology.orchestra.run.smithi005.stdout:460 trunc from 0x931b7af to 0xe2d91c8 2024-05-11T23:01:37.131 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:37.131 INFO:teuthology.orchestra.run.smithi005.stdout:464 read 0xc67a9f3 thru 0xc6808a5 (0x5eb3 bytes) 2024-05-11T23:01:37.131 INFO:teuthology.orchestra.run.smithi005.stdout:465 read 0xbf45f88 thru 0xbf4f732 (0x97ab bytes) 2024-05-11T23:01:37.141 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:37.141 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:37.147 INFO:teuthology.orchestra.run.smithi005.stdout:466 trunc from 0xe2d91c8 to 0x7bd579 2024-05-11T23:01:37.157 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:01:37.161 INFO:teuthology.orchestra.run.smithi005.stdout:468 read 0x6a596 thru 0x75ccd (0xb738 bytes) 2024-05-11T23:01:37.176 INFO:teuthology.orchestra.run.smithi005.stdout:469 write 0x785a172 thru 0x786135d (0x71ec bytes) 2024-05-11T23:01:37.193 INFO:teuthology.orchestra.run.smithi005.stdout:470 write 0xdb0742e thru 0xdb0c973 (0x5546 bytes) 2024-05-11T23:01:37.197 INFO:teuthology.orchestra.run.smithi005.stdout:472 write 0x34327bb thru 0x343d53c (0xad82 bytes) 2024-05-11T23:01:37.199 INFO:teuthology.orchestra.run.smithi005.stdout:473 trunc from 0xdb0c974 to 0xd487839 2024-05-11T23:01:37.207 INFO:teuthology.orchestra.run.smithi005.stdout:474 write 0x4d68117 thru 0x4d756a7 (0xd591 bytes) 2024-05-11T23:01:37.208 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:37.208+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:37.209 INFO:teuthology.orchestra.run.smithi005.stdout:475 write 0xb65a2c5 thru 0xb667357 (0xd093 bytes) 2024-05-11T23:01:37.213 INFO:teuthology.orchestra.run.smithi005.stdout:476 writesame 0x59c54e6 thru 0x59c626b data_size 0x3(0xd86 bytes) 2024-05-11T23:01:37.215 INFO:teuthology.orchestra.run.smithi005.stdout:477 write 0x4003d85 thru 0x4010cb6 (0xcf32 bytes) 2024-05-11T23:01:37.217 INFO:teuthology.orchestra.run.smithi005.stdout:478 punch from 0x78bb43a to 0x78bc0ed, (0xcb3 bytes) 2024-05-11T23:01:37.217 INFO:teuthology.orchestra.run.smithi005.stdout:479 punch from 0xa530ab to 0xa611b5, (0xe10a bytes) 2024-05-11T23:01:37.217 INFO:teuthology.orchestra.run.smithi005.stdout:480 punch from 0xb5ba76a to 0xb5bb222, (0xab8 bytes) 2024-05-11T23:01:37.217 INFO:teuthology.orchestra.run.smithi005.stdout:482 read 0x82e1e39 thru 0x82efaef (0xdcb7 bytes) 2024-05-11T23:01:37.217 INFO:teuthology.orchestra.run.smithi005.stdout:483 write 0x2c8b887 thru 0x2c91daf (0x6529 bytes) 2024-05-11T23:01:37.230 INFO:teuthology.orchestra.run.smithi005.stdout:484 trunc from 0xd487839 to 0x37d8667 2024-05-11T23:01:37.244 INFO:teuthology.orchestra.run.smithi005.stdout:486 read 0x346ef6f thru 0x347dda1 (0xee33 bytes) 2024-05-11T23:01:37.255 INFO:teuthology.orchestra.run.smithi005.stdout:488 write 0x59d50a5 thru 0x59e4666 (0xf5c2 bytes) 2024-05-11T23:01:37.269 INFO:teuthology.orchestra.run.smithi005.stdout:489 trunc from 0x59e4667 to 0xe1828db 2024-05-11T23:01:37.274 INFO:teuthology.orchestra.run.smithi005.stdout:490 read 0x8ff68a8 thru 0x8ffb5b2 (0x4d0b bytes) 2024-05-11T23:01:37.274 INFO:teuthology.orchestra.run.smithi005.stdout:491 punch from 0x9be6ec0 to 0x9bef71e, (0x885e bytes) 2024-05-11T23:01:37.291 INFO:teuthology.orchestra.run.smithi005.stdout:492 trunc from 0xe1828db to 0x5a8b37 2024-05-11T23:01:37.301 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:01:37.302 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:01:37.307 INFO:teuthology.orchestra.run.smithi005.stdout:493 read 0x2b7683 thru 0x2bd2c8 (0x5c46 bytes) 2024-05-11T23:01:37.307 INFO:teuthology.orchestra.run.smithi005.stdout:494 punch from 0x35e551 to 0x364153, (0x5c02 bytes) 2024-05-11T23:01:37.310 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:37.310+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:37.316 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:37.316 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:37.323 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:01:37.323 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:01:37.330 INFO:teuthology.orchestra.run.smithi005.stdout:495 write 0xed91c5c thru 0xeda123d (0xf5e2 bytes) 2024-05-11T23:01:37.332 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:01:37.334 INFO:teuthology.orchestra.run.smithi005.stdout:496 writesame 0xccefb53 thru 0xccfa936 data_size 0x4(0xade4 bytes) 2024-05-11T23:01:37.341 INFO:teuthology.orchestra.run.smithi005.stdout:497 read 0xe6ffda1 thru 0xe70e972 (0xebd2 bytes) 2024-05-11T23:01:37.341 INFO:teuthology.orchestra.run.smithi005.stdout:498 writesame 0x287d2c5 thru 0x288a3b1 data_size 0x13(0xd0ed bytes) 2024-05-11T23:01:37.343 INFO:teuthology.orchestra.run.smithi005.stdout:499 punch from 0x8e037e3 to 0x8e0d494, (0x9cb1 bytes) 2024-05-11T23:01:37.343 INFO:teuthology.orchestra.run.smithi005.stdout:502 write 0x73b1e45 thru 0x73b9079 (0x7235 bytes) 2024-05-11T23:01:37.345 INFO:teuthology.orchestra.run.smithi005.stdout:505 read 0x54b5438 thru 0x54b5f84 (0xb4d bytes) 2024-05-11T23:01:37.345 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:01:37.346 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:01:37.346 INFO:teuthology.orchestra.run.smithi005.stdout:506 read 0x5a51cff thru 0x5a60a58 (0xed5a bytes) 2024-05-11T23:01:37.346 INFO:teuthology.orchestra.run.smithi005.stdout:508 write 0x5192839 thru 0x519f515 (0xccdd bytes) 2024-05-11T23:01:37.350 INFO:teuthology.orchestra.run.smithi005.stdout:510 read 0xac845fe thru 0xac86816 (0x2219 bytes) 2024-05-11T23:01:37.355 INFO:teuthology.orchestra.run.smithi005.stdout:513 trunc from 0xeda123e to 0xa9124bf 2024-05-11T23:01:37.367 INFO:teuthology.orchestra.run.smithi005.stdout:514 read 0x43b91b1 thru 0x43bc939 (0x3789 bytes) 2024-05-11T23:01:37.367 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:01:37.368 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:01:37.368 INFO:teuthology.orchestra.run.smithi005.stdout:515 read 0x2c1631c thru 0x2c1f245 (0x8f2a bytes) 2024-05-11T23:01:37.368 INFO:teuthology.orchestra.run.smithi005.stdout:516 write 0x15d3f46 thru 0x15e0d55 (0xce10 bytes) 2024-05-11T23:01:37.372 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:37.372 INFO:teuthology.orchestra.run.smithi005.stdout:519 write 0x8a2e43e thru 0x8a3bbe1 (0xd7a4 bytes) 2024-05-11T23:01:37.375 INFO:teuthology.orchestra.run.smithi005.stdout:520 write 0x59fae1 thru 0x5ade64 (0xe384 bytes) 2024-05-11T23:01:37.377 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:37.383 INFO:teuthology.orchestra.run.smithi005.stdout:522 trunc from 0xa9124bf to 0x54d83ec 2024-05-11T23:01:37.385 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:01:37.386 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:01:37.409 INFO:teuthology.orchestra.run.smithi005.stdout:524 trunc from 0x54d83ec to 0x22e9e2a 2024-05-11T23:01:37.410 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:37.410+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:37.422 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:01:37.423 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:01:37.436 INFO:teuthology.orchestra.run.smithi005.stdout:526 writesame 0x8cdd3fc thru 0x8ce1542 data_size 0x1(0x4147 bytes) 2024-05-11T23:01:37.437 INFO:teuthology.orchestra.run.smithi005.stdout:527 punch from 0x86040d0 to 0x8608e01, (0x4d31 bytes) 2024-05-11T23:01:37.441 INFO:teuthology.orchestra.run.smithi005.stdout:528 trunc from 0x8ce1543 to 0xce23a6e 2024-05-11T23:01:37.446 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:01:37.446 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:01:37.447 INFO:teuthology.orchestra.run.smithi005.stdout:529 punch from 0x47c519f to 0x47d30e8, (0xdf49 bytes) 2024-05-11T23:01:37.447 INFO:teuthology.orchestra.run.smithi005.stdout:530 write 0x8aa35b6 thru 0x8aa41f7 (0xc42 bytes) 2024-05-11T23:01:37.450 INFO:teuthology.orchestra.run.smithi005.stdout:533 write 0x2ec8be4 thru 0x2ed6e25 (0xe242 bytes) 2024-05-11T23:01:37.458 INFO:teuthology.orchestra.run.smithi005.stdout:535 writesame 0xd302d55 thru 0xd30ec49 data_size 0x5(0xbef5 bytes) 2024-05-11T23:01:37.458 INFO:teuthology.orchestra.run.smithi005.stdout:536 read 0x73f24e4 thru 0x73f9d15 (0x7832 bytes) 2024-05-11T23:01:37.458 INFO:teuthology.orchestra.run.smithi005.stdout:538 read 0xa514573 thru 0xa522ca6 (0xe734 bytes) 2024-05-11T23:01:37.458 INFO:teuthology.orchestra.run.smithi005.stdout:539 write 0xbd3fac7 thru 0xbd45dad (0x62e7 bytes) 2024-05-11T23:01:37.466 INFO:teuthology.orchestra.run.smithi005.stdout:541 write 0xd435a81 thru 0xd43c171 (0x66f1 bytes) 2024-05-11T23:01:37.469 INFO:teuthology.orchestra.run.smithi005.stdout:542 read 0xbb0f951 thru 0xbb1c000 (0xc6b0 bytes) 2024-05-11T23:01:37.469 INFO:teuthology.orchestra.run.smithi005.stdout:543 writesame 0x9531973 thru 0x953750e data_size 0x34(0x5b9c bytes) 2024-05-11T23:01:37.472 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:37.472 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:01:37.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:01:37.472 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:01:37.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.472 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:2802 3.4 3:20d95d3e:::rbd_data.111e7a935520.000000000000000b:4 [read 548864~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:37.472 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.707301+0000", 2024-05-11T23:01:37.472 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.763598338, 2024-05-11T23:01:37.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0036211030000000001, 2024-05-11T23:01:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2802 2024-05-11T23:01:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707301+0000", 2024-05-11T23:01:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:37.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707301+0000", 2024-05-11T23:01:37.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:37.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707301+0000", 2024-05-11T23:01:37.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4700000000000002e-07 2024-05-11T23:01:37.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:37.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707304+0000", 2024-05-11T23:01:37.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4030000000000001e-06 2024-05-11T23:01:37.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:37.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707310+0000", 2024-05-11T23:01:37.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6610000000000003e-06 2024-05-11T23:01:37.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:37.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707315+0000", 2024-05-11T23:01:37.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8289999999999998e-06 2024-05-11T23:01:37.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:37.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.709874+0000", 2024-05-11T23:01:37.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0025591820000000001 2024-05-11T23:01:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.709994+0000", 2024-05-11T23:01:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011917999999999999 2024-05-11T23:01:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.710922+0000", 2024-05-11T23:01:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00092840100000000003 2024-05-11T23:01:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:2834 3.4 3:20d95d3e:::rbd_data.111e7a935520.000000000000000b:4 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.707725+0000", 2024-05-11T23:01:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.76317397, 2024-05-11T23:01:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0027582219999999998, 2024-05-11T23:01:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2834 2024-05-11T23:01:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707725+0000", 2024-05-11T23:01:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707725+0000", 2024-05-11T23:01:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707726+0000", 2024-05-11T23:01:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.58e-07 2024-05-11T23:01:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707728+0000", 2024-05-11T23:01:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.401e-06 2024-05-11T23:01:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707735+0000", 2024-05-11T23:01:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5150000000000003e-06 2024-05-11T23:01:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707739+0000", 2024-05-11T23:01:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8330000000000001e-06 2024-05-11T23:01:37.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.710306+0000", 2024-05-11T23:01:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002566741 2024-05-11T23:01:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.710390+0000", 2024-05-11T23:01:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3869000000000006e-05 2024-05-11T23:01:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.710483+0000", 2024-05-11T23:01:37.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3405000000000004e-05 2024-05-11T23:01:37.481 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.481 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.481 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.481 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:2839 3.19 3:9a7df64e:::rbd_data.111e7a935520.0000000000000007:4 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:37.481 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.707746+0000", 2024-05-11T23:01:37.481 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7631530560000002, 2024-05-11T23:01:37.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0027190399999999998, 2024-05-11T23:01:37.481 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:37.481 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:37.482 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:37.482 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:37.482 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:37.482 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:37.482 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2839 2024-05-11T23:01:37.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.482 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:37.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:37.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707746+0000", 2024-05-11T23:01:37.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707746+0000", 2024-05-11T23:01:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707747+0000", 2024-05-11T23:01:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9099999999999999e-07 2024-05-11T23:01:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707749+0000", 2024-05-11T23:01:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0700000000000001e-06 2024-05-11T23:01:37.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707755+0000", 2024-05-11T23:01:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0220000000000001e-06 2024-05-11T23:01:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707759+0000", 2024-05-11T23:01:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7600000000000002e-06 2024-05-11T23:01:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.710290+0000", 2024-05-11T23:01:37.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0025309820000000002 2024-05-11T23:01:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.710372+0000", 2024-05-11T23:01:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2109999999999998e-05 2024-05-11T23:01:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.710465+0000", 2024-05-11T23:01:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2705000000000001e-05 2024-05-11T23:01:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:2843 3.19 3:9a7df64e:::rbd_data.111e7a935520.0000000000000007:4 [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.707766+0000", 2024-05-11T23:01:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7631332309999999, 2024-05-11T23:01:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003421281, 2024-05-11T23:01:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2843 2024-05-11T23:01:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.486 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707766+0000", 2024-05-11T23:01:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707766+0000", 2024-05-11T23:01:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707766+0000", 2024-05-11T23:01:37.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2599999999999998e-07 2024-05-11T23:01:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707769+0000", 2024-05-11T23:01:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3010000000000001e-06 2024-05-11T23:01:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707775+0000", 2024-05-11T23:01:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0329999999999997e-06 2024-05-11T23:01:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:37.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707779+0000", 2024-05-11T23:01:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7140000000000001e-06 2024-05-11T23:01:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.710746+0000", 2024-05-11T23:01:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0029666039999999999 2024-05-11T23:01:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.710831+0000", 2024-05-11T23:01:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5208000000000001e-05 2024-05-11T23:01:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.711187+0000", 2024-05-11T23:01:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00035599500000000001 2024-05-11T23:01:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:2846 3.4 3:20d95d3e:::rbd_data.111e7a935520.000000000000000b:4 [read 98304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.707785+0000", 2024-05-11T23:01:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.763113723, 2024-05-11T23:01:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0028916720000000001, 2024-05-11T23:01:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:37.490 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2846 2024-05-11T23:01:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707785+0000", 2024-05-11T23:01:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707785+0000", 2024-05-11T23:01:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707786+0000", 2024-05-11T23:01:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:01:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707788+0000", 2024-05-11T23:01:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.012e-06 2024-05-11T23:01:37.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707795+0000", 2024-05-11T23:01:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4699999999999996e-06 2024-05-11T23:01:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.707800+0000", 2024-05-11T23:01:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9860000000000002e-06 2024-05-11T23:01:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.710509+0000", 2024-05-11T23:01:37.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0027091049999999998 2024-05-11T23:01:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.710587+0000", 2024-05-11T23:01:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7725000000000005e-05 2024-05-11T23:01:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.710677+0000", 2024-05-11T23:01:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9983999999999998e-05 2024-05-11T23:01:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:2918 3.4 3:20d95d3e:::rbd_data.111e7a935520.000000000000000b:4 [read 491520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.713413+0000", 2024-05-11T23:01:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7574866099999999, 2024-05-11T23:01:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0027761460000000002, 2024-05-11T23:01:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 2918 2024-05-11T23:01:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:37.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.713413+0000", 2024-05-11T23:01:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.713413+0000", 2024-05-11T23:01:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.713413+0000", 2024-05-11T23:01:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.89e-07 2024-05-11T23:01:37.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.713416+0000", 2024-05-11T23:01:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8490000000000002e-06 2024-05-11T23:01:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.713424+0000", 2024-05-11T23:01:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5939999999999996e-06 2024-05-11T23:01:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.713429+0000", 2024-05-11T23:01:37.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7459999999999997e-06 2024-05-11T23:01:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.715955+0000", 2024-05-11T23:01:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0025256829999999999 2024-05-11T23:01:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.716046+0000", 2024-05-11T23:01:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1169999999999996e-05 2024-05-11T23:01:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:37.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.716189+0000", 2024-05-11T23:01:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00014261499999999999 2024-05-11T23:01:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3194 3.19 3:9a7df64e:::rbd_data.111e7a935520.0000000000000007:4 [read 638976~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.728120+0000", 2024-05-11T23:01:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7427788500000001, 2024-05-11T23:01:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0028304160000000001, 2024-05-11T23:01:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:37.499 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3194 2024-05-11T23:01:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728120+0000", 2024-05-11T23:01:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728120+0000", 2024-05-11T23:01:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728121+0000", 2024-05-11T23:01:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6600000000000002e-07 2024-05-11T23:01:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728123+0000", 2024-05-11T23:01:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.419e-06 2024-05-11T23:01:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728131+0000", 2024-05-11T23:01:37.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7360000000000001e-06 2024-05-11T23:01:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728136+0000", 2024-05-11T23:01:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2689999999999999e-06 2024-05-11T23:01:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.730776+0000", 2024-05-11T23:01:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0026402890000000001 2024-05-11T23:01:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.730858+0000", 2024-05-11T23:01:37.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2008999999999994e-05 2024-05-11T23:01:37.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:37.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.730951+0000", 2024-05-11T23:01:37.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2328000000000001e-05 2024-05-11T23:01:37.503 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.503 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.503 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.504 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3207 3.19 3:9a7df64e:::rbd_data.111e7a935520.0000000000000007:4 [read 704512~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:37.504 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.728165+0000", 2024-05-11T23:01:37.504 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.742734424, 2024-05-11T23:01:37.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0029804110000000001, 2024-05-11T23:01:37.504 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:37.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:37.504 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:37.504 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:37.504 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:37.504 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:37.505 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3207 2024-05-11T23:01:37.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.505 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:37.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:37.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728165+0000", 2024-05-11T23:01:37.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:37.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728165+0000", 2024-05-11T23:01:37.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:37.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728165+0000", 2024-05-11T23:01:37.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.34e-07 2024-05-11T23:01:37.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:37.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728167+0000", 2024-05-11T23:01:37.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0439999999999998e-06 2024-05-11T23:01:37.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:37.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728174+0000", 2024-05-11T23:01:37.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6120000000000004e-06 2024-05-11T23:01:37.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:37.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728181+0000", 2024-05-11T23:01:37.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1110000000000002e-06 2024-05-11T23:01:37.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:37.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.730980+0000", 2024-05-11T23:01:37.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0027989629999999998 2024-05-11T23:01:37.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:37.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.731059+0000", 2024-05-11T23:01:37.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9023999999999994e-05 2024-05-11T23:01:37.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:37.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.731145+0000", 2024-05-11T23:01:37.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6223000000000004e-05 2024-05-11T23:01:37.511 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.511 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.511 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.512 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3214 3.19 3:9a7df64e:::rbd_data.111e7a935520.0000000000000007:4 [read 57344~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:37.512 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.728188+0000", 2024-05-11T23:01:37.512 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7427112139999998, 2024-05-11T23:01:37.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0031613000000000001, 2024-05-11T23:01:37.512 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:37.512 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:37.513 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:37.513 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:37.513 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:37.513 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:37.513 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3214 2024-05-11T23:01:37.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.513 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:37.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:37.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728188+0000", 2024-05-11T23:01:37.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:37.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728188+0000", 2024-05-11T23:01:37.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:37.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728188+0000", 2024-05-11T23:01:37.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9900000000000001e-07 2024-05-11T23:01:37.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:37.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728190+0000", 2024-05-11T23:01:37.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0619999999999999e-06 2024-05-11T23:01:37.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:37.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728199+0000", 2024-05-11T23:01:37.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1980000000000001e-06 2024-05-11T23:01:37.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:37.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728204+0000", 2024-05-11T23:01:37.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1220000000000003e-06 2024-05-11T23:01:37.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:37.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.731176+0000", 2024-05-11T23:01:37.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0029717810000000002 2024-05-11T23:01:37.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:37.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.731252+0000", 2024-05-11T23:01:37.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6278000000000004e-05 2024-05-11T23:01:37.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:37.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.731349+0000", 2024-05-11T23:01:37.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7460000000000005e-05 2024-05-11T23:01:37.519 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.519 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.519 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.519 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3227 3.19 3:9a7df64e:::rbd_data.111e7a935520.0000000000000007:4 [read 122880~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.728232+0000", 2024-05-11T23:01:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7426669930000003, 2024-05-11T23:01:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0033291029999999999, 2024-05-11T23:01:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3227 2024-05-11T23:01:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:37.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728232+0000", 2024-05-11T23:01:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728232+0000", 2024-05-11T23:01:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728233+0000", 2024-05-11T23:01:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1800000000000001e-07 2024-05-11T23:01:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728235+0000", 2024-05-11T23:01:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0339999999999999e-06 2024-05-11T23:01:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728241+0000", 2024-05-11T23:01:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6239999999999996e-06 2024-05-11T23:01:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728246+0000", 2024-05-11T23:01:37.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0699999999999997e-06 2024-05-11T23:01:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.731384+0000", 2024-05-11T23:01:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0031379260000000001 2024-05-11T23:01:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.731460+0000", 2024-05-11T23:01:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6093999999999993e-05 2024-05-11T23:01:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.731561+0000", 2024-05-11T23:01:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000100937 2024-05-11T23:01:37.523 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3250 3.19 3:9a7df64e:::rbd_data.111e7a935520.0000000000000007:4 [read 253952~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.728294+0000", 2024-05-11T23:01:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7426051220000001, 2024-05-11T23:01:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003476341, 2024-05-11T23:01:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:37.524 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3250 2024-05-11T23:01:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728294+0000", 2024-05-11T23:01:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728294+0000", 2024-05-11T23:01:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:37.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728295+0000", 2024-05-11T23:01:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:01:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728297+0000", 2024-05-11T23:01:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1500000000000002e-06 2024-05-11T23:01:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728303+0000", 2024-05-11T23:01:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6939999999999998e-06 2024-05-11T23:01:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:37.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728308+0000", 2024-05-11T23:01:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9470000000000002e-06 2024-05-11T23:01:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.731591+0000", 2024-05-11T23:01:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0032831420000000002 2024-05-11T23:01:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.731677+0000", 2024-05-11T23:01:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5797000000000005e-05 2024-05-11T23:01:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.731770+0000", 2024-05-11T23:01:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3182000000000003e-05 2024-05-11T23:01:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3252 3.19 3:9a7df64e:::rbd_data.111e7a935520.0000000000000007:4 [read 188416~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.728315+0000", 2024-05-11T23:01:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7425840629999998, 2024-05-11T23:01:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0036785780000000001, 2024-05-11T23:01:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:37.528 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3252 2024-05-11T23:01:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:37.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728315+0000", 2024-05-11T23:01:37.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728315+0000", 2024-05-11T23:01:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728315+0000", 2024-05-11T23:01:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3700000000000001e-07 2024-05-11T23:01:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728318+0000", 2024-05-11T23:01:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.159e-06 2024-05-11T23:01:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728325+0000", 2024-05-11T23:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3170000000000004e-06 2024-05-11T23:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728330+0000", 2024-05-11T23:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1359999999999996e-06 2024-05-11T23:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.731803+0000", 2024-05-11T23:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0034730120000000001 2024-05-11T23:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.731894+0000", 2024-05-11T23:01:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0766999999999993e-05 2024-05-11T23:01:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.731994+0000", 2024-05-11T23:01:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9850000000000001e-05 2024-05-11T23:01:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3262 3.19 3:9a7df64e:::rbd_data.111e7a935520.0000000000000007:4 [read 319488~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.728362+0000", 2024-05-11T23:01:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7425368240000001, 2024-05-11T23:01:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003865285, 2024-05-11T23:01:37.533 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3262 2024-05-11T23:01:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728362+0000", 2024-05-11T23:01:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728362+0000", 2024-05-11T23:01:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728363+0000", 2024-05-11T23:01:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2e-07 2024-05-11T23:01:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728365+0000", 2024-05-11T23:01:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.784e-06 2024-05-11T23:01:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:37.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728371+0000", 2024-05-11T23:01:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5350000000000001e-06 2024-05-11T23:01:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728376+0000", 2024-05-11T23:01:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8969999999999999e-06 2024-05-11T23:01:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.732041+0000", 2024-05-11T23:01:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003664591 2024-05-11T23:01:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:37.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.732128+0000", 2024-05-11T23:01:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7748000000000001e-05 2024-05-11T23:01:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.732228+0000", 2024-05-11T23:01:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9309999999999996e-05 2024-05-11T23:01:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3275 3.19 3:9a7df64e:::rbd_data.111e7a935520.0000000000000007:4 [read 385024~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.728408+0000", 2024-05-11T23:01:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7424914249999999, 2024-05-11T23:01:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.004066783, 2024-05-11T23:01:37.537 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3275 2024-05-11T23:01:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728408+0000", 2024-05-11T23:01:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728408+0000", 2024-05-11T23:01:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728408+0000", 2024-05-11T23:01:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7599999999999997e-07 2024-05-11T23:01:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728410+0000", 2024-05-11T23:01:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1639999999999999e-06 2024-05-11T23:01:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:37.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728417+0000", 2024-05-11T23:01:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5760000000000002e-06 2024-05-11T23:01:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728422+0000", 2024-05-11T23:01:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1019999999999996e-06 2024-05-11T23:01:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.732262+0000", 2024-05-11T23:01:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0038402459999999999 2024-05-11T23:01:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:37.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.732352+0000", 2024-05-11T23:01:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0011000000000003e-05 2024-05-11T23:01:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.732475+0000", 2024-05-11T23:01:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012220800000000001 2024-05-11T23:01:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3287 3.19 3:9a7df64e:::rbd_data.111e7a935520.0000000000000007:4 [read 450560~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.728450+0000", 2024-05-11T23:01:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7424492379999998, 2024-05-11T23:01:37.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0042984590000000001, 2024-05-11T23:01:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3287 2024-05-11T23:01:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728450+0000", 2024-05-11T23:01:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728450+0000", 2024-05-11T23:01:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728450+0000", 2024-05-11T23:01:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8799999999999998e-07 2024-05-11T23:01:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728452+0000", 2024-05-11T23:01:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9999999999999999e-06 2024-05-11T23:01:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728459+0000", 2024-05-11T23:01:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4509999999999997e-06 2024-05-11T23:01:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728464+0000", 2024-05-11T23:01:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.994e-06 2024-05-11T23:01:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.732524+0000", 2024-05-11T23:01:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.004059811 2024-05-11T23:01:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:37.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.732642+0000", 2024-05-11T23:01:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118504 2024-05-11T23:01:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.732748+0000", 2024-05-11T23:01:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010631100000000001 2024-05-11T23:01:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3299 3.19 3:9a7df64e:::rbd_data.111e7a935520.0000000000000007:4 [read 516096~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.728559+0000", 2024-05-11T23:01:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.742340392, 2024-05-11T23:01:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0044327189999999999, 2024-05-11T23:01:37.545 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3299 2024-05-11T23:01:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728559+0000", 2024-05-11T23:01:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728559+0000", 2024-05-11T23:01:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728559+0000", 2024-05-11T23:01:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6199999999999998e-07 2024-05-11T23:01:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728563+0000", 2024-05-11T23:01:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.303e-06 2024-05-11T23:01:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:37.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728571+0000", 2024-05-11T23:01:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3960000000000006e-06 2024-05-11T23:01:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728579+0000", 2024-05-11T23:01:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.52e-06 2024-05-11T23:01:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.732783+0000", 2024-05-11T23:01:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0042043879999999999 2024-05-11T23:01:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:37.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.732876+0000", 2024-05-11T23:01:37.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3450999999999997e-05 2024-05-11T23:01:37.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:37.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.732992+0000", 2024-05-11T23:01:37.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000115099 2024-05-11T23:01:37.550 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.550 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.550 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.550 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3311 3.19 3:9a7df64e:::rbd_data.111e7a935520.0000000000000007:4 [read 581632~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:37.550 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.728610+0000", 2024-05-11T23:01:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7422891009999999, 2024-05-11T23:01:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046331840000000003, 2024-05-11T23:01:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3311 2024-05-11T23:01:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.551 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728610+0000", 2024-05-11T23:01:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728610+0000", 2024-05-11T23:01:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728610+0000", 2024-05-11T23:01:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6899999999999998e-07 2024-05-11T23:01:37.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728614+0000", 2024-05-11T23:01:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.134e-06 2024-05-11T23:01:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728620+0000", 2024-05-11T23:01:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3980000000000004e-06 2024-05-11T23:01:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728625+0000", 2024-05-11T23:01:37.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8919999999999999e-06 2024-05-11T23:01:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.733046+0000", 2024-05-11T23:01:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0044207930000000001 2024-05-11T23:01:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.733141+0000", 2024-05-11T23:01:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4840000000000007e-05 2024-05-11T23:01:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:37.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.733243+0000", 2024-05-11T23:01:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010275799999999999 2024-05-11T23:01:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3323 3.19 3:9a7df64e:::rbd_data.111e7a935520.0000000000000007:4 [read 647168~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.728655+0000", 2024-05-11T23:01:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7422442399999998, 2024-05-11T23:01:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0047986030000000002, 2024-05-11T23:01:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:37.555 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3323 2024-05-11T23:01:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728655+0000", 2024-05-11T23:01:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728655+0000", 2024-05-11T23:01:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728655+0000", 2024-05-11T23:01:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9799999999999999e-07 2024-05-11T23:01:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728657+0000", 2024-05-11T23:01:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1009999999999999e-06 2024-05-11T23:01:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728667+0000", 2024-05-11T23:01:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.251e-06 2024-05-11T23:01:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728672+0000", 2024-05-11T23:01:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3890000000000004e-06 2024-05-11T23:01:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.733274+0000", 2024-05-11T23:01:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046023219999999998 2024-05-11T23:01:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.733355+0000", 2024-05-11T23:01:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0302999999999993e-05 2024-05-11T23:01:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.733454+0000", 2024-05-11T23:01:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8838999999999993e-05 2024-05-11T23:01:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3327 3.4 3:20d95d3e:::rbd_data.111e7a935520.000000000000000b:4 [read 647168~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.728680+0000", 2024-05-11T23:01:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7422196149999998, 2024-05-11T23:01:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0035129269999999999, 2024-05-11T23:01:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:37.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3327 2024-05-11T23:01:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728680+0000", 2024-05-11T23:01:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728680+0000", 2024-05-11T23:01:37.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728680+0000", 2024-05-11T23:01:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9100000000000004e-07 2024-05-11T23:01:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728686+0000", 2024-05-11T23:01:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3319999999999997e-06 2024-05-11T23:01:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728696+0000", 2024-05-11T23:01:37.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4909999999999994e-06 2024-05-11T23:01:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728703+0000", 2024-05-11T23:01:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4370000000000001e-06 2024-05-11T23:01:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.732030+0000", 2024-05-11T23:01:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0033268970000000001 2024-05-11T23:01:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:37.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.732108+0000", 2024-05-11T23:01:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7504999999999997e-05 2024-05-11T23:01:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.732193+0000", 2024-05-11T23:01:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4773999999999998e-05 2024-05-11T23:01:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3335 3.19 3:9a7df64e:::rbd_data.111e7a935520.0000000000000007:4 [read 712704~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.728712+0000", 2024-05-11T23:01:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7421872249999999, 2024-05-11T23:01:37.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0049667069999999999, 2024-05-11T23:01:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3335 2024-05-11T23:01:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728712+0000", 2024-05-11T23:01:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728712+0000", 2024-05-11T23:01:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728712+0000", 2024-05-11T23:01:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:01:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728715+0000", 2024-05-11T23:01:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.216e-06 2024-05-11T23:01:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728722+0000", 2024-05-11T23:01:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7840000000000004e-06 2024-05-11T23:01:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.728728+0000", 2024-05-11T23:01:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3569999999999997e-06 2024-05-11T23:01:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.733483+0000", 2024-05-11T23:01:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0047553880000000001 2024-05-11T23:01:37.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.733574+0000", 2024-05-11T23:01:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0977000000000007e-05 2024-05-11T23:01:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.733679+0000", 2024-05-11T23:01:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010456 2024-05-11T23:01:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:37.567 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:37.567 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:37.568 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:37.568 INFO:teuthology.orchestra.run.smithi005.stdout:546 punch from 0x2587741 to 0x258b1df, (0x3a9e bytes) 2024-05-11T23:01:37.568 INFO:teuthology.orchestra.run.smithi005.stdout:547 punch from 0x3453a02 to 0x3459796, (0x5d94 bytes) 2024-05-11T23:01:37.568 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:37.569 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:37.569 INFO:teuthology.orchestra.run.smithi005.stdout:551 writesame 0x29ac8e0 thru 0x29b32af data_size 0x8(0x69d0 bytes) 2024-05-11T23:01:37.569 INFO:teuthology.orchestra.run.smithi005.stdout:553 write 0x7447ab5 thru 0x7449800 (0x1d4c bytes) 2024-05-11T23:01:37.570 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:01:37.570 INFO:teuthology.orchestra.run.smithi005.stdout:554 writesame 0xe134439 thru 0xe139849 data_size 0x1(0x5411 bytes) 2024-05-11T23:01:37.571 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:01:37.571 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:01:37.571 INFO:teuthology.orchestra.run.smithi005.stdout:556 punch from 0x661d8ac to 0x662a6d4, (0xce28 bytes) 2024-05-11T23:01:37.571 INFO:teuthology.orchestra.run.smithi005.stdout:557 write 0x4b9c195 thru 0x4b9fe02 (0x3c6e bytes) 2024-05-11T23:01:37.572 INFO:teuthology.orchestra.run.smithi005.stdout:558 write 0x9b3f368 thru 0x9b4bcef (0xc988 bytes) 2024-05-11T23:01:37.572 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-11T23:01:37.573 INFO:teuthology.orchestra.run.smithi005.stdout:559 trunc from 0xe13984a to 0x1a0a21 2024-05-11T23:01:37.573 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:37.573 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:37.574 INFO:teuthology.orchestra.run.smithi005.stdout:moved 0 / 82 (0%) 2024-05-11T23:01:37.574 INFO:teuthology.orchestra.run.smithi005.stdout:avg 10.25 2024-05-11T23:01:37.574 INFO:teuthology.orchestra.run.smithi005.stdout:stddev 3.26917 -> 3.26917 (expected baseline 2.99479) 2024-05-11T23:01:37.574 INFO:teuthology.orchestra.run.smithi005.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-05-11T23:01:37.574 INFO:teuthology.orchestra.run.smithi005.stdout:max osd.7 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-05-11T23:01:37.574 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:01:37.574 INFO:teuthology.orchestra.run.smithi005.stdout:oload 1000 2024-05-11T23:01:37.574 INFO:teuthology.orchestra.run.smithi005.stdout:max_change 0.05 2024-05-11T23:01:37.574 INFO:teuthology.orchestra.run.smithi005.stdout:max_change_osds 4 2024-05-11T23:01:37.574 INFO:teuthology.orchestra.run.smithi005.stdout:average_utilization 0.0003 2024-05-11T23:01:37.575 INFO:teuthology.orchestra.run.smithi005.stdout:overload_utilization 0.0033 2024-05-11T23:01:37.575 INFO:teuthology.orchestra.run.smithi005.stderr:no change 2024-05-11T23:01:37.575 INFO:teuthology.orchestra.run.smithi005.stdout:563 write 0xd6440e0 thru 0xd6530f3 (0xf014 bytes) 2024-05-11T23:01:37.576 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-05-11T23:01:37.576 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-05-11T23:01:37.577 INFO:teuthology.orchestra.run.smithi005.stdout:564 write 0xdb2d3d3 thru 0xdb3948c (0xc0ba bytes) 2024-05-11T23:01:37.577 INFO:teuthology.orchestra.run.smithi005.stdout:566 punch from 0x26babbc to 0x26c9c00, (0xf044 bytes) 2024-05-11T23:01:37.577 INFO:teuthology.orchestra.run.smithi005.stdout:567 read 0x7b3feb0 thru 0x7b4634f (0x64a0 bytes) 2024-05-11T23:01:37.577 INFO:teuthology.orchestra.run.smithi005.stdout:569 read 0x1d324a thru 0x1dbb77 (0x892e bytes) 2024-05-11T23:01:37.577 INFO:teuthology.orchestra.run.smithi005.stdout:570 read 0x2868a18 thru 0x2876636 (0xdc1f bytes) 2024-05-11T23:01:37.578 INFO:teuthology.orchestra.run.smithi005.stdout:571 trunc from 0xdb3948d to 0x1e4381f 2024-05-11T23:01:37.579 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:37.578+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:37.613 INFO:teuthology.orchestra.run.smithi005.stdout:573 writesame 0xec8984d thru 0xec98eeb data_size 0x73(0xf69f bytes) 2024-05-11T23:01:37.613 INFO:teuthology.orchestra.run.smithi005.stdout:574 read 0x1a95a2d thru 0x1a99119 (0x36ed bytes) 2024-05-11T23:01:37.613 INFO:teuthology.orchestra.run.smithi005.stdout:578 writesame 0x7bc8fe5 thru 0x7bd63cf data_size 0x1(0xd3eb bytes) 2024-05-11T23:01:37.655 INFO:teuthology.orchestra.run.smithi005.stdout:579 trunc from 0xec98eec to 0x1d0846f 2024-05-11T23:01:37.669 INFO:teuthology.orchestra.run.smithi005.stdout:581 trunc from 0x1d0846f to 0x28bb46b 2024-05-11T23:01:37.673 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:37.673+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:37.675 INFO:teuthology.orchestra.run.smithi005.stdout:582 read 0x26170 thru 0x282e6 (0x2177 bytes) 2024-05-11T23:01:37.692 INFO:teuthology.orchestra.run.smithi005.stdout:583 writesame 0xa14d62b thru 0xa156f4f data_size 0x1(0x9925 bytes) 2024-05-11T23:01:37.692 INFO:teuthology.orchestra.run.smithi005.stdout:584 punch from 0x784f1eb to 0x7858017, (0x8e2c bytes) 2024-05-11T23:01:37.696 INFO:teuthology.orchestra.run.smithi005.stdout:585 trunc from 0xa156f50 to 0x62d5870 2024-05-11T23:01:37.705 INFO:teuthology.orchestra.run.smithi005.stdout:587 read 0x272b92a thru 0x273ad8b (0xf462 bytes) 2024-05-11T23:01:37.708 INFO:teuthology.orchestra.run.smithi005.stdout:589 trunc from 0x62d5870 to 0x8fab166 2024-05-11T23:01:37.715 INFO:teuthology.orchestra.run.smithi005.stdout:590 punch from 0x8cb652f to 0x8cc0420, (0x9ef1 bytes) 2024-05-11T23:01:37.726 INFO:teuthology.orchestra.run.smithi005.stdout:591 writesame 0xcbd9dfa thru 0xcbdf596 data_size 0x1(0x579d bytes) 2024-05-11T23:01:37.726 INFO:teuthology.orchestra.run.smithi005.stdout:592 read 0x3a67606 thru 0x3a75f78 (0xe973 bytes) 2024-05-11T23:01:37.726 INFO:teuthology.orchestra.run.smithi005.stdout:593 write 0x5048318 thru 0x50581e9 (0xfed2 bytes) 2024-05-11T23:01:37.729 INFO:teuthology.orchestra.run.smithi005.stdout:594 write 0xb2bc559 thru 0xb2c4681 (0x8129 bytes) 2024-05-11T23:01:37.731 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:37.732 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:37.732 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:01:37.732 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:37.732 INFO:teuthology.orchestra.run.smithi005.stdout:595 write 0x7e4baaa thru 0x7e52ac8 (0x701f bytes) 2024-05-11T23:01:37.734 INFO:teuthology.orchestra.run.smithi005.stdout:597 writesame 0x3d46005 thru 0x3d53c9a data_size 0x5(0xdc96 bytes) 2024-05-11T23:01:37.739 INFO:teuthology.orchestra.run.smithi005.stdout:599 writesame 0x37f19cd thru 0x37f85b2 data_size 0x7(0x6be6 bytes) 2024-05-11T23:01:37.744 INFO:teuthology.orchestra.run.smithi005.stdout:600 write 0xb25e0f8 thru 0xb26a725 (0xc62e bytes) 2024-05-11T23:01:37.745 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:01:37.748 INFO:teuthology.orchestra.run.smithi005.stdout:604 read 0x200fc99 thru 0x201a3ea (0xa752 bytes) 2024-05-11T23:01:37.748 INFO:teuthology.orchestra.run.smithi005.stdout:605 read 0xca0c3a1 thru 0xca0c8bc (0x51c bytes) 2024-05-11T23:01:37.748 INFO:teuthology.orchestra.run.smithi005.stdout:606 writesame 0xae03cfc thru 0xae10cd1 data_size 0x25(0xcfd6 bytes) 2024-05-11T23:01:37.751 INFO:teuthology.orchestra.run.smithi005.stdout:608 read 0x3d8c31a thru 0x3d8c6d0 (0x3b7 bytes) 2024-05-11T23:01:37.758 INFO:teuthology.orchestra.run.smithi005.stdout:610 trunc from 0xcbdf597 to 0x724ee4a 2024-05-11T23:01:37.775 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:37.775+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:37.777 INFO:teuthology.orchestra.run.smithi005.stdout:611 trunc from 0x724ee4a to 0x45ad2f7 2024-05-11T23:01:37.785 INFO:teuthology.orchestra.run.smithi005.stdout:612 punch from 0xcde197 to 0xce2b31, (0x499a bytes) 2024-05-11T23:01:37.785 INFO:teuthology.orchestra.run.smithi005.stdout:614 read 0x234336e thru 0x234ce03 (0x9a96 bytes) 2024-05-11T23:01:37.786 INFO:teuthology.orchestra.run.smithi005.stdout:616 punch from 0x405dbd0 to 0x4060c47, (0x3077 bytes) 2024-05-11T23:01:37.795 INFO:teuthology.orchestra.run.smithi005.stdout:618 trunc from 0x45ad2f7 to 0xb2fcdfb 2024-05-11T23:01:37.802 INFO:teuthology.orchestra.run.smithi005.stdout:619 trunc from 0xb2fcdfb to 0xb5e1160 2024-05-11T23:01:37.807 INFO:teuthology.orchestra.run.smithi005.stdout:620 write 0x18172a7 thru 0x181fc6e (0x89c8 bytes) 2024-05-11T23:01:37.809 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:37.809 INFO:teuthology.orchestra.run.smithi005.stdout:625 punch from 0x24738a2 to 0x24763c2, (0x2b20 bytes) 2024-05-11T23:01:37.809 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:37.810 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:37.810 INFO:teuthology.orchestra.run.smithi005.stdout:629 punch from 0x5b6278c to 0x5b706c6, (0xdf3a bytes) 2024-05-11T23:01:37.819 INFO:teuthology.orchestra.run.smithi005.stdout:630 write 0xd6991c2 thru 0xd6a3414 (0xa253 bytes) 2024-05-11T23:01:37.823 INFO:teuthology.orchestra.run.smithi005.stdout:633 read 0x5a9be4 thru 0x5b7126 (0xd543 bytes) 2024-05-11T23:01:37.823 INFO:teuthology.orchestra.run.smithi005.stdout:634 read 0x67a32f4 thru 0x67a4b5c (0x1869 bytes) 2024-05-11T23:01:37.823 INFO:teuthology.orchestra.run.smithi005.stdout:636 punch from 0x4c58b36 to 0x4c5fa57, (0x6f21 bytes) 2024-05-11T23:01:37.824 INFO:teuthology.orchestra.run.smithi005.stdout:637 punch from 0x9b41f25 to 0x9b42612, (0x6ed bytes) 2024-05-11T23:01:37.824 INFO:teuthology.orchestra.run.smithi005.stdout:639 read 0xd1f320a thru 0xd1f961e (0x6415 bytes) 2024-05-11T23:01:37.824 INFO:teuthology.orchestra.run.smithi005.stdout:640 read 0x6fe032b thru 0x6fe1e7e (0x1b54 bytes) 2024-05-11T23:01:37.824 INFO:teuthology.orchestra.run.smithi005.stdout:642 read 0x337a5c2 thru 0x337dc7f (0x36be bytes) 2024-05-11T23:01:37.824 INFO:teuthology.orchestra.run.smithi005.stdout:643 punch from 0xd62a900 to 0xd6320a3, (0x77a3 bytes) 2024-05-11T23:01:37.824 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:37.824 INFO:teuthology.orchestra.run.smithi005.stdout:647 punch from 0xc257a4a to 0xc25b7d0, (0x3d86 bytes) 2024-05-11T23:01:37.824 INFO:teuthology.orchestra.run.smithi005.stdout:649 read 0x2289614 thru 0x228a3d0 (0xdbd bytes) 2024-05-11T23:01:37.824 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:37.824 INFO:teuthology.orchestra.run.smithi005.stdout:652 write 0x6f314e2 thru 0x6f33703 (0x2222 bytes) 2024-05-11T23:01:37.826 INFO:teuthology.orchestra.run.smithi005.stdout:653 punch from 0x37b3f13 to 0x37b9af3, (0x5be0 bytes) 2024-05-11T23:01:37.826 INFO:teuthology.orchestra.run.smithi005.stdout:655 write 0xcf161dd thru 0xcf181da (0x1ffe bytes) 2024-05-11T23:01:37.828 INFO:teuthology.orchestra.run.smithi005.stdout:656 read 0x83738c6 thru 0x83748df (0x101a bytes) 2024-05-11T23:01:37.828 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:37.828 INFO:teuthology.orchestra.run.smithi005.stdout:658 writesame 0x8ad3d07 thru 0x8ae0974 data_size 0x89(0xcc6e bytes) 2024-05-11T23:01:37.840 INFO:teuthology.orchestra.run.smithi005.stdout:659 write 0xed0a58c thru 0xed1343d (0x8eb2 bytes) 2024-05-11T23:01:37.854 INFO:teuthology.orchestra.run.smithi005.stdout:660 trunc from 0xed1343e to 0x49d456e 2024-05-11T23:01:37.875 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:37.875+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:37.885 INFO:teuthology.orchestra.run.smithi005.stdout:663 write 0xe75605b thru 0xe75765e (0x1604 bytes) 2024-05-11T23:01:37.890 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:37.891 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:37.891 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:01:37.891 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:01:37.891 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:37.905 INFO:teuthology.orchestra.run.smithi005.stdout:664 trunc from 0xe75765f to 0x635ebd 2024-05-11T23:01:37.907 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:01:37.930 INFO:teuthology.orchestra.run.smithi005.stdout:665 trunc from 0x635ebd to 0xd439177 2024-05-11T23:01:37.937 INFO:teuthology.orchestra.run.smithi005.stdout:666 write 0x7b58274 thru 0x7b63c9f (0xba2c bytes) 2024-05-11T23:01:37.941 INFO:teuthology.orchestra.run.smithi005.stdout:667 writesame 0xdad310 thru 0xdb443b data_size 0x2(0x712c bytes) 2024-05-11T23:01:37.949 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:37.949+0000 7fdcf2021640 -1 mgr.server reply reply (3) No such process no change 2024-05-11T23:01:37.950 INFO:teuthology.orchestra.run.smithi005.stdout:moved 4 / 82 (4.87805%) 2024-05-11T23:01:37.950 INFO:teuthology.orchestra.run.smithi005.stdout:avg 10.25 2024-05-11T23:01:37.950 INFO:teuthology.orchestra.run.smithi005.stdout:stddev 3.26917 -> 3.30719 (expected baseline 2.99479) 2024-05-11T23:01:37.950 INFO:teuthology.orchestra.run.smithi005.stdout:min osd.2 with 5 -> 6 pgs (0.487805 -> 0.585366 * mean) 2024-05-11T23:01:37.950 INFO:teuthology.orchestra.run.smithi005.stdout:max osd.7 with 14 -> 16 pgs (1.36585 -> 1.56098 * mean) 2024-05-11T23:01:37.950 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:01:37.950 INFO:teuthology.orchestra.run.smithi005.stdout:oload 110 2024-05-11T23:01:37.950 INFO:teuthology.orchestra.run.smithi005.stdout:max_change 1 2024-05-11T23:01:37.951 INFO:teuthology.orchestra.run.smithi005.stdout:max_change_osds 4 2024-05-11T23:01:37.951 INFO:teuthology.orchestra.run.smithi005.stdout:average_utilization 0.0003 2024-05-11T23:01:37.951 INFO:teuthology.orchestra.run.smithi005.stdout:overload_utilization 0.0004 2024-05-11T23:01:37.951 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 weight 1.0000 -> 0.8615 2024-05-11T23:01:37.951 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 weight 1.0000 -> 0.8649 2024-05-11T23:01:37.951 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 weight 1.0000 -> 0.8694 2024-05-11T23:01:37.951 INFO:teuthology.orchestra.run.smithi005.stderr:no change 2024-05-11T23:01:37.952 INFO:teuthology.orchestra.run.smithi005.stdout:669 trunc from 0xd439177 to 0x8c5414f 2024-05-11T23:01:37.961 INFO:teuthology.orchestra.run.smithi005.stdout:671 writesame 0xf701ad thru 0xf7ee81 data_size 0x1(0xecd5 bytes) 2024-05-11T23:01:37.964 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-05-11T23:01:37.965 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-05-11T23:01:37.976 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:37.975+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:37.987 INFO:teuthology.orchestra.run.smithi005.stdout:673 clone 6 order 24 su 65536 sc 4 2024-05-11T23:01:38.049 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:38.049 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:01:38.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:01:38.049 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:01:38.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.049 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:1390 3.0 e24/19 3:03c4b8c1:::rbd_data.111e50384849.000000000000004b:head v 24'2, mlcod=0'0)", 2024-05-11T23:01:38.049 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:32.930428+0000", 2024-05-11T23:01:38.049 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.1175568739999999, 2024-05-11T23:01:38.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00084374099999999996, 2024-05-11T23:01:38.049 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.049 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.930428+0000", 2024-05-11T23:01:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.930428+0000", 2024-05-11T23:01:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.930429+0000", 2024-05-11T23:01:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2780000000000001e-06 2024-05-11T23:01:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.930447+0000", 2024-05-11T23:01:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7592e-05 2024-05-11T23:01:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.930464+0000", 2024-05-11T23:01:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6747000000000001e-05 2024-05-11T23:01:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.930473+0000", 2024-05-11T23:01:38.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2909999999999996e-06 2024-05-11T23:01:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.930530+0000", 2024-05-11T23:01:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7401000000000002e-05 2024-05-11T23:01:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.930549+0000", 2024-05-11T23:01:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8785000000000001e-05 2024-05-11T23:01:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.931236+0000", 2024-05-11T23:01:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00068625900000000004 2024-05-11T23:01:38.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:32.931272+0000", 2024-05-11T23:01:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6387999999999998e-05 2024-05-11T23:01:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:1670 3.0 e25/19 3:03c4b8c1:::rbd_data.111e50384849.000000000000004b:head v 25'4, mlcod=24'1)", 2024-05-11T23:01:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.554035+0000", 2024-05-11T23:01:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.4939502109999996, 2024-05-11T23:01:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0011790030000000001, 2024-05-11T23:01:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.053 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.554035+0000", 2024-05-11T23:01:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.554035+0000", 2024-05-11T23:01:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.554035+0000", 2024-05-11T23:01:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9900000000000001e-07 2024-05-11T23:01:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.554042+0000", 2024-05-11T23:01:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.939e-06 2024-05-11T23:01:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.554062+0000", 2024-05-11T23:01:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0256000000000001e-05 2024-05-11T23:01:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.554072+0000", 2024-05-11T23:01:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4760000000000004e-06 2024-05-11T23:01:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.554148+0000", 2024-05-11T23:01:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5865999999999995e-05 2024-05-11T23:01:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.554173+0000", 2024-05-11T23:01:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4839000000000001e-05 2024-05-11T23:01:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:38.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.555166+0000", 2024-05-11T23:01:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00099379299999999998 2024-05-11T23:01:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.555214+0000", 2024-05-11T23:01:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7435000000000002e-05 2024-05-11T23:01:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "pg_update_log_missing(3.4 epoch 25/19 rep_tid 71 entries 25'17 (0'0) error 3:21f195d1:::rbd_data.111e50384849.0000000000000059:head by client.4382.0:1859 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 23'14)", 2024-05-11T23:01:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.569432+0000", 2024-05-11T23:01:38.057 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.4785535169999999, 2024-05-11T23:01:38.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00080634299999999995, 2024-05-11T23:01:38.058 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.058 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:38.058 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569432+0000", 2024-05-11T23:01:38.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569432+0000", 2024-05-11T23:01:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569433+0000", 2024-05-11T23:01:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.658e-06 2024-05-11T23:01:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569446+0000", 2024-05-11T23:01:38.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2679e-05 2024-05-11T23:01:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569478+0000", 2024-05-11T23:01:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2008000000000003e-05 2024-05-11T23:01:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569496+0000", 2024-05-11T23:01:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8479999999999999e-05 2024-05-11T23:01:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.569602+0000", 2024-05-11T23:01:38.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010609 2024-05-11T23:01:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.570238+0000", 2024-05-11T23:01:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00063542799999999995 2024-05-11T23:01:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "pg_update_log_missing(3.0 epoch 25/19 rep_tid 74 entries 25'5 (0'0) error 3:0097cebb:::rbd_data.111e50384849.0000000000000062:head by client.4382.0:2022 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 24'2)", 2024-05-11T23:01:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.583075+0000", 2024-05-11T23:01:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.4649103400000003, 2024-05-11T23:01:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001200989, 2024-05-11T23:01:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.061 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.062 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:38.062 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583075+0000", 2024-05-11T23:01:38.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583075+0000", 2024-05-11T23:01:38.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583076+0000", 2024-05-11T23:01:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.215e-06 2024-05-11T23:01:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583090+0000", 2024-05-11T23:01:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3798000000000001e-05 2024-05-11T23:01:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583119+0000", 2024-05-11T23:01:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9669999999999999e-05 2024-05-11T23:01:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583139+0000", 2024-05-11T23:01:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.98e-05 2024-05-11T23:01:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583225+0000", 2024-05-11T23:01:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6113999999999996e-05 2024-05-11T23:01:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.584276+0000", 2024-05-11T23:01:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001050392 2024-05-11T23:01:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:2023 3.4 e25/19 3:25486f9d:::rbd_data.111e50384849.000000000000005e:head v 25'20, mlcod=25'18)", 2024-05-11T23:01:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.583356+0000", 2024-05-11T23:01:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.4646287759999996, 2024-05-11T23:01:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00095195800000000004, 2024-05-11T23:01:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:38.065 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583356+0000", 2024-05-11T23:01:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583356+0000", 2024-05-11T23:01:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583357+0000", 2024-05-11T23:01:38.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5499999999999998e-07 2024-05-11T23:01:38.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583362+0000", 2024-05-11T23:01:38.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4890000000000003e-06 2024-05-11T23:01:38.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583371+0000", 2024-05-11T23:01:38.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7960000000000001e-06 2024-05-11T23:01:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583377+0000", 2024-05-11T23:01:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9150000000000001e-06 2024-05-11T23:01:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583429+0000", 2024-05-11T23:01:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1795000000000001e-05 2024-05-11T23:01:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.583448+0000", 2024-05-11T23:01:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9511999999999999e-05 2024-05-11T23:01:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.584273+0000", 2024-05-11T23:01:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00082449499999999996 2024-05-11T23:01:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.584308+0000", 2024-05-11T23:01:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5500999999999998e-05 2024-05-11T23:01:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.071 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.072 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:2112 3.0 e25/19 3:0097cebb:::rbd_data.111e50384849.0000000000000062:head v 25'6, mlcod=25'4)", 2024-05-11T23:01:38.072 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.588426+0000", 2024-05-11T23:01:38.072 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.4595591900000002, 2024-05-11T23:01:38.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00079788199999999997, 2024-05-11T23:01:38.072 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.072 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.072 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:38.072 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.588426+0000", 2024-05-11T23:01:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.588426+0000", 2024-05-11T23:01:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.588427+0000", 2024-05-11T23:01:38.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.432e-06 2024-05-11T23:01:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.588435+0000", 2024-05-11T23:01:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.746e-06 2024-05-11T23:01:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.588455+0000", 2024-05-11T23:01:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9573999999999999e-05 2024-05-11T23:01:38.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.588464+0000", 2024-05-11T23:01:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1169999999999993e-06 2024-05-11T23:01:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.588525+0000", 2024-05-11T23:01:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0781e-05 2024-05-11T23:01:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.588548+0000", 2024-05-11T23:01:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3110000000000001e-05 2024-05-11T23:01:38.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.589179+0000", 2024-05-11T23:01:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00063125200000000001 2024-05-11T23:01:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.589224+0000", 2024-05-11T23:01:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4870000000000002e-05 2024-05-11T23:01:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:3773 3.0 e25/19 3:07bd8f02:::rbd_data.111e89cdde7d.0000000000000033:head v 25'12, mlcod=25'10)", 2024-05-11T23:01:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:34.185354+0000", 2024-05-11T23:01:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.862631038, 2024-05-11T23:01:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00080682200000000001, 2024-05-11T23:01:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.185354+0000", 2024-05-11T23:01:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.185354+0000", 2024-05-11T23:01:38.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.185355+0000", 2024-05-11T23:01:38.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9100000000000002e-07 2024-05-11T23:01:38.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.185364+0000", 2024-05-11T23:01:38.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0580000000000004e-06 2024-05-11T23:01:38.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.185376+0000", 2024-05-11T23:01:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1936999999999999e-05 2024-05-11T23:01:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.185382+0000", 2024-05-11T23:01:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7359999999999998e-06 2024-05-11T23:01:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.185450+0000", 2024-05-11T23:01:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8825999999999995e-05 2024-05-11T23:01:38.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.185477+0000", 2024-05-11T23:01:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6081e-05 2024-05-11T23:01:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.186133+0000", 2024-05-11T23:01:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00065617900000000001 2024-05-11T23:01:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.186161+0000", 2024-05-11T23:01:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8113999999999999e-05 2024-05-11T23:01:38.080 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:3828 3.0 e25/19 3:05412a4a:::rbd_data.111e89cdde7d.0000000000000005:head v 25'13, mlcod=25'11)", 2024-05-11T23:01:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:34.218355+0000", 2024-05-11T23:01:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.8296302949999999, 2024-05-11T23:01:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001051424, 2024-05-11T23:01:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.218355+0000", 2024-05-11T23:01:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.218355+0000", 2024-05-11T23:01:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.218356+0000", 2024-05-11T23:01:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6390000000000001e-06 2024-05-11T23:01:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.218373+0000", 2024-05-11T23:01:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6735000000000002e-05 2024-05-11T23:01:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.218394+0000", 2024-05-11T23:01:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0771000000000002e-05 2024-05-11T23:01:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.218404+0000", 2024-05-11T23:01:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0169999999999999e-05 2024-05-11T23:01:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.218498+0000", 2024-05-11T23:01:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4186999999999998e-05 2024-05-11T23:01:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.218525+0000", 2024-05-11T23:01:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7123000000000001e-05 2024-05-11T23:01:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.219360+0000", 2024-05-11T23:01:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00083427999999999996 2024-05-11T23:01:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.219406+0000", 2024-05-11T23:01:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6519e-05 2024-05-11T23:01:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:3880 3.9 e25/19 3:930a78aa:::rbd_data.111e89cdde7d.0000000000000017:head v 25'3, mlcod=25'1)", 2024-05-11T23:01:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:34.261218+0000", 2024-05-11T23:01:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7867666450000002, 2024-05-11T23:01:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078769699999999996, 2024-05-11T23:01:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.085 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.261218+0000", 2024-05-11T23:01:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.261218+0000", 2024-05-11T23:01:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.261220+0000", 2024-05-11T23:01:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3260000000000001e-06 2024-05-11T23:01:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.261238+0000", 2024-05-11T23:01:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8763999999999999e-05 2024-05-11T23:01:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.261257+0000", 2024-05-11T23:01:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8329999999999999e-05 2024-05-11T23:01:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.261267+0000", 2024-05-11T23:01:38.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8490000000000001e-06 2024-05-11T23:01:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.261330+0000", 2024-05-11T23:01:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2972000000000006e-05 2024-05-11T23:01:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.261355+0000", 2024-05-11T23:01:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5114e-05 2024-05-11T23:01:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:38.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.261974+0000", 2024-05-11T23:01:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00061926099999999997 2024-05-11T23:01:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.262006+0000", 2024-05-11T23:01:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2081e-05 2024-05-11T23:01:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:3956 3.0 e25/19 3:04edcdf9:::rbd_data.111e89cdde7d.00000000000000af:head v 25'15, mlcod=25'13)", 2024-05-11T23:01:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:34.341621+0000", 2024-05-11T23:01:38.089 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7063639290000001, 2024-05-11T23:01:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00095440800000000001, 2024-05-11T23:01:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.341621+0000", 2024-05-11T23:01:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.341621+0000", 2024-05-11T23:01:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.341622+0000", 2024-05-11T23:01:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.327e-06 2024-05-11T23:01:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.341641+0000", 2024-05-11T23:01:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8592000000000001e-05 2024-05-11T23:01:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.341660+0000", 2024-05-11T23:01:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9426999999999999e-05 2024-05-11T23:01:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.341670+0000", 2024-05-11T23:01:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5340000000000005e-06 2024-05-11T23:01:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.341736+0000", 2024-05-11T23:01:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6377999999999994e-05 2024-05-11T23:01:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.341758+0000", 2024-05-11T23:01:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1469999999999999e-05 2024-05-11T23:01:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.342534+0000", 2024-05-11T23:01:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00077617599999999995 2024-05-11T23:01:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.342576+0000", 2024-05-11T23:01:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1504000000000001e-05 2024-05-11T23:01:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:3977 3.4 e25/19 3:25610a0f:::rbd_data.111e89cdde7d.00000000000000a3:head v 25'21, mlcod=25'19)", 2024-05-11T23:01:38.093 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:34.357524+0000", 2024-05-11T23:01:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.6904609640000001, 2024-05-11T23:01:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00082257399999999996, 2024-05-11T23:01:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.357524+0000", 2024-05-11T23:01:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.357524+0000", 2024-05-11T23:01:38.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.357525+0000", 2024-05-11T23:01:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.083e-06 2024-05-11T23:01:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.357543+0000", 2024-05-11T23:01:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7441000000000001e-05 2024-05-11T23:01:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.357557+0000", 2024-05-11T23:01:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4466e-05 2024-05-11T23:01:38.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.357566+0000", 2024-05-11T23:01:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8929999999999994e-06 2024-05-11T23:01:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.357621+0000", 2024-05-11T23:01:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5494000000000001e-05 2024-05-11T23:01:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.357644+0000", 2024-05-11T23:01:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2833999999999999e-05 2024-05-11T23:01:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.358300+0000", 2024-05-11T23:01:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00065602200000000003 2024-05-11T23:01:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.358347+0000", 2024-05-11T23:01:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6341000000000002e-05 2024-05-11T23:01:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:3995 3.4 e25/19 3:25610a0f:::rbd_data.111e89cdde7d.00000000000000a3:head v 25'22, mlcod=25'20)", 2024-05-11T23:01:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:34.370337+0000", 2024-05-11T23:01:38.097 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.6776481739999998, 2024-05-11T23:01:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078695000000000002, 2024-05-11T23:01:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.370337+0000", 2024-05-11T23:01:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.370337+0000", 2024-05-11T23:01:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.370338+0000", 2024-05-11T23:01:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6080000000000001e-06 2024-05-11T23:01:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.370347+0000", 2024-05-11T23:01:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9660000000000002e-06 2024-05-11T23:01:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.370368+0000", 2024-05-11T23:01:38.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0360000000000002e-05 2024-05-11T23:01:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.370380+0000", 2024-05-11T23:01:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1958e-05 2024-05-11T23:01:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.370453+0000", 2024-05-11T23:01:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3200000000000004e-05 2024-05-11T23:01:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.370481+0000", 2024-05-11T23:01:38.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8192e-05 2024-05-11T23:01:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.371079+0000", 2024-05-11T23:01:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00059805100000000005 2024-05-11T23:01:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.371124+0000", 2024-05-11T23:01:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4614999999999999e-05 2024-05-11T23:01:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:4025 3.0 e25/19 3:07bd8f02:::rbd_data.111e89cdde7d.0000000000000033:head v 25'18, mlcod=25'16)", 2024-05-11T23:01:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:34.386815+0000", 2024-05-11T23:01:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.661170083, 2024-05-11T23:01:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00083478700000000003, 2024-05-11T23:01:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.386815+0000", 2024-05-11T23:01:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.386815+0000", 2024-05-11T23:01:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.386816+0000", 2024-05-11T23:01:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.248e-06 2024-05-11T23:01:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.386853+0000", 2024-05-11T23:01:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6884000000000001e-05 2024-05-11T23:01:38.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.386869+0000", 2024-05-11T23:01:38.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5954999999999999e-05 2024-05-11T23:01:38.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.386878+0000", 2024-05-11T23:01:38.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9400000000000008e-06 2024-05-11T23:01:38.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.386951+0000", 2024-05-11T23:01:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2673e-05 2024-05-11T23:01:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.386974+0000", 2024-05-11T23:01:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2836e-05 2024-05-11T23:01:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.387610+0000", 2024-05-11T23:01:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00063691799999999999 2024-05-11T23:01:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.387650+0000", 2024-05-11T23:01:38.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9332999999999998e-05 2024-05-11T23:01:38.106 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.106 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.106 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.106 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:4079 3.0 e25/19 3:00b71501:::rbd_data.111e89cdde7d.000000000000002b:head v 25'19, mlcod=25'17)", 2024-05-11T23:01:38.106 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:34.449448+0000", 2024-05-11T23:01:38.106 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.5985371289999999, 2024-05-11T23:01:38.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00084739400000000001, 2024-05-11T23:01:38.106 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.106 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:38.107 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.449448+0000", 2024-05-11T23:01:38.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.449448+0000", 2024-05-11T23:01:38.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.449449+0000", 2024-05-11T23:01:38.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.579e-06 2024-05-11T23:01:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.449460+0000", 2024-05-11T23:01:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0706e-05 2024-05-11T23:01:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.449480+0000", 2024-05-11T23:01:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0006000000000001e-05 2024-05-11T23:01:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.449490+0000", 2024-05-11T23:01:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0049e-05 2024-05-11T23:01:38.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.449576+0000", 2024-05-11T23:01:38.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6062000000000004e-05 2024-05-11T23:01:38.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.449604+0000", 2024-05-11T23:01:38.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7654999999999999e-05 2024-05-11T23:01:38.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:38.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.450246+0000", 2024-05-11T23:01:38.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00064220599999999998 2024-05-11T23:01:38.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.450295+0000", 2024-05-11T23:01:38.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9131000000000001e-05 2024-05-11T23:01:38.110 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.110 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.110 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.110 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:4080 3.9 e25/19 3:9205b2af:::rbd_data.111e89cdde7d.000000000000002c:head v 25'6, mlcod=25'4)", 2024-05-11T23:01:38.110 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:34.449721+0000", 2024-05-11T23:01:38.110 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.5982636509999999, 2024-05-11T23:01:38.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00085231100000000002, 2024-05-11T23:01:38.110 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.111 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.111 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:38.111 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.449721+0000", 2024-05-11T23:01:38.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.449721+0000", 2024-05-11T23:01:38.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.449722+0000", 2024-05-11T23:01:38.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1099999999999995e-07 2024-05-11T23:01:38.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.449729+0000", 2024-05-11T23:01:38.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6869999999999997e-06 2024-05-11T23:01:38.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.449743+0000", 2024-05-11T23:01:38.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.448e-05 2024-05-11T23:01:38.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.449750+0000", 2024-05-11T23:01:38.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6069999999999996e-06 2024-05-11T23:01:38.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.449865+0000", 2024-05-11T23:01:38.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000115089 2024-05-11T23:01:38.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.449895+0000", 2024-05-11T23:01:38.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0093000000000001e-05 2024-05-11T23:01:38.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:38.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.450533+0000", 2024-05-11T23:01:38.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00063759400000000001 2024-05-11T23:01:38.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.450574+0000", 2024-05-11T23:01:38.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1149999999999997e-05 2024-05-11T23:01:38.114 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.114 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.114 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.114 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:4087 3.0 e25/19 3:07bd8f02:::rbd_data.111e89cdde7d.0000000000000033:head v 25'20, mlcod=25'18)", 2024-05-11T23:01:38.114 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:34.450984+0000", 2024-05-11T23:01:38.114 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.5970015169999998, 2024-05-11T23:01:38.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00081335900000000004, 2024-05-11T23:01:38.115 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.115 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.115 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:38.115 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.450984+0000", 2024-05-11T23:01:38.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.450984+0000", 2024-05-11T23:01:38.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.450984+0000", 2024-05-11T23:01:38.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9299999999999997e-07 2024-05-11T23:01:38.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.450990+0000", 2024-05-11T23:01:38.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.446e-06 2024-05-11T23:01:38.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.451003+0000", 2024-05-11T23:01:38.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3611000000000001e-05 2024-05-11T23:01:38.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.451010+0000", 2024-05-11T23:01:38.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8190000000000003e-06 2024-05-11T23:01:38.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.451075+0000", 2024-05-11T23:01:38.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5220000000000002e-05 2024-05-11T23:01:38.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.451096+0000", 2024-05-11T23:01:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0835999999999998e-05 2024-05-11T23:01:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.451757+0000", 2024-05-11T23:01:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00066034399999999999 2024-05-11T23:01:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.451797+0000", 2024-05-11T23:01:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0290000000000002e-05 2024-05-11T23:01:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.118 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.119 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.119 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:4209 3.9 e25/19 3:92dfc630:::rbd_data.111e89cdde7d.00000000000000ed:head v 25'10, mlcod=25'8)", 2024-05-11T23:01:38.119 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:34.553012+0000", 2024-05-11T23:01:38.119 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.4949733250000001, 2024-05-11T23:01:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00091672100000000001, 2024-05-11T23:01:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.553012+0000", 2024-05-11T23:01:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.553012+0000", 2024-05-11T23:01:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.553013+0000", 2024-05-11T23:01:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.362e-06 2024-05-11T23:01:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.553034+0000", 2024-05-11T23:01:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0976e-05 2024-05-11T23:01:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.553054+0000", 2024-05-11T23:01:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9599999999999999e-05 2024-05-11T23:01:38.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.553064+0000", 2024-05-11T23:01:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0156000000000001e-05 2024-05-11T23:01:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.553116+0000", 2024-05-11T23:01:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2095000000000002e-05 2024-05-11T23:01:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.553131+0000", 2024-05-11T23:01:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5041e-05 2024-05-11T23:01:38.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.553894+0000", 2024-05-11T23:01:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000762559 2024-05-11T23:01:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.553928+0000", 2024-05-11T23:01:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4931999999999997e-05 2024-05-11T23:01:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.123 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:6279 3.9 e28/19 3:9601c7aa:::rbd_data.111e65d1e80f.0000000000000019:head v 28'13, mlcod=25'11)", 2024-05-11T23:01:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:37.356781+0000", 2024-05-11T23:01:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.69120361500000005, 2024-05-11T23:01:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00091867999999999995, 2024-05-11T23:01:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.356781+0000", 2024-05-11T23:01:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.356781+0000", 2024-05-11T23:01:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.356784+0000", 2024-05-11T23:01:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2079999999999999e-06 2024-05-11T23:01:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.356797+0000", 2024-05-11T23:01:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3291e-05 2024-05-11T23:01:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.356821+0000", 2024-05-11T23:01:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4318999999999999e-05 2024-05-11T23:01:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.356832+0000", 2024-05-11T23:01:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1114000000000001e-05 2024-05-11T23:01:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.356944+0000", 2024-05-11T23:01:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000111853 2024-05-11T23:01:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.356974+0000", 2024-05-11T23:01:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9454e-05 2024-05-11T23:01:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.357641+0000", 2024-05-11T23:01:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00066727300000000002 2024-05-11T23:01:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.357700+0000", 2024-05-11T23:01:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9168e-05 2024-05-11T23:01:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:6298 3.9 e28/19 3:9601c7aa:::rbd_data.111e65d1e80f.0000000000000019:head v 28'14, mlcod=28'12)", 2024-05-11T23:01:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:37.386839+0000", 2024-05-11T23:01:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.66114583599999999, 2024-05-11T23:01:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00078938499999999998, 2024-05-11T23:01:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.386839+0000", 2024-05-11T23:01:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.386839+0000", 2024-05-11T23:01:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.386841+0000", 2024-05-11T23:01:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.505e-06 2024-05-11T23:01:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.386850+0000", 2024-05-11T23:01:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2939999999999994e-06 2024-05-11T23:01:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.386871+0000", 2024-05-11T23:01:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1188000000000001e-05 2024-05-11T23:01:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.386882+0000", 2024-05-11T23:01:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0338e-05 2024-05-11T23:01:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.386962+0000", 2024-05-11T23:01:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0222000000000005e-05 2024-05-11T23:01:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.386985+0000", 2024-05-11T23:01:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3362000000000001e-05 2024-05-11T23:01:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.387583+0000", 2024-05-11T23:01:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00059802300000000004 2024-05-11T23:01:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.387629+0000", 2024-05-11T23:01:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5453e-05 2024-05-11T23:01:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.131 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:6403 3.9 e28/19 3:9601c7aa:::rbd_data.111e65d1e80f.0000000000000019:head v 28'15, mlcod=28'13)", 2024-05-11T23:01:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:37.488426+0000", 2024-05-11T23:01:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.559558895, 2024-05-11T23:01:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00080037800000000003, 2024-05-11T23:01:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.488426+0000", 2024-05-11T23:01:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.488426+0000", 2024-05-11T23:01:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.488427+0000", 2024-05-11T23:01:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.13e-06 2024-05-11T23:01:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.488471+0000", 2024-05-11T23:01:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3628000000000003e-05 2024-05-11T23:01:38.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.488486+0000", 2024-05-11T23:01:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4620999999999999e-05 2024-05-11T23:01:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.488493+0000", 2024-05-11T23:01:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0910000000000004e-06 2024-05-11T23:01:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.488548+0000", 2024-05-11T23:01:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5117000000000001e-05 2024-05-11T23:01:38.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.488564+0000", 2024-05-11T23:01:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6489000000000001e-05 2024-05-11T23:01:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.489189+0000", 2024-05-11T23:01:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00062461199999999998 2024-05-11T23:01:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:37.489227+0000", 2024-05-11T23:01:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.769e-05 2024-05-11T23:01:38.135 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.136 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.136 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.136 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.136 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.136 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:38.137 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:01:38.138 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:38.137+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:38.238 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:38.238+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:38.248 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:38.248 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:38.248 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:01:38.248 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:38.257 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:01:38.290 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-05-11T23:01:38.289+0000 7fdcf2021640 -1 mgr.server reply reply (3) No such process no change 2024-05-11T23:01:38.290 INFO:teuthology.orchestra.run.smithi005.stdout:moved 1 / 82 (1.21951%) 2024-05-11T23:01:38.290 INFO:teuthology.orchestra.run.smithi005.stdout:avg 10.25 2024-05-11T23:01:38.290 INFO:teuthology.orchestra.run.smithi005.stdout:stddev 3.26917 -> 3.34477 (expected baseline 2.99479) 2024-05-11T23:01:38.290 INFO:teuthology.orchestra.run.smithi005.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-05-11T23:01:38.290 INFO:teuthology.orchestra.run.smithi005.stdout:max osd.7 with 14 -> 15 pgs (1.36585 -> 1.46341 * mean) 2024-05-11T23:01:38.290 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:01:38.290 INFO:teuthology.orchestra.run.smithi005.stdout:oload 110 2024-05-11T23:01:38.290 INFO:teuthology.orchestra.run.smithi005.stdout:max_change 0.05 2024-05-11T23:01:38.291 INFO:teuthology.orchestra.run.smithi005.stdout:max_change_osds 4 2024-05-11T23:01:38.291 INFO:teuthology.orchestra.run.smithi005.stdout:average_utilization 0.0003 2024-05-11T23:01:38.291 INFO:teuthology.orchestra.run.smithi005.stdout:overload_utilization 0.0004 2024-05-11T23:01:38.291 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 weight 1.0000 -> 0.9500 2024-05-11T23:01:38.291 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 weight 1.0000 -> 0.9500 2024-05-11T23:01:38.291 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 weight 1.0000 -> 0.9500 2024-05-11T23:01:38.291 INFO:teuthology.orchestra.run.smithi005.stderr:no change 2024-05-11T23:01:38.303 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-05-11T23:01:38.303 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-05-11T23:01:38.339 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:38.339+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:38.368 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:38.368 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:38.368 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:01:38.369 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:01:38.369 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:38.378 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:01:38.462 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:38.462+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:38.532 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:38.532 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:01:38.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:01:38.532 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:01:38.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.532 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:1966 3.7 3:e5c0e1bc:::rbd_data.111e50384849.0000000000000051:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:38.532 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.578077+0000", 2024-05-11T23:01:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.9532635889999996, 2024-05-11T23:01:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012474607, 2024-05-11T23:01:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1966 2024-05-11T23:01:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578077+0000", 2024-05-11T23:01:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578077+0000", 2024-05-11T23:01:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578078+0000", 2024-05-11T23:01:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5600000000000005e-07 2024-05-11T23:01:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578081+0000", 2024-05-11T23:01:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6169999999999998e-06 2024-05-11T23:01:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578091+0000", 2024-05-11T23:01:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7070000000000004e-06 2024-05-11T23:01:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578097+0000", 2024-05-11T23:01:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2929999999999997e-06 2024-05-11T23:01:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.581103+0000", 2024-05-11T23:01:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0030059240000000001 2024-05-11T23:01:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.581179+0000", 2024-05-11T23:01:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6038000000000006e-05 2024-05-11T23:01:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 7", 2024-05-11T23:01:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.581434+0000", 2024-05-11T23:01:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025452 2024-05-11T23:01:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.584458+0000", 2024-05-11T23:01:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0030242889999999999 2024-05-11T23:01:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.590448+0000", 2024-05-11T23:01:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0059902200000000001 2024-05-11T23:01:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.590509+0000", 2024-05-11T23:01:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0673e-05 2024-05-11T23:01:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.590552+0000", 2024-05-11T23:01:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2670000000000003e-05 2024-05-11T23:01:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3119 3.1e 3:78cd6643:::rbd_data.111e7a935520.0000000000000004:4 [read 245760~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749195+0000", 2024-05-11T23:01:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7821453280000004, 2024-05-11T23:01:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012256286999999999, 2024-05-11T23:01:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:38.538 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3119 2024-05-11T23:01:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749195+0000", 2024-05-11T23:01:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749195+0000", 2024-05-11T23:01:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749196+0000", 2024-05-11T23:01:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4099999999999999e-07 2024-05-11T23:01:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749198+0000", 2024-05-11T23:01:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1409999999999999e-06 2024-05-11T23:01:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749203+0000", 2024-05-11T23:01:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.366e-06 2024-05-11T23:01:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749207+0000", 2024-05-11T23:01:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9349999999999996e-06 2024-05-11T23:01:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761351+0000", 2024-05-11T23:01:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012143426000000001 2024-05-11T23:01:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761396+0000", 2024-05-11T23:01:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5361999999999997e-05 2024-05-11T23:01:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761452+0000", 2024-05-11T23:01:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5516000000000002e-05 2024-05-11T23:01:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.543 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3129 3.1e 3:7877a10d:::rbd_data.111e7a935520.0000000000000001:4 [read 311296~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:38.543 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749226+0000", 2024-05-11T23:01:38.543 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7821146109999999, 2024-05-11T23:01:38.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012371007, 2024-05-11T23:01:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3129 2024-05-11T23:01:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749226+0000", 2024-05-11T23:01:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749226+0000", 2024-05-11T23:01:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749227+0000", 2024-05-11T23:01:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8500000000000002e-07 2024-05-11T23:01:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749229+0000", 2024-05-11T23:01:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2970000000000002e-06 2024-05-11T23:01:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749235+0000", 2024-05-11T23:01:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5810000000000002e-06 2024-05-11T23:01:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749240+0000", 2024-05-11T23:01:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6829999999999999e-06 2024-05-11T23:01:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761485+0000", 2024-05-11T23:01:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012245056000000001 2024-05-11T23:01:38.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761535+0000", 2024-05-11T23:01:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9589000000000002e-05 2024-05-11T23:01:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761597+0000", 2024-05-11T23:01:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2316000000000004e-05 2024-05-11T23:01:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3133 3.1e 3:78cd6643:::rbd_data.111e7a935520.0000000000000004:4 [read 311296~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:38.547 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749291+0000", 2024-05-11T23:01:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7820500719999997, 2024-05-11T23:01:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012427446, 2024-05-11T23:01:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3133 2024-05-11T23:01:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749291+0000", 2024-05-11T23:01:38.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749291+0000", 2024-05-11T23:01:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749291+0000", 2024-05-11T23:01:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4600000000000005e-07 2024-05-11T23:01:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749293+0000", 2024-05-11T23:01:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2180000000000002e-06 2024-05-11T23:01:38.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749299+0000", 2024-05-11T23:01:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.93e-06 2024-05-11T23:01:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749305+0000", 2024-05-11T23:01:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4199999999999998e-06 2024-05-11T23:01:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761614+0000", 2024-05-11T23:01:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012309621999999999 2024-05-11T23:01:38.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761663+0000", 2024-05-11T23:01:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.884e-05 2024-05-11T23:01:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761718+0000", 2024-05-11T23:01:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4870000000000002e-05 2024-05-11T23:01:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.551 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3141 3.1e 3:7877a10d:::rbd_data.111e7a935520.0000000000000001:4 [read 376832~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749311+0000", 2024-05-11T23:01:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7820293810000001, 2024-05-11T23:01:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012575493, 2024-05-11T23:01:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3141 2024-05-11T23:01:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749311+0000", 2024-05-11T23:01:38.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749311+0000", 2024-05-11T23:01:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749312+0000", 2024-05-11T23:01:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:01:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749314+0000", 2024-05-11T23:01:38.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0549999999999998e-06 2024-05-11T23:01:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749321+0000", 2024-05-11T23:01:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5340000000000002e-06 2024-05-11T23:01:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749325+0000", 2024-05-11T23:01:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8990000000000003e-06 2024-05-11T23:01:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761757+0000", 2024-05-11T23:01:38.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012431413000000001 2024-05-11T23:01:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761828+0000", 2024-05-11T23:01:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1741000000000003e-05 2024-05-11T23:01:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761887+0000", 2024-05-11T23:01:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8468000000000003e-05 2024-05-11T23:01:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.555 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3144 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749351+0000", 2024-05-11T23:01:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7819898920000004, 2024-05-11T23:01:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012554991, 2024-05-11T23:01:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3144 2024-05-11T23:01:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749351+0000", 2024-05-11T23:01:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749351+0000", 2024-05-11T23:01:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749351+0000", 2024-05-11T23:01:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0299999999999999e-07 2024-05-11T23:01:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749353+0000", 2024-05-11T23:01:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9609999999999999e-06 2024-05-11T23:01:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749359+0000", 2024-05-11T23:01:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9359999999999996e-06 2024-05-11T23:01:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749364+0000", 2024-05-11T23:01:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4379999999999997e-06 2024-05-11T23:01:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761901+0000", 2024-05-11T23:01:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012537074 2024-05-11T23:01:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761906+0000", 2024-05-11T23:01:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.079e-06 2024-05-11T23:01:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.559 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3152 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749388+0000", 2024-05-11T23:01:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7819526129999996, 2024-05-11T23:01:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012551034000000001, 2024-05-11T23:01:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3152 2024-05-11T23:01:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.560 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749388+0000", 2024-05-11T23:01:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749388+0000", 2024-05-11T23:01:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749389+0000", 2024-05-11T23:01:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4300000000000002e-07 2024-05-11T23:01:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749393+0000", 2024-05-11T23:01:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8619999999999996e-06 2024-05-11T23:01:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749402+0000", 2024-05-11T23:01:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1050000000000001e-06 2024-05-11T23:01:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749430+0000", 2024-05-11T23:01:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8039e-05 2024-05-11T23:01:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761929+0000", 2024-05-11T23:01:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012498746999999999 2024-05-11T23:01:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761939+0000", 2024-05-11T23:01:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0538e-05 2024-05-11T23:01:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.563 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3155 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749503+0000", 2024-05-11T23:01:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7818374449999999, 2024-05-11T23:01:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012482586, 2024-05-11T23:01:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3155 2024-05-11T23:01:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749503+0000", 2024-05-11T23:01:38.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749503+0000", 2024-05-11T23:01:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749504+0000", 2024-05-11T23:01:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0299999999999999e-07 2024-05-11T23:01:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749511+0000", 2024-05-11T23:01:38.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4560000000000003e-06 2024-05-11T23:01:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749519+0000", 2024-05-11T23:01:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6009999999999997e-06 2024-05-11T23:01:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749525+0000", 2024-05-11T23:01:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8699999999999997e-06 2024-05-11T23:01:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761976+0000", 2024-05-11T23:01:38.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012451234 2024-05-11T23:01:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761986+0000", 2024-05-11T23:01:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8220000000000002e-06 2024-05-11T23:01:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3164 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749534+0000", 2024-05-11T23:01:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7818072220000003, 2024-05-11T23:01:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012486199, 2024-05-11T23:01:38.567 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3164 2024-05-11T23:01:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749534+0000", 2024-05-11T23:01:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749534+0000", 2024-05-11T23:01:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749534+0000", 2024-05-11T23:01:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7700000000000004e-07 2024-05-11T23:01:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749536+0000", 2024-05-11T23:01:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0789999999999999e-06 2024-05-11T23:01:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749542+0000", 2024-05-11T23:01:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:01:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749548+0000", 2024-05-11T23:01:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0090000000000004e-06 2024-05-11T23:01:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762014+0000", 2024-05-11T23:01:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012466090000000001 2024-05-11T23:01:38.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762020+0000", 2024-05-11T23:01:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4099999999999999e-06 2024-05-11T23:01:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3167 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749577+0000", 2024-05-11T23:01:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7817634409999998, 2024-05-11T23:01:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012471350000000001, 2024-05-11T23:01:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.571 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3167 2024-05-11T23:01:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749577+0000", 2024-05-11T23:01:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749577+0000", 2024-05-11T23:01:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749578+0000", 2024-05-11T23:01:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2800000000000002e-07 2024-05-11T23:01:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749580+0000", 2024-05-11T23:01:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.058e-06 2024-05-11T23:01:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749587+0000", 2024-05-11T23:01:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4089999999999997e-06 2024-05-11T23:01:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749591+0000", 2024-05-11T23:01:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3050000000000003e-06 2024-05-11T23:01:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762040+0000", 2024-05-11T23:01:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012448489 2024-05-11T23:01:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762049+0000", 2024-05-11T23:01:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6610000000000006e-06 2024-05-11T23:01:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3176 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749637+0000", 2024-05-11T23:01:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7817037730000003, 2024-05-11T23:01:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012425937999999999, 2024-05-11T23:01:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.575 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3176 2024-05-11T23:01:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749637+0000", 2024-05-11T23:01:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749637+0000", 2024-05-11T23:01:38.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749637+0000", 2024-05-11T23:01:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0900000000000002e-07 2024-05-11T23:01:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749640+0000", 2024-05-11T23:01:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6070000000000002e-06 2024-05-11T23:01:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749649+0000", 2024-05-11T23:01:38.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5629999999999992e-06 2024-05-11T23:01:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749654+0000", 2024-05-11T23:01:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5570000000000003e-06 2024-05-11T23:01:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762059+0000", 2024-05-11T23:01:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012404567 2024-05-11T23:01:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762063+0000", 2024-05-11T23:01:38.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1350000000000002e-06 2024-05-11T23:01:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3179 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749695+0000", 2024-05-11T23:01:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7816459479999995, 2024-05-11T23:01:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012391855, 2024-05-11T23:01:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:38.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3179 2024-05-11T23:01:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749695+0000", 2024-05-11T23:01:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749695+0000", 2024-05-11T23:01:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749695+0000", 2024-05-11T23:01:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.63e-07 2024-05-11T23:01:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749698+0000", 2024-05-11T23:01:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.83e-06 2024-05-11T23:01:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749707+0000", 2024-05-11T23:01:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4810000000000006e-06 2024-05-11T23:01:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749711+0000", 2024-05-11T23:01:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6859999999999997e-06 2024-05-11T23:01:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762079+0000", 2024-05-11T23:01:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012367515000000001 2024-05-11T23:01:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762087+0000", 2024-05-11T23:01:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8800000000000008e-06 2024-05-11T23:01:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.582 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3188 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749747+0000", 2024-05-11T23:01:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7815938989999998, 2024-05-11T23:01:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012374299, 2024-05-11T23:01:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:38.583 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3188 2024-05-11T23:01:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749747+0000", 2024-05-11T23:01:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749747+0000", 2024-05-11T23:01:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749747+0000", 2024-05-11T23:01:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1199999999999998e-07 2024-05-11T23:01:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749750+0000", 2024-05-11T23:01:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2869999999999999e-06 2024-05-11T23:01:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749756+0000", 2024-05-11T23:01:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8199999999999999e-06 2024-05-11T23:01:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749763+0000", 2024-05-11T23:01:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3149999999999997e-06 2024-05-11T23:01:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762116+0000", 2024-05-11T23:01:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012353207999999999 2024-05-11T23:01:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762121+0000", 2024-05-11T23:01:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2569999999999998e-06 2024-05-11T23:01:38.586 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3191 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749794+0000", 2024-05-11T23:01:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7815462709999998, 2024-05-11T23:01:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012357628000000001, 2024-05-11T23:01:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3191 2024-05-11T23:01:38.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749794+0000", 2024-05-11T23:01:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749794+0000", 2024-05-11T23:01:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749795+0000", 2024-05-11T23:01:38.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:01:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749800+0000", 2024-05-11T23:01:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7129999999999996e-06 2024-05-11T23:01:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749807+0000", 2024-05-11T23:01:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3039999999999999e-06 2024-05-11T23:01:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749812+0000", 2024-05-11T23:01:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3569999999999997e-06 2024-05-11T23:01:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762147+0000", 2024-05-11T23:01:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012335084 2024-05-11T23:01:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762152+0000", 2024-05-11T23:01:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6909999999999996e-06 2024-05-11T23:01:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3201 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749845+0000", 2024-05-11T23:01:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7814955499999998, 2024-05-11T23:01:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012333903, 2024-05-11T23:01:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3201 2024-05-11T23:01:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.591 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749845+0000", 2024-05-11T23:01:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749845+0000", 2024-05-11T23:01:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749846+0000", 2024-05-11T23:01:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4999999999999998e-07 2024-05-11T23:01:38.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749852+0000", 2024-05-11T23:01:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5729999999999996e-06 2024-05-11T23:01:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749861+0000", 2024-05-11T23:01:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0080000000000008e-06 2024-05-11T23:01:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749866+0000", 2024-05-11T23:01:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6720000000000003e-06 2024-05-11T23:01:38.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762175+0000", 2024-05-11T23:01:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012308852 2024-05-11T23:01:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762179+0000", 2024-05-11T23:01:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3479999999999997e-06 2024-05-11T23:01:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3204 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749889+0000", 2024-05-11T23:01:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7814512999999996, 2024-05-11T23:01:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012314227000000001, 2024-05-11T23:01:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3204 2024-05-11T23:01:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749889+0000", 2024-05-11T23:01:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749889+0000", 2024-05-11T23:01:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749890+0000", 2024-05-11T23:01:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:01:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749892+0000", 2024-05-11T23:01:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9970000000000001e-06 2024-05-11T23:01:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749897+0000", 2024-05-11T23:01:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5500000000000002e-06 2024-05-11T23:01:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749902+0000", 2024-05-11T23:01:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3409999999999996e-06 2024-05-11T23:01:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762199+0000", 2024-05-11T23:01:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012297577000000001 2024-05-11T23:01:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762204+0000", 2024-05-11T23:01:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5310000000000003e-06 2024-05-11T23:01:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3209 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:38.598 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749907+0000", 2024-05-11T23:01:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7814333810000003, 2024-05-11T23:01:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012319788999999999, 2024-05-11T23:01:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3209 2024-05-11T23:01:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749907+0000", 2024-05-11T23:01:38.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749907+0000", 2024-05-11T23:01:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749908+0000", 2024-05-11T23:01:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-05-11T23:01:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749916+0000", 2024-05-11T23:01:38.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8550000000000008e-06 2024-05-11T23:01:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749923+0000", 2024-05-11T23:01:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7850000000000003e-06 2024-05-11T23:01:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749928+0000", 2024-05-11T23:01:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8609999999999997e-06 2024-05-11T23:01:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762223+0000", 2024-05-11T23:01:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012294589999999999 2024-05-11T23:01:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762227+0000", 2024-05-11T23:01:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4510000000000002e-06 2024-05-11T23:01:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3212 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749956+0000", 2024-05-11T23:01:38.602 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7813849949999998, 2024-05-11T23:01:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012295808, 2024-05-11T23:01:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3212 2024-05-11T23:01:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749956+0000", 2024-05-11T23:01:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749956+0000", 2024-05-11T23:01:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749956+0000", 2024-05-11T23:01:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:01:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749959+0000", 2024-05-11T23:01:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7879999999999998e-06 2024-05-11T23:01:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749965+0000", 2024-05-11T23:01:38.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2269999999999998e-06 2024-05-11T23:01:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749970+0000", 2024-05-11T23:01:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0849999999999996e-06 2024-05-11T23:01:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762247+0000", 2024-05-11T23:01:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012276989 2024-05-11T23:01:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762252+0000", 2024-05-11T23:01:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3669999999999999e-06 2024-05-11T23:01:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3221 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749998+0000", 2024-05-11T23:01:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7813424260000001, 2024-05-11T23:01:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012277692999999999, 2024-05-11T23:01:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3221 2024-05-11T23:01:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749998+0000", 2024-05-11T23:01:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749998+0000", 2024-05-11T23:01:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749999+0000", 2024-05-11T23:01:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3300000000000002e-07 2024-05-11T23:01:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750001+0000", 2024-05-11T23:01:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.007e-06 2024-05-11T23:01:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750007+0000", 2024-05-11T23:01:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2160000000000003e-06 2024-05-11T23:01:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750012+0000", 2024-05-11T23:01:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0200000000000002e-06 2024-05-11T23:01:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762271+0000", 2024-05-11T23:01:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012258626999999999 2024-05-11T23:01:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762276+0000", 2024-05-11T23:01:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2900000000000002e-06 2024-05-11T23:01:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3225 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.750040+0000", 2024-05-11T23:01:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7813006150000001, 2024-05-11T23:01:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012260699999999999, 2024-05-11T23:01:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3225 2024-05-11T23:01:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750040+0000", 2024-05-11T23:01:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750040+0000", 2024-05-11T23:01:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750040+0000", 2024-05-11T23:01:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-05-11T23:01:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750043+0000", 2024-05-11T23:01:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.672e-06 2024-05-11T23:01:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750049+0000", 2024-05-11T23:01:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3230000000000003e-06 2024-05-11T23:01:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750055+0000", 2024-05-11T23:01:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.609e-06 2024-05-11T23:01:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762297+0000", 2024-05-11T23:01:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012241432 2024-05-11T23:01:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762301+0000", 2024-05-11T23:01:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2939999999999999e-06 2024-05-11T23:01:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:38.613 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:38.615 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:01:38.615 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:38.615+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:38.716 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:38.715+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:38.724 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:38.724 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:38.724 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:01:38.724 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:38.733 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:01:38.776 INFO:teuthology.orchestra.run.smithi005.stdout:moved 1 / 82 (1.21951%) 2024-05-11T23:01:38.776 INFO:teuthology.orchestra.run.smithi005.stdout:avg 10.25 2024-05-11T23:01:38.776 INFO:teuthology.orchestra.run.smithi005.stdout:stddev 3.26917 -> 3.34477 (expected baseline 2.99479) 2024-05-11T23:01:38.776 INFO:teuthology.orchestra.run.smithi005.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-05-11T23:01:38.776 INFO:teuthology.orchestra.run.smithi005.stdout:max osd.7 with 14 -> 15 pgs (1.36585 -> 1.46341 * mean) 2024-05-11T23:01:38.777 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:01:38.777 INFO:teuthology.orchestra.run.smithi005.stdout:oload 110 2024-05-11T23:01:38.777 INFO:teuthology.orchestra.run.smithi005.stdout:max_change 0.05 2024-05-11T23:01:38.777 INFO:teuthology.orchestra.run.smithi005.stdout:max_change_osds 4 2024-05-11T23:01:38.777 INFO:teuthology.orchestra.run.smithi005.stdout:average_utilization 0.0003 2024-05-11T23:01:38.777 INFO:teuthology.orchestra.run.smithi005.stdout:overload_utilization 0.0004 2024-05-11T23:01:38.777 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 weight 1.0000 -> 0.9500 2024-05-11T23:01:38.777 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 weight 1.0000 -> 0.9500 2024-05-11T23:01:38.777 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 weight 1.0000 -> 0.9500 2024-05-11T23:01:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-05-11T23:01:38.793 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-05-11T23:01:38.793 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-05-11T23:01:38.817 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:38.817+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:38.841 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone5 from 0x8c5414f (overlap 0x1a0a21) to 0x53539 2024-05-11T23:01:38.886 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:01:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:01:38.887 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:38.895 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:01:38.911 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-05-11T23:01:38.927 INFO:teuthology.orchestra.run.smithi005.stdout:674 write 0xc059b76 thru 0xc061ac9 (0x7f54 bytes) 2024-05-11T23:01:38.932 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:38.936 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:38.936+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:38.937 INFO:teuthology.orchestra.run.smithi005.stdout:676 trunc from 0xc061aca to 0x833ab32 2024-05-11T23:01:38.958 INFO:teuthology.orchestra.run.smithi005.stdout:677 writesame 0x53af858 thru 0x53b9917 data_size 0x20(0xa0c0 bytes) 2024-05-11T23:01:38.982 INFO:teuthology.orchestra.run.smithi005.stdout:679 trunc from 0x833ab32 to 0x15e2114 2024-05-11T23:01:39.018 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:39.018+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:39.023 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:39.035 INFO:teuthology.orchestra.run.smithi005.stdout:681 trunc from 0x15e2114 to 0xb83961c 2024-05-11T23:01:39.039 INFO:teuthology.orchestra.run.smithi005.stdout:683 read 0x2c07d4 thru 0x2c9ab7 (0x92e4 bytes) 2024-05-11T23:01:39.040 INFO:teuthology.orchestra.run.smithi005.stdout:684 write 0x962d839 thru 0x9633a38 (0x6200 bytes) 2024-05-11T23:01:39.044 INFO:teuthology.orchestra.run.smithi005.stdout:685 write 0xa13ecf0 thru 0xa13f22b (0x53c bytes) 2024-05-11T23:01:39.053 INFO:teuthology.orchestra.run.smithi005.stdout:686 writesame 0xdeb7bf3 thru 0xdebcf8c data_size 0x12(0x539a bytes) 2024-05-11T23:01:39.053 INFO:teuthology.orchestra.run.smithi005.stdout:690 write 0x93b6175 thru 0x93c5669 (0xf4f5 bytes) 2024-05-11T23:01:39.056 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:01:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:01:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:01:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-11T23:01:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:29.463901+0000", 2024-05-11T23:01:39.057 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.5911563510000004, 2024-05-11T23:01:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0077045129999999996, 2024-05-11T23:01:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.058 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9 2024-05-11T23:01:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463901+0000", 2024-05-11T23:01:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463901+0000", 2024-05-11T23:01:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463902+0000", 2024-05-11T23:01:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.367e-06 2024-05-11T23:01:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463908+0000", 2024-05-11T23:01:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0789999999999997e-06 2024-05-11T23:01:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463921+0000", 2024-05-11T23:01:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2734e-05 2024-05-11T23:01:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463928+0000", 2024-05-11T23:01:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2429999999999999e-06 2024-05-11T23:01:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463973+0000", 2024-05-11T23:01:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4116999999999998e-05 2024-05-11T23:01:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op must wait for map", 2024-05-11T23:01:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463977+0000", 2024-05-11T23:01:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3030000000000001e-06 2024-05-11T23:01:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.465607+0000", 2024-05-11T23:01:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016304119999999999 2024-05-11T23:01:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.465688+0000", 2024-05-11T23:01:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0957000000000004e-05 2024-05-11T23:01:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-05-11T23:01:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.465822+0000", 2024-05-11T23:01:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013396099999999999 2024-05-11T23:01:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.466978+0000", 2024-05-11T23:01:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001155664 2024-05-11T23:01:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.471543+0000", 2024-05-11T23:01:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045646870000000004 2024-05-11T23:01:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.471580+0000", 2024-05-11T23:01:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7398999999999999e-05 2024-05-11T23:01:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.471606+0000", 2024-05-11T23:01:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5590000000000001e-05 2024-05-11T23:01:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:2223 3.1a 3:5bdb11a3:::rbd_data.111e50384849.000000000000003e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 180224] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.608039+0000", 2024-05-11T23:01:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.4470187039999995, 2024-05-11T23:01:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0078733559999999998, 2024-05-11T23:01:39.065 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2223 2024-05-11T23:01:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608039+0000", 2024-05-11T23:01:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608039+0000", 2024-05-11T23:01:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608040+0000", 2024-05-11T23:01:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3900000000000004e-07 2024-05-11T23:01:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608044+0000", 2024-05-11T23:01:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0380000000000001e-06 2024-05-11T23:01:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608053+0000", 2024-05-11T23:01:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3449999999999995e-06 2024-05-11T23:01:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608058+0000", 2024-05-11T23:01:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2809999999999999e-06 2024-05-11T23:01:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608106+0000", 2024-05-11T23:01:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7552000000000001e-05 2024-05-11T23:01:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608191+0000", 2024-05-11T23:01:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4981999999999994e-05 2024-05-11T23:01:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-05-11T23:01:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608307+0000", 2024-05-11T23:01:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011595 2024-05-11T23:01:39.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.609447+0000", 2024-05-11T23:01:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011405600000000001 2024-05-11T23:01:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.615854+0000", 2024-05-11T23:01:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0064062959999999997 2024-05-11T23:01:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.615887+0000", 2024-05-11T23:01:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3456e-05 2024-05-11T23:01:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.615912+0000", 2024-05-11T23:01:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5057000000000001e-05 2024-05-11T23:01:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3651 3.1a 3:5d63366f:::rbd_data.111e89cdde7d.000000000000000c:head [set-alloc-hint object_size 1048576 write_size 1048576,write 785032~6745] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:39.071 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.063156+0000", 2024-05-11T23:01:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.9919015709999996, 2024-05-11T23:01:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060821240000000004, 2024-05-11T23:01:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3651 2024-05-11T23:01:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063156+0000", 2024-05-11T23:01:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063156+0000", 2024-05-11T23:01:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063157+0000", 2024-05-11T23:01:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.094e-06 2024-05-11T23:01:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063166+0000", 2024-05-11T23:01:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5189999999999998e-06 2024-05-11T23:01:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063181+0000", 2024-05-11T23:01:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4691999999999999e-05 2024-05-11T23:01:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063189+0000", 2024-05-11T23:01:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3810000000000001e-06 2024-05-11T23:01:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063244+0000", 2024-05-11T23:01:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5081999999999999e-05 2024-05-11T23:01:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063365+0000", 2024-05-11T23:01:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012095300000000001 2024-05-11T23:01:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-05-11T23:01:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063496+0000", 2024-05-11T23:01:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013151100000000001 2024-05-11T23:01:39.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.064622+0000", 2024-05-11T23:01:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00112612 2024-05-11T23:01:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.069157+0000", 2024-05-11T23:01:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045349739999999998 2024-05-11T23:01:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.069202+0000", 2024-05-11T23:01:39.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5238999999999998e-05 2024-05-11T23:01:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.069238+0000", 2024-05-11T23:01:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5559000000000003e-05 2024-05-11T23:01:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4208 3.e e25/19 3:774edc63:::rbd_object_map.111e89cdde7d:head v 25'259, mlcod=25'255)", 2024-05-11T23:01:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.552088+0000", 2024-05-11T23:01:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.5029690860000002, 2024-05-11T23:01:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071901839999999996, 2024-05-11T23:01:39.077 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552088+0000", 2024-05-11T23:01:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552088+0000", 2024-05-11T23:01:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552089+0000", 2024-05-11T23:01:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4900000000000001e-07 2024-05-11T23:01:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552093+0000", 2024-05-11T23:01:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7190000000000001e-06 2024-05-11T23:01:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552102+0000", 2024-05-11T23:01:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6319999999999997e-06 2024-05-11T23:01:39.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552107+0000", 2024-05-11T23:01:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3980000000000001e-06 2024-05-11T23:01:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552147+0000", 2024-05-11T23:01:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0423999999999998e-05 2024-05-11T23:01:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552160+0000", 2024-05-11T23:01:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3246e-05 2024-05-11T23:01:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559245+0000", 2024-05-11T23:01:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0070852620000000002 2024-05-11T23:01:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559278+0000", 2024-05-11T23:01:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3053999999999999e-05 2024-05-11T23:01:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.081 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4209 3.9 3:92dfc630:::rbd_data.111e89cdde7d.00000000000000ed:head [set-alloc-hint object_size 1048576 write_size 1048576,write 9891~6493] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.552603+0000", 2024-05-11T23:01:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.5024545539999998, 2024-05-11T23:01:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0067329449999999997, 2024-05-11T23:01:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.082 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4209 2024-05-11T23:01:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552603+0000", 2024-05-11T23:01:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552603+0000", 2024-05-11T23:01:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552603+0000", 2024-05-11T23:01:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2799999999999996e-07 2024-05-11T23:01:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552610+0000", 2024-05-11T23:01:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.472e-06 2024-05-11T23:01:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552618+0000", 2024-05-11T23:01:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3299999999999999e-06 2024-05-11T23:01:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552623+0000", 2024-05-11T23:01:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8010000000000003e-06 2024-05-11T23:01:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552670+0000", 2024-05-11T23:01:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6686e-05 2024-05-11T23:01:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552782+0000", 2024-05-11T23:01:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112389 2024-05-11T23:01:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:01:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552923+0000", 2024-05-11T23:01:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000141225 2024-05-11T23:01:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554063+0000", 2024-05-11T23:01:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011397130000000001 2024-05-11T23:01:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559270+0000", 2024-05-11T23:01:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052066589999999998 2024-05-11T23:01:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559306+0000", 2024-05-11T23:01:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6133000000000001e-05 2024-05-11T23:01:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559336+0000", 2024-05-11T23:01:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9909000000000001e-05 2024-05-11T23:01:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.087 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4210 3.3 3:c0d177bc:::rbd_data.111e89cdde7d.00000000000000ee:head [set-alloc-hint object_size 1048576 write_size 1048576,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.554387+0000", 2024-05-11T23:01:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.5006707260000001, 2024-05-11T23:01:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080170269999999995, 2024-05-11T23:01:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.088 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4210 2024-05-11T23:01:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554387+0000", 2024-05-11T23:01:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554387+0000", 2024-05-11T23:01:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554387+0000", 2024-05-11T23:01:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6199999999999997e-07 2024-05-11T23:01:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554395+0000", 2024-05-11T23:01:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8720000000000002e-06 2024-05-11T23:01:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554405+0000", 2024-05-11T23:01:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5889999999999992e-06 2024-05-11T23:01:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554410+0000", 2024-05-11T23:01:39.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0980000000000001e-06 2024-05-11T23:01:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554458+0000", 2024-05-11T23:01:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8189999999999998e-05 2024-05-11T23:01:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554533+0000", 2024-05-11T23:01:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4900000000000005e-05 2024-05-11T23:01:39.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554615+0000", 2024-05-11T23:01:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2363999999999999e-05 2024-05-11T23:01:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.555352+0000", 2024-05-11T23:01:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00073687199999999996 2024-05-11T23:01:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.562311+0000", 2024-05-11T23:01:39.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0069588610000000002 2024-05-11T23:01:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.562363+0000", 2024-05-11T23:01:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2058999999999997e-05 2024-05-11T23:01:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.562404+0000", 2024-05-11T23:01:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0460000000000002e-05 2024-05-11T23:01:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.094 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4245 3.3 3:c52a1aee:::rbd_data.111e89cdde7d.0000000000000080:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598236+0000", 2024-05-11T23:01:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.4568218179999999, 2024-05-11T23:01:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0075491519999999999, 2024-05-11T23:01:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.095 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4245 2024-05-11T23:01:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598236+0000", 2024-05-11T23:01:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598236+0000", 2024-05-11T23:01:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598236+0000", 2024-05-11T23:01:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9899999999999999e-07 2024-05-11T23:01:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598240+0000", 2024-05-11T23:01:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5200000000000002e-06 2024-05-11T23:01:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598250+0000", 2024-05-11T23:01:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.764e-06 2024-05-11T23:01:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598257+0000", 2024-05-11T23:01:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2640000000000002e-06 2024-05-11T23:01:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598309+0000", 2024-05-11T23:01:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2410999999999997e-05 2024-05-11T23:01:39.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598388+0000", 2024-05-11T23:01:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9120000000000001e-05 2024-05-11T23:01:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598490+0000", 2024-05-11T23:01:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101285 2024-05-11T23:01:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.599363+0000", 2024-05-11T23:01:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00087332100000000004 2024-05-11T23:01:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605664+0000", 2024-05-11T23:01:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063005889999999997 2024-05-11T23:01:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605744+0000", 2024-05-11T23:01:39.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0507999999999996e-05 2024-05-11T23:01:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605785+0000", 2024-05-11T23:01:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0571000000000002e-05 2024-05-11T23:01:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4246 3.3 3:c5bb5d23:::rbd_data.111e89cdde7d.0000000000000081:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598285+0000", 2024-05-11T23:01:39.101 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.456772548, 2024-05-11T23:01:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0075911599999999996, 2024-05-11T23:01:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4246 2024-05-11T23:01:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598285+0000", 2024-05-11T23:01:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598285+0000", 2024-05-11T23:01:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598285+0000", 2024-05-11T23:01:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4700000000000001e-07 2024-05-11T23:01:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598288+0000", 2024-05-11T23:01:39.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0120000000000001e-06 2024-05-11T23:01:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598296+0000", 2024-05-11T23:01:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7000000000000008e-06 2024-05-11T23:01:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598302+0000", 2024-05-11T23:01:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4790000000000004e-06 2024-05-11T23:01:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598716+0000", 2024-05-11T23:01:39.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00041448 2024-05-11T23:01:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598784+0000", 2024-05-11T23:01:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7689999999999997e-05 2024-05-11T23:01:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598881+0000", 2024-05-11T23:01:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7163000000000005e-05 2024-05-11T23:01:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.599467+0000", 2024-05-11T23:01:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00058593100000000004 2024-05-11T23:01:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605792+0000", 2024-05-11T23:01:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063255739999999996 2024-05-11T23:01:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605832+0000", 2024-05-11T23:01:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9301000000000002e-05 2024-05-11T23:01:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605876+0000", 2024-05-11T23:01:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4283000000000003e-05 2024-05-11T23:01:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4247 3.1 e25/19 3:80c18ebf:::rbd_data.111e89cdde7d.0000000000000082:head v 25'53, mlcod=25'51)", 2024-05-11T23:01:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598556+0000", 2024-05-11T23:01:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.4565010850000002, 2024-05-11T23:01:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072492870000000001, 2024-05-11T23:01:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598556+0000", 2024-05-11T23:01:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598556+0000", 2024-05-11T23:01:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598557+0000", 2024-05-11T23:01:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-07 2024-05-11T23:01:39.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598562+0000", 2024-05-11T23:01:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6999999999999999e-06 2024-05-11T23:01:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598571+0000", 2024-05-11T23:01:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5629999999999998e-06 2024-05-11T23:01:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598585+0000", 2024-05-11T23:01:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4066000000000001e-05 2024-05-11T23:01:39.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598632+0000", 2024-05-11T23:01:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6471000000000003e-05 2024-05-11T23:01:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598650+0000", 2024-05-11T23:01:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8593999999999998e-05 2024-05-11T23:01:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605754+0000", 2024-05-11T23:01:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071034740000000002 2024-05-11T23:01:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605806+0000", 2024-05-11T23:01:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1820999999999997e-05 2024-05-11T23:01:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4248 3.f e25/19 3:f1322f7b:::rbd_data.111e89cdde7d.0000000000000083:head v 25'15, mlcod=25'13)", 2024-05-11T23:01:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598632+0000", 2024-05-11T23:01:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.4564251720000003, 2024-05-11T23:01:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071071980000000003, 2024-05-11T23:01:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.113 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598632+0000", 2024-05-11T23:01:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598632+0000", 2024-05-11T23:01:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598633+0000", 2024-05-11T23:01:39.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-05-11T23:01:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598638+0000", 2024-05-11T23:01:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5040000000000002e-06 2024-05-11T23:01:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598657+0000", 2024-05-11T23:01:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8195000000000001e-05 2024-05-11T23:01:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598664+0000", 2024-05-11T23:01:39.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0729999999999999e-06 2024-05-11T23:01:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598714+0000", 2024-05-11T23:01:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0766000000000001e-05 2024-05-11T23:01:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598734+0000", 2024-05-11T23:01:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9544000000000001e-05 2024-05-11T23:01:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605676+0000", 2024-05-11T23:01:39.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0069418090000000002 2024-05-11T23:01:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605739+0000", 2024-05-11T23:01:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3590999999999995e-05 2024-05-11T23:01:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4250 3.1d e25/19 3:be073325:::rbd_data.111e89cdde7d.0000000000000085:head v 25'18, mlcod=25'16)", 2024-05-11T23:01:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598711+0000", 2024-05-11T23:01:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.4563465659999997, 2024-05-11T23:01:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0070702610000000004, 2024-05-11T23:01:39.117 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598711+0000", 2024-05-11T23:01:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598711+0000", 2024-05-11T23:01:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598711+0000", 2024-05-11T23:01:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.68e-07 2024-05-11T23:01:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598717+0000", 2024-05-11T23:01:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4009999999999996e-06 2024-05-11T23:01:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598726+0000", 2024-05-11T23:01:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1400000000000006e-06 2024-05-11T23:01:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598732+0000", 2024-05-11T23:01:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4479999999999999e-06 2024-05-11T23:01:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598777+0000", 2024-05-11T23:01:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.473e-05 2024-05-11T23:01:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598799+0000", 2024-05-11T23:01:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1651999999999999e-05 2024-05-11T23:01:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605692+0000", 2024-05-11T23:01:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0068933680000000004 2024-05-11T23:01:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605781+0000", 2024-05-11T23:01:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8954000000000002e-05 2024-05-11T23:01:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4249 3.1c e25/19 3:3e719808:::rbd_data.111e89cdde7d.0000000000000084:head v 25'28, mlcod=25'26)", 2024-05-11T23:01:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598858+0000", 2024-05-11T23:01:39.121 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.4561993300000005, 2024-05-11T23:01:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.007008294, 2024-05-11T23:01:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598858+0000", 2024-05-11T23:01:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598858+0000", 2024-05-11T23:01:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598859+0000", 2024-05-11T23:01:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5200000000000002e-07 2024-05-11T23:01:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598862+0000", 2024-05-11T23:01:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5489999999999998e-06 2024-05-11T23:01:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598868+0000", 2024-05-11T23:01:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2770000000000002e-06 2024-05-11T23:01:39.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598872+0000", 2024-05-11T23:01:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9650000000000002e-06 2024-05-11T23:01:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598912+0000", 2024-05-11T23:01:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0064999999999997e-05 2024-05-11T23:01:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598930+0000", 2024-05-11T23:01:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8014999999999999e-05 2024-05-11T23:01:39.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605703+0000", 2024-05-11T23:01:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0067726169999999999 2024-05-11T23:01:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605866+0000", 2024-05-11T23:01:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016335399999999999 2024-05-11T23:01:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:6308 3.13 e28/19 3:cade1bdc:::rbd_data.111e65d1e80f.0000000000000008:head v 28'32, mlcod=28'30)", 2024-05-11T23:01:39.125 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:37.395189+0000", 2024-05-11T23:01:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.659868774, 2024-05-11T23:01:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060546920000000004, 2024-05-11T23:01:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.395189+0000", 2024-05-11T23:01:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.395189+0000", 2024-05-11T23:01:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.395189+0000", 2024-05-11T23:01:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8999999999999996e-07 2024-05-11T23:01:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.395195+0000", 2024-05-11T23:01:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3909999999999997e-06 2024-05-11T23:01:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.395205+0000", 2024-05-11T23:01:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0185e-05 2024-05-11T23:01:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.395212+0000", 2024-05-11T23:01:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8299999999999998e-06 2024-05-11T23:01:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.395474+0000", 2024-05-11T23:01:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026272699999999998 2024-05-11T23:01:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.395528+0000", 2024-05-11T23:01:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3737999999999999e-05 2024-05-11T23:01:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.401211+0000", 2024-05-11T23:01:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056827750000000002 2024-05-11T23:01:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.401243+0000", 2024-05-11T23:01:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2356e-05 2024-05-11T23:01:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.129 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:6502 3.14 3:284c4678:::rbd_data.111e65d1e80f.0000000000000016:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-11T23:01:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:37.616415+0000", 2024-05-11T23:01:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.438642048, 2024-05-11T23:01:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022765519000000001, 2024-05-11T23:01:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 6502 2024-05-11T23:01:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.130 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616415+0000", 2024-05-11T23:01:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616415+0000", 2024-05-11T23:01:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616417+0000", 2024-05-11T23:01:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8899999999999999e-06 2024-05-11T23:01:39.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616427+0000", 2024-05-11T23:01:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7759999999999992e-06 2024-05-11T23:01:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616454+0000", 2024-05-11T23:01:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6596000000000001e-05 2024-05-11T23:01:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616467+0000", 2024-05-11T23:01:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3114e-05 2024-05-11T23:01:39.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616557+0000", 2024-05-11T23:01:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9823999999999999e-05 2024-05-11T23:01:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616600+0000", 2024-05-11T23:01:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3822999999999997e-05 2024-05-11T23:01:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:01:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.620723+0000", 2024-05-11T23:01:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041222430000000003 2024-05-11T23:01:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.639100+0000", 2024-05-11T23:01:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018377853999999999 2024-05-11T23:01:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.639113+0000", 2024-05-11T23:01:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2198000000000001e-05 2024-05-11T23:01:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.639147+0000", 2024-05-11T23:01:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4344000000000002e-05 2024-05-11T23:01:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.639181+0000", 2024-05-11T23:01:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3856999999999998e-05 2024-05-11T23:01:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4382.0:6502 3.14 e28/19 ondisk, result = 0)", 2024-05-11T23:01:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:37.632885+0000", 2024-05-11T23:01:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.4221727529999999, 2024-05-11T23:01:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062216509999999999, 2024-05-11T23:01:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:39.135 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.632885+0000", 2024-05-11T23:01:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.632885+0000", 2024-05-11T23:01:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.632885+0000", 2024-05-11T23:01:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7700000000000004e-07 2024-05-11T23:01:39.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.632890+0000", 2024-05-11T23:01:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4549999999999997e-06 2024-05-11T23:01:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.632902+0000", 2024-05-11T23:01:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2089999999999999e-05 2024-05-11T23:01:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.632911+0000", 2024-05-11T23:01:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0489999999999993e-06 2024-05-11T23:01:39.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.639087+0000", 2024-05-11T23:01:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061764430000000002 2024-05-11T23:01:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.639096+0000", 2024-05-11T23:01:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5099999999999998e-06 2024-05-11T23:01:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.639106+0000", 2024-05-11T23:01:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0327e-05 2024-05-11T23:01:39.138 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:7522 3.14 3:298c29ea:::rbd_data.111e65d1e80f.0000000000000002:8 [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-05-11T23:01:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:38.991623+0000", 2024-05-11T23:01:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.063434849000000001, 2024-05-11T23:01:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061460769999999998, 2024-05-11T23:01:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 7522 2024-05-11T23:01:39.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991623+0000", 2024-05-11T23:01:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991623+0000", 2024-05-11T23:01:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991623+0000", 2024-05-11T23:01:39.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9200000000000002e-07 2024-05-11T23:01:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991624+0000", 2024-05-11T23:01:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6899999999999996e-07 2024-05-11T23:01:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991627+0000", 2024-05-11T23:01:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7970000000000001e-06 2024-05-11T23:01:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991629+0000", 2024-05-11T23:01:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5019999999999999e-06 2024-05-11T23:01:39.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.997581+0000", 2024-05-11T23:01:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005951797 2024-05-11T23:01:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.997662+0000", 2024-05-11T23:01:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0956000000000002e-05 2024-05-11T23:01:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.997769+0000", 2024-05-11T23:01:39.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106764 2024-05-11T23:01:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:7523 3.14 3:298c29ea:::rbd_data.111e65d1e80f.0000000000000002:8 [sparse-read 3801088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-05-11T23:01:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:38.991632+0000", 2024-05-11T23:01:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.063425199000000002, 2024-05-11T23:01:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063546940000000001, 2024-05-11T23:01:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.143 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 7523 2024-05-11T23:01:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991632+0000", 2024-05-11T23:01:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991632+0000", 2024-05-11T23:01:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991632+0000", 2024-05-11T23:01:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-05-11T23:01:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991633+0000", 2024-05-11T23:01:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0750000000000001e-06 2024-05-11T23:01:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991636+0000", 2024-05-11T23:01:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8250000000000001e-06 2024-05-11T23:01:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991639+0000", 2024-05-11T23:01:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4899999999999999e-06 2024-05-11T23:01:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.997807+0000", 2024-05-11T23:01:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061682910000000002 2024-05-11T23:01:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.997890+0000", 2024-05-11T23:01:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2758999999999998e-05 2024-05-11T23:01:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.997987+0000", 2024-05-11T23:01:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7051000000000003e-05 2024-05-11T23:01:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:7524 3.14 3:298c29ea:::rbd_data.111e65d1e80f.0000000000000002:8 [sparse-read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-05-11T23:01:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:38.991642+0000", 2024-05-11T23:01:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.063415585999999996, 2024-05-11T23:01:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0065093379999999999, 2024-05-11T23:01:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.147 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 7524 2024-05-11T23:01:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991642+0000", 2024-05-11T23:01:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991642+0000", 2024-05-11T23:01:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991642+0000", 2024-05-11T23:01:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0800000000000001e-07 2024-05-11T23:01:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991643+0000", 2024-05-11T23:01:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9300000000000006e-07 2024-05-11T23:01:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991646+0000", 2024-05-11T23:01:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.779e-06 2024-05-11T23:01:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991648+0000", 2024-05-11T23:01:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6369999999999999e-06 2024-05-11T23:01:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998022+0000", 2024-05-11T23:01:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063738090000000002 2024-05-11T23:01:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998073+0000", 2024-05-11T23:01:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0994e-05 2024-05-11T23:01:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998151+0000", 2024-05-11T23:01:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7917999999999995e-05 2024-05-11T23:01:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:7525 3.14 3:298c29ea:::rbd_data.111e65d1e80f.0000000000000002:8 [sparse-read 4325376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-05-11T23:01:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:38.991652+0000", 2024-05-11T23:01:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.063405886999999994, 2024-05-11T23:01:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0066895660000000001, 2024-05-11T23:01:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.151 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 7525 2024-05-11T23:01:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991652+0000", 2024-05-11T23:01:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991652+0000", 2024-05-11T23:01:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991652+0000", 2024-05-11T23:01:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1500000000000001e-07 2024-05-11T23:01:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991653+0000", 2024-05-11T23:01:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7100000000000011e-07 2024-05-11T23:01:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991655+0000", 2024-05-11T23:01:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6989999999999999e-06 2024-05-11T23:01:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991658+0000", 2024-05-11T23:01:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0939999999999999e-06 2024-05-11T23:01:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998180+0000", 2024-05-11T23:01:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006521135 2024-05-11T23:01:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998247+0000", 2024-05-11T23:01:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7362999999999999e-05 2024-05-11T23:01:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998341+0000", 2024-05-11T23:01:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4089e-05 2024-05-11T23:01:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:7526 3.14 3:298c29ea:::rbd_data.111e65d1e80f.0000000000000002:8 [sparse-read 4587520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-05-11T23:01:39.155 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:38.991662+0000", 2024-05-11T23:01:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.063395800000000002, 2024-05-11T23:01:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0069244689999999999, 2024-05-11T23:01:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 7526 2024-05-11T23:01:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991662+0000", 2024-05-11T23:01:39.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991662+0000", 2024-05-11T23:01:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991662+0000", 2024-05-11T23:01:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:01:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991665+0000", 2024-05-11T23:01:39.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9579999999999999e-06 2024-05-11T23:01:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991668+0000", 2024-05-11T23:01:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.01e-06 2024-05-11T23:01:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991670+0000", 2024-05-11T23:01:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5440000000000001e-06 2024-05-11T23:01:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998388+0000", 2024-05-11T23:01:39.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0067174000000000001 2024-05-11T23:01:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998479+0000", 2024-05-11T23:01:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1629e-05 2024-05-11T23:01:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998586+0000", 2024-05-11T23:01:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106708 2024-05-11T23:01:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.160 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:39.162 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:01:39.162 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:39.161+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:39.163 INFO:teuthology.orchestra.run.smithi005.stdout:691 write 0x8a0915c thru 0x8a09820 (0x6c5 bytes) 2024-05-11T23:01:39.164 INFO:teuthology.orchestra.run.smithi005.stdout:693 trunc from 0xdebcf8d to 0x9cfc4bd 2024-05-11T23:01:39.164 INFO:teuthology.orchestra.run.smithi005.stdout:truncating to largest ever: 0xed4aefa 2024-05-11T23:01:39.164 INFO:teuthology.orchestra.run.smithi005.stdout:694 trunc from 0x9cfc4bd to 0xed4aefa 2024-05-11T23:01:39.164 INFO:teuthology.orchestra.run.smithi005.stdout:695 punch from 0xbdac9dc to 0xbdb1def, (0x5413 bytes) 2024-05-11T23:01:39.164 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:39.164 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:39.164 INFO:teuthology.orchestra.run.smithi005.stdout:700 read 0x595ef74 thru 0x596c307 (0xd394 bytes) 2024-05-11T23:01:39.164 INFO:teuthology.orchestra.run.smithi005.stdout:701 writesame 0xde54af6 thru 0xde56ed1 data_size 0x1(0x23dc bytes) 2024-05-11T23:01:39.164 INFO:teuthology.orchestra.run.smithi005.stdout:702 writesame 0xd0f036 thru 0xd11233 data_size 0x2(0x21fe bytes) 2024-05-11T23:01:39.164 INFO:teuthology.orchestra.run.smithi005.stdout:703 read 0x8924f88 thru 0x8932eac (0xdf25 bytes) 2024-05-11T23:01:39.164 INFO:teuthology.orchestra.run.smithi005.stdout:705 trunc from 0xed4aefa to 0x4674d0b 2024-05-11T23:01:39.164 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:39.164 INFO:teuthology.orchestra.run.smithi005.stdout:707 write 0x24a0c88 thru 0x24a8f2d (0x82a6 bytes) 2024-05-11T23:01:39.164 INFO:teuthology.orchestra.run.smithi005.stdout:708 write 0xd07ed19 thru 0xd08c567 (0xd84f bytes) 2024-05-11T23:01:39.164 INFO:teuthology.orchestra.run.smithi005.stdout:709 punch from 0x76491ff to 0x7656c64, (0xda65 bytes) 2024-05-11T23:01:39.165 INFO:teuthology.orchestra.run.smithi005.stdout:710 punch from 0x2be8055 to 0x2bf4203, (0xc1ae bytes) 2024-05-11T23:01:39.165 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:39.165 INFO:teuthology.orchestra.run.smithi005.stdout:712 read 0xbd10d9e thru 0xbd1a29e (0x9501 bytes) 2024-05-11T23:01:39.165 INFO:teuthology.orchestra.run.smithi005.stdout:713 read 0x4094467 thru 0x409febe (0xba58 bytes) 2024-05-11T23:01:39.165 INFO:teuthology.orchestra.run.smithi005.stdout:714 write 0xfca050 thru 0xfd88fb (0xe8ac bytes) 2024-05-11T23:01:39.165 INFO:teuthology.orchestra.run.smithi005.stdout:715 clone 7 order 22 su 131072 sc 2 2024-05-11T23:01:39.262 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:39.262+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:39.308 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:39.308 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:39.308 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:01:39.308 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:39.316 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:01:39.363 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:39.363+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:39.422 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:01:39.422 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:01:39.422 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:39.431 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:01:39.472 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:39.472+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:39.565 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:39.565+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:39.573 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:01:39.577 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:01:39.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:01:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:01:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:1499 3.c 3:30f012b2:::rbd_data.111e50384849.0000000000000049:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.543409+0000", 2024-05-11T23:01:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.0328739569999996, 2024-05-11T23:01:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0075691669999999999, 2024-05-11T23:01:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.578 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1499 2024-05-11T23:01:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543409+0000", 2024-05-11T23:01:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543409+0000", 2024-05-11T23:01:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543410+0000", 2024-05-11T23:01:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6700000000000003e-07 2024-05-11T23:01:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543413+0000", 2024-05-11T23:01:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.574e-06 2024-05-11T23:01:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543422+0000", 2024-05-11T23:01:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6049999999999998e-06 2024-05-11T23:01:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543428+0000", 2024-05-11T23:01:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6460000000000004e-06 2024-05-11T23:01:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.544103+0000", 2024-05-11T23:01:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00067411900000000004 2024-05-11T23:01:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.544216+0000", 2024-05-11T23:01:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011385800000000001 2024-05-11T23:01:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.550978+0000", 2024-05-11T23:01:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006761698 2024-05-11T23:01:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:1681 3.c 3:30f012b2:::rbd_data.111e50384849.0000000000000049:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.557367+0000", 2024-05-11T23:01:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.0189160829999997, 2024-05-11T23:01:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056449150000000003, 2024-05-11T23:01:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.582 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1681 2024-05-11T23:01:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557367+0000", 2024-05-11T23:01:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557367+0000", 2024-05-11T23:01:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557367+0000", 2024-05-11T23:01:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0200000000000003e-07 2024-05-11T23:01:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557370+0000", 2024-05-11T23:01:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5799999999999999e-06 2024-05-11T23:01:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557376+0000", 2024-05-11T23:01:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.319e-06 2024-05-11T23:01:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557381+0000", 2024-05-11T23:01:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6999999999999999e-06 2024-05-11T23:01:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.558331+0000", 2024-05-11T23:01:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00094994600000000002 2024-05-11T23:01:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.558380+0000", 2024-05-11T23:01:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9675000000000001e-05 2024-05-11T23:01:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:01:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.558482+0000", 2024-05-11T23:01:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101438 2024-05-11T23:01:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.562440+0000", 2024-05-11T23:01:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039585209999999996 2024-05-11T23:01:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.562852+0000", 2024-05-11T23:01:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00041132499999999998 2024-05-11T23:01:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.562971+0000", 2024-05-11T23:01:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011910500000000001 2024-05-11T23:01:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.563012+0000", 2024-05-11T23:01:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0904e-05 2024-05-11T23:01:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:1853 3.17 e25/19 3:ef0f2dd1:::rbd_object_map.111e50384849.0000000000000005:head v 25'5, mlcod=25'2)", 2024-05-11T23:01:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.568898+0000", 2024-05-11T23:01:39.587 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 6.0073847999999996, 2024-05-11T23:01:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005500095, 2024-05-11T23:01:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568898+0000", 2024-05-11T23:01:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568898+0000", 2024-05-11T23:01:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568899+0000", 2024-05-11T23:01:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5000000000000003e-07 2024-05-11T23:01:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568904+0000", 2024-05-11T23:01:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4580000000000001e-06 2024-05-11T23:01:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568915+0000", 2024-05-11T23:01:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1088e-05 2024-05-11T23:01:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568922+0000", 2024-05-11T23:01:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6900000000000003e-06 2024-05-11T23:01:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568977+0000", 2024-05-11T23:01:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5299e-05 2024-05-11T23:01:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568995+0000", 2024-05-11T23:01:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8173999999999999e-05 2024-05-11T23:01:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574370+0000", 2024-05-11T23:01:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005374864 2024-05-11T23:01:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574398+0000", 2024-05-11T23:01:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7971999999999999e-05 2024-05-11T23:01:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "pg_update_log_missing(3.1f epoch 25/19 rep_tid 142 entries 25'64 (0'0) error 3:fc8394ee:::rbd_data.111e50384849.000000000000005d:head by client.4382.0:1942 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'61)", 2024-05-11T23:01:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.577276+0000", 2024-05-11T23:01:39.591 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.9990072679999997, 2024-05-11T23:01:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0085889989999999999, 2024-05-11T23:01:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577276+0000", 2024-05-11T23:01:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577276+0000", 2024-05-11T23:01:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577276+0000", 2024-05-11T23:01:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5299999999999999e-07 2024-05-11T23:01:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577281+0000", 2024-05-11T23:01:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3489999999999999e-06 2024-05-11T23:01:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577292+0000", 2024-05-11T23:01:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0813e-05 2024-05-11T23:01:39.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577300+0000", 2024-05-11T23:01:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5209999999999997e-06 2024-05-11T23:01:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577377+0000", 2024-05-11T23:01:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7197e-05 2024-05-11T23:01:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.585865+0000", 2024-05-11T23:01:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0084876659999999996 2024-05-11T23:01:39.594 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3456 3.16 3:68b574d8:::rbd_data.111e89cdde7d.000000000000003c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.888719+0000", 2024-05-11T23:01:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.6875641789999998, 2024-05-11T23:01:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0057862640000000002, 2024-05-11T23:01:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3456 2024-05-11T23:01:39.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888719+0000", 2024-05-11T23:01:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888719+0000", 2024-05-11T23:01:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888720+0000", 2024-05-11T23:01:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8100000000000001e-07 2024-05-11T23:01:39.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888724+0000", 2024-05-11T23:01:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3769999999999998e-06 2024-05-11T23:01:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888736+0000", 2024-05-11T23:01:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2237e-05 2024-05-11T23:01:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888743+0000", 2024-05-11T23:01:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6000000000000003e-06 2024-05-11T23:01:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888795+0000", 2024-05-11T23:01:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2206999999999997e-05 2024-05-11T23:01:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888873+0000", 2024-05-11T23:01:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8388999999999998e-05 2024-05-11T23:01:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:01:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888956+0000", 2024-05-11T23:01:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2183999999999998e-05 2024-05-11T23:01:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.891425+0000", 2024-05-11T23:01:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002469691 2024-05-11T23:01:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894453+0000", 2024-05-11T23:01:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030273259999999999 2024-05-11T23:01:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894481+0000", 2024-05-11T23:01:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8683999999999998e-05 2024-05-11T23:01:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894505+0000", 2024-05-11T23:01:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3688000000000001e-05 2024-05-11T23:01:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3457 3.c 3:353bdd7c:::rbd_data.111e89cdde7d.0000000000000050:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.888792+0000", 2024-05-11T23:01:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.6874910879999998, 2024-05-11T23:01:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054713679999999999, 2024-05-11T23:01:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.600 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3457 2024-05-11T23:01:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888792+0000", 2024-05-11T23:01:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888792+0000", 2024-05-11T23:01:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888792+0000", 2024-05-11T23:01:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5300000000000004e-07 2024-05-11T23:01:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888795+0000", 2024-05-11T23:01:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0750000000000002e-06 2024-05-11T23:01:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888805+0000", 2024-05-11T23:01:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4790000000000002e-06 2024-05-11T23:01:39.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888811+0000", 2024-05-11T23:01:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5459999999999999e-06 2024-05-11T23:01:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888857+0000", 2024-05-11T23:01:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6007000000000002e-05 2024-05-11T23:01:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888906+0000", 2024-05-11T23:01:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9496999999999997e-05 2024-05-11T23:01:39.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:01:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.889004+0000", 2024-05-11T23:01:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8270999999999995e-05 2024-05-11T23:01:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.889869+0000", 2024-05-11T23:01:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00086435500000000003 2024-05-11T23:01:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894165+0000", 2024-05-11T23:01:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042960259999999997 2024-05-11T23:01:39.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894215+0000", 2024-05-11T23:01:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9876999999999997e-05 2024-05-11T23:01:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894263+0000", 2024-05-11T23:01:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8581999999999997e-05 2024-05-11T23:01:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3852 3.1d 3:bde7e891:::rbd_data.111e89cdde7d.00000000000000b9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:39.605 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.243857+0000", 2024-05-11T23:01:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.3324259669999998, 2024-05-11T23:01:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061537099999999997, 2024-05-11T23:01:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3852 2024-05-11T23:01:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243857+0000", 2024-05-11T23:01:39.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243857+0000", 2024-05-11T23:01:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243858+0000", 2024-05-11T23:01:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9400000000000005e-07 2024-05-11T23:01:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243863+0000", 2024-05-11T23:01:39.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.237e-06 2024-05-11T23:01:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243877+0000", 2024-05-11T23:01:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-05-11T23:01:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243884+0000", 2024-05-11T23:01:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4340000000000003e-06 2024-05-11T23:01:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243943+0000", 2024-05-11T23:01:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8705999999999997e-05 2024-05-11T23:01:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244022+0000", 2024-05-11T23:01:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9559000000000002e-05 2024-05-11T23:01:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244121+0000", 2024-05-11T23:01:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8227000000000007e-05 2024-05-11T23:01:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.246909+0000", 2024-05-11T23:01:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002788198 2024-05-11T23:01:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.249917+0000", 2024-05-11T23:01:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030083839999999998 2024-05-11T23:01:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.249968+0000", 2024-05-11T23:01:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.113e-05 2024-05-11T23:01:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.250011+0000", 2024-05-11T23:01:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2386999999999999e-05 2024-05-11T23:01:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:3855 3.17 e25/19 3:eb11d2b0:::rbd_data.111e89cdde7d.00000000000000bc:head v 25'17, mlcod=25'15)", 2024-05-11T23:01:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.244299+0000", 2024-05-11T23:01:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.3319836860000001, 2024-05-11T23:01:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054219239999999998, 2024-05-11T23:01:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244299+0000", 2024-05-11T23:01:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244299+0000", 2024-05-11T23:01:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244300+0000", 2024-05-11T23:01:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1799999999999995e-07 2024-05-11T23:01:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244305+0000", 2024-05-11T23:01:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4009999999999996e-06 2024-05-11T23:01:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244314+0000", 2024-05-11T23:01:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7770000000000008e-06 2024-05-11T23:01:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244323+0000", 2024-05-11T23:01:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9959999999999999e-06 2024-05-11T23:01:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244383+0000", 2024-05-11T23:01:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9867000000000001e-05 2024-05-11T23:01:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244403+0000", 2024-05-11T23:01:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9885999999999999e-05 2024-05-11T23:01:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.249691+0000", 2024-05-11T23:01:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052880150000000001 2024-05-11T23:01:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.249721+0000", 2024-05-11T23:01:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0463999999999998e-05 2024-05-11T23:01:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.614 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4107 3.1c 3:3e719808:::rbd_data.111e89cdde7d.0000000000000084:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.457074+0000", 2024-05-11T23:01:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.119209143, 2024-05-11T23:01:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080074340000000008, 2024-05-11T23:01:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4107 2024-05-11T23:01:39.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457074+0000", 2024-05-11T23:01:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457074+0000", 2024-05-11T23:01:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457075+0000", 2024-05-11T23:01:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9899999999999999e-07 2024-05-11T23:01:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457079+0000", 2024-05-11T23:01:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.194e-06 2024-05-11T23:01:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457090+0000", 2024-05-11T23:01:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.154e-05 2024-05-11T23:01:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457096+0000", 2024-05-11T23:01:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3890000000000004e-06 2024-05-11T23:01:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457145+0000", 2024-05-11T23:01:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9245e-05 2024-05-11T23:01:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457196+0000", 2024-05-11T23:01:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1084000000000001e-05 2024-05-11T23:01:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457269+0000", 2024-05-11T23:01:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2712000000000004e-05 2024-05-11T23:01:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.462365+0000", 2024-05-11T23:01:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050963850000000002 2024-05-11T23:01:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.464998+0000", 2024-05-11T23:01:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026330799999999999 2024-05-11T23:01:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465047+0000", 2024-05-11T23:01:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8754000000000001e-05 2024-05-11T23:01:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465081+0000", 2024-05-11T23:01:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4251999999999997e-05 2024-05-11T23:01:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4108 3.1d 3:be073325:::rbd_data.111e89cdde7d.0000000000000085:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.457110+0000", 2024-05-11T23:01:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.1191727199999999, 2024-05-11T23:01:39.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079569480000000001, 2024-05-11T23:01:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4108 2024-05-11T23:01:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457110+0000", 2024-05-11T23:01:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457110+0000", 2024-05-11T23:01:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457111+0000", 2024-05-11T23:01:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6699999999999999e-07 2024-05-11T23:01:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457113+0000", 2024-05-11T23:01:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6220000000000001e-06 2024-05-11T23:01:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457120+0000", 2024-05-11T23:01:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8909999999999998e-06 2024-05-11T23:01:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457124+0000", 2024-05-11T23:01:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6660000000000001e-06 2024-05-11T23:01:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457169+0000", 2024-05-11T23:01:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5439999999999999e-05 2024-05-11T23:01:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457256+0000", 2024-05-11T23:01:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7131000000000003e-05 2024-05-11T23:01:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457357+0000", 2024-05-11T23:01:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100655 2024-05-11T23:01:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.462328+0000", 2024-05-11T23:01:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049712630000000001 2024-05-11T23:01:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.464981+0000", 2024-05-11T23:01:39.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026531160000000001 2024-05-11T23:01:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465030+0000", 2024-05-11T23:01:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8659999999999998e-05 2024-05-11T23:01:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465067+0000", 2024-05-11T23:01:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7036999999999998e-05 2024-05-11T23:01:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4110 3.1c 3:3e4e9183:::rbd_data.111e89cdde7d.00000000000000a8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.457578+0000", 2024-05-11T23:01:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.118704492, 2024-05-11T23:01:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00756685, 2024-05-11T23:01:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4110 2024-05-11T23:01:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457578+0000", 2024-05-11T23:01:39.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457578+0000", 2024-05-11T23:01:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457579+0000", 2024-05-11T23:01:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2200000000000004e-07 2024-05-11T23:01:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457583+0000", 2024-05-11T23:01:39.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9079999999999997e-06 2024-05-11T23:01:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457592+0000", 2024-05-11T23:01:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4290000000000007e-06 2024-05-11T23:01:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457599+0000", 2024-05-11T23:01:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3790000000000003e-06 2024-05-11T23:01:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457697+0000", 2024-05-11T23:01:39.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8492000000000005e-05 2024-05-11T23:01:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457768+0000", 2024-05-11T23:01:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0798999999999998e-05 2024-05-11T23:01:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457869+0000", 2024-05-11T23:01:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101118 2024-05-11T23:01:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.462402+0000", 2024-05-11T23:01:39.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045324400000000004 2024-05-11T23:01:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465088+0000", 2024-05-11T23:01:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026866260000000001 2024-05-11T23:01:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465117+0000", 2024-05-11T23:01:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8517000000000002e-05 2024-05-11T23:01:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465145+0000", 2024-05-11T23:01:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8520000000000001e-05 2024-05-11T23:01:39.630 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4111 3.2 e25/19 3:45191464:::rbd_data.111e89cdde7d.00000000000000a9:head v 25'50, mlcod=25'48)", 2024-05-11T23:01:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.457624+0000", 2024-05-11T23:01:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.1186591809999999, 2024-05-11T23:01:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0074051719999999998, 2024-05-11T23:01:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457624+0000", 2024-05-11T23:01:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457624+0000", 2024-05-11T23:01:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457624+0000", 2024-05-11T23:01:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6100000000000001e-07 2024-05-11T23:01:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457629+0000", 2024-05-11T23:01:39.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4580000000000003e-06 2024-05-11T23:01:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457636+0000", 2024-05-11T23:01:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5360000000000004e-06 2024-05-11T23:01:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457641+0000", 2024-05-11T23:01:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0760000000000002e-06 2024-05-11T23:01:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457689+0000", 2024-05-11T23:01:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8019999999999998e-05 2024-05-11T23:01:39.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457715+0000", 2024-05-11T23:01:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5647999999999999e-05 2024-05-11T23:01:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.464999+0000", 2024-05-11T23:01:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072837329999999997 2024-05-11T23:01:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465029+0000", 2024-05-11T23:01:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0239999999999998e-05 2024-05-11T23:01:39.634 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4109 3.17 e25/19 3:ec04b13d:::rbd_data.111e89cdde7d.0000000000000086:head v 25'22, mlcod=25'20)", 2024-05-11T23:01:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.457665+0000", 2024-05-11T23:01:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.1186178460000002, 2024-05-11T23:01:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.007399699, 2024-05-11T23:01:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457665+0000", 2024-05-11T23:01:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457665+0000", 2024-05-11T23:01:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457665+0000", 2024-05-11T23:01:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1199999999999999e-07 2024-05-11T23:01:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457670+0000", 2024-05-11T23:01:39.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:01:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457677+0000", 2024-05-11T23:01:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6959999999999999e-06 2024-05-11T23:01:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457682+0000", 2024-05-11T23:01:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7910000000000004e-06 2024-05-11T23:01:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457723+0000", 2024-05-11T23:01:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1607999999999999e-05 2024-05-11T23:01:39.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457740+0000", 2024-05-11T23:01:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6643e-05 2024-05-11T23:01:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465033+0000", 2024-05-11T23:01:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072929639999999999 2024-05-11T23:01:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465065+0000", 2024-05-11T23:01:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1801000000000003e-05 2024-05-11T23:01:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.638 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4158 3.1c 3:3e719808:::rbd_data.111e89cdde7d.0000000000000084:head [set-alloc-hint object_size 1048576 write_size 1048576,write 622592~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.498662+0000", 2024-05-11T23:01:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.0776204009999999, 2024-05-11T23:01:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0058631339999999999, 2024-05-11T23:01:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4158 2024-05-11T23:01:39.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498662+0000", 2024-05-11T23:01:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498662+0000", 2024-05-11T23:01:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498664+0000", 2024-05-11T23:01:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.886e-06 2024-05-11T23:01:39.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498676+0000", 2024-05-11T23:01:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1919999999999999e-05 2024-05-11T23:01:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498692+0000", 2024-05-11T23:01:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5576000000000001e-05 2024-05-11T23:01:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498701+0000", 2024-05-11T23:01:39.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0580000000000004e-06 2024-05-11T23:01:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498763+0000", 2024-05-11T23:01:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2181000000000007e-05 2024-05-11T23:01:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498839+0000", 2024-05-11T23:01:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6261999999999996e-05 2024-05-11T23:01:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498969+0000", 2024-05-11T23:01:39.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000129176 2024-05-11T23:01:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.501285+0000", 2024-05-11T23:01:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023169559999999998 2024-05-11T23:01:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504450+0000", 2024-05-11T23:01:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0031641920000000001 2024-05-11T23:01:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504493+0000", 2024-05-11T23:01:39.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3442999999999997e-05 2024-05-11T23:01:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504526+0000", 2024-05-11T23:01:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2484000000000003e-05 2024-05-11T23:01:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4159 3.1d 3:be073325:::rbd_data.111e89cdde7d.0000000000000085:head [set-alloc-hint object_size 1048576 write_size 1048576,write 622592~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.498840+0000", 2024-05-11T23:01:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.0774428819999997, 2024-05-11T23:01:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056856160000000001, 2024-05-11T23:01:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.644 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4159 2024-05-11T23:01:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498840+0000", 2024-05-11T23:01:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498840+0000", 2024-05-11T23:01:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498841+0000", 2024-05-11T23:01:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5199999999999995e-07 2024-05-11T23:01:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498854+0000", 2024-05-11T23:01:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3502e-05 2024-05-11T23:01:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498865+0000", 2024-05-11T23:01:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0394000000000001e-05 2024-05-11T23:01:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498873+0000", 2024-05-11T23:01:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8029999999999997e-06 2024-05-11T23:01:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498933+0000", 2024-05-11T23:01:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0955000000000002e-05 2024-05-11T23:01:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499008+0000", 2024-05-11T23:01:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4210999999999998e-05 2024-05-11T23:01:39.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499165+0000", 2024-05-11T23:01:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015752100000000001 2024-05-11T23:01:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.501518+0000", 2024-05-11T23:01:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023527180000000002 2024-05-11T23:01:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504432+0000", 2024-05-11T23:01:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029144700000000002 2024-05-11T23:01:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504485+0000", 2024-05-11T23:01:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2639999999999997e-05 2024-05-11T23:01:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504526+0000", 2024-05-11T23:01:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0550000000000003e-05 2024-05-11T23:01:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4249 3.1c 3:3e719808:::rbd_data.111e89cdde7d.0000000000000084:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598332+0000", 2024-05-11T23:01:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.9779510780000003, 2024-05-11T23:01:39.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0077041669999999996, 2024-05-11T23:01:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4249 2024-05-11T23:01:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598332+0000", 2024-05-11T23:01:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598332+0000", 2024-05-11T23:01:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598333+0000", 2024-05-11T23:01:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7499999999999998e-07 2024-05-11T23:01:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598337+0000", 2024-05-11T23:01:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6609999999999999e-06 2024-05-11T23:01:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598350+0000", 2024-05-11T23:01:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2999e-05 2024-05-11T23:01:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598358+0000", 2024-05-11T23:01:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9500000000000001e-06 2024-05-11T23:01:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598409+0000", 2024-05-11T23:01:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0192000000000003e-05 2024-05-11T23:01:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598489+0000", 2024-05-11T23:01:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0605000000000005e-05 2024-05-11T23:01:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598592+0000", 2024-05-11T23:01:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010261 2024-05-11T23:01:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.603246+0000", 2024-05-11T23:01:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0046541619999999999 2024-05-11T23:01:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605952+0000", 2024-05-11T23:01:39.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027065169999999999 2024-05-11T23:01:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605999+0000", 2024-05-11T23:01:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6283999999999999e-05 2024-05-11T23:01:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606036+0000", 2024-05-11T23:01:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7212000000000002e-05 2024-05-11T23:01:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4250 3.1d 3:be073325:::rbd_data.111e89cdde7d.0000000000000085:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598376+0000", 2024-05-11T23:01:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.9779064420000001, 2024-05-11T23:01:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0076523390000000002, 2024-05-11T23:01:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4250 2024-05-11T23:01:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598376+0000", 2024-05-11T23:01:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598376+0000", 2024-05-11T23:01:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598377+0000", 2024-05-11T23:01:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9200000000000001e-07 2024-05-11T23:01:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598380+0000", 2024-05-11T23:01:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8899999999999999e-06 2024-05-11T23:01:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598388+0000", 2024-05-11T23:01:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8210000000000002e-06 2024-05-11T23:01:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598396+0000", 2024-05-11T23:01:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1079999999999993e-06 2024-05-11T23:01:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598436+0000", 2024-05-11T23:01:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0219000000000002e-05 2024-05-11T23:01:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598514+0000", 2024-05-11T23:01:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7918999999999997e-05 2024-05-11T23:01:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598613+0000", 2024-05-11T23:01:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9335999999999999e-05 2024-05-11T23:01:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.603227+0000", 2024-05-11T23:01:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0046141799999999998 2024-05-11T23:01:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605932+0000", 2024-05-11T23:01:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002704438 2024-05-11T23:01:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605981+0000", 2024-05-11T23:01:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8806e-05 2024-05-11T23:01:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606029+0000", 2024-05-11T23:01:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8130000000000002e-05 2024-05-11T23:01:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4253 3.1d 3:ba7bd5e3:::rbd_data.111e89cdde7d.00000000000000e3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.600882+0000", 2024-05-11T23:01:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.9754010099999997, 2024-05-11T23:01:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054413350000000003, 2024-05-11T23:01:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.660 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4253 2024-05-11T23:01:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600882+0000", 2024-05-11T23:01:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600882+0000", 2024-05-11T23:01:39.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600883+0000", 2024-05-11T23:01:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5400000000000001e-07 2024-05-11T23:01:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600887+0000", 2024-05-11T23:01:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.092e-06 2024-05-11T23:01:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600897+0000", 2024-05-11T23:01:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0093e-05 2024-05-11T23:01:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600902+0000", 2024-05-11T23:01:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1399999999999999e-06 2024-05-11T23:01:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600951+0000", 2024-05-11T23:01:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9407000000000003e-05 2024-05-11T23:01:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.601034+0000", 2024-05-11T23:01:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3108000000000005e-05 2024-05-11T23:01:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.601137+0000", 2024-05-11T23:01:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010276999999999999 2024-05-11T23:01:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606041+0000", 2024-05-11T23:01:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049042649999999997 2024-05-11T23:01:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606242+0000", 2024-05-11T23:01:39.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020061300000000001 2024-05-11T23:01:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606287+0000", 2024-05-11T23:01:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4824000000000003e-05 2024-05-11T23:01:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606323+0000", 2024-05-11T23:01:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6368999999999997e-05 2024-05-11T23:01:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.665 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:6733 3.1c 3:3bac25fd:::rbd_data.111e65d1e80f.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-11T23:01:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:37.962181+0000", 2024-05-11T23:01:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.614101421, 2024-05-11T23:01:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025925762000000002, 2024-05-11T23:01:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 6733 2024-05-11T23:01:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962181+0000", 2024-05-11T23:01:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962181+0000", 2024-05-11T23:01:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962182+0000", 2024-05-11T23:01:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2699999999999998e-07 2024-05-11T23:01:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962187+0000", 2024-05-11T23:01:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3050000000000003e-06 2024-05-11T23:01:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962197+0000", 2024-05-11T23:01:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0010000000000001e-05 2024-05-11T23:01:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962201+0000", 2024-05-11T23:01:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8709999999999996e-06 2024-05-11T23:01:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962276+0000", 2024-05-11T23:01:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5004000000000002e-05 2024-05-11T23:01:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962325+0000", 2024-05-11T23:01:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8093999999999997e-05 2024-05-11T23:01:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.966865+0000", 2024-05-11T23:01:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045404720000000003 2024-05-11T23:01:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.987960+0000", 2024-05-11T23:01:39.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021095442999999998 2024-05-11T23:01:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.988032+0000", 2024-05-11T23:01:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2013000000000003e-05 2024-05-11T23:01:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.988070+0000", 2024-05-11T23:01:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7981000000000001e-05 2024-05-11T23:01:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.988107+0000", 2024-05-11T23:01:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6742000000000002e-05 2024-05-11T23:01:39.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4382.0:6733 3.1c e28/19 ondisk, result = 0)", 2024-05-11T23:01:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:37.982065+0000", 2024-05-11T23:01:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.594217859, 2024-05-11T23:01:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060490049999999997, 2024-05-11T23:01:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.982065+0000", 2024-05-11T23:01:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.982065+0000", 2024-05-11T23:01:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.982066+0000", 2024-05-11T23:01:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.156e-06 2024-05-11T23:01:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.982071+0000", 2024-05-11T23:01:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:01:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.982083+0000", 2024-05-11T23:01:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1742e-05 2024-05-11T23:01:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.982108+0000", 2024-05-11T23:01:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5001999999999999e-05 2024-05-11T23:01:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.988021+0000", 2024-05-11T23:01:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059133149999999997 2024-05-11T23:01:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:39.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.988028+0000", 2024-05-11T23:01:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4649999999999997e-06 2024-05-11T23:01:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.988114+0000", 2024-05-11T23:01:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5538000000000007e-05 2024-05-11T23:01:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:39.674 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:39.674 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:39.676 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:01:39.676 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:39.675+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:39.704 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:01:39.705 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:01:39.722 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:01:39.722 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:01:39.739 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:01:39.739 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:01:39.756 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:01:39.757 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:01:39.775 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:01:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:01:39.776 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:39.775+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:39.777 INFO:teuthology.orchestra.run.smithi005.stdout:moved 0 / 82 (0%) 2024-05-11T23:01:39.777 INFO:teuthology.orchestra.run.smithi005.stdout:avg 10.25 2024-05-11T23:01:39.777 INFO:teuthology.orchestra.run.smithi005.stdout:stddev 3.34477 -> 3.34477 (expected baseline 2.99479) 2024-05-11T23:01:39.777 INFO:teuthology.orchestra.run.smithi005.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-05-11T23:01:39.777 INFO:teuthology.orchestra.run.smithi005.stdout:max osd.7 with 15 -> 15 pgs (1.46341 -> 1.46341 * mean) 2024-05-11T23:01:39.777 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:01:39.778 INFO:teuthology.orchestra.run.smithi005.stdout:oload 110 2024-05-11T23:01:39.778 INFO:teuthology.orchestra.run.smithi005.stdout:max_change 0.05 2024-05-11T23:01:39.778 INFO:teuthology.orchestra.run.smithi005.stdout:max_change_osds 4 2024-05-11T23:01:39.778 INFO:teuthology.orchestra.run.smithi005.stdout:average_utilization 0.0003 2024-05-11T23:01:39.778 INFO:teuthology.orchestra.run.smithi005.stdout:overload_utilization 0.0004 2024-05-11T23:01:39.778 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 weight 0.9500 -> 0.9000 2024-05-11T23:01:39.802 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:01:39.803 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:01:39.820 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:01:39.820 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:01:39.833 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:39.834 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:39.840 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:01:39.840 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:01:39.844 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:01:39.844 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone6 from 0xd08c568 (overlap 0x15e2114) to 0x1181a76 2024-05-11T23:01:39.893 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:39.893+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:39.934 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-05-11T23:01:39.950 INFO:teuthology.orchestra.run.smithi005.stdout:716 trunc from 0xd08c568 to 0xac13c8 2024-05-11T23:01:39.974 INFO:teuthology.orchestra.run.smithi005.stdout:717 trunc from 0xac13c8 to 0xaa5f72f 2024-05-11T23:01:39.977 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:39.977+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:39.980 INFO:teuthology.orchestra.run.smithi005.stdout:718 punch from 0xa9deab9 to 0xa9e55f5, (0x6b3c bytes) 2024-05-11T23:01:39.980 INFO:teuthology.orchestra.run.smithi005.stdout:719 punch from 0x38ae005 to 0x38b0c2d, (0x2c28 bytes) 2024-05-11T23:01:39.980 INFO:teuthology.orchestra.run.smithi005.stdout:720 write 0x944b34f thru 0x944eb8a (0x383c bytes) 2024-05-11T23:01:39.983 INFO:teuthology.orchestra.run.smithi005.stdout:723 write 0x5bf5a2 thru 0x5c832a (0x8d89 bytes) 2024-05-11T23:01:39.994 INFO:teuthology.orchestra.run.smithi005.stdout:725 read 0x8a42764 thru 0x8a4a4f4 (0x7d91 bytes) 2024-05-11T23:01:39.994 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:39.994 INFO:teuthology.orchestra.run.smithi005.stdout:728 read 0x308b673 thru 0x3098aca (0xd458 bytes) 2024-05-11T23:01:39.994 INFO:teuthology.orchestra.run.smithi005.stdout:729 read 0x9d59939 thru 0x9d655fd (0xbcc5 bytes) 2024-05-11T23:01:39.994 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:39.994 INFO:teuthology.orchestra.run.smithi005.stdout:733 read 0x869f02d thru 0x86a827b (0x924f bytes) 2024-05-11T23:01:39.994 INFO:teuthology.orchestra.run.smithi005.stdout:734 read 0xa583112 thru 0xa591076 (0xdf65 bytes) 2024-05-11T23:01:39.995 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:39.995 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:39.995 INFO:teuthology.orchestra.run.smithi005.stdout:735 trunc from 0xaa5f72f to 0xbffd613 2024-05-11T23:01:40.001 INFO:teuthology.orchestra.run.smithi005.stdout:737 write 0x8fc3f8f thru 0x8fcc14c (0x81be bytes) 2024-05-11T23:01:40.004 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:01:40.006 INFO:teuthology.orchestra.run.smithi005.stdout:738 write 0x3b65c35 thru 0x3b68554 (0x2920 bytes) 2024-05-11T23:01:40.010 INFO:teuthology.orchestra.run.smithi005.stdout:739 punch from 0x949f221 to 0x94a9503, (0xa2e2 bytes) 2024-05-11T23:01:40.010 INFO:teuthology.orchestra.run.smithi005.stdout:740 read 0x701fde2 thru 0x7024e44 (0x5063 bytes) 2024-05-11T23:01:40.011 INFO:teuthology.orchestra.run.smithi005.stdout:741 write 0xb66d380 thru 0xb67a19a (0xce1b bytes) 2024-05-11T23:01:40.022 INFO:teuthology.orchestra.run.smithi005.stdout:742 trunc from 0xbffd613 to 0x40c3f5e 2024-05-11T23:01:40.031 INFO:teuthology.orchestra.run.smithi005.stdout:743 punch from 0x25ed9b1 to 0x25fac1a, (0xd269 bytes) 2024-05-11T23:01:40.041 INFO:teuthology.orchestra.run.smithi005.stdout:745 writesame 0x7148464 thru 0x714a384 data_size 0x1(0x1f21 bytes) 2024-05-11T23:01:40.041 INFO:teuthology.orchestra.run.smithi005.stdout:746 read 0x4234005 thru 0x4234279 (0x275 bytes) 2024-05-11T23:01:40.041 INFO:teuthology.orchestra.run.smithi005.stdout:748 punch from 0x3b29a53 to 0x3b2fb17, (0x60c4 bytes) 2024-05-11T23:01:40.041 INFO:teuthology.orchestra.run.smithi005.stdout:749 read 0x18de31e thru 0x18e8c34 (0xa917 bytes) 2024-05-11T23:01:40.041 INFO:teuthology.orchestra.run.smithi005.stdout:752 read 0x5756c56 thru 0x576518e (0xe539 bytes) 2024-05-11T23:01:40.041 INFO:teuthology.orchestra.run.smithi005.stdout:754 punch from 0x53cfa13 to 0x53da2b8, (0xa8a5 bytes) 2024-05-11T23:01:40.041 INFO:teuthology.orchestra.run.smithi005.stdout:755 read 0x4b7dae8 thru 0x4b8ca7c (0xef95 bytes) 2024-05-11T23:01:40.041 INFO:teuthology.orchestra.run.smithi005.stdout:756 read 0x4b16d69 thru 0x4b1b9e6 (0x4c7e bytes) 2024-05-11T23:01:40.041 INFO:teuthology.orchestra.run.smithi005.stdout:761 read 0x37daf66 thru 0x37de719 (0x37b4 bytes) 2024-05-11T23:01:40.041 INFO:teuthology.orchestra.run.smithi005.stdout:762 read 0x6d01137 thru 0x6d0f7f1 (0xe6bb bytes) 2024-05-11T23:01:40.041 INFO:teuthology.orchestra.run.smithi005.stdout:763 punch from 0xf12923 to 0xf22489, (0xfb66 bytes) 2024-05-11T23:01:40.041 INFO:teuthology.orchestra.run.smithi005.stdout:764 write 0x3e052d7 thru 0x3e0617c (0xea6 bytes) 2024-05-11T23:01:40.053 INFO:teuthology.orchestra.run.smithi005.stdout:765 write 0xb81ee5d thru 0xb824949 (0x5aed bytes) 2024-05-11T23:01:40.056 INFO:teuthology.orchestra.run.smithi005.stdout:767 write 0x19bd480 thru 0x19bf18f (0x1d10 bytes) 2024-05-11T23:01:40.058 INFO:teuthology.orchestra.run.smithi005.stdout:768 write 0x7b85241 thru 0x7b91fd3 (0xcd93 bytes) 2024-05-11T23:01:40.061 INFO:teuthology.orchestra.run.smithi005.stdout:769 punch from 0x2190445 to 0x219629f, (0x5e5a bytes) 2024-05-11T23:01:40.061 INFO:teuthology.orchestra.run.smithi005.stdout:770 write 0x111745c thru 0x111b826 (0x43cb bytes) 2024-05-11T23:01:40.064 INFO:teuthology.orchestra.run.smithi005.stdout:771 read 0x8321aec thru 0x8330a7d (0xef92 bytes) 2024-05-11T23:01:40.064 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.064 INFO:teuthology.orchestra.run.smithi005.stdout:774 punch from 0xace632b to 0xacf12f3, (0xafc8 bytes) 2024-05-11T23:01:40.064 INFO:teuthology.orchestra.run.smithi005.stdout:775 read 0x5097973 thru 0x50a073d (0x8dcb bytes) 2024-05-11T23:01:40.064 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.064 INFO:teuthology.orchestra.run.smithi005.stdout:778 writesame 0x1344252 thru 0x1345c73 data_size 0xf(0x1a22 bytes) 2024-05-11T23:01:40.076 INFO:teuthology.orchestra.run.smithi005.stdout:779 trunc from 0xb82494a to 0x203113d 2024-05-11T23:01:40.078 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:40.078+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:40.086 INFO:teuthology.orchestra.run.smithi005.stdout:780 read 0x400a6 thru 0x4e037 (0xdf92 bytes) 2024-05-11T23:01:40.086 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.086 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.086 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.099 INFO:teuthology.orchestra.run.smithi005.stdout:788 write 0x70dd101 thru 0x70ea7f5 (0xd6f5 bytes) 2024-05-11T23:01:40.102 INFO:teuthology.orchestra.run.smithi005.stdout:792 punch from 0x4b9b5d to 0x4bbc4e, (0x20f1 bytes) 2024-05-11T23:01:40.112 INFO:teuthology.orchestra.run.smithi005.stdout:793 write 0xa1feb7d thru 0xa1fed22 (0x1a6 bytes) 2024-05-11T23:01:40.115 INFO:teuthology.orchestra.run.smithi005.stdout:795 read 0x9376c80 thru 0x9380731 (0x9ab2 bytes) 2024-05-11T23:01:40.115 INFO:teuthology.orchestra.run.smithi005.stdout:796 punch from 0x584aad6 to 0x584fbd8, (0x5102 bytes) 2024-05-11T23:01:40.115 INFO:teuthology.orchestra.run.smithi005.stdout:797 read 0x9de7274 thru 0x9df5842 (0xe5cf bytes) 2024-05-11T23:01:40.115 INFO:teuthology.orchestra.run.smithi005.stdout:798 write 0x57b8796 thru 0x57b89f9 (0x264 bytes) 2024-05-11T23:01:40.116 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:40.116 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:40.117 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.125 INFO:teuthology.orchestra.run.smithi005.stdout:802 write 0xc46b4de thru 0xc470d32 (0x5855 bytes) 2024-05-11T23:01:40.126 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:01:40.128 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.128 INFO:teuthology.orchestra.run.smithi005.stdout:804 punch from 0x3a55c64 to 0x3a60ab8, (0xae54 bytes) 2024-05-11T23:01:40.128 INFO:teuthology.orchestra.run.smithi005.stdout:805 writesame 0xa174acb thru 0xa1782ad data_size 0x13(0x37e3 bytes) 2024-05-11T23:01:40.130 INFO:teuthology.orchestra.run.smithi005.stdout:806 write 0x115f569 thru 0x1165b73 (0x660b bytes) 2024-05-11T23:01:40.132 INFO:teuthology.orchestra.run.smithi005.stdout:807 write 0x77e09b2 thru 0x77ea8be (0x9f0d bytes) 2024-05-11T23:01:40.144 INFO:teuthology.orchestra.run.smithi005.stdout:810 trunc from 0xc470d33 to 0x2ea1d2f 2024-05-11T23:01:40.172 INFO:teuthology.orchestra.run.smithi005.stdout:811 writesame 0xd880a46 thru 0xd88fc9c data_size 0x1(0xf257 bytes) 2024-05-11T23:01:40.179 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:40.179+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:40.181 INFO:teuthology.orchestra.run.smithi005.stdout:812 trunc from 0xd88fc9d to 0x55c2a23 2024-05-11T23:01:40.187 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.187 INFO:teuthology.orchestra.run.smithi005.stdout:814 read 0xccd2c4 thru 0xcd2be7 (0x5924 bytes) 2024-05-11T23:01:40.187 INFO:teuthology.orchestra.run.smithi005.stdout:816 read 0x53bc05b thru 0x53bff2d (0x3ed3 bytes) 2024-05-11T23:01:40.187 INFO:teuthology.orchestra.run.smithi005.stdout:817 read 0x89c13f thru 0x8a12f2 (0x51b4 bytes) 2024-05-11T23:01:40.197 INFO:teuthology.orchestra.run.smithi005.stdout:818 write 0x88a1871 thru 0x88b04f6 (0xec86 bytes) 2024-05-11T23:01:40.200 INFO:teuthology.orchestra.run.smithi005.stdout:819 punch from 0x3c15f30 to 0x3c160e9, (0x1b9 bytes) 2024-05-11T23:01:40.203 INFO:teuthology.orchestra.run.smithi005.stdout:820 trunc from 0x88b04f7 to 0xb552ca0 2024-05-11T23:01:40.209 INFO:teuthology.orchestra.run.smithi005.stdout:822 write 0x876b9e7 thru 0x877a103 (0xe71d bytes) 2024-05-11T23:01:40.212 INFO:teuthology.orchestra.run.smithi005.stdout:824 writesame 0x188ee6e thru 0x1897cb2 data_size 0x1(0x8e45 bytes) 2024-05-11T23:01:40.228 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.228 INFO:teuthology.orchestra.run.smithi005.stdout:827 write 0x31dc802 thru 0x31e12c2 (0x4ac1 bytes) 2024-05-11T23:01:40.231 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.231 INFO:teuthology.orchestra.run.smithi005.stdout:829 read 0x29bd73c thru 0x29c3421 (0x5ce6 bytes) 2024-05-11T23:01:40.232 INFO:teuthology.orchestra.run.smithi005.stdout:830 read 0x3d2c443 thru 0x3d39427 (0xcfe5 bytes) 2024-05-11T23:01:40.232 INFO:teuthology.orchestra.run.smithi005.stdout:831 write 0x3d16f1f thru 0x3d1bc89 (0x4d6b bytes) 2024-05-11T23:01:40.234 INFO:teuthology.orchestra.run.smithi005.stdout:834 punch from 0x71174e7 to 0x7117702, (0x21b bytes) 2024-05-11T23:01:40.234 INFO:teuthology.orchestra.run.smithi005.stdout:836 punch from 0x2acd6e0 to 0x2ad776c, (0xa08c bytes) 2024-05-11T23:01:40.238 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:40.238 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:40.241 INFO:teuthology.orchestra.run.smithi005.stdout:837 trunc from 0xb552ca0 to 0x4b26002 2024-05-11T23:01:40.247 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:01:40.260 INFO:teuthology.orchestra.run.smithi005.stdout:839 write 0x8487f61 thru 0x8488d9c (0xe3c bytes) 2024-05-11T23:01:40.263 INFO:teuthology.orchestra.run.smithi005.stdout:840 read 0xd274f9 thru 0xd2925b (0x1d63 bytes) 2024-05-11T23:01:40.263 INFO:teuthology.orchestra.run.smithi005.stdout:841 punch from 0x38f22b4 to 0x38ff12d, (0xce79 bytes) 2024-05-11T23:01:40.264 INFO:teuthology.orchestra.run.smithi005.stdout:842 read 0x495be1c thru 0x495f155 (0x333a bytes) 2024-05-11T23:01:40.264 INFO:teuthology.orchestra.run.smithi005.stdout:844 read 0x7295784 thru 0x729acfb (0x5578 bytes) 2024-05-11T23:01:40.264 INFO:teuthology.orchestra.run.smithi005.stdout:845 read 0xc17c8 thru 0xcccde (0xb517 bytes) 2024-05-11T23:01:40.264 INFO:teuthology.orchestra.run.smithi005.stdout:846 write 0x3145d1e thru 0x314b5dc (0x58bf bytes) 2024-05-11T23:01:40.275 INFO:teuthology.orchestra.run.smithi005.stdout:847 write 0xcaee701 thru 0xcaf4892 (0x6192 bytes) 2024-05-11T23:01:40.278 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.278 INFO:teuthology.orchestra.run.smithi005.stdout:850 read 0xae54d97 thru 0xae62bd8 (0xde42 bytes) 2024-05-11T23:01:40.278 INFO:teuthology.orchestra.run.smithi005.stdout:851 read 0xb587187 thru 0xb595603 (0xe47d bytes) 2024-05-11T23:01:40.279 INFO:teuthology.orchestra.run.smithi005.stdout:854 write 0x9c93c0b thru 0x9c9e34e (0xa744 bytes) 2024-05-11T23:01:40.286 INFO:teuthology.orchestra.run.smithi005.stdout:856 write 0xcc034bb thru 0xcc0a72a (0x7270 bytes) 2024-05-11T23:01:40.290 INFO:teuthology.orchestra.run.smithi005.stdout:858 read 0xb3891b1 thru 0xb397b6d (0xe9bd bytes) 2024-05-11T23:01:40.290 INFO:teuthology.orchestra.run.smithi005.stdout:859 punch from 0x644b99f to 0x64516d7, (0x5d38 bytes) 2024-05-11T23:01:40.290 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.290 INFO:teuthology.orchestra.run.smithi005.stdout:862 write 0xb4facf8 thru 0xb4fc852 (0x1b5b bytes) 2024-05-11T23:01:40.292 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:40.292+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:40.299 INFO:teuthology.orchestra.run.smithi005.stdout:863 writesame 0xd69f16a thru 0xd6a7b85 data_size 0x2(0x8a1c bytes) 2024-05-11T23:01:40.300 INFO:teuthology.orchestra.run.smithi005.stdout:864 read 0x6e7b17c thru 0x6e81652 (0x64d7 bytes) 2024-05-11T23:01:40.300 INFO:teuthology.orchestra.run.smithi005.stdout:865 writesame 0x6bc9a5b thru 0x6bcafa5 data_size 0x3(0x154b bytes) 2024-05-11T23:01:40.302 INFO:teuthology.orchestra.run.smithi005.stdout:867 read 0x6904b0a thru 0x6912327 (0xd81e bytes) 2024-05-11T23:01:40.303 INFO:teuthology.orchestra.run.smithi005.stdout:868 punch from 0x8af835a to 0x8b02f87, (0xac2d bytes) 2024-05-11T23:01:40.303 INFO:teuthology.orchestra.run.smithi005.stdout:869 punch from 0x47801b2 to 0x4783989, (0x37d7 bytes) 2024-05-11T23:01:40.303 INFO:teuthology.orchestra.run.smithi005.stdout:870 read 0x5d69c25 thru 0x5d78d71 (0xf14d bytes) 2024-05-11T23:01:40.304 INFO:teuthology.orchestra.run.smithi005.stdout:871 write 0xc729a25 thru 0xc73623d (0xc819 bytes) 2024-05-11T23:01:40.307 INFO:teuthology.orchestra.run.smithi005.stdout:872 punch from 0xd06122b to 0xd066308, (0x50dd bytes) 2024-05-11T23:01:40.307 INFO:teuthology.orchestra.run.smithi005.stdout:873 writesame 0xff56c3 thru 0x1000573 data_size 0x3(0xaeb1 bytes) 2024-05-11T23:01:40.313 INFO:teuthology.orchestra.run.smithi005.stdout:874 write 0x4cb4a40 thru 0x4cc4184 (0xf745 bytes) 2024-05-11T23:01:40.317 INFO:teuthology.orchestra.run.smithi005.stdout:875 punch from 0x7755c91 to 0x7765477, (0xf7e6 bytes) 2024-05-11T23:01:40.317 INFO:teuthology.orchestra.run.smithi005.stdout:876 write 0xbe36fef thru 0xbe3b1e3 (0x41f5 bytes) 2024-05-11T23:01:40.319 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.319 INFO:teuthology.orchestra.run.smithi005.stdout:878 punch from 0x1b1c25f to 0x1b28a43, (0xc7e4 bytes) 2024-05-11T23:01:40.334 INFO:teuthology.orchestra.run.smithi005.stdout:879 trunc from 0xd6a7b86 to 0xbc09ad 2024-05-11T23:01:40.361 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:40.361 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:40.366 INFO:teuthology.orchestra.run.smithi005.stdout:880 writesame 0xba63c44 thru 0xba6d83e data_size 0x1(0x9bfb bytes) 2024-05-11T23:01:40.366 INFO:teuthology.orchestra.run.smithi005.stdout:882 read 0x61a183c thru 0x61b0ae7 (0xf2ac bytes) 2024-05-11T23:01:40.366 INFO:teuthology.orchestra.run.smithi005.stdout:883 write 0xda5350 thru 0xdb23cf (0xd080 bytes) 2024-05-11T23:01:40.367 INFO:teuthology.orchestra.run.smithi005.stdout:884 write 0x9937995 thru 0x9943865 (0xbed1 bytes) 2024-05-11T23:01:40.370 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:01:40.380 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:40.380+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:40.384 INFO:teuthology.orchestra.run.smithi005.stdout:886 trunc from 0xba6d83f to 0x474f1a 2024-05-11T23:01:40.394 INFO:teuthology.orchestra.run.smithi005.stdout:887 read 0x1f584d thru 0x1fd3c4 (0x7b78 bytes) 2024-05-11T23:01:40.395 INFO:teuthology.orchestra.run.smithi005.stdout:890 trunc from 0x474f1a to 0x13ef8cc 2024-05-11T23:01:40.410 INFO:teuthology.orchestra.run.smithi005.stdout:891 write 0x553b0f1 thru 0x5543e40 (0x8d50 bytes) 2024-05-11T23:01:40.413 INFO:teuthology.orchestra.run.smithi005.stdout:892 write 0x1a84403 thru 0x1a911d1 (0xcdcf bytes) 2024-05-11T23:01:40.426 INFO:teuthology.orchestra.run.smithi005.stdout:893 writesame 0x8d3bdf9 thru 0x8d45ca3 data_size 0x1(0x9eab bytes) 2024-05-11T23:01:40.426 INFO:teuthology.orchestra.run.smithi005.stdout:894 writesame 0x2f505c9 thru 0x2f53a1a data_size 0x2(0x3452 bytes) 2024-05-11T23:01:40.432 INFO:teuthology.orchestra.run.smithi005.stdout:895 trunc from 0x8d45ca4 to 0x77b3c3c 2024-05-11T23:01:40.438 INFO:teuthology.orchestra.run.smithi005.stdout:896 read 0x33ea9eb thru 0x33f1394 (0x69aa bytes) 2024-05-11T23:01:40.451 INFO:teuthology.orchestra.run.smithi005.stdout:899 write 0xe1b8a42 thru 0xe1c18bb (0x8e7a bytes) 2024-05-11T23:01:40.454 INFO:teuthology.orchestra.run.smithi005.stdout:900 write 0x51633ac thru 0x5166ea4 (0x3af9 bytes) 2024-05-11T23:01:40.456 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.456 INFO:teuthology.orchestra.run.smithi005.stdout:904 write 0xc0b472f thru 0xc0b8150 (0x3a22 bytes) 2024-05-11T23:01:40.458 INFO:teuthology.orchestra.run.smithi005.stdout:906 write 0x97a5127 thru 0x97ae831 (0x970b bytes) 2024-05-11T23:01:40.474 INFO:teuthology.orchestra.run.smithi005.stdout:907 trunc from 0xe1c18bc to 0x2a66df4 2024-05-11T23:01:40.481 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:40.481+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:40.487 INFO:teuthology.orchestra.run.smithi005.stdout:908 read 0x2477343 thru 0x2481888 (0xa546 bytes) 2024-05-11T23:01:40.497 INFO:teuthology.orchestra.run.smithi005.stdout:910 writesame 0x6b2dba9 thru 0x6b377d8 data_size 0x88(0x9c30 bytes) 2024-05-11T23:01:40.497 INFO:teuthology.orchestra.run.smithi005.stdout:911 writesame 0x3338ed4 thru 0x3344fb0 data_size 0x1(0xc0dd bytes) 2024-05-11T23:01:40.514 INFO:teuthology.orchestra.run.smithi005.stdout:912 read 0x203dcc5 thru 0x2048508 (0xa844 bytes) 2024-05-11T23:01:40.514 INFO:teuthology.orchestra.run.smithi005.stdout:913 read 0x2542369 thru 0x254f154 (0xcdec bytes) 2024-05-11T23:01:40.514 INFO:teuthology.orchestra.run.smithi005.stdout:914 write 0x3a405ae thru 0x3a4a15d (0x9bb0 bytes) 2024-05-11T23:01:40.517 INFO:teuthology.orchestra.run.smithi005.stdout:915 write 0x22e35ce thru 0x22e802f (0x4a62 bytes) 2024-05-11T23:01:40.520 INFO:teuthology.orchestra.run.smithi005.stdout:919 write 0x599141d thru 0x59a0661 (0xf245 bytes) 2024-05-11T23:01:40.522 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:40.522 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:40.525 INFO:teuthology.orchestra.run.smithi005.stdout:921 read 0x6079eda thru 0x60850d1 (0xb1f8 bytes) 2024-05-11T23:01:40.525 INFO:teuthology.orchestra.run.smithi005.stdout:922 punch from 0x385625f to 0x385a0b4, (0x3e55 bytes) 2024-05-11T23:01:40.526 INFO:teuthology.orchestra.run.smithi005.stdout:923 writesame 0x5b2be41 thru 0x5b327f0 data_size 0x59(0x69b0 bytes) 2024-05-11T23:01:40.527 INFO:teuthology.orchestra.run.smithi005.stdout:924 read 0x52cf314 thru 0x52d52e5 (0x5fd2 bytes) 2024-05-11T23:01:40.532 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:01:40.533 INFO:teuthology.orchestra.run.smithi005.stdout:925 write 0x6e9d936 thru 0x6ea70f6 (0x97c1 bytes) 2024-05-11T23:01:40.536 INFO:teuthology.orchestra.run.smithi005.stdout:926 write 0x1efc54f thru 0x1eff37a (0x2e2c bytes) 2024-05-11T23:01:40.539 INFO:teuthology.orchestra.run.smithi005.stdout:927 trunc from 0x6ea70f7 to 0x73049f8 2024-05-11T23:01:40.543 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.543 INFO:teuthology.orchestra.run.smithi005.stdout:930 write 0x6d63bdc thru 0x6d6be54 (0x8279 bytes) 2024-05-11T23:01:40.544 INFO:teuthology.orchestra.run.smithi005.stdout:931 write 0x428881 thru 0x42b4ec (0x2c6c bytes) 2024-05-11T23:01:40.546 INFO:teuthology.orchestra.run.smithi005.stdout:932 write 0x107ed1 thru 0x10cf6f (0x509f bytes) 2024-05-11T23:01:40.547 INFO:teuthology.orchestra.run.smithi005.stdout:933 punch from 0x1a3f4b0 to 0x1a3fcab, (0x7fb bytes) 2024-05-11T23:01:40.559 INFO:teuthology.orchestra.run.smithi005.stdout:934 write 0xc5e6ec3 thru 0xc5f6669 (0xf7a7 bytes) 2024-05-11T23:01:40.564 INFO:teuthology.orchestra.run.smithi005.stdout:935 trunc from 0xc5f666a to 0xe68642c 2024-05-11T23:01:40.569 INFO:teuthology.orchestra.run.smithi005.stdout:938 write 0x6235754 thru 0x623738c (0x1c39 bytes) 2024-05-11T23:01:40.572 INFO:teuthology.orchestra.run.smithi005.stdout:940 punch from 0xa00c416 to 0xa014bdd, (0x87c7 bytes) 2024-05-11T23:01:40.572 INFO:teuthology.orchestra.run.smithi005.stdout:941 punch from 0x1478754 to 0x1483108, (0xa9b4 bytes) 2024-05-11T23:01:40.572 INFO:teuthology.orchestra.run.smithi005.stdout:942 read 0x8c81af2 thru 0x8c89fcc (0x84db bytes) 2024-05-11T23:01:40.572 INFO:teuthology.orchestra.run.smithi005.stdout:943 writesame 0xad2e236 thru 0xad39b45 data_size 0x70(0xb910 bytes) 2024-05-11T23:01:40.575 INFO:teuthology.orchestra.run.smithi005.stdout:945 writesame 0xaa471f0 thru 0xaa4ecc2 data_size 0x3(0x7ad3 bytes) 2024-05-11T23:01:40.582 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:40.582+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:40.582 INFO:teuthology.orchestra.run.smithi005.stdout:947 trunc from 0xe68642c to 0xd13437f 2024-05-11T23:01:40.587 INFO:teuthology.orchestra.run.smithi005.stdout:949 read 0x5fa3287 thru 0x5fab9a8 (0x8722 bytes) 2024-05-11T23:01:40.587 INFO:teuthology.orchestra.run.smithi005.stdout:954 read 0x61dc380 thru 0x61de50f (0x2190 bytes) 2024-05-11T23:01:40.594 INFO:teuthology.orchestra.run.smithi005.stdout:956 trunc from 0xd13437f to 0x7ca272c 2024-05-11T23:01:40.605 INFO:teuthology.orchestra.run.smithi005.stdout:957 trunc from 0x7ca272c to 0x9ae2269 2024-05-11T23:01:40.610 INFO:teuthology.orchestra.run.smithi005.stdout:958 write 0x49a42f1 thru 0x49af19b (0xaeab bytes) 2024-05-11T23:01:40.614 INFO:teuthology.orchestra.run.smithi005.stdout:959 punch from 0x29a69f8 to 0x29a7b8b, (0x1193 bytes) 2024-05-11T23:01:40.614 INFO:teuthology.orchestra.run.smithi005.stdout:960 writesame 0x5d77bb4 thru 0x5d86674 data_size 0x1(0xeac1 bytes) 2024-05-11T23:01:40.631 INFO:teuthology.orchestra.run.smithi005.stdout:962 read 0x6ca9dd9 thru 0x6cb080d (0x6a35 bytes) 2024-05-11T23:01:40.631 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.632 INFO:teuthology.orchestra.run.smithi005.stdout:966 trunc from 0x9ae2269 to 0x8b97aa7 2024-05-11T23:01:40.637 INFO:teuthology.orchestra.run.smithi005.stdout:968 read 0x2a7034a thru 0x2a79f17 (0x9bce bytes) 2024-05-11T23:01:40.646 INFO:teuthology.orchestra.run.smithi005.stdout:971 write 0xbb380a4 thru 0xbb3fbc8 (0x7b25 bytes) 2024-05-11T23:01:40.649 INFO:teuthology.orchestra.run.smithi005.stdout:973 read 0x8f58e80 thru 0x8f5cb5a (0x3cdb bytes) 2024-05-11T23:01:40.649 INFO:teuthology.orchestra.run.smithi005.stdout:974 write 0x9058b5b thru 0x90626fe (0x9ba4 bytes) 2024-05-11T23:01:40.652 INFO:teuthology.orchestra.run.smithi005.stdout:977 punch from 0x26fb08c to 0x26fe563, (0x34d7 bytes) 2024-05-11T23:01:40.652 INFO:teuthology.orchestra.run.smithi005.stdout:978 read 0x94bbf98 thru 0x94c4ab9 (0x8b22 bytes) 2024-05-11T23:01:40.653 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:40.653 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:40.657 INFO:teuthology.orchestra.run.smithi005.stdout:980 trunc from 0xbb3fbc9 to 0x7bd4468 2024-05-11T23:01:40.664 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:01:40.666 INFO:teuthology.orchestra.run.smithi005.stdout:981 read 0x5aecedd thru 0x5afb7e8 (0xe90c bytes) 2024-05-11T23:01:40.668 INFO:teuthology.orchestra.run.smithi005.stdout:983 write 0x612ec73 thru 0x61375bb (0x8949 bytes) 2024-05-11T23:01:40.669 INFO:teuthology.orchestra.run.smithi005.stdout:984 punch from 0x250ba4a to 0x25107af, (0x4d65 bytes) 2024-05-11T23:01:40.669 INFO:teuthology.orchestra.run.smithi005.stdout:985 write 0x363dfa5 thru 0x364080a (0x2866 bytes) 2024-05-11T23:01:40.670 INFO:teuthology.orchestra.run.smithi005.stdout:986 write 0x21119a7 thru 0x211cf78 (0xb5d2 bytes) 2024-05-11T23:01:40.673 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.673 INFO:teuthology.orchestra.run.smithi005.stdout:988 write 0x4c5d270 thru 0x4c6aea8 (0xdc39 bytes) 2024-05-11T23:01:40.678 INFO:teuthology.orchestra.run.smithi005.stdout:989 trunc from 0x7bd4468 to 0x630126f 2024-05-11T23:01:40.683 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:40.683+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:40.688 INFO:teuthology.orchestra.run.smithi005.stdout:990 write 0xd28c7a thru 0xd32d9a (0xa121 bytes) 2024-05-11T23:01:40.703 INFO:teuthology.orchestra.run.smithi005.stdout:991 writesame 0xc8dd6a2 thru 0xc8e1bff data_size 0xd(0x455e bytes) 2024-05-11T23:01:40.703 INFO:teuthology.orchestra.run.smithi005.stdout:992 punch from 0x6e812af to 0x6e8744c, (0x619d bytes) 2024-05-11T23:01:40.703 INFO:teuthology.orchestra.run.smithi005.stdout:993 write 0xa6c2c47 thru 0xa6cf0d4 (0xc48e bytes) 2024-05-11T23:01:40.706 INFO:teuthology.orchestra.run.smithi005.stdout:994 punch from 0xb78b1d6 to 0xb7930c2, (0x7eec bytes) 2024-05-11T23:01:40.706 INFO:teuthology.orchestra.run.smithi005.stdout:995 read 0x28b3963 thru 0x28b6c6f (0x330d bytes) 2024-05-11T23:01:40.706 INFO:teuthology.orchestra.run.smithi005.stdout:996 read 0x8942da thru 0x89b5b9 (0x72e0 bytes) 2024-05-11T23:01:40.712 INFO:teuthology.orchestra.run.smithi005.stdout:997 trunc from 0xc8e1c00 to 0x71feb0c 2024-05-11T23:01:40.720 INFO:teuthology.orchestra.run.smithi005.stdout:999 write 0x3da8229 thru 0x3db5cdd (0xdab5 bytes) 2024-05-11T23:01:40.723 INFO:teuthology.orchestra.run.smithi005.stdout:1000 read 0x4e6405a thru 0x4e66f62 (0x2f09 bytes) 2024-05-11T23:01:40.723 INFO:teuthology.orchestra.run.smithi005.stdout:1001 read 0x5ea88a4 thru 0x5eab8a6 (0x3003 bytes) 2024-05-11T23:01:40.731 INFO:teuthology.orchestra.run.smithi005.stdout:1002 trunc from 0x71feb0c to 0xd92c3f0 2024-05-11T23:01:40.736 INFO:teuthology.orchestra.run.smithi005.stdout:1003 punch from 0x7f26df2 to 0x7f320be, (0xb2cc bytes) 2024-05-11T23:01:40.736 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.736 INFO:teuthology.orchestra.run.smithi005.stdout:1005 read 0x8a3d73a thru 0x8a485ae (0xae75 bytes) 2024-05-11T23:01:40.736 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.737 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.737 INFO:teuthology.orchestra.run.smithi005.stdout:1009 punch from 0x2d05d46 to 0x2d083b4, (0x266e bytes) 2024-05-11T23:01:40.740 INFO:teuthology.orchestra.run.smithi005.stdout:1010 trunc from 0xd92c3f0 to 0xab9caf9 2024-05-11T23:01:40.747 INFO:teuthology.orchestra.run.smithi005.stdout:1011 trunc from 0xab9caf9 to 0xc8929ab 2024-05-11T23:01:40.752 INFO:teuthology.orchestra.run.smithi005.stdout:1013 writesame 0x4f26135 thru 0x4f2f8a1 data_size 0x1(0x976d bytes) 2024-05-11T23:01:40.770 INFO:teuthology.orchestra.run.smithi005.stdout:1014 read 0x1a3fae8 thru 0x1a4e8ea (0xee03 bytes) 2024-05-11T23:01:40.770 INFO:teuthology.orchestra.run.smithi005.stdout:1015 read 0xb1bdf43 thru 0xb1c904b (0xb109 bytes) 2024-05-11T23:01:40.778 INFO:teuthology.orchestra.run.smithi005.stdout:1016 write 0xe1fc720 thru 0xe20728f (0xab70 bytes) 2024-05-11T23:01:40.781 INFO:teuthology.orchestra.run.smithi005.stdout:1017 write 0xbb63999 thru 0xbb64145 (0x7ad bytes) 2024-05-11T23:01:40.783 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:40.783+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:40.784 INFO:teuthology.orchestra.run.smithi005.stdout:1018 write 0xad8abfd thru 0xad8fafa (0x4efe bytes) 2024-05-11T23:01:40.786 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.786 INFO:teuthology.orchestra.run.smithi005.stdout:1020 writesame 0x6b19cfb thru 0x6b2692c data_size 0x2(0xcc32 bytes) 2024-05-11T23:01:40.796 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-11T23:01:40.796 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:40.797 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:40.799 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.799 INFO:teuthology.orchestra.run.smithi005.stdout:1022 read 0xd4b9862 thru 0xd4bcc48 (0x33e7 bytes) 2024-05-11T23:01:40.806 INFO:teuthology.orchestra.run.smithi005.stdout:1025 write 0xe228f35 thru 0xe2304e2 (0x75ae bytes) 2024-05-11T23:01:40.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:01:40.808 INFO:teuthology.orchestra.run.smithi005.stdout:1027 read 0x58f623e thru 0x58ffbfd (0x99c0 bytes) 2024-05-11T23:01:40.809 INFO:teuthology.orchestra.run.smithi005.stdout:1029 write 0x31de926 thru 0x31e9716 (0xadf1 bytes) 2024-05-11T23:01:40.811 INFO:teuthology.orchestra.run.smithi005.stdout:1031 read 0x864bdfb thru 0x864dc36 (0x1e3c bytes) 2024-05-11T23:01:40.811 INFO:teuthology.orchestra.run.smithi005.stdout:1034 read 0xb4a7b0f thru 0xb4afa7a (0x7f6c bytes) 2024-05-11T23:01:40.811 INFO:teuthology.orchestra.run.smithi005.stdout:1036 write 0x4c6e418 thru 0x4c70900 (0x24e9 bytes) 2024-05-11T23:01:40.812 INFO:teuthology.orchestra.run.smithi005.stdout:1037 write 0x80cac41 thru 0x80d4e22 (0xa1e2 bytes) 2024-05-11T23:01:40.815 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.815 INFO:teuthology.orchestra.run.smithi005.stdout:1040 writesame 0xc47325b thru 0xc47ec8d data_size 0x1(0xba33 bytes) 2024-05-11T23:01:40.851 INFO:teuthology.orchestra.run.smithi005.stdout:1043 trunc from 0xe2304e3 to 0x2b2c12f 2024-05-11T23:01:40.866 INFO:teuthology.orchestra.run.smithi005.stdout:1044 trunc from 0x2b2c12f to 0x3fe90cb 2024-05-11T23:01:40.871 INFO:teuthology.orchestra.run.smithi005.stdout:1045 read 0x2455d96 thru 0x2457223 (0x148e bytes) 2024-05-11T23:01:40.872 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.872 INFO:teuthology.orchestra.run.smithi005.stdout:1047 read 0xfe7f2 thru 0x101408 (0x2c17 bytes) 2024-05-11T23:01:40.872 INFO:teuthology.orchestra.run.smithi005.stdout:1048 read 0xafa1f4 thru 0xafbdb7 (0x1bc4 bytes) 2024-05-11T23:01:40.872 INFO:teuthology.orchestra.run.smithi005.stdout:1049 read 0xd00986 thru 0xd028d8 (0x1f53 bytes) 2024-05-11T23:01:40.882 INFO:teuthology.orchestra.run.smithi005.stdout:1050 writesame 0x69da3e3 thru 0x69dee01 data_size 0x4b(0x4a1f bytes) 2024-05-11T23:01:40.882 INFO:teuthology.orchestra.run.smithi005.stdout:1052 punch from 0x2c00303 to 0x2c008a4, (0x5a1 bytes) 2024-05-11T23:01:40.884 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:40.884+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:40.894 INFO:teuthology.orchestra.run.smithi005.stdout:1055 writesame 0xb65f723 thru 0xb662b05 data_size 0x1(0x33e3 bytes) 2024-05-11T23:01:40.894 INFO:teuthology.orchestra.run.smithi005.stdout:1056 write 0x6d25ada thru 0x6d2a9b9 (0x4ee0 bytes) 2024-05-11T23:01:40.897 INFO:teuthology.orchestra.run.smithi005.stdout:1057 read 0x9ff0506 thru 0x9ff2328 (0x1e23 bytes) 2024-05-11T23:01:40.897 INFO:teuthology.orchestra.run.smithi005.stdout:1059 read 0x3f44fe9 thru 0x3f4d44d (0x8465 bytes) 2024-05-11T23:01:40.897 INFO:teuthology.orchestra.run.smithi005.stdout:1061 punch from 0x814d9b4 to 0x815367f, (0x5ccb bytes) 2024-05-11T23:01:40.897 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.897 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.897 INFO:teuthology.orchestra.run.smithi005.stdout:1064 write 0x3ebb87e thru 0x3ec768b (0xbe0e bytes) 2024-05-11T23:01:40.902 INFO:teuthology.orchestra.run.smithi005.stdout:1065 read 0x14983cd thru 0x14a4560 (0xc194 bytes) 2024-05-11T23:01:40.902 INFO:teuthology.orchestra.run.smithi005.stdout:1066 write 0x5a12527 thru 0x5a13452 (0xf2c bytes) 2024-05-11T23:01:40.905 INFO:teuthology.orchestra.run.smithi005.stdout:1067 read 0x24a5ed3 thru 0x24b0484 (0xa5b2 bytes) 2024-05-11T23:01:40.906 INFO:teuthology.orchestra.run.smithi005.stdout:1068 read 0x7a5aaa0 thru 0x7a5ac61 (0x1c2 bytes) 2024-05-11T23:01:40.906 INFO:teuthology.orchestra.run.smithi005.stdout:1070 punch from 0x5db986c to 0x5dbee2a, (0x55be bytes) 2024-05-11T23:01:40.906 INFO:teuthology.orchestra.run.smithi005.stdout:1071 read 0x6a0c4f2 thru 0x6a0c6f3 (0x202 bytes) 2024-05-11T23:01:40.906 INFO:teuthology.orchestra.run.smithi005.stdout:1073 punch from 0x295328e to 0x295e1f0, (0xaf62 bytes) 2024-05-11T23:01:40.906 INFO:teuthology.orchestra.run.smithi005.stdout:1074 punch from 0x44a46f1 to 0x44adaaa, (0x93b9 bytes) 2024-05-11T23:01:40.906 INFO:teuthology.orchestra.run.smithi005.stdout:1076 write 0x7fc20e7 thru 0x7fccefe (0xae18 bytes) 2024-05-11T23:01:40.908 INFO:teuthology.orchestra.run.smithi005.stdout:1077 writesame 0x25da107 thru 0x25e7632 data_size 0xe(0xd52c bytes) 2024-05-11T23:01:40.914 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:40.914 INFO:teuthology.orchestra.run.smithi005.stdout:1079 read 0x3cadc03 thru 0x3cb9776 (0xbb74 bytes) 2024-05-11T23:01:40.914 INFO:teuthology.orchestra.run.smithi005.stdout:1080 read 0x417b1e6 thru 0x4187bf3 (0xca0e bytes) 2024-05-11T23:01:40.915 INFO:teuthology.orchestra.run.smithi005.stdout:1081 punch from 0x482dcc2 to 0x4831d84, (0x40c2 bytes) 2024-05-11T23:01:40.915 INFO:teuthology.orchestra.run.smithi005.stdout:1083 write 0xa898c34 thru 0xa8a1a5f (0x8e2c bytes) 2024-05-11T23:01:40.918 INFO:teuthology.orchestra.run.smithi005.stdout:1086 write 0x865a903 thru 0x8660e9d (0x659b bytes) 2024-05-11T23:01:40.922 INFO:teuthology.orchestra.run.smithi005.stdout:1087 punch from 0x6ae5239 to 0x6ae6cef, (0x1ab6 bytes) 2024-05-11T23:01:40.937 INFO:teuthology.orchestra.run.smithi005.stdout:1088 trunc from 0xb662b06 to 0x5a38b2 2024-05-11T23:01:40.948 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:40.948 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:40.949 INFO:teuthology.orchestra.run.smithi005.stdout:1090 read 0x5a019 thru 0x62f35 (0x8f1d bytes) 2024-05-11T23:01:40.949 INFO:teuthology.orchestra.run.smithi005.stdout:1091 read 0x5914c thru 0x5f2da (0x618f bytes) 2024-05-11T23:01:40.962 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:01:40.968 INFO:teuthology.orchestra.run.smithi005.stdout:1092 writesame 0xaf7513b thru 0xaf7c1c8 data_size 0x1(0x708e bytes) 2024-05-11T23:01:40.968 INFO:teuthology.orchestra.run.smithi005.stdout:1093 punch from 0x7c6b69e to 0x7c7854e, (0xceb0 bytes) 2024-05-11T23:01:40.968 INFO:teuthology.orchestra.run.smithi005.stdout:1094 read 0x1d6c8e3 thru 0x1d792c4 (0xc9e2 bytes) 2024-05-11T23:01:40.968 INFO:teuthology.orchestra.run.smithi005.stdout:1095 punch from 0x43632f9 to 0x43698cd, (0x65d4 bytes) 2024-05-11T23:01:40.968 INFO:teuthology.orchestra.run.smithi005.stdout:1096 read 0x8fb9852 thru 0x8fc0778 (0x6f27 bytes) 2024-05-11T23:01:40.968 INFO:teuthology.orchestra.run.smithi005.stdout:1098 punch from 0x282f3be to 0x283aa16, (0xb658 bytes) 2024-05-11T23:01:40.974 INFO:teuthology.orchestra.run.smithi005.stdout:1099 trunc from 0xaf7c1c9 to 0x50734f0 2024-05-11T23:01:40.984 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:40.984+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:40.985 INFO:teuthology.orchestra.run.smithi005.stdout:1101 write 0x5811246 thru 0x581e211 (0xcfcc bytes) 2024-05-11T23:01:40.988 INFO:teuthology.orchestra.run.smithi005.stdout:1102 write 0x4513567 thru 0x45148ec (0x1386 bytes) 2024-05-11T23:01:41.000 INFO:teuthology.orchestra.run.smithi005.stdout:1104 write 0x88e7933 thru 0x88e9435 (0x1b03 bytes) 2024-05-11T23:01:41.013 INFO:teuthology.orchestra.run.smithi005.stdout:1105 writesame 0xc8b2f7c thru 0xc8b6d50 data_size 0x1(0x3dd5 bytes) 2024-05-11T23:01:41.013 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:41.016 INFO:teuthology.orchestra.run.smithi005.stdout:1107 trunc from 0xc8b6d51 to 0xa21a291 2024-05-11T23:01:41.021 INFO:teuthology.orchestra.run.smithi005.stdout:1111 read 0x7c70969 thru 0x7c7102b (0x6c3 bytes) 2024-05-11T23:01:41.022 INFO:teuthology.orchestra.run.smithi005.stdout:1112 write 0x4221ae6 thru 0x422b3c2 (0x98dd bytes) 2024-05-11T23:01:41.025 INFO:teuthology.orchestra.run.smithi005.stdout:1114 writesame 0x3e3d454 thru 0x3e418dc data_size 0x1d(0x4489 bytes) 2024-05-11T23:01:41.029 INFO:teuthology.orchestra.run.smithi005.stdout:1115 writesame 0x66cafa4 thru 0x66d29af data_size 0x6b(0x7a0c bytes) 2024-05-11T23:01:41.032 INFO:teuthology.orchestra.run.smithi005.stdout:1116 read 0x9b144b9 thru 0x9b21bb0 (0xd6f8 bytes) 2024-05-11T23:01:41.032 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:41.032 INFO:teuthology.orchestra.run.smithi005.stdout:1120 write 0x5dec175 thru 0x5dfae2a (0xecb6 bytes) 2024-05-11T23:01:41.035 INFO:teuthology.orchestra.run.smithi005.stdout:1121 writesame 0x5a4c21 thru 0x5ac9bc data_size 0x2(0x7d9c bytes) 2024-05-11T23:01:41.048 INFO:teuthology.orchestra.run.smithi005.stdout:1122 writesame 0xa6146a5 thru 0xa6163f4 data_size 0x38(0x1d50 bytes) 2024-05-11T23:01:41.059 INFO:teuthology.orchestra.run.smithi005.stdout:1123 writesame 0xdffa396 thru 0xe007ec7 data_size 0x1(0xdb32 bytes) 2024-05-11T23:01:41.059 INFO:teuthology.orchestra.run.smithi005.stdout:1127 read 0xcabaf1f thru 0xcabc0bb (0x119d bytes) 2024-05-11T23:01:41.059 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:41.059 INFO:teuthology.orchestra.run.smithi005.stdout:1129 trunc from 0xe007ec8 to 0xdceb912 2024-05-11T23:01:41.064 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:41.064 INFO:teuthology.orchestra.run.smithi005.stdout:1131 write 0x1dfc4b3 thru 0x1e068d3 (0xa421 bytes) 2024-05-11T23:01:41.075 INFO:teuthology.orchestra.run.smithi005.stdout:1132 trunc from 0xdceb912 to 0xb8f0419 2024-05-11T23:01:41.080 INFO:teuthology.orchestra.run.smithi005.stdout:1133 read 0x3ba99f7 thru 0x3bb0a5f (0x7069 bytes) 2024-05-11T23:01:41.080 INFO:teuthology.orchestra.run.smithi005.stdout:1134 writesame 0x6e7420e thru 0x6e7ae9d data_size 0x48(0x6c90 bytes) 2024-05-11T23:01:41.083 INFO:teuthology.orchestra.run.smithi005.stdout:1135 write 0x19c4c44 thru 0x19c6fb4 (0x2371 bytes) 2024-05-11T23:01:41.085 INFO:teuthology.orchestra.run.smithi005.stdout:1136 punch from 0x2a3c038 to 0x2a4916e, (0xd136 bytes) 2024-05-11T23:01:41.085 INFO:teuthology.orchestra.run.smithi005.stdout:1138 read 0xa306ae8 thru 0xa30c43f (0x5958 bytes) 2024-05-11T23:01:41.085 INFO:teuthology.orchestra.run.smithi005.stdout:1140 punch from 0x89d8a3b to 0x89e61b6, (0xd77b bytes) 2024-05-11T23:01:41.085 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:41.085+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:41.087 INFO:teuthology.orchestra.run.smithi005.stdout:1141 trunc from 0xb8f0419 to 0xd65e6a4 2024-05-11T23:01:41.100 INFO:teuthology.orchestra.run.smithi005.stdout:1142 writesame 0xe873a4d thru 0xe8839fa data_size 0x3(0xffae bytes) 2024-05-11T23:01:41.101 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:41.101 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:41.115 INFO:teuthology.orchestra.run.smithi005.stdout:1145 trunc from 0xe8839fb to 0x2fe1319 2024-05-11T23:01:41.123 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:01:41.141 INFO:teuthology.orchestra.run.smithi005.stdout:1147 write 0xa716a25 thru 0xa71be14 (0x53f0 bytes) 2024-05-11T23:01:41.154 INFO:teuthology.orchestra.run.smithi005.stdout:1148 writesame 0xe645dd4 thru 0xe64631d data_size 0x1(0x54a bytes) 2024-05-11T23:01:41.154 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:41.154 INFO:teuthology.orchestra.run.smithi005.stdout:1150 write 0xca3e54e thru 0xca4309d (0x4b50 bytes) 2024-05-11T23:01:41.157 INFO:teuthology.orchestra.run.smithi005.stdout:1151 writesame 0xd5f2247 thru 0xd5ff4a9 data_size 0xd(0xd263 bytes) 2024-05-11T23:01:41.169 INFO:teuthology.orchestra.run.smithi005.stdout:1153 trunc from 0xe64631e to 0x731cfc4 2024-05-11T23:01:41.186 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:41.186+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:41.187 INFO:teuthology.orchestra.run.smithi005.stdout:1154 trunc from 0x731cfc4 to 0xab5da 2024-05-11T23:01:41.209 INFO:teuthology.orchestra.run.smithi005.stdout:1156 write 0x4f4fc3d thru 0x4f59322 (0x96e6 bytes) 2024-05-11T23:01:41.213 INFO:teuthology.orchestra.run.smithi005.stdout:1157 punch from 0x1d2d7a1 to 0x1d38895, (0xb0f4 bytes) 2024-05-11T23:01:41.213 INFO:teuthology.orchestra.run.smithi005.stdout:1158 read 0x387e0f2 thru 0x388bc76 (0xdb85 bytes) 2024-05-11T23:01:41.213 INFO:teuthology.orchestra.run.smithi005.stdout:1159 write 0x4c6360 thru 0x4c7922 (0x15c3 bytes) 2024-05-11T23:01:41.213 INFO:teuthology.orchestra.run.smithi005.stdout:1160 read 0x4d96985 thru 0x4d99fc9 (0x3645 bytes) 2024-05-11T23:01:41.214 INFO:teuthology.orchestra.run.smithi005.stdout:1161 read 0xf822c9 thru 0xf85da4 (0x3adc bytes) 2024-05-11T23:01:41.225 INFO:teuthology.orchestra.run.smithi005.stdout:1162 write 0xa4c60e4 thru 0xa4cea5d (0x897a bytes) 2024-05-11T23:01:41.227 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:41.227 INFO:teuthology.orchestra.run.smithi005.stdout:1164 writesame 0x3a790a8 thru 0x3a7ae80 data_size 0x1b(0x1dd9 bytes) 2024-05-11T23:01:41.230 INFO:teuthology.orchestra.run.smithi005.stdout:1165 punch from 0x5bf0bd2 to 0x5bfb3a7, (0xa7d5 bytes) 2024-05-11T23:01:41.230 INFO:teuthology.orchestra.run.smithi005.stdout:1166 write 0x1d30254 thru 0x1d3f4b5 (0xf262 bytes) 2024-05-11T23:01:41.233 INFO:teuthology.orchestra.run.smithi005.stdout:1167 writesame 0x4d47502 thru 0x4d4c6d1 data_size 0x38(0x51d0 bytes) 2024-05-11T23:01:41.234 INFO:teuthology.orchestra.run.smithi005.stdout:1168 writesame 0x559f799 thru 0x55ac6b7 data_size 0x1(0xcf1f bytes) 2024-05-11T23:01:41.260 INFO:teuthology.orchestra.run.smithi005.stdout:1169 read 0x118264f thru 0x118619a (0x3b4c bytes) 2024-05-11T23:01:41.260 INFO:teuthology.orchestra.run.smithi005.stdout:1170 writesame 0x413204a thru 0x4138e05 data_size 0x6(0x6dbc bytes) 2024-05-11T23:01:41.265 INFO:teuthology.orchestra.run.smithi005.stdout:1171 punch from 0x987830d to 0x9881b60, (0x9853 bytes) 2024-05-11T23:01:41.265 INFO:teuthology.orchestra.run.smithi005.stdout:1174 punch from 0xa07678a to 0xa07c6ad, (0x5f23 bytes) 2024-05-11T23:01:41.265 INFO:teuthology.orchestra.run.smithi005.stdout:1176 writesame 0x8a5b9d5 thru 0x8a67df0 data_size 0x1c(0xc41c bytes) 2024-05-11T23:01:41.269 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:41.269 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:41.271 INFO:teuthology.orchestra.run.smithi005.stdout:1179 punch from 0x606f691 to 0x60797e2, (0xa151 bytes) 2024-05-11T23:01:41.271 INFO:teuthology.orchestra.run.smithi005.stdout:1180 punch from 0x9fb3698 to 0x9fb4011, (0x979 bytes) 2024-05-11T23:01:41.271 INFO:teuthology.orchestra.run.smithi005.stdout:1181 punch from 0x88b6de5 to 0x88c6be6, (0xfe01 bytes) 2024-05-11T23:01:41.271 INFO:teuthology.orchestra.run.smithi005.stdout:1182 punch from 0x48731cb to 0x4881a45, (0xe87a bytes) 2024-05-11T23:01:41.271 INFO:teuthology.orchestra.run.smithi005.stdout:1183 write 0x94bdeba thru 0x94c6b62 (0x8ca9 bytes) 2024-05-11T23:01:41.281 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:01:41.282 INFO:teuthology.orchestra.run.smithi005.stdout:1184 write 0xc7aeef0 thru 0xc7baedd (0xbfee bytes) 2024-05-11T23:01:41.285 INFO:teuthology.orchestra.run.smithi005.stdout:1185 read 0xf8180 thru 0xffaf4 (0x7975 bytes) 2024-05-11T23:01:41.285 INFO:teuthology.orchestra.run.smithi005.stdout:1186 write 0x9f95a2e thru 0x9fa25ad (0xcb80 bytes) 2024-05-11T23:01:41.286 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:41.285+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:41.289 INFO:teuthology.orchestra.run.smithi005.stdout:1187 write 0x2bf34d7 thru 0x2c00a2b (0xd555 bytes) 2024-05-11T23:01:41.293 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:41.293 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:41.293 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:41.294 INFO:teuthology.orchestra.run.smithi005.stdout:1193 read 0x7ba1112 thru 0x7ba1ba0 (0xa8f bytes) 2024-05-11T23:01:41.294 INFO:teuthology.orchestra.run.smithi005.stdout:1194 write 0x22e1dd9 thru 0x22f01aa (0xe3d2 bytes) 2024-05-11T23:01:41.301 INFO:teuthology.orchestra.run.smithi005.stdout:1196 trunc from 0xc7baede to 0x832b176 2024-05-11T23:01:41.321 INFO:teuthology.orchestra.run.smithi005.stdout:1198 writesame 0xbe0cf96 thru 0xbe144bd data_size 0x5c(0x7528 bytes) 2024-05-11T23:01:41.321 INFO:teuthology.orchestra.run.smithi005.stdout:1199 writesame 0x9710b0 thru 0x978c7a data_size 0x2b(0x7bcb bytes) 2024-05-11T23:01:41.324 INFO:teuthology.orchestra.run.smithi005.stdout:1201 writesame 0x188197e thru 0x188876a data_size 0x1(0x6ded bytes) 2024-05-11T23:01:41.338 INFO:teuthology.orchestra.run.smithi005.stdout:1202 punch from 0x51fafdd to 0x5201ded, (0x6e10 bytes) 2024-05-11T23:01:41.339 INFO:teuthology.orchestra.run.smithi005.stdout:1203 read 0x3e3af53 thru 0x3e3b34c (0x3fa bytes) 2024-05-11T23:01:41.339 INFO:teuthology.orchestra.run.smithi005.stdout:1204 write 0x2672d95 thru 0x2672fff (0x26b bytes) 2024-05-11T23:01:41.340 INFO:teuthology.orchestra.run.smithi005.stdout:1205 read 0x222affa thru 0x2232b99 (0x7ba0 bytes) 2024-05-11T23:01:41.340 INFO:teuthology.orchestra.run.smithi005.stdout:1206 punch from 0x57f9165 to 0x5800da3, (0x7c3e bytes) 2024-05-11T23:01:41.340 INFO:teuthology.orchestra.run.smithi005.stdout:1207 write 0x9998aa9 thru 0x99a4202 (0xb75a bytes) 2024-05-11T23:01:41.344 INFO:teuthology.orchestra.run.smithi005.stdout:1208 punch from 0xb86e30b to 0xb86f7f0, (0x14e5 bytes) 2024-05-11T23:01:41.344 INFO:teuthology.orchestra.run.smithi005.stdout:1210 write 0xba2925f thru 0xba2d44b (0x41ed bytes) 2024-05-11T23:01:41.347 INFO:teuthology.orchestra.run.smithi005.stdout:1211 punch from 0x343ac65 to 0x343bc19, (0xfb4 bytes) 2024-05-11T23:01:41.347 INFO:teuthology.orchestra.run.smithi005.stdout:1212 read 0xb7a471d thru 0xb7a4ce6 (0x5ca bytes) 2024-05-11T23:01:41.347 INFO:teuthology.orchestra.run.smithi005.stdout:1213 trunc from 0xbe144be to 0xc0a4fee 2024-05-11T23:01:41.352 INFO:teuthology.orchestra.run.smithi005.stdout:1214 write 0x6d17224 thru 0x6d201a0 (0x8f7d bytes) 2024-05-11T23:01:41.355 INFO:teuthology.orchestra.run.smithi005.stdout:1216 write 0x53221c0 thru 0x533044a (0xe28b bytes) 2024-05-11T23:01:41.364 INFO:teuthology.orchestra.run.smithi005.stdout:1218 trunc from 0xc0a4fee to 0x5901845 2024-05-11T23:01:41.373 INFO:teuthology.orchestra.run.smithi005.stdout:1220 read 0x33f1ec2 thru 0x33f323b (0x137a bytes) 2024-05-11T23:01:41.373 INFO:teuthology.orchestra.run.smithi005.stdout:1222 read 0x7f1877 thru 0x800f69 (0xf6f3 bytes) 2024-05-11T23:01:41.379 INFO:teuthology.orchestra.run.smithi005.stdout:1224 write 0x65758f0 thru 0x6584ce9 (0xf3fa bytes) 2024-05-11T23:01:41.383 INFO:teuthology.orchestra.run.smithi005.stdout:1226 punch from 0x56bde67 to 0x56cb52d, (0xd6c6 bytes) 2024-05-11T23:01:41.383 INFO:teuthology.orchestra.run.smithi005.stdout:1227 read 0x6b0b1c thru 0x6b5236 (0x471b bytes) 2024-05-11T23:01:41.387 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:41.387+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:41.392 INFO:teuthology.orchestra.run.smithi005.stdout:1228 trunc from 0x6584cea to 0xe2a69d9 2024-05-11T23:01:41.397 INFO:teuthology.orchestra.run.smithi005.stdout:1229 punch from 0x746e133 to 0x746e8a3, (0x770 bytes) 2024-05-11T23:01:41.397 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:41.397 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:41.397 INFO:teuthology.orchestra.run.smithi005.stdout:1233 punch from 0x492895c to 0x4930377, (0x7a1b bytes) 2024-05-11T23:01:41.397 INFO:teuthology.orchestra.run.smithi005.stdout:1234 punch from 0x42be129 to 0x42c2895, (0x476c bytes) 2024-05-11T23:01:41.397 INFO:teuthology.orchestra.run.smithi005.stdout:1235 write 0xd6c04d7 thru 0xd6c5d48 (0x5872 bytes) 2024-05-11T23:01:41.399 INFO:teuthology.orchestra.run.smithi005.stdout:1237 read 0x2a84f24 thru 0x2a93baa (0xec87 bytes) 2024-05-11T23:01:41.400 INFO:teuthology.orchestra.run.smithi005.stdout:1238 writesame 0x2725e56 thru 0x27284e0 data_size 0x17(0x268b bytes) 2024-05-11T23:01:41.402 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:41.402 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:41.413 INFO:teuthology.orchestra.run.smithi005.stdout:1239 trunc from 0xe2a69d9 to 0x3e0d5b5 2024-05-11T23:01:41.417 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:01:41.424 INFO:teuthology.orchestra.run.smithi005.stdout:1240 read 0x324d0d thru 0x327e59 (0x314d bytes) 2024-05-11T23:01:41.434 INFO:teuthology.orchestra.run.smithi005.stdout:1242 trunc from 0x3e0d5b5 to 0xc4da9c6 2024-05-11T23:01:41.439 INFO:teuthology.orchestra.run.smithi005.stdout:1243 read 0x2f40618 thru 0x2f44992 (0x437b bytes) 2024-05-11T23:01:41.439 INFO:teuthology.orchestra.run.smithi005.stdout:1244 write 0x20ed977 thru 0x20f5bc7 (0x8251 bytes) 2024-05-11T23:01:41.440 INFO:teuthology.orchestra.run.smithi005.stdout:1245 read 0x1d2a04 thru 0x1d9025 (0x6622 bytes) 2024-05-11T23:01:41.441 INFO:teuthology.orchestra.run.smithi005.stdout:1246 write 0xb53e2b6 thru 0xb544a18 (0x6763 bytes) 2024-05-11T23:01:41.444 INFO:teuthology.orchestra.run.smithi005.stdout:1247 read 0x29e8c01 thru 0x29f4220 (0xb620 bytes) 2024-05-11T23:01:41.444 INFO:teuthology.orchestra.run.smithi005.stdout:1248 punch from 0x6edea99 to 0x6ee9074, (0xa5db bytes) 2024-05-11T23:01:41.446 INFO:teuthology.orchestra.run.smithi005.stdout:1249 trunc from 0xc4da9c6 to 0xb1ebc68 2024-05-11T23:01:41.452 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:41.452 INFO:teuthology.orchestra.run.smithi005.stdout:1252 punch from 0x4bd198b to 0x4bd2919, (0xf8e bytes) 2024-05-11T23:01:41.452 INFO:teuthology.orchestra.run.smithi005.stdout:1253 punch from 0x61a1627 to 0x61ab7b9, (0xa192 bytes) 2024-05-11T23:01:41.452 INFO:teuthology.orchestra.run.smithi005.stdout:1254 write 0xb09602b thru 0xb09e0f4 (0x80ca bytes) 2024-05-11T23:01:41.454 INFO:teuthology.orchestra.run.smithi005.stdout:1255 read 0x112f9d8 thru 0x1133f7d (0x45a6 bytes) 2024-05-11T23:01:41.456 INFO:teuthology.orchestra.run.smithi005.stdout:1256 trunc from 0xb1ebc68 to 0x8f8f89b 2024-05-11T23:01:41.466 INFO:teuthology.orchestra.run.smithi005.stdout:1257 read 0x4d4bf57 thru 0x4d4d994 (0x1a3e bytes) 2024-05-11T23:01:41.466 INFO:teuthology.orchestra.run.smithi005.stdout:1258 read 0x5675fb8 thru 0x5679567 (0x35b0 bytes) 2024-05-11T23:01:41.467 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:41.467 INFO:teuthology.orchestra.run.smithi005.stdout:1261 clone 8 order 18 su 131072 sc 14 2024-05-11T23:01:41.488 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:41.488+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:41.540 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:41.540 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:41.549 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:01:41.590 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:41.590+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:41.691 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:41.690+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:41.699 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:41.699 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:41.708 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:01:41.792 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:41.791+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:41.792 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-05-11T23:01:41.824 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:41.824 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:41.834 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:01:41.852 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:01:41.872 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone7 from 0x8f8f89b (overlap 0xab5da) to 0x0 2024-05-11T23:01:41.893 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:41.892+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:41.977 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-05-11T23:01:41.992 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:41.992 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:41.993 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:41.993+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:42.003 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:01:42.004 INFO:teuthology.orchestra.run.smithi005.stdout:1263 write 0x6f41da thru 0x6f7888 (0x36af bytes) 2024-05-11T23:01:42.010 INFO:teuthology.orchestra.run.smithi005.stdout:1264 write 0x71b40d9 thru 0x71c2f3f (0xee67 bytes) 2024-05-11T23:01:42.014 INFO:teuthology.orchestra.run.smithi005.stdout:1265 write 0x7f2c5f4 thru 0x7f37b29 (0xb536 bytes) 2024-05-11T23:01:42.016 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:01:42.017 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:01:42.018 INFO:teuthology.orchestra.run.smithi005.stdout:1266 punch from 0x57e3db7 to 0x57efd76, (0xbfbf bytes) 2024-05-11T23:01:42.025 INFO:teuthology.orchestra.run.smithi005.stdout:1267 trunc from 0x8f8f89b to 0x2a9dc88 2024-05-11T23:01:42.040 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:01:42.040 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:01:42.045 INFO:teuthology.orchestra.run.smithi005.stdout:1268 trunc from 0x2a9dc88 to 0x1096455 2024-05-11T23:01:42.064 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:01:42.064 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:01:42.065 INFO:teuthology.orchestra.run.smithi005.stdout:1269 punch from 0x6a27c5 to 0x6addb4, (0xb5ef bytes) 2024-05-11T23:01:42.065 INFO:teuthology.orchestra.run.smithi005.stdout:1270 read 0x12474 thru 0x21d3c (0xf8c9 bytes) 2024-05-11T23:01:42.083 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:01:42.083 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:01:42.084 INFO:teuthology.orchestra.run.smithi005.stdout:1271 write 0xc124122 thru 0xc129c1d (0x5afc bytes) 2024-05-11T23:01:42.088 INFO:teuthology.orchestra.run.smithi005.stdout:1272 punch from 0x9fea76d to 0x9ff63d2, (0xbc65 bytes) 2024-05-11T23:01:42.090 INFO:teuthology.orchestra.run.smithi005.stdout:1274 trunc from 0xc129c1e to 0xd65de3a 2024-05-11T23:01:42.094 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:42.094+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:42.096 INFO:teuthology.orchestra.run.smithi005.stdout:1275 write 0xb9a2e6f thru 0xb9ad1a2 (0xa334 bytes) 2024-05-11T23:01:42.104 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:01:42.104 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:01:42.105 INFO:teuthology.orchestra.run.smithi005.stdout:1276 write 0xe86bc6e thru 0xe8734d6 (0x7869 bytes) 2024-05-11T23:01:42.108 INFO:teuthology.orchestra.run.smithi005.stdout:1277 write 0x1d81997 thru 0x1d83a0c (0x2076 bytes) 2024-05-11T23:01:42.111 INFO:teuthology.orchestra.run.smithi005.stdout:1278 read 0xbfb9fcb thru 0xbfbe255 (0x428b bytes) 2024-05-11T23:01:42.111 INFO:teuthology.orchestra.run.smithi005.stdout:1279 write 0x2e307a4 thru 0x2e3af89 (0xa7e6 bytes) 2024-05-11T23:01:42.115 INFO:teuthology.orchestra.run.smithi005.stdout:1281 trunc from 0xe8734d7 to 0xd5f4a8b 2024-05-11T23:01:42.124 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:01:42.124 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:01:42.125 INFO:teuthology.orchestra.run.smithi005.stdout:1282 write 0x12b6e27 thru 0x12ba99b (0x3b75 bytes) 2024-05-11T23:01:42.127 INFO:teuthology.orchestra.run.smithi005.stdout:1286 write 0x442bd64 thru 0x443a892 (0xeb2f bytes) 2024-05-11T23:01:42.130 INFO:teuthology.orchestra.run.smithi005.stdout:1287 write 0x4ae95d5 thru 0x4af9142 (0xfb6e bytes) 2024-05-11T23:01:42.133 INFO:teuthology.orchestra.run.smithi005.stdout:1289 writesame 0x3b5c75b thru 0x3b6754d data_size 0x1(0xadf3 bytes) 2024-05-11T23:01:42.147 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:01:42.147 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:01:42.148 INFO:teuthology.orchestra.run.smithi005.stdout:1290 read 0x97be3d5 thru 0x97c391a (0x5546 bytes) 2024-05-11T23:01:42.148 INFO:teuthology.orchestra.run.smithi005.stdout:1291 punch from 0xa3cb022 to 0xa3d3e44, (0x8e22 bytes) 2024-05-11T23:01:42.148 INFO:teuthology.orchestra.run.smithi005.stdout:1292 read 0x42fb2ed thru 0x42fd85e (0x2572 bytes) 2024-05-11T23:01:42.148 INFO:teuthology.orchestra.run.smithi005.stdout:1294 read 0x5ac9077 thru 0x5ad5985 (0xc90f bytes) 2024-05-11T23:01:42.148 INFO:teuthology.orchestra.run.smithi005.stdout:1295 punch from 0x3bd7498 to 0x3bdb1d9, (0x3d41 bytes) 2024-05-11T23:01:42.148 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:42.148 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:42.148 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:01:42.149 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:01:42.149 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:42.157 INFO:teuthology.orchestra.run.smithi005.stdout:1296 trunc from 0xd5f4a8b to 0x4d50aa0 2024-05-11T23:01:42.164 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:01:42.167 INFO:teuthology.orchestra.run.smithi005.stdout:1297 punch from 0x357cd4c to 0x3589d30, (0xcfe4 bytes) 2024-05-11T23:01:42.167 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:42.169 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:01:42.169 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:01:42.172 INFO:teuthology.orchestra.run.smithi005.stdout:1299 trunc from 0x4d50aa0 to 0x9e8c747 2024-05-11T23:01:42.178 INFO:teuthology.orchestra.run.smithi005.stdout:1300 write 0x2510fb0 thru 0x2513155 (0x21a6 bytes) 2024-05-11T23:01:42.181 INFO:teuthology.orchestra.run.smithi005.stdout:1301 write 0x4e06f2c thru 0x4e0b212 (0x42e7 bytes) 2024-05-11T23:01:42.184 INFO:teuthology.orchestra.run.smithi005.stdout:1303 read 0x6b77c56 thru 0x6b820d8 (0xa483 bytes) 2024-05-11T23:01:42.184 INFO:teuthology.orchestra.run.smithi005.stdout:1304 read 0x7b8a514 thru 0x7b94c0d (0xa6fa bytes) 2024-05-11T23:01:42.186 INFO:teuthology.orchestra.run.smithi005.stdout:1306 trunc from 0x9e8c747 to 0x73e9364 2024-05-11T23:01:42.195 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:42.195+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:42.205 INFO:teuthology.orchestra.run.smithi005.stdout:1308 writesame 0xdac7c52 thru 0xdad48d0 data_size 0x1(0xcc7f bytes) 2024-05-11T23:01:42.206 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:42.209 INFO:teuthology.orchestra.run.smithi005.stdout:1310 trunc from 0xdad48d1 to 0xac30d91 2024-05-11T23:01:42.215 INFO:teuthology.orchestra.run.smithi005.stdout:1311 flatten 2024-05-11T23:01:42.229 INFO:teuthology.orchestra.run.smithi005.stdout:1312 read 0x263e83a thru 0x2647ca1 (0x9468 bytes) 2024-05-11T23:01:42.229 INFO:teuthology.orchestra.run.smithi005.stdout:1313 punch from 0x643e74 to 0x64d2e5, (0x9471 bytes) 2024-05-11T23:01:42.233 INFO:teuthology.orchestra.run.smithi005.stdout:1314 trunc from 0xac30d91 to 0xdabecdf 2024-05-11T23:01:42.239 INFO:teuthology.orchestra.run.smithi005.stdout:1315 write 0x1ab3e9f thru 0x1ac1a72 (0xdbd4 bytes) 2024-05-11T23:01:42.242 INFO:teuthology.orchestra.run.smithi005.stdout:1316 read 0x4123748 thru 0x412aa39 (0x72f2 bytes) 2024-05-11T23:01:42.252 INFO:teuthology.orchestra.run.smithi005.stdout:1317 trunc from 0xdabecdf to 0x58cc8d3 2024-05-11T23:01:42.257 INFO:teuthology.orchestra.run.smithi005.stdout:1318 punch from 0x500627c to 0x500c602, (0x6386 bytes) 2024-05-11T23:01:42.260 INFO:teuthology.orchestra.run.smithi005.stdout:1319 trunc from 0x58cc8d3 to 0x823871a 2024-05-11T23:01:42.266 INFO:teuthology.orchestra.run.smithi005.stdout:1320 punch from 0x2b635f9 to 0x2b731b2, (0xfbb9 bytes) 2024-05-11T23:01:42.271 INFO:teuthology.orchestra.run.smithi005.stdout:1321 write 0x8601f83 thru 0x8611795 (0xf813 bytes) 2024-05-11T23:01:42.281 INFO:teuthology.orchestra.run.smithi005.stdout:1322 write 0x9fee304 thru 0x9ffaae9 (0xc7e6 bytes) 2024-05-11T23:01:42.284 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:42.285 INFO:teuthology.orchestra.run.smithi005.stdout:1327 punch from 0x2776a0f to 0x277d8cc, (0x6ebd bytes) 2024-05-11T23:01:42.285 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:42.285 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:42.285 INFO:teuthology.orchestra.run.smithi005.stdout:1330 read 0x30ff887 thru 0x31038d0 (0x404a bytes) 2024-05-11T23:01:42.289 INFO:teuthology.orchestra.run.smithi005.stdout:1333 trunc from 0x9ffaaea to 0xd9e1e0f 2024-05-11T23:01:42.294 INFO:teuthology.orchestra.run.smithi005.stdout:1334 write 0x434d952 thru 0x4350298 (0x2947 bytes) 2024-05-11T23:01:42.295 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:42.295+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:42.297 INFO:teuthology.orchestra.run.smithi005.stdout:1335 write 0xc1f376d thru 0xc1f3e1e (0x6b2 bytes) 2024-05-11T23:01:42.298 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:42.298 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:01:42.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:01:42.298 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:01:42.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.298 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:1966 3.7 3:e5c0e1bc:::rbd_data.111e50384849.0000000000000051:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.298 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.578077+0000", 2024-05-11T23:01:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7196162820000005, 2024-05-11T23:01:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012474607, 2024-05-11T23:01:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1966 2024-05-11T23:01:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:42.299 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578077+0000", 2024-05-11T23:01:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578077+0000", 2024-05-11T23:01:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578078+0000", 2024-05-11T23:01:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5600000000000005e-07 2024-05-11T23:01:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:42.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578081+0000", 2024-05-11T23:01:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6169999999999998e-06 2024-05-11T23:01:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578091+0000", 2024-05-11T23:01:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7070000000000004e-06 2024-05-11T23:01:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.578097+0000", 2024-05-11T23:01:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2929999999999997e-06 2024-05-11T23:01:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:42.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.581103+0000", 2024-05-11T23:01:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0030059240000000001 2024-05-11T23:01:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.581179+0000", 2024-05-11T23:01:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6038000000000006e-05 2024-05-11T23:01:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 7", 2024-05-11T23:01:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.581434+0000", 2024-05-11T23:01:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00025452 2024-05-11T23:01:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:42.302 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.584458+0000", 2024-05-11T23:01:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0030242889999999999 2024-05-11T23:01:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.590448+0000", 2024-05-11T23:01:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0059902200000000001 2024-05-11T23:01:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.590509+0000", 2024-05-11T23:01:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0673e-05 2024-05-11T23:01:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:42.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.590552+0000", 2024-05-11T23:01:42.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2670000000000003e-05 2024-05-11T23:01:42.304 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.304 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.304 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.304 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3119 3.1e 3:78cd6643:::rbd_data.111e7a935520.0000000000000004:4 [read 245760~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.304 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749195+0000", 2024-05-11T23:01:42.304 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5484980210000003, 2024-05-11T23:01:42.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012256286999999999, 2024-05-11T23:01:42.304 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:42.304 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:42.304 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:42.304 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:42.304 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:42.305 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.305 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3119 2024-05-11T23:01:42.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.305 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:42.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:42.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749195+0000", 2024-05-11T23:01:42.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:42.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749195+0000", 2024-05-11T23:01:42.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:42.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749196+0000", 2024-05-11T23:01:42.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4099999999999999e-07 2024-05-11T23:01:42.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:42.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749198+0000", 2024-05-11T23:01:42.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1409999999999999e-06 2024-05-11T23:01:42.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:42.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749203+0000", 2024-05-11T23:01:42.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.366e-06 2024-05-11T23:01:42.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.307 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749207+0000", 2024-05-11T23:01:42.307 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9349999999999996e-06 2024-05-11T23:01:42.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.307 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:42.307 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761351+0000", 2024-05-11T23:01:42.307 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012143426000000001 2024-05-11T23:01:42.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.307 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:42.307 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761396+0000", 2024-05-11T23:01:42.307 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5361999999999997e-05 2024-05-11T23:01:42.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.307 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:42.307 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761452+0000", 2024-05-11T23:01:42.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5516000000000002e-05 2024-05-11T23:01:42.308 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.308 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.308 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.308 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3129 3.1e 3:7877a10d:::rbd_data.111e7a935520.0000000000000001:4 [read 311296~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.308 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749226+0000", 2024-05-11T23:01:42.308 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5484673040000008, 2024-05-11T23:01:42.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012371007, 2024-05-11T23:01:42.308 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:42.308 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:42.308 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:42.309 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:42.309 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:42.309 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.309 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3129 2024-05-11T23:01:42.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.309 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:42.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.309 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:42.309 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749226+0000", 2024-05-11T23:01:42.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.310 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:42.310 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749226+0000", 2024-05-11T23:01:42.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.310 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:42.310 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749227+0000", 2024-05-11T23:01:42.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8500000000000002e-07 2024-05-11T23:01:42.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:42.311 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749229+0000", 2024-05-11T23:01:42.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2970000000000002e-06 2024-05-11T23:01:42.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:42.311 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749235+0000", 2024-05-11T23:01:42.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5810000000000002e-06 2024-05-11T23:01:42.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.311 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749240+0000", 2024-05-11T23:01:42.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6829999999999999e-06 2024-05-11T23:01:42.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:42.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761485+0000", 2024-05-11T23:01:42.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012245056000000001 2024-05-11T23:01:42.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:42.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761535+0000", 2024-05-11T23:01:42.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9589000000000002e-05 2024-05-11T23:01:42.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:42.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761597+0000", 2024-05-11T23:01:42.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2316000000000004e-05 2024-05-11T23:01:42.312 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.313 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.313 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.313 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3133 3.1e 3:78cd6643:::rbd_data.111e7a935520.0000000000000004:4 [read 311296~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.313 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749291+0000", 2024-05-11T23:01:42.313 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5484027650000005, 2024-05-11T23:01:42.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012427446, 2024-05-11T23:01:42.313 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:42.313 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:42.313 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:42.313 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:42.313 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:42.313 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.313 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3133 2024-05-11T23:01:42.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.314 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:42.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:42.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749291+0000", 2024-05-11T23:01:42.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:42.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749291+0000", 2024-05-11T23:01:42.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:42.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749291+0000", 2024-05-11T23:01:42.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4600000000000005e-07 2024-05-11T23:01:42.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:42.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749293+0000", 2024-05-11T23:01:42.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2180000000000002e-06 2024-05-11T23:01:42.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:42.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749299+0000", 2024-05-11T23:01:42.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.93e-06 2024-05-11T23:01:42.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749305+0000", 2024-05-11T23:01:42.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4199999999999998e-06 2024-05-11T23:01:42.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:42.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761614+0000", 2024-05-11T23:01:42.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012309621999999999 2024-05-11T23:01:42.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:42.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761663+0000", 2024-05-11T23:01:42.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.884e-05 2024-05-11T23:01:42.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:42.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761718+0000", 2024-05-11T23:01:42.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4870000000000002e-05 2024-05-11T23:01:42.317 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.317 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.317 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.317 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3141 3.1e 3:7877a10d:::rbd_data.111e7a935520.0000000000000001:4 [read 376832~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.317 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749311+0000", 2024-05-11T23:01:42.317 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5483820739999992, 2024-05-11T23:01:42.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012575493, 2024-05-11T23:01:42.317 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:42.317 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:42.317 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:42.317 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:42.317 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:42.318 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.318 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3141 2024-05-11T23:01:42.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.318 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:42.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:42.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749311+0000", 2024-05-11T23:01:42.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:42.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749311+0000", 2024-05-11T23:01:42.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:42.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749312+0000", 2024-05-11T23:01:42.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:01:42.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:42.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749314+0000", 2024-05-11T23:01:42.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0549999999999998e-06 2024-05-11T23:01:42.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:42.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749321+0000", 2024-05-11T23:01:42.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5340000000000002e-06 2024-05-11T23:01:42.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.320 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749325+0000", 2024-05-11T23:01:42.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8990000000000003e-06 2024-05-11T23:01:42.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:42.320 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761757+0000", 2024-05-11T23:01:42.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012431413000000001 2024-05-11T23:01:42.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:42.320 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761828+0000", 2024-05-11T23:01:42.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1741000000000003e-05 2024-05-11T23:01:42.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:42.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761887+0000", 2024-05-11T23:01:42.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8468000000000003e-05 2024-05-11T23:01:42.321 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.321 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.321 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.321 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3144 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.321 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749351+0000", 2024-05-11T23:01:42.321 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5483425850000003, 2024-05-11T23:01:42.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012554991, 2024-05-11T23:01:42.321 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:42.321 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:42.321 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:42.321 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:42.322 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:42.322 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.322 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3144 2024-05-11T23:01:42.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.322 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:42.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:42.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749351+0000", 2024-05-11T23:01:42.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:42.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749351+0000", 2024-05-11T23:01:42.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:42.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749351+0000", 2024-05-11T23:01:42.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0299999999999999e-07 2024-05-11T23:01:42.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:42.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749353+0000", 2024-05-11T23:01:42.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9609999999999999e-06 2024-05-11T23:01:42.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:42.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749359+0000", 2024-05-11T23:01:42.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9359999999999996e-06 2024-05-11T23:01:42.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749364+0000", 2024-05-11T23:01:42.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4379999999999997e-06 2024-05-11T23:01:42.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:42.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761901+0000", 2024-05-11T23:01:42.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012537074 2024-05-11T23:01:42.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:42.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761906+0000", 2024-05-11T23:01:42.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.079e-06 2024-05-11T23:01:42.325 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.325 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.325 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.325 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3152 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.325 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749388+0000", 2024-05-11T23:01:42.325 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5483053059999996, 2024-05-11T23:01:42.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012551034000000001, 2024-05-11T23:01:42.325 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:42.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:42.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:42.326 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:42.326 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:42.326 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.326 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3152 2024-05-11T23:01:42.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.326 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:42.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:42.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749388+0000", 2024-05-11T23:01:42.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:42.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749388+0000", 2024-05-11T23:01:42.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:42.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749389+0000", 2024-05-11T23:01:42.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4300000000000002e-07 2024-05-11T23:01:42.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:42.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749393+0000", 2024-05-11T23:01:42.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8619999999999996e-06 2024-05-11T23:01:42.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:42.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749402+0000", 2024-05-11T23:01:42.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1050000000000001e-06 2024-05-11T23:01:42.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749430+0000", 2024-05-11T23:01:42.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8039e-05 2024-05-11T23:01:42.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:42.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761929+0000", 2024-05-11T23:01:42.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012498746999999999 2024-05-11T23:01:42.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:42.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761939+0000", 2024-05-11T23:01:42.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0538e-05 2024-05-11T23:01:42.329 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.329 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.329 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.329 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3155 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.329 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749503+0000", 2024-05-11T23:01:42.329 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5481901380000007, 2024-05-11T23:01:42.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012482586, 2024-05-11T23:01:42.329 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:42.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:42.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:42.329 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:42.330 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:42.330 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.330 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3155 2024-05-11T23:01:42.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.330 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:42.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:42.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749503+0000", 2024-05-11T23:01:42.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:42.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749503+0000", 2024-05-11T23:01:42.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:42.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749504+0000", 2024-05-11T23:01:42.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0299999999999999e-07 2024-05-11T23:01:42.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:42.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749511+0000", 2024-05-11T23:01:42.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4560000000000003e-06 2024-05-11T23:01:42.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:42.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749519+0000", 2024-05-11T23:01:42.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6009999999999997e-06 2024-05-11T23:01:42.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749525+0000", 2024-05-11T23:01:42.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8699999999999997e-06 2024-05-11T23:01:42.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:42.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761976+0000", 2024-05-11T23:01:42.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012451234 2024-05-11T23:01:42.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:42.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.761986+0000", 2024-05-11T23:01:42.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8220000000000002e-06 2024-05-11T23:01:42.332 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.333 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.333 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.333 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3164 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.333 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749534+0000", 2024-05-11T23:01:42.333 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5481599149999994, 2024-05-11T23:01:42.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012486199, 2024-05-11T23:01:42.333 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:42.333 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:42.333 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:42.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:42.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:42.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3164 2024-05-11T23:01:42.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.334 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:42.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:42.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749534+0000", 2024-05-11T23:01:42.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:42.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749534+0000", 2024-05-11T23:01:42.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:42.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749534+0000", 2024-05-11T23:01:42.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7700000000000004e-07 2024-05-11T23:01:42.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:42.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749536+0000", 2024-05-11T23:01:42.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0789999999999999e-06 2024-05-11T23:01:42.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:42.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749542+0000", 2024-05-11T23:01:42.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:01:42.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749548+0000", 2024-05-11T23:01:42.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0090000000000004e-06 2024-05-11T23:01:42.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:42.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762014+0000", 2024-05-11T23:01:42.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012466090000000001 2024-05-11T23:01:42.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:42.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762020+0000", 2024-05-11T23:01:42.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4099999999999999e-06 2024-05-11T23:01:42.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.336 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.337 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3167 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.337 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749577+0000", 2024-05-11T23:01:42.337 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5481161340000007, 2024-05-11T23:01:42.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012471350000000001, 2024-05-11T23:01:42.337 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:42.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:42.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:42.337 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:42.337 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:42.337 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.337 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3167 2024-05-11T23:01:42.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.337 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:42.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:42.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749577+0000", 2024-05-11T23:01:42.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:42.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749577+0000", 2024-05-11T23:01:42.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:42.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749578+0000", 2024-05-11T23:01:42.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2800000000000002e-07 2024-05-11T23:01:42.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749580+0000", 2024-05-11T23:01:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.058e-06 2024-05-11T23:01:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749587+0000", 2024-05-11T23:01:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4089999999999997e-06 2024-05-11T23:01:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749591+0000", 2024-05-11T23:01:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3050000000000003e-06 2024-05-11T23:01:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:42.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762040+0000", 2024-05-11T23:01:42.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012448489 2024-05-11T23:01:42.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:42.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762049+0000", 2024-05-11T23:01:42.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6610000000000006e-06 2024-05-11T23:01:42.340 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.340 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.340 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.340 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3176 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.340 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749637+0000", 2024-05-11T23:01:42.340 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5480564660000002, 2024-05-11T23:01:42.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012425937999999999, 2024-05-11T23:01:42.341 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:42.341 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:42.341 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:42.341 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:42.341 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:42.341 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.341 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3176 2024-05-11T23:01:42.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.341 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:42.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:42.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749637+0000", 2024-05-11T23:01:42.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:42.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749637+0000", 2024-05-11T23:01:42.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:42.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749637+0000", 2024-05-11T23:01:42.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0900000000000002e-07 2024-05-11T23:01:42.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:42.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749640+0000", 2024-05-11T23:01:42.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6070000000000002e-06 2024-05-11T23:01:42.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:42.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749649+0000", 2024-05-11T23:01:42.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5629999999999992e-06 2024-05-11T23:01:42.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749654+0000", 2024-05-11T23:01:42.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5570000000000003e-06 2024-05-11T23:01:42.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:42.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762059+0000", 2024-05-11T23:01:42.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012404567 2024-05-11T23:01:42.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:42.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762063+0000", 2024-05-11T23:01:42.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1350000000000002e-06 2024-05-11T23:01:42.344 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.344 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.344 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.344 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3179 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.344 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749695+0000", 2024-05-11T23:01:42.344 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5479986409999995, 2024-05-11T23:01:42.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012391855, 2024-05-11T23:01:42.344 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:42.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:42.344 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:42.344 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:42.344 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:42.345 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.345 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3179 2024-05-11T23:01:42.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.345 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:42.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:42.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749695+0000", 2024-05-11T23:01:42.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:42.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749695+0000", 2024-05-11T23:01:42.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:42.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749695+0000", 2024-05-11T23:01:42.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.63e-07 2024-05-11T23:01:42.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:42.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749698+0000", 2024-05-11T23:01:42.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.83e-06 2024-05-11T23:01:42.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:42.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749707+0000", 2024-05-11T23:01:42.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4810000000000006e-06 2024-05-11T23:01:42.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749711+0000", 2024-05-11T23:01:42.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6859999999999997e-06 2024-05-11T23:01:42.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:42.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762079+0000", 2024-05-11T23:01:42.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012367515000000001 2024-05-11T23:01:42.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:42.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762087+0000", 2024-05-11T23:01:42.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8800000000000008e-06 2024-05-11T23:01:42.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.348 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.348 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3188 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.348 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749747+0000", 2024-05-11T23:01:42.348 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5479465920000006, 2024-05-11T23:01:42.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012374299, 2024-05-11T23:01:42.348 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:42.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:42.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:42.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:42.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:42.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3188 2024-05-11T23:01:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749747+0000", 2024-05-11T23:01:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749747+0000", 2024-05-11T23:01:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:42.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749747+0000", 2024-05-11T23:01:42.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1199999999999998e-07 2024-05-11T23:01:42.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:42.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749750+0000", 2024-05-11T23:01:42.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2869999999999999e-06 2024-05-11T23:01:42.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:42.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749756+0000", 2024-05-11T23:01:42.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8199999999999999e-06 2024-05-11T23:01:42.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749763+0000", 2024-05-11T23:01:42.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3149999999999997e-06 2024-05-11T23:01:42.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:42.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762116+0000", 2024-05-11T23:01:42.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012353207999999999 2024-05-11T23:01:42.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:42.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762121+0000", 2024-05-11T23:01:42.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2569999999999998e-06 2024-05-11T23:01:42.351 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.352 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.352 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.352 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3191 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.352 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749794+0000", 2024-05-11T23:01:42.352 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5478989639999998, 2024-05-11T23:01:42.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012357628000000001, 2024-05-11T23:01:42.352 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:42.352 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:42.352 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:42.352 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:42.352 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:42.352 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.353 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3191 2024-05-11T23:01:42.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.353 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:42.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:42.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749794+0000", 2024-05-11T23:01:42.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:42.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749794+0000", 2024-05-11T23:01:42.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:42.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749795+0000", 2024-05-11T23:01:42.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:01:42.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:42.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749800+0000", 2024-05-11T23:01:42.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7129999999999996e-06 2024-05-11T23:01:42.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:42.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749807+0000", 2024-05-11T23:01:42.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3039999999999999e-06 2024-05-11T23:01:42.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749812+0000", 2024-05-11T23:01:42.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3569999999999997e-06 2024-05-11T23:01:42.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:42.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762147+0000", 2024-05-11T23:01:42.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012335084 2024-05-11T23:01:42.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:42.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762152+0000", 2024-05-11T23:01:42.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6909999999999996e-06 2024-05-11T23:01:42.356 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.356 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.356 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.356 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3201 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.356 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749845+0000", 2024-05-11T23:01:42.356 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5478482430000007, 2024-05-11T23:01:42.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012333903, 2024-05-11T23:01:42.356 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:42.356 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:42.356 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:42.356 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:42.356 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:42.356 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.357 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3201 2024-05-11T23:01:42.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.357 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:42.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:42.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749845+0000", 2024-05-11T23:01:42.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:42.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749845+0000", 2024-05-11T23:01:42.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:42.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749846+0000", 2024-05-11T23:01:42.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4999999999999998e-07 2024-05-11T23:01:42.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:42.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749852+0000", 2024-05-11T23:01:42.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5729999999999996e-06 2024-05-11T23:01:42.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:42.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749861+0000", 2024-05-11T23:01:42.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0080000000000008e-06 2024-05-11T23:01:42.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749866+0000", 2024-05-11T23:01:42.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6720000000000003e-06 2024-05-11T23:01:42.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:42.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762175+0000", 2024-05-11T23:01:42.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012308852 2024-05-11T23:01:42.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:42.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762179+0000", 2024-05-11T23:01:42.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3479999999999997e-06 2024-05-11T23:01:42.359 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.359 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.360 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.360 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3204 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.360 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749889+0000", 2024-05-11T23:01:42.360 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5478039930000005, 2024-05-11T23:01:42.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012314227000000001, 2024-05-11T23:01:42.360 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:42.360 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:42.360 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:42.360 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:42.360 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:42.360 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.360 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3204 2024-05-11T23:01:42.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.361 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:42.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:42.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749889+0000", 2024-05-11T23:01:42.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:42.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749889+0000", 2024-05-11T23:01:42.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:42.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749890+0000", 2024-05-11T23:01:42.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:01:42.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:42.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749892+0000", 2024-05-11T23:01:42.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9970000000000001e-06 2024-05-11T23:01:42.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:42.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749897+0000", 2024-05-11T23:01:42.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5500000000000002e-06 2024-05-11T23:01:42.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749902+0000", 2024-05-11T23:01:42.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3409999999999996e-06 2024-05-11T23:01:42.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:42.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762199+0000", 2024-05-11T23:01:42.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012297577000000001 2024-05-11T23:01:42.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:42.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762204+0000", 2024-05-11T23:01:42.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5310000000000003e-06 2024-05-11T23:01:42.363 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.363 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.363 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.363 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3209 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.363 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749907+0000", 2024-05-11T23:01:42.363 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5477860739999993, 2024-05-11T23:01:42.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012319788999999999, 2024-05-11T23:01:42.364 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:42.364 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:42.364 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:42.364 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:42.364 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:42.364 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.364 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3209 2024-05-11T23:01:42.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.364 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:42.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:42.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749907+0000", 2024-05-11T23:01:42.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749907+0000", 2024-05-11T23:01:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749908+0000", 2024-05-11T23:01:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4699999999999998e-07 2024-05-11T23:01:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749916+0000", 2024-05-11T23:01:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8550000000000008e-06 2024-05-11T23:01:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:42.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749923+0000", 2024-05-11T23:01:42.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7850000000000003e-06 2024-05-11T23:01:42.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749928+0000", 2024-05-11T23:01:42.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8609999999999997e-06 2024-05-11T23:01:42.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:42.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762223+0000", 2024-05-11T23:01:42.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012294589999999999 2024-05-11T23:01:42.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:42.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762227+0000", 2024-05-11T23:01:42.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4510000000000002e-06 2024-05-11T23:01:42.367 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.367 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.367 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.367 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3212 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.367 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749956+0000", 2024-05-11T23:01:42.367 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5477376879999998, 2024-05-11T23:01:42.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012295808, 2024-05-11T23:01:42.367 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:42.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:42.368 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:42.368 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:42.368 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:42.368 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.368 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3212 2024-05-11T23:01:42.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.368 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:42.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:42.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749956+0000", 2024-05-11T23:01:42.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:42.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749956+0000", 2024-05-11T23:01:42.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:42.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749956+0000", 2024-05-11T23:01:42.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:01:42.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:42.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749959+0000", 2024-05-11T23:01:42.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7879999999999998e-06 2024-05-11T23:01:42.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:42.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749965+0000", 2024-05-11T23:01:42.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2269999999999998e-06 2024-05-11T23:01:42.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749970+0000", 2024-05-11T23:01:42.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0849999999999996e-06 2024-05-11T23:01:42.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:42.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762247+0000", 2024-05-11T23:01:42.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012276989 2024-05-11T23:01:42.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:42.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762252+0000", 2024-05-11T23:01:42.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3669999999999999e-06 2024-05-11T23:01:42.371 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.371 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.371 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.371 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3221 3.1e 3.b085ee1e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.371 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.749998+0000", 2024-05-11T23:01:42.371 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5476951190000001, 2024-05-11T23:01:42.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012277692999999999, 2024-05-11T23:01:42.371 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:42.371 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:42.371 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:42.371 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:42.372 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:42.372 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.372 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3221 2024-05-11T23:01:42.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.372 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:42.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:42.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749998+0000", 2024-05-11T23:01:42.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:42.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749998+0000", 2024-05-11T23:01:42.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:42.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.749999+0000", 2024-05-11T23:01:42.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3300000000000002e-07 2024-05-11T23:01:42.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:42.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750001+0000", 2024-05-11T23:01:42.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.007e-06 2024-05-11T23:01:42.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:42.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750007+0000", 2024-05-11T23:01:42.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2160000000000003e-06 2024-05-11T23:01:42.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750012+0000", 2024-05-11T23:01:42.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0200000000000002e-06 2024-05-11T23:01:42.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:42.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762271+0000", 2024-05-11T23:01:42.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012258626999999999 2024-05-11T23:01:42.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:42.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762276+0000", 2024-05-11T23:01:42.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2900000000000002e-06 2024-05-11T23:01:42.375 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.375 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.375 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.375 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3225 3.1e 3.c266b31e (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.375 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.750040+0000", 2024-05-11T23:01:42.375 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.5476533079999992, 2024-05-11T23:01:42.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012260699999999999, 2024-05-11T23:01:42.375 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:42.375 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:42.375 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:42.375 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:42.375 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:42.376 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.376 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3225 2024-05-11T23:01:42.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.376 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:42.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:42.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750040+0000", 2024-05-11T23:01:42.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:42.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750040+0000", 2024-05-11T23:01:42.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:42.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:42.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750040+0000", 2024-05-11T23:01:42.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-05-11T23:01:42.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:42.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750043+0000", 2024-05-11T23:01:42.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.672e-06 2024-05-11T23:01:42.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:42.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750049+0000", 2024-05-11T23:01:42.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3230000000000003e-06 2024-05-11T23:01:42.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.750055+0000", 2024-05-11T23:01:42.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.609e-06 2024-05-11T23:01:42.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:42.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762297+0000", 2024-05-11T23:01:42.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012241432 2024-05-11T23:01:42.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:42.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:42.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:42.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.762301+0000", 2024-05-11T23:01:42.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2939999999999999e-06 2024-05-11T23:01:42.380 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.380 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.380 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.380 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:42.380 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:42.380 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:42.381 INFO:teuthology.orchestra.run.smithi005.stdout:1336 read 0x81be60f thru 0x81c9671 (0xb063 bytes) 2024-05-11T23:01:42.381 INFO:teuthology.orchestra.run.smithi005.stdout:1337 trunc from 0xd9e1e0f to 0x8297e66 2024-05-11T23:01:42.382 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:01:42.385 INFO:teuthology.orchestra.run.smithi005.stdout:1338 write 0x7cca202 thru 0x7cd4f1d (0xad1c bytes) 2024-05-11T23:01:42.386 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:42.386 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:42.386 INFO:teuthology.orchestra.run.smithi005.stdout:1342 punch from 0x280afbc to 0x280ec82, (0x3cc6 bytes) 2024-05-11T23:01:42.386 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:42.386 INFO:teuthology.orchestra.run.smithi005.stdout:1345 trunc from 0x8297e66 to 0x7b01d76 2024-05-11T23:01:42.386 INFO:teuthology.orchestra.run.smithi005.stdout:1346 write 0xd662248 thru 0xd665386 (0x313f bytes) 2024-05-11T23:01:42.386 INFO:teuthology.orchestra.run.smithi005.stdout:1347 read 0x9a859b7 thru 0x9a8aef3 (0x553d bytes) 2024-05-11T23:01:42.386 INFO:teuthology.orchestra.run.smithi005.stdout:1348 write 0x7a92319 thru 0x7a99ba0 (0x7888 bytes) 2024-05-11T23:01:42.386 INFO:teuthology.orchestra.run.smithi005.stdout:1350 read 0x558c9fb thru 0x5596e57 (0xa45d bytes) 2024-05-11T23:01:42.386 INFO:teuthology.orchestra.run.smithi005.stdout:1352 read 0xcdf76f1 thru 0xcdf8671 (0xf81 bytes) 2024-05-11T23:01:42.386 INFO:teuthology.orchestra.run.smithi005.stdout:1353 read 0x2a30167 thru 0x2a3a347 (0xa1e1 bytes) 2024-05-11T23:01:42.386 INFO:teuthology.orchestra.run.smithi005.stdout:1354 read 0xaa8ddb4 thru 0xaa931fb (0x5448 bytes) 2024-05-11T23:01:42.387 INFO:teuthology.orchestra.run.smithi005.stdout:1355 write 0x62508db thru 0x6259479 (0x8b9f bytes) 2024-05-11T23:01:42.387 INFO:teuthology.orchestra.run.smithi005.stdout:1356 read 0x96287c0 thru 0x9637b61 (0xf3a2 bytes) 2024-05-11T23:01:42.387 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:42.387 INFO:teuthology.orchestra.run.smithi005.stdout:1359 write 0x240d71a thru 0x240db67 (0x44e bytes) 2024-05-11T23:01:42.387 INFO:teuthology.orchestra.run.smithi005.stdout:1362 write 0x85d0ca7 thru 0x85d3ac0 (0x2e1a bytes) 2024-05-11T23:01:42.387 INFO:teuthology.orchestra.run.smithi005.stdout:1363 read 0x6ee10f1 thru 0x6ee8bc3 (0x7ad3 bytes) 2024-05-11T23:01:42.387 INFO:teuthology.orchestra.run.smithi005.stdout:1364 writesame 0x365d685 thru 0x365fd1a data_size 0xb(0x2696 bytes) 2024-05-11T23:01:42.387 INFO:teuthology.orchestra.run.smithi005.stdout:1365 read 0x76106e9 thru 0x7616452 (0x5d6a bytes) 2024-05-11T23:01:42.387 INFO:teuthology.orchestra.run.smithi005.stdout:1367 read 0xa512f32 thru 0xa51ab53 (0x7c22 bytes) 2024-05-11T23:01:42.387 INFO:teuthology.orchestra.run.smithi005.stdout:1368 write 0x2f1c50e thru 0x2f2b50d (0xf000 bytes) 2024-05-11T23:01:42.387 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:42.387 INFO:teuthology.orchestra.run.smithi005.stdout:1371 write 0x23c55b5 thru 0x23cb8bd (0x6309 bytes) 2024-05-11T23:01:42.387 INFO:teuthology.orchestra.run.smithi005.stdout:1374 write 0xb4126ad thru 0xb420655 (0xdfa9 bytes) 2024-05-11T23:01:42.387 INFO:teuthology.orchestra.run.smithi005.stdout:1375 writesame 0xb6b2e8e thru 0xb6b4e2a data_size 0x1(0x1f9d bytes) 2024-05-11T23:01:42.387 INFO:teuthology.orchestra.run.smithi005.stdout:1377 write 0xa1951f7 thru 0xa1a2c03 (0xda0d bytes) 2024-05-11T23:01:42.387 INFO:teuthology.orchestra.run.smithi005.stdout:1378 punch from 0x8e59e03 to 0x8e5aab8, (0xcb5 bytes) 2024-05-11T23:01:42.387 INFO:teuthology.orchestra.run.smithi005.stdout:1380 read 0xce5eaa7 thru 0xce63a07 (0x4f61 bytes) 2024-05-11T23:01:42.387 INFO:teuthology.orchestra.run.smithi005.stdout:1381 write 0x6b8b176 thru 0x6b8ff0d (0x4d98 bytes) 2024-05-11T23:01:42.388 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:42.388 INFO:teuthology.orchestra.run.smithi005.stdout:1383 read 0x3dd3dbd thru 0x3dd8946 (0x4b8a bytes) 2024-05-11T23:01:42.388 INFO:teuthology.orchestra.run.smithi005.stdout:1384 writesame 0x7567b92 thru 0x756ebf8 data_size 0x5(0x7067 bytes) 2024-05-11T23:01:42.388 INFO:teuthology.orchestra.run.smithi005.stdout:1385 read 0x573f8a thru 0x5802c5 (0xc33c bytes) 2024-05-11T23:01:42.388 INFO:teuthology.orchestra.run.smithi005.stdout:1386 write 0x2b51a94 thru 0x2b51af9 (0x66 bytes) 2024-05-11T23:01:42.388 INFO:teuthology.orchestra.run.smithi005.stdout:1387 writesame 0x980d447 thru 0x98166b7 data_size 0x1(0x9271 bytes) 2024-05-11T23:01:42.400 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:42.401 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:42.401 INFO:teuthology.orchestra.run.smithi005.stdout:1393 clone 9 order 23 su 65536 sc 12 2024-05-11T23:01:42.431 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:42.431+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:42.492 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:42.492 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:42.493 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:01:42.493 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:42.497 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:42.497+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:42.501 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:01:42.598 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:42.598+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:42.646 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:42.646 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:42.646 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:01:42.647 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:01:42.647 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:42.655 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:01:42.699 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:42.699+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:42.800 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:42.800+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:42.804 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:42.804 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:01:42.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:01:42.804 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:01:42.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.804 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-11T23:01:42.804 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:29.463901+0000", 2024-05-11T23:01:42.805 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.339349158999999, 2024-05-11T23:01:42.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0077045129999999996, 2024-05-11T23:01:42.805 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:42.805 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:42.805 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:42.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:42.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:42.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.805 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9 2024-05-11T23:01:42.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.805 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:42.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:42.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463901+0000", 2024-05-11T23:01:42.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:42.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463901+0000", 2024-05-11T23:01:42.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:42.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463902+0000", 2024-05-11T23:01:42.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.367e-06 2024-05-11T23:01:42.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:42.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463908+0000", 2024-05-11T23:01:42.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0789999999999997e-06 2024-05-11T23:01:42.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:42.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463921+0000", 2024-05-11T23:01:42.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2734e-05 2024-05-11T23:01:42.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463928+0000", 2024-05-11T23:01:42.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2429999999999999e-06 2024-05-11T23:01:42.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463973+0000", 2024-05-11T23:01:42.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4116999999999998e-05 2024-05-11T23:01:42.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op must wait for map", 2024-05-11T23:01:42.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463977+0000", 2024-05-11T23:01:42.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3030000000000001e-06 2024-05-11T23:01:42.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.465607+0000", 2024-05-11T23:01:42.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016304119999999999 2024-05-11T23:01:42.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:42.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.465688+0000", 2024-05-11T23:01:42.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0957000000000004e-05 2024-05-11T23:01:42.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-05-11T23:01:42.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.465822+0000", 2024-05-11T23:01:42.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013396099999999999 2024-05-11T23:01:42.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:42.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.466978+0000", 2024-05-11T23:01:42.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001155664 2024-05-11T23:01:42.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:42.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.471543+0000", 2024-05-11T23:01:42.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045646870000000004 2024-05-11T23:01:42.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:42.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.471580+0000", 2024-05-11T23:01:42.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7398999999999999e-05 2024-05-11T23:01:42.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:42.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.471606+0000", 2024-05-11T23:01:42.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5590000000000001e-05 2024-05-11T23:01:42.811 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.811 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.811 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.811 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:2223 3.1a 3:5bdb11a3:::rbd_data.111e50384849.000000000000003e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 180224] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.811 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.608039+0000", 2024-05-11T23:01:42.811 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.1952115120000002, 2024-05-11T23:01:42.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0078733559999999998, 2024-05-11T23:01:42.811 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:42.811 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:42.811 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:42.812 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:42.812 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:42.812 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.812 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2223 2024-05-11T23:01:42.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.812 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:42.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:42.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608039+0000", 2024-05-11T23:01:42.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:42.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608039+0000", 2024-05-11T23:01:42.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:42.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608040+0000", 2024-05-11T23:01:42.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3900000000000004e-07 2024-05-11T23:01:42.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:42.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608044+0000", 2024-05-11T23:01:42.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0380000000000001e-06 2024-05-11T23:01:42.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:42.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608053+0000", 2024-05-11T23:01:42.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3449999999999995e-06 2024-05-11T23:01:42.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608058+0000", 2024-05-11T23:01:42.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2809999999999999e-06 2024-05-11T23:01:42.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608106+0000", 2024-05-11T23:01:42.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7552000000000001e-05 2024-05-11T23:01:42.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:42.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608191+0000", 2024-05-11T23:01:42.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4981999999999994e-05 2024-05-11T23:01:42.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-05-11T23:01:42.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608307+0000", 2024-05-11T23:01:42.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011595 2024-05-11T23:01:42.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:42.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.609447+0000", 2024-05-11T23:01:42.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011405600000000001 2024-05-11T23:01:42.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:42.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.615854+0000", 2024-05-11T23:01:42.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0064062959999999997 2024-05-11T23:01:42.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:42.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.615887+0000", 2024-05-11T23:01:42.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3456e-05 2024-05-11T23:01:42.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:42.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.615912+0000", 2024-05-11T23:01:42.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5057000000000001e-05 2024-05-11T23:01:42.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.816 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.817 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3651 3.1a 3:5d63366f:::rbd_data.111e89cdde7d.000000000000000c:head [set-alloc-hint object_size 1048576 write_size 1048576,write 785032~6745] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.817 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.063156+0000", 2024-05-11T23:01:42.817 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.7400943790000003, 2024-05-11T23:01:42.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060821240000000004, 2024-05-11T23:01:42.817 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:42.817 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:42.817 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:42.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:42.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:42.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.817 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3651 2024-05-11T23:01:42.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.818 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:42.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:42.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063156+0000", 2024-05-11T23:01:42.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:42.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063156+0000", 2024-05-11T23:01:42.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:42.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063157+0000", 2024-05-11T23:01:42.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.094e-06 2024-05-11T23:01:42.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:42.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063166+0000", 2024-05-11T23:01:42.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5189999999999998e-06 2024-05-11T23:01:42.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:42.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063181+0000", 2024-05-11T23:01:42.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4691999999999999e-05 2024-05-11T23:01:42.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063189+0000", 2024-05-11T23:01:42.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3810000000000001e-06 2024-05-11T23:01:42.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063244+0000", 2024-05-11T23:01:42.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5081999999999999e-05 2024-05-11T23:01:42.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:42.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063365+0000", 2024-05-11T23:01:42.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012095300000000001 2024-05-11T23:01:42.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-05-11T23:01:42.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063496+0000", 2024-05-11T23:01:42.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013151100000000001 2024-05-11T23:01:42.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:42.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.064622+0000", 2024-05-11T23:01:42.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00112612 2024-05-11T23:01:42.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:42.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.069157+0000", 2024-05-11T23:01:42.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045349739999999998 2024-05-11T23:01:42.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:42.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.069202+0000", 2024-05-11T23:01:42.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5238999999999998e-05 2024-05-11T23:01:42.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:42.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.069238+0000", 2024-05-11T23:01:42.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5559000000000003e-05 2024-05-11T23:01:42.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.822 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.822 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4208 3.e e25/19 3:774edc63:::rbd_object_map.111e89cdde7d:head v 25'259, mlcod=25'255)", 2024-05-11T23:01:42.822 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.552088+0000", 2024-05-11T23:01:42.823 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.2511618939999991, 2024-05-11T23:01:42.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071901839999999996, 2024-05-11T23:01:42.823 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:42.823 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:42.823 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:42.823 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:42.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:42.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552088+0000", 2024-05-11T23:01:42.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:42.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552088+0000", 2024-05-11T23:01:42.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:42.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552089+0000", 2024-05-11T23:01:42.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4900000000000001e-07 2024-05-11T23:01:42.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:42.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552093+0000", 2024-05-11T23:01:42.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7190000000000001e-06 2024-05-11T23:01:42.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:42.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552102+0000", 2024-05-11T23:01:42.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6319999999999997e-06 2024-05-11T23:01:42.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552107+0000", 2024-05-11T23:01:42.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3980000000000001e-06 2024-05-11T23:01:42.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552147+0000", 2024-05-11T23:01:42.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0423999999999998e-05 2024-05-11T23:01:42.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:42.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552160+0000", 2024-05-11T23:01:42.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3246e-05 2024-05-11T23:01:42.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:42.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559245+0000", 2024-05-11T23:01:42.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0070852620000000002 2024-05-11T23:01:42.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:42.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559278+0000", 2024-05-11T23:01:42.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3053999999999999e-05 2024-05-11T23:01:42.826 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.826 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.826 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.827 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4209 3.9 3:92dfc630:::rbd_data.111e89cdde7d.00000000000000ed:head [set-alloc-hint object_size 1048576 write_size 1048576,write 9891~6493] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.827 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.552603+0000", 2024-05-11T23:01:42.827 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.2506473620000005, 2024-05-11T23:01:42.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0067329449999999997, 2024-05-11T23:01:42.827 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:42.827 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:42.827 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:42.827 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:42.827 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:42.827 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.827 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4209 2024-05-11T23:01:42.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.827 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:42.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:42.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552603+0000", 2024-05-11T23:01:42.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:42.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552603+0000", 2024-05-11T23:01:42.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:42.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552603+0000", 2024-05-11T23:01:42.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2799999999999996e-07 2024-05-11T23:01:42.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:42.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552610+0000", 2024-05-11T23:01:42.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.472e-06 2024-05-11T23:01:42.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:42.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552618+0000", 2024-05-11T23:01:42.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3299999999999999e-06 2024-05-11T23:01:42.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552623+0000", 2024-05-11T23:01:42.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8010000000000003e-06 2024-05-11T23:01:42.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552670+0000", 2024-05-11T23:01:42.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6686e-05 2024-05-11T23:01:42.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:42.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552782+0000", 2024-05-11T23:01:42.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112389 2024-05-11T23:01:42.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:01:42.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552923+0000", 2024-05-11T23:01:42.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000141225 2024-05-11T23:01:42.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:42.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554063+0000", 2024-05-11T23:01:42.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011397130000000001 2024-05-11T23:01:42.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:42.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559270+0000", 2024-05-11T23:01:42.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052066589999999998 2024-05-11T23:01:42.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:42.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559306+0000", 2024-05-11T23:01:42.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6133000000000001e-05 2024-05-11T23:01:42.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:42.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559336+0000", 2024-05-11T23:01:42.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9909000000000001e-05 2024-05-11T23:01:42.832 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.832 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.832 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.832 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4210 3.3 3:c0d177bc:::rbd_data.111e89cdde7d.00000000000000ee:head [set-alloc-hint object_size 1048576 write_size 1048576,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.832 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.554387+0000", 2024-05-11T23:01:42.832 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.2488635339999998, 2024-05-11T23:01:42.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080170269999999995, 2024-05-11T23:01:42.832 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:42.832 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:42.833 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:42.833 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:42.833 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:42.833 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.833 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4210 2024-05-11T23:01:42.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.833 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:42.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:42.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554387+0000", 2024-05-11T23:01:42.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:42.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554387+0000", 2024-05-11T23:01:42.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:42.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554387+0000", 2024-05-11T23:01:42.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6199999999999997e-07 2024-05-11T23:01:42.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:42.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554395+0000", 2024-05-11T23:01:42.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8720000000000002e-06 2024-05-11T23:01:42.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:42.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554405+0000", 2024-05-11T23:01:42.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5889999999999992e-06 2024-05-11T23:01:42.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554410+0000", 2024-05-11T23:01:42.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0980000000000001e-06 2024-05-11T23:01:42.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554458+0000", 2024-05-11T23:01:42.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8189999999999998e-05 2024-05-11T23:01:42.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:42.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554533+0000", 2024-05-11T23:01:42.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4900000000000005e-05 2024-05-11T23:01:42.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:42.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554615+0000", 2024-05-11T23:01:42.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2363999999999999e-05 2024-05-11T23:01:42.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:42.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.555352+0000", 2024-05-11T23:01:42.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00073687199999999996 2024-05-11T23:01:42.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:42.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.562311+0000", 2024-05-11T23:01:42.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0069588610000000002 2024-05-11T23:01:42.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:42.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.562363+0000", 2024-05-11T23:01:42.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2058999999999997e-05 2024-05-11T23:01:42.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:42.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.562404+0000", 2024-05-11T23:01:42.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0460000000000002e-05 2024-05-11T23:01:42.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.837 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.838 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4245 3.3 3:c52a1aee:::rbd_data.111e89cdde7d.0000000000000080:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.838 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598236+0000", 2024-05-11T23:01:42.838 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.2050146260000005, 2024-05-11T23:01:42.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0075491519999999999, 2024-05-11T23:01:42.838 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:42.838 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:42.838 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:42.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:42.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:42.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.838 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4245 2024-05-11T23:01:42.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.838 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:42.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:42.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598236+0000", 2024-05-11T23:01:42.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:42.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598236+0000", 2024-05-11T23:01:42.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:42.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598236+0000", 2024-05-11T23:01:42.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9899999999999999e-07 2024-05-11T23:01:42.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:42.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598240+0000", 2024-05-11T23:01:42.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5200000000000002e-06 2024-05-11T23:01:42.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:42.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598250+0000", 2024-05-11T23:01:42.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.764e-06 2024-05-11T23:01:42.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598257+0000", 2024-05-11T23:01:42.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2640000000000002e-06 2024-05-11T23:01:42.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598309+0000", 2024-05-11T23:01:42.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2410999999999997e-05 2024-05-11T23:01:42.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:42.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598388+0000", 2024-05-11T23:01:42.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9120000000000001e-05 2024-05-11T23:01:42.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:42.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598490+0000", 2024-05-11T23:01:42.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101285 2024-05-11T23:01:42.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:42.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.599363+0000", 2024-05-11T23:01:42.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00087332100000000004 2024-05-11T23:01:42.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:42.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605664+0000", 2024-05-11T23:01:42.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063005889999999997 2024-05-11T23:01:42.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:42.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605744+0000", 2024-05-11T23:01:42.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0507999999999996e-05 2024-05-11T23:01:42.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:42.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605785+0000", 2024-05-11T23:01:42.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0571000000000002e-05 2024-05-11T23:01:42.843 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.843 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.843 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.843 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4246 3.3 3:c5bb5d23:::rbd_data.111e89cdde7d.0000000000000081:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:42.843 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598285+0000", 2024-05-11T23:01:42.843 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.2049653560000007, 2024-05-11T23:01:42.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0075911599999999996, 2024-05-11T23:01:42.843 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:42.843 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:42.843 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:42.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:42.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:42.844 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.844 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4246 2024-05-11T23:01:42.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.844 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:42.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:42.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598285+0000", 2024-05-11T23:01:42.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:42.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598285+0000", 2024-05-11T23:01:42.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:42.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598285+0000", 2024-05-11T23:01:42.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4700000000000001e-07 2024-05-11T23:01:42.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:42.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598288+0000", 2024-05-11T23:01:42.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0120000000000001e-06 2024-05-11T23:01:42.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:42.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598296+0000", 2024-05-11T23:01:42.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7000000000000008e-06 2024-05-11T23:01:42.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598302+0000", 2024-05-11T23:01:42.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4790000000000004e-06 2024-05-11T23:01:42.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598716+0000", 2024-05-11T23:01:42.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00041448 2024-05-11T23:01:42.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:42.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598784+0000", 2024-05-11T23:01:42.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7689999999999997e-05 2024-05-11T23:01:42.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:42.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598881+0000", 2024-05-11T23:01:42.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7163000000000005e-05 2024-05-11T23:01:42.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:42.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.599467+0000", 2024-05-11T23:01:42.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00058593100000000004 2024-05-11T23:01:42.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:42.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605792+0000", 2024-05-11T23:01:42.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063255739999999996 2024-05-11T23:01:42.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:42.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605832+0000", 2024-05-11T23:01:42.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9301000000000002e-05 2024-05-11T23:01:42.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:42.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605876+0000", 2024-05-11T23:01:42.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4283000000000003e-05 2024-05-11T23:01:42.848 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.848 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.848 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.848 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4247 3.1 e25/19 3:80c18ebf:::rbd_data.111e89cdde7d.0000000000000082:head v 25'53, mlcod=25'51)", 2024-05-11T23:01:42.848 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598556+0000", 2024-05-11T23:01:42.848 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.204693893, 2024-05-11T23:01:42.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072492870000000001, 2024-05-11T23:01:42.848 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:42.849 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:42.849 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:42.849 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:42.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:42.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598556+0000", 2024-05-11T23:01:42.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:42.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598556+0000", 2024-05-11T23:01:42.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:42.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598557+0000", 2024-05-11T23:01:42.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-07 2024-05-11T23:01:42.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:42.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598562+0000", 2024-05-11T23:01:42.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6999999999999999e-06 2024-05-11T23:01:42.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:42.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598571+0000", 2024-05-11T23:01:42.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5629999999999998e-06 2024-05-11T23:01:42.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598585+0000", 2024-05-11T23:01:42.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4066000000000001e-05 2024-05-11T23:01:42.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598632+0000", 2024-05-11T23:01:42.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6471000000000003e-05 2024-05-11T23:01:42.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:42.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598650+0000", 2024-05-11T23:01:42.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8593999999999998e-05 2024-05-11T23:01:42.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:42.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605754+0000", 2024-05-11T23:01:42.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071034740000000002 2024-05-11T23:01:42.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:42.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605806+0000", 2024-05-11T23:01:42.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1820999999999997e-05 2024-05-11T23:01:42.852 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.852 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.852 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.852 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4248 3.f e25/19 3:f1322f7b:::rbd_data.111e89cdde7d.0000000000000083:head v 25'15, mlcod=25'13)", 2024-05-11T23:01:42.852 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598632+0000", 2024-05-11T23:01:42.852 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.2046179800000001, 2024-05-11T23:01:42.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071071980000000003, 2024-05-11T23:01:42.852 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:42.853 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:42.853 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:42.853 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:42.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:42.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598632+0000", 2024-05-11T23:01:42.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:42.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598632+0000", 2024-05-11T23:01:42.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:42.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598633+0000", 2024-05-11T23:01:42.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-05-11T23:01:42.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:42.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598638+0000", 2024-05-11T23:01:42.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5040000000000002e-06 2024-05-11T23:01:42.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:42.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598657+0000", 2024-05-11T23:01:42.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8195000000000001e-05 2024-05-11T23:01:42.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598664+0000", 2024-05-11T23:01:42.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0729999999999999e-06 2024-05-11T23:01:42.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598714+0000", 2024-05-11T23:01:42.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0766000000000001e-05 2024-05-11T23:01:42.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:42.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598734+0000", 2024-05-11T23:01:42.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9544000000000001e-05 2024-05-11T23:01:42.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605676+0000", 2024-05-11T23:01:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0069418090000000002 2024-05-11T23:01:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605739+0000", 2024-05-11T23:01:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3590999999999995e-05 2024-05-11T23:01:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4250 3.1d e25/19 3:be073325:::rbd_data.111e89cdde7d.0000000000000085:head v 25'18, mlcod=25'16)", 2024-05-11T23:01:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598711+0000", 2024-05-11T23:01:42.856 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.2045393739999994, 2024-05-11T23:01:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0070702610000000004, 2024-05-11T23:01:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598711+0000", 2024-05-11T23:01:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598711+0000", 2024-05-11T23:01:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598711+0000", 2024-05-11T23:01:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.68e-07 2024-05-11T23:01:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598717+0000", 2024-05-11T23:01:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4009999999999996e-06 2024-05-11T23:01:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598726+0000", 2024-05-11T23:01:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1400000000000006e-06 2024-05-11T23:01:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598732+0000", 2024-05-11T23:01:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4479999999999999e-06 2024-05-11T23:01:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598777+0000", 2024-05-11T23:01:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.473e-05 2024-05-11T23:01:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598799+0000", 2024-05-11T23:01:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1651999999999999e-05 2024-05-11T23:01:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605692+0000", 2024-05-11T23:01:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0068933680000000004 2024-05-11T23:01:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605781+0000", 2024-05-11T23:01:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8954000000000002e-05 2024-05-11T23:01:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4249 3.1c e25/19 3:3e719808:::rbd_data.111e89cdde7d.0000000000000084:head v 25'28, mlcod=25'26)", 2024-05-11T23:01:42.860 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598858+0000", 2024-05-11T23:01:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.2043921379999993, 2024-05-11T23:01:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.007008294, 2024-05-11T23:01:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598858+0000", 2024-05-11T23:01:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598858+0000", 2024-05-11T23:01:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598859+0000", 2024-05-11T23:01:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5200000000000002e-07 2024-05-11T23:01:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598862+0000", 2024-05-11T23:01:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5489999999999998e-06 2024-05-11T23:01:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598868+0000", 2024-05-11T23:01:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2770000000000002e-06 2024-05-11T23:01:42.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598872+0000", 2024-05-11T23:01:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9650000000000002e-06 2024-05-11T23:01:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598912+0000", 2024-05-11T23:01:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0064999999999997e-05 2024-05-11T23:01:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598930+0000", 2024-05-11T23:01:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8014999999999999e-05 2024-05-11T23:01:42.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605703+0000", 2024-05-11T23:01:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0067726169999999999 2024-05-11T23:01:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605866+0000", 2024-05-11T23:01:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016335399999999999 2024-05-11T23:01:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:6502 3.14 3:284c4678:::rbd_data.111e65d1e80f.0000000000000016:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-11T23:01:42.864 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:37.616415+0000", 2024-05-11T23:01:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.1868348559999999, 2024-05-11T23:01:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022765519000000001, 2024-05-11T23:01:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 6502 2024-05-11T23:01:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616415+0000", 2024-05-11T23:01:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616415+0000", 2024-05-11T23:01:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616417+0000", 2024-05-11T23:01:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8899999999999999e-06 2024-05-11T23:01:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616427+0000", 2024-05-11T23:01:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7759999999999992e-06 2024-05-11T23:01:42.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616454+0000", 2024-05-11T23:01:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6596000000000001e-05 2024-05-11T23:01:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616467+0000", 2024-05-11T23:01:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3114e-05 2024-05-11T23:01:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616557+0000", 2024-05-11T23:01:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9823999999999999e-05 2024-05-11T23:01:42.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616600+0000", 2024-05-11T23:01:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3822999999999997e-05 2024-05-11T23:01:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:01:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.620723+0000", 2024-05-11T23:01:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041222430000000003 2024-05-11T23:01:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.639100+0000", 2024-05-11T23:01:42.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018377853999999999 2024-05-11T23:01:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.639113+0000", 2024-05-11T23:01:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2198000000000001e-05 2024-05-11T23:01:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.639147+0000", 2024-05-11T23:01:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4344000000000002e-05 2024-05-11T23:01:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.639181+0000", 2024-05-11T23:01:42.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3856999999999998e-05 2024-05-11T23:01:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4382.0:6502 3.14 e28/19 ondisk, result = 0)", 2024-05-11T23:01:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:37.632885+0000", 2024-05-11T23:01:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.1703655609999997, 2024-05-11T23:01:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062216509999999999, 2024-05-11T23:01:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:42.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.632885+0000", 2024-05-11T23:01:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.632885+0000", 2024-05-11T23:01:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.632885+0000", 2024-05-11T23:01:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7700000000000004e-07 2024-05-11T23:01:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.632890+0000", 2024-05-11T23:01:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4549999999999997e-06 2024-05-11T23:01:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.632902+0000", 2024-05-11T23:01:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2089999999999999e-05 2024-05-11T23:01:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.632911+0000", 2024-05-11T23:01:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0489999999999993e-06 2024-05-11T23:01:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.639087+0000", 2024-05-11T23:01:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061764430000000002 2024-05-11T23:01:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.639096+0000", 2024-05-11T23:01:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5099999999999998e-06 2024-05-11T23:01:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.639106+0000", 2024-05-11T23:01:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0327e-05 2024-05-11T23:01:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.873 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:7522 3.14 3:298c29ea:::rbd_data.111e65d1e80f.0000000000000002:8 [sparse-read 3538944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-05-11T23:01:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:38.991623+0000", 2024-05-11T23:01:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.8116276569999998, 2024-05-11T23:01:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061460769999999998, 2024-05-11T23:01:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 7522 2024-05-11T23:01:42.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991623+0000", 2024-05-11T23:01:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991623+0000", 2024-05-11T23:01:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991623+0000", 2024-05-11T23:01:42.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9200000000000002e-07 2024-05-11T23:01:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991624+0000", 2024-05-11T23:01:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6899999999999996e-07 2024-05-11T23:01:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991627+0000", 2024-05-11T23:01:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7970000000000001e-06 2024-05-11T23:01:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991629+0000", 2024-05-11T23:01:42.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5019999999999999e-06 2024-05-11T23:01:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.997581+0000", 2024-05-11T23:01:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005951797 2024-05-11T23:01:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.997662+0000", 2024-05-11T23:01:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0956000000000002e-05 2024-05-11T23:01:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.997769+0000", 2024-05-11T23:01:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106764 2024-05-11T23:01:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.877 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:7523 3.14 3:298c29ea:::rbd_data.111e65d1e80f.0000000000000002:8 [sparse-read 3801088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-05-11T23:01:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:38.991632+0000", 2024-05-11T23:01:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.8116180069999999, 2024-05-11T23:01:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063546940000000001, 2024-05-11T23:01:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.878 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 7523 2024-05-11T23:01:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991632+0000", 2024-05-11T23:01:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991632+0000", 2024-05-11T23:01:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991632+0000", 2024-05-11T23:01:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.03e-07 2024-05-11T23:01:42.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991633+0000", 2024-05-11T23:01:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0750000000000001e-06 2024-05-11T23:01:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991636+0000", 2024-05-11T23:01:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8250000000000001e-06 2024-05-11T23:01:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991639+0000", 2024-05-11T23:01:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4899999999999999e-06 2024-05-11T23:01:42.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.997807+0000", 2024-05-11T23:01:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061682910000000002 2024-05-11T23:01:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.997890+0000", 2024-05-11T23:01:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2758999999999998e-05 2024-05-11T23:01:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.997987+0000", 2024-05-11T23:01:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7051000000000003e-05 2024-05-11T23:01:42.881 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:7524 3.14 3:298c29ea:::rbd_data.111e65d1e80f.0000000000000002:8 [sparse-read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-05-11T23:01:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:38.991642+0000", 2024-05-11T23:01:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.8116083939999998, 2024-05-11T23:01:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0065093379999999999, 2024-05-11T23:01:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.882 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 7524 2024-05-11T23:01:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991642+0000", 2024-05-11T23:01:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991642+0000", 2024-05-11T23:01:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991642+0000", 2024-05-11T23:01:42.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0800000000000001e-07 2024-05-11T23:01:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991643+0000", 2024-05-11T23:01:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9300000000000006e-07 2024-05-11T23:01:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991646+0000", 2024-05-11T23:01:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.779e-06 2024-05-11T23:01:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991648+0000", 2024-05-11T23:01:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6369999999999999e-06 2024-05-11T23:01:42.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998022+0000", 2024-05-11T23:01:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063738090000000002 2024-05-11T23:01:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998073+0000", 2024-05-11T23:01:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0994e-05 2024-05-11T23:01:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998151+0000", 2024-05-11T23:01:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7917999999999995e-05 2024-05-11T23:01:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.885 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:7525 3.14 3:298c29ea:::rbd_data.111e65d1e80f.0000000000000002:8 [sparse-read 4325376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-05-11T23:01:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:38.991652+0000", 2024-05-11T23:01:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.8115986949999998, 2024-05-11T23:01:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0066895660000000001, 2024-05-11T23:01:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 7525 2024-05-11T23:01:42.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991652+0000", 2024-05-11T23:01:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991652+0000", 2024-05-11T23:01:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991652+0000", 2024-05-11T23:01:42.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1500000000000001e-07 2024-05-11T23:01:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991653+0000", 2024-05-11T23:01:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7100000000000011e-07 2024-05-11T23:01:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991655+0000", 2024-05-11T23:01:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6989999999999999e-06 2024-05-11T23:01:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991658+0000", 2024-05-11T23:01:42.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0939999999999999e-06 2024-05-11T23:01:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998180+0000", 2024-05-11T23:01:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006521135 2024-05-11T23:01:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998247+0000", 2024-05-11T23:01:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7362999999999999e-05 2024-05-11T23:01:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998341+0000", 2024-05-11T23:01:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4089e-05 2024-05-11T23:01:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:7526 3.14 3:298c29ea:::rbd_data.111e65d1e80f.0000000000000002:8 [sparse-read 4587520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-05-11T23:01:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:38.991662+0000", 2024-05-11T23:01:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.8115886080000001, 2024-05-11T23:01:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0069244689999999999, 2024-05-11T23:01:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:42.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 7526 2024-05-11T23:01:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991662+0000", 2024-05-11T23:01:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991662+0000", 2024-05-11T23:01:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991662+0000", 2024-05-11T23:01:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:01:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991665+0000", 2024-05-11T23:01:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9579999999999999e-06 2024-05-11T23:01:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991668+0000", 2024-05-11T23:01:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.01e-06 2024-05-11T23:01:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991670+0000", 2024-05-11T23:01:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5440000000000001e-06 2024-05-11T23:01:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998388+0000", 2024-05-11T23:01:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0067174000000000001 2024-05-11T23:01:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:42.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998479+0000", 2024-05-11T23:01:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1629e-05 2024-05-11T23:01:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998586+0000", 2024-05-11T23:01:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106708 2024-05-11T23:01:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:10072 3.1a 3:5fbbf0ff:::rbd_data.111e1e89614a.0000000000000266:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-05-11T23:01:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:42.385140+0000", 2024-05-11T23:01:42.894 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.41811004400000001, 2024-05-11T23:01:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016594922000000002, 2024-05-11T23:01:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10072 2024-05-11T23:01:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.385140+0000", 2024-05-11T23:01:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.385140+0000", 2024-05-11T23:01:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.385141+0000", 2024-05-11T23:01:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1790000000000001e-06 2024-05-11T23:01:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.385146+0000", 2024-05-11T23:01:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4440000000000001e-06 2024-05-11T23:01:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.385156+0000", 2024-05-11T23:01:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0456e-05 2024-05-11T23:01:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.385162+0000", 2024-05-11T23:01:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4569999999999996e-06 2024-05-11T23:01:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.385209+0000", 2024-05-11T23:01:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7759999999999997e-05 2024-05-11T23:01:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:42.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.385293+0000", 2024-05-11T23:01:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3059000000000006e-05 2024-05-11T23:01:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-05-11T23:01:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.388259+0000", 2024-05-11T23:01:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029662019999999998 2024-05-11T23:01:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.401614+0000", 2024-05-11T23:01:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01335504 2024-05-11T23:01:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:42.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.401631+0000", 2024-05-11T23:01:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6867999999999998e-05 2024-05-11T23:01:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.401687+0000", 2024-05-11T23:01:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6648999999999999e-05 2024-05-11T23:01:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.401735+0000", 2024-05-11T23:01:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7808e-05 2024-05-11T23:01:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:42.899 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:42.900 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:42.901 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:01:42.902 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:42.901+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:42.984 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-05-11T23:01:42.985 INFO:teuthology.orchestra.run.smithi005.stdout:1394 write 0xa17fc81 thru 0xa18a06e (0xa3ee bytes) 2024-05-11T23:01:42.995 INFO:teuthology.orchestra.run.smithi005.stdout:1397 trunc from 0xd665387 to 0xb7f9a6d 2024-05-11T23:01:43.002 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:43.002+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:43.014 INFO:teuthology.orchestra.run.smithi005.stdout:1398 read 0xa6ce6d thru 0xa74b05 (0x7c99 bytes) 2024-05-11T23:01:43.014 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:43.014 INFO:teuthology.orchestra.run.smithi005.stdout:1401 write 0x389962d thru 0x38a5694 (0xc068 bytes) 2024-05-11T23:01:43.018 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:43.018 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:01:43.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.018 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:10296 3.16 3:6dc0aec3:::rbd_data.111e520e7062.0000000000000005:head [call rbd.sparse_copyup in=18548b,set-alloc-hint object_size 8388608 write_size 8388608,write 4953645~27091 in=27091b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-05-11T23:01:43.018 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:43.017468+0000", 2024-05-11T23:01:43.018 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0011118790000000001, 2024-05-11T23:01:43.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011335099999999999, 2024-05-11T23:01:43.018 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.018 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.018 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for sub ops", 2024-05-11T23:01:43.018 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.019 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.019 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.019 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10296 2024-05-11T23:01:43.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.019 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.017468+0000", 2024-05-11T23:01:43.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.017468+0000", 2024-05-11T23:01:43.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.017469+0000", 2024-05-11T23:01:43.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9599999999999999e-07 2024-05-11T23:01:43.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.017538+0000", 2024-05-11T23:01:43.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9198999999999999e-05 2024-05-11T23:01:43.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.017549+0000", 2024-05-11T23:01:43.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0385e-05 2024-05-11T23:01:43.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.017555+0000", 2024-05-11T23:01:43.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2990000000000002e-06 2024-05-11T23:01:43.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.017596+0000", 2024-05-11T23:01:43.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0893999999999998e-05 2024-05-11T23:01:43.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.017681+0000", 2024-05-11T23:01:43.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5431000000000002e-05 2024-05-11T23:01:43.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:01:43.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.017799+0000", 2024-05-11T23:01:43.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011738799999999999 2024-05-11T23:01:43.022 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.022 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.022 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.022 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.022 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-11T23:01:43.022 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 1 2024-05-11T23:01:43.022 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:43.023 INFO:teuthology.orchestra.run.smithi005.stdout:1402 punch from 0xa456f04 to 0xa4667ba, (0xf8b6 bytes) 2024-05-11T23:01:43.027 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:01:43.028 INFO:teuthology.orchestra.run.smithi005.stdout:1404 write 0xd968b77 thru 0xd96d114 (0x459e bytes) 2024-05-11T23:01:43.032 INFO:teuthology.orchestra.run.smithi005.stdout:1405 punch from 0xd005f55 to 0xd01570e, (0xf7b9 bytes) 2024-05-11T23:01:43.032 INFO:teuthology.orchestra.run.smithi005.stdout:1406 writesame 0xb6d73b0 thru 0xb6d9c73 data_size 0x2(0x28c4 bytes) 2024-05-11T23:01:43.035 INFO:teuthology.orchestra.run.smithi005.stdout:1407 writesame 0x9ac2a37 thru 0x9ac937d data_size 0x1(0x6947 bytes) 2024-05-11T23:01:43.047 INFO:teuthology.orchestra.run.smithi005.stdout:1408 write 0x8ea5e6a thru 0x8ea8568 (0x26ff bytes) 2024-05-11T23:01:43.048 INFO:teuthology.orchestra.run.smithi005.stdout:1409 writesame 0xd0b9dd0 thru 0xd0c751a data_size 0x49(0xd74b bytes) 2024-05-11T23:01:43.052 INFO:teuthology.orchestra.run.smithi005.stdout:1412 writesame 0x5f75ad3 thru 0x5f7be99 data_size 0x29(0x63c7 bytes) 2024-05-11T23:01:43.056 INFO:teuthology.orchestra.run.smithi005.stdout:1413 read 0xd5ebedf thru 0xd5f7321 (0xb443 bytes) 2024-05-11T23:01:43.056 INFO:teuthology.orchestra.run.smithi005.stdout:1414 writesame 0x66da8d0 thru 0x66dd974 data_size 0x7(0x30a5 bytes) 2024-05-11T23:01:43.058 INFO:teuthology.orchestra.run.smithi005.stdout:1415 write 0x1ed0c4a thru 0x1ed8892 (0x7c49 bytes) 2024-05-11T23:01:43.076 INFO:teuthology.orchestra.run.smithi005.stdout:1418 trunc from 0xd96d115 to 0x1ac333b 2024-05-11T23:01:43.103 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:43.103+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:43.105 INFO:teuthology.orchestra.run.smithi005.stdout:1421 punch from 0x12771eb to 0x1285e81, (0xec96 bytes) 2024-05-11T23:01:43.111 INFO:teuthology.orchestra.run.smithi005.stdout:1422 write 0x2e5d146 thru 0x2e60ddb (0x3c96 bytes) 2024-05-11T23:01:43.113 INFO:teuthology.orchestra.run.smithi005.stdout:1423 flatten 2024-05-11T23:01:43.128 INFO:teuthology.orchestra.run.smithi005.stdout:1424 write 0x73c65ac thru 0x73cf0de (0x8b33 bytes) 2024-05-11T23:01:43.131 INFO:teuthology.orchestra.run.smithi005.stdout:1425 read 0x6a4033f thru 0x6a4a263 (0x9f25 bytes) 2024-05-11T23:01:43.131 INFO:teuthology.orchestra.run.smithi005.stdout:1427 write 0x32ba795 thru 0x32c99cc (0xf238 bytes) 2024-05-11T23:01:43.143 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:43.143 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:43.143 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:01:43.143 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:01:43.143 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:43.147 INFO:teuthology.orchestra.run.smithi005.stdout:1428 write 0xe829831 thru 0xe82d7c1 (0x3f91 bytes) 2024-05-11T23:01:43.150 INFO:teuthology.orchestra.run.smithi005.stdout:1429 clone 10 order 20 su 262144 sc 2 2024-05-11T23:01:43.153 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:01:43.204 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:43.204+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:43.300 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:43.300 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:01:43.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:01:43.300 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:01:43.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.300 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:1499 3.c 3:30f012b2:::rbd_data.111e50384849.0000000000000049:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.300 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.543409+0000", 2024-05-11T23:01:43.300 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.755605933, 2024-05-11T23:01:43.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0075691669999999999, 2024-05-11T23:01:43.301 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.301 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.301 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:43.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.301 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1499 2024-05-11T23:01:43.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.301 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543409+0000", 2024-05-11T23:01:43.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543409+0000", 2024-05-11T23:01:43.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543410+0000", 2024-05-11T23:01:43.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6700000000000003e-07 2024-05-11T23:01:43.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543413+0000", 2024-05-11T23:01:43.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.574e-06 2024-05-11T23:01:43.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543422+0000", 2024-05-11T23:01:43.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6049999999999998e-06 2024-05-11T23:01:43.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543428+0000", 2024-05-11T23:01:43.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6460000000000004e-06 2024-05-11T23:01:43.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.544103+0000", 2024-05-11T23:01:43.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00067411900000000004 2024-05-11T23:01:43.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.544216+0000", 2024-05-11T23:01:43.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011385800000000001 2024-05-11T23:01:43.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.550978+0000", 2024-05-11T23:01:43.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006761698 2024-05-11T23:01:43.304 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.304 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.304 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.305 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:1681 3.c 3:30f012b2:::rbd_data.111e50384849.0000000000000049:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.305 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.557367+0000", 2024-05-11T23:01:43.305 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.7416480589999992, 2024-05-11T23:01:43.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056449150000000003, 2024-05-11T23:01:43.305 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.305 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.305 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.305 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.305 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.305 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.305 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1681 2024-05-11T23:01:43.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.306 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557367+0000", 2024-05-11T23:01:43.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557367+0000", 2024-05-11T23:01:43.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557367+0000", 2024-05-11T23:01:43.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0200000000000003e-07 2024-05-11T23:01:43.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557370+0000", 2024-05-11T23:01:43.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5799999999999999e-06 2024-05-11T23:01:43.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557376+0000", 2024-05-11T23:01:43.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.319e-06 2024-05-11T23:01:43.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557381+0000", 2024-05-11T23:01:43.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6999999999999999e-06 2024-05-11T23:01:43.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.558331+0000", 2024-05-11T23:01:43.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00094994600000000002 2024-05-11T23:01:43.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.558380+0000", 2024-05-11T23:01:43.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9675000000000001e-05 2024-05-11T23:01:43.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:01:43.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.558482+0000", 2024-05-11T23:01:43.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101438 2024-05-11T23:01:43.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.562440+0000", 2024-05-11T23:01:43.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039585209999999996 2024-05-11T23:01:43.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.562852+0000", 2024-05-11T23:01:43.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00041132499999999998 2024-05-11T23:01:43.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.562971+0000", 2024-05-11T23:01:43.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011910500000000001 2024-05-11T23:01:43.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.563012+0000", 2024-05-11T23:01:43.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0904e-05 2024-05-11T23:01:43.310 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.311 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.311 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.311 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:1853 3.17 e25/19 3:ef0f2dd1:::rbd_object_map.111e50384849.0000000000000005:head v 25'5, mlcod=25'2)", 2024-05-11T23:01:43.311 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.568898+0000", 2024-05-11T23:01:43.311 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.7301167759999991, 2024-05-11T23:01:43.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005500095, 2024-05-11T23:01:43.311 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.311 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.311 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.311 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568898+0000", 2024-05-11T23:01:43.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568898+0000", 2024-05-11T23:01:43.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568899+0000", 2024-05-11T23:01:43.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5000000000000003e-07 2024-05-11T23:01:43.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568904+0000", 2024-05-11T23:01:43.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4580000000000001e-06 2024-05-11T23:01:43.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568915+0000", 2024-05-11T23:01:43.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1088e-05 2024-05-11T23:01:43.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568922+0000", 2024-05-11T23:01:43.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6900000000000003e-06 2024-05-11T23:01:43.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568977+0000", 2024-05-11T23:01:43.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5299e-05 2024-05-11T23:01:43.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568995+0000", 2024-05-11T23:01:43.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8173999999999999e-05 2024-05-11T23:01:43.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574370+0000", 2024-05-11T23:01:43.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005374864 2024-05-11T23:01:43.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574398+0000", 2024-05-11T23:01:43.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7971999999999999e-05 2024-05-11T23:01:43.315 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.315 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.315 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.315 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "pg_update_log_missing(3.1f epoch 25/19 rep_tid 142 entries 25'64 (0'0) error 3:fc8394ee:::rbd_data.111e50384849.000000000000005d:head by client.4382.0:1942 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'61)", 2024-05-11T23:01:43.315 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.577276+0000", 2024-05-11T23:01:43.315 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.7217392440000001, 2024-05-11T23:01:43.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0085889989999999999, 2024-05-11T23:01:43.316 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.316 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.316 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:43.316 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577276+0000", 2024-05-11T23:01:43.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577276+0000", 2024-05-11T23:01:43.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577276+0000", 2024-05-11T23:01:43.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5299999999999999e-07 2024-05-11T23:01:43.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577281+0000", 2024-05-11T23:01:43.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3489999999999999e-06 2024-05-11T23:01:43.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577292+0000", 2024-05-11T23:01:43.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0813e-05 2024-05-11T23:01:43.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577300+0000", 2024-05-11T23:01:43.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5209999999999997e-06 2024-05-11T23:01:43.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577377+0000", 2024-05-11T23:01:43.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7197e-05 2024-05-11T23:01:43.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.585865+0000", 2024-05-11T23:01:43.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0084876659999999996 2024-05-11T23:01:43.318 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.318 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.318 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.319 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3456 3.16 3:68b574d8:::rbd_data.111e89cdde7d.000000000000003c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.319 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.888719+0000", 2024-05-11T23:01:43.319 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.4102961549999993, 2024-05-11T23:01:43.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0057862640000000002, 2024-05-11T23:01:43.319 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.319 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.319 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.319 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.319 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.319 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.319 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3456 2024-05-11T23:01:43.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.319 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888719+0000", 2024-05-11T23:01:43.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888719+0000", 2024-05-11T23:01:43.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888720+0000", 2024-05-11T23:01:43.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8100000000000001e-07 2024-05-11T23:01:43.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888724+0000", 2024-05-11T23:01:43.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3769999999999998e-06 2024-05-11T23:01:43.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888736+0000", 2024-05-11T23:01:43.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2237e-05 2024-05-11T23:01:43.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888743+0000", 2024-05-11T23:01:43.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6000000000000003e-06 2024-05-11T23:01:43.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888795+0000", 2024-05-11T23:01:43.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2206999999999997e-05 2024-05-11T23:01:43.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888873+0000", 2024-05-11T23:01:43.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8388999999999998e-05 2024-05-11T23:01:43.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:01:43.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888956+0000", 2024-05-11T23:01:43.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2183999999999998e-05 2024-05-11T23:01:43.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.891425+0000", 2024-05-11T23:01:43.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002469691 2024-05-11T23:01:43.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894453+0000", 2024-05-11T23:01:43.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030273259999999999 2024-05-11T23:01:43.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894481+0000", 2024-05-11T23:01:43.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8683999999999998e-05 2024-05-11T23:01:43.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894505+0000", 2024-05-11T23:01:43.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3688000000000001e-05 2024-05-11T23:01:43.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.324 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.324 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3457 3.c 3:353bdd7c:::rbd_data.111e89cdde7d.0000000000000050:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.324 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.888792+0000", 2024-05-11T23:01:43.324 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.4102230640000002, 2024-05-11T23:01:43.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054713679999999999, 2024-05-11T23:01:43.324 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.324 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.325 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.325 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3457 2024-05-11T23:01:43.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.325 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888792+0000", 2024-05-11T23:01:43.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888792+0000", 2024-05-11T23:01:43.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888792+0000", 2024-05-11T23:01:43.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5300000000000004e-07 2024-05-11T23:01:43.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888795+0000", 2024-05-11T23:01:43.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0750000000000002e-06 2024-05-11T23:01:43.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888805+0000", 2024-05-11T23:01:43.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4790000000000002e-06 2024-05-11T23:01:43.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888811+0000", 2024-05-11T23:01:43.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5459999999999999e-06 2024-05-11T23:01:43.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888857+0000", 2024-05-11T23:01:43.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6007000000000002e-05 2024-05-11T23:01:43.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888906+0000", 2024-05-11T23:01:43.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9496999999999997e-05 2024-05-11T23:01:43.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:01:43.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.889004+0000", 2024-05-11T23:01:43.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8270999999999995e-05 2024-05-11T23:01:43.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.889869+0000", 2024-05-11T23:01:43.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00086435500000000003 2024-05-11T23:01:43.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894165+0000", 2024-05-11T23:01:43.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042960259999999997 2024-05-11T23:01:43.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894215+0000", 2024-05-11T23:01:43.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9876999999999997e-05 2024-05-11T23:01:43.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894263+0000", 2024-05-11T23:01:43.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8581999999999997e-05 2024-05-11T23:01:43.329 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.330 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.330 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.330 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3852 3.1d 3:bde7e891:::rbd_data.111e89cdde7d.00000000000000b9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.330 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.243857+0000", 2024-05-11T23:01:43.330 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.0551579429999993, 2024-05-11T23:01:43.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061537099999999997, 2024-05-11T23:01:43.330 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.330 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.330 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.330 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.331 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.331 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.331 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3852 2024-05-11T23:01:43.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.331 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243857+0000", 2024-05-11T23:01:43.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243857+0000", 2024-05-11T23:01:43.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243858+0000", 2024-05-11T23:01:43.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9400000000000005e-07 2024-05-11T23:01:43.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243863+0000", 2024-05-11T23:01:43.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.237e-06 2024-05-11T23:01:43.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243877+0000", 2024-05-11T23:01:43.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-05-11T23:01:43.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243884+0000", 2024-05-11T23:01:43.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4340000000000003e-06 2024-05-11T23:01:43.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243943+0000", 2024-05-11T23:01:43.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8705999999999997e-05 2024-05-11T23:01:43.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244022+0000", 2024-05-11T23:01:43.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9559000000000002e-05 2024-05-11T23:01:43.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:43.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244121+0000", 2024-05-11T23:01:43.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8227000000000007e-05 2024-05-11T23:01:43.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.246909+0000", 2024-05-11T23:01:43.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002788198 2024-05-11T23:01:43.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.249917+0000", 2024-05-11T23:01:43.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030083839999999998 2024-05-11T23:01:43.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.249968+0000", 2024-05-11T23:01:43.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.113e-05 2024-05-11T23:01:43.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.250011+0000", 2024-05-11T23:01:43.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2386999999999999e-05 2024-05-11T23:01:43.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.335 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.335 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:3855 3.17 e25/19 3:eb11d2b0:::rbd_data.111e89cdde7d.00000000000000bc:head v 25'17, mlcod=25'15)", 2024-05-11T23:01:43.335 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.244299+0000", 2024-05-11T23:01:43.335 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.0547156619999996, 2024-05-11T23:01:43.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054219239999999998, 2024-05-11T23:01:43.336 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.336 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.336 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.336 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244299+0000", 2024-05-11T23:01:43.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244299+0000", 2024-05-11T23:01:43.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244300+0000", 2024-05-11T23:01:43.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1799999999999995e-07 2024-05-11T23:01:43.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244305+0000", 2024-05-11T23:01:43.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4009999999999996e-06 2024-05-11T23:01:43.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244314+0000", 2024-05-11T23:01:43.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7770000000000008e-06 2024-05-11T23:01:43.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244323+0000", 2024-05-11T23:01:43.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9959999999999999e-06 2024-05-11T23:01:43.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244383+0000", 2024-05-11T23:01:43.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9867000000000001e-05 2024-05-11T23:01:43.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244403+0000", 2024-05-11T23:01:43.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9885999999999999e-05 2024-05-11T23:01:43.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.249691+0000", 2024-05-11T23:01:43.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052880150000000001 2024-05-11T23:01:43.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.249721+0000", 2024-05-11T23:01:43.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0463999999999998e-05 2024-05-11T23:01:43.339 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.339 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.339 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.340 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4107 3.1c 3:3e719808:::rbd_data.111e89cdde7d.0000000000000084:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.340 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.457074+0000", 2024-05-11T23:01:43.340 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.8419411189999995, 2024-05-11T23:01:43.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080074340000000008, 2024-05-11T23:01:43.340 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.340 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.340 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.340 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.340 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.340 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.340 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4107 2024-05-11T23:01:43.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.340 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457074+0000", 2024-05-11T23:01:43.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457074+0000", 2024-05-11T23:01:43.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457075+0000", 2024-05-11T23:01:43.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9899999999999999e-07 2024-05-11T23:01:43.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457079+0000", 2024-05-11T23:01:43.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.194e-06 2024-05-11T23:01:43.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457090+0000", 2024-05-11T23:01:43.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.154e-05 2024-05-11T23:01:43.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457096+0000", 2024-05-11T23:01:43.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3890000000000004e-06 2024-05-11T23:01:43.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457145+0000", 2024-05-11T23:01:43.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9245e-05 2024-05-11T23:01:43.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457196+0000", 2024-05-11T23:01:43.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1084000000000001e-05 2024-05-11T23:01:43.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:43.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457269+0000", 2024-05-11T23:01:43.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2712000000000004e-05 2024-05-11T23:01:43.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.462365+0000", 2024-05-11T23:01:43.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050963850000000002 2024-05-11T23:01:43.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.464998+0000", 2024-05-11T23:01:43.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026330799999999999 2024-05-11T23:01:43.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465047+0000", 2024-05-11T23:01:43.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8754000000000001e-05 2024-05-11T23:01:43.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465081+0000", 2024-05-11T23:01:43.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4251999999999997e-05 2024-05-11T23:01:43.344 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.344 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.344 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.345 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4108 3.1d 3:be073325:::rbd_data.111e89cdde7d.0000000000000085:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.345 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.457110+0000", 2024-05-11T23:01:43.345 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.8419046960000003, 2024-05-11T23:01:43.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079569480000000001, 2024-05-11T23:01:43.345 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.345 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.345 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.345 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4108 2024-05-11T23:01:43.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.346 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457110+0000", 2024-05-11T23:01:43.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457110+0000", 2024-05-11T23:01:43.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457111+0000", 2024-05-11T23:01:43.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6699999999999999e-07 2024-05-11T23:01:43.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457113+0000", 2024-05-11T23:01:43.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6220000000000001e-06 2024-05-11T23:01:43.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457120+0000", 2024-05-11T23:01:43.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8909999999999998e-06 2024-05-11T23:01:43.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457124+0000", 2024-05-11T23:01:43.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6660000000000001e-06 2024-05-11T23:01:43.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457169+0000", 2024-05-11T23:01:43.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5439999999999999e-05 2024-05-11T23:01:43.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457256+0000", 2024-05-11T23:01:43.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7131000000000003e-05 2024-05-11T23:01:43.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:43.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457357+0000", 2024-05-11T23:01:43.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100655 2024-05-11T23:01:43.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.462328+0000", 2024-05-11T23:01:43.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049712630000000001 2024-05-11T23:01:43.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.464981+0000", 2024-05-11T23:01:43.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026531160000000001 2024-05-11T23:01:43.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465030+0000", 2024-05-11T23:01:43.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8659999999999998e-05 2024-05-11T23:01:43.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465067+0000", 2024-05-11T23:01:43.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7036999999999998e-05 2024-05-11T23:01:43.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.350 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.350 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4110 3.1c 3:3e4e9183:::rbd_data.111e89cdde7d.00000000000000a8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.350 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.457578+0000", 2024-05-11T23:01:43.350 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.8414364679999995, 2024-05-11T23:01:43.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00756685, 2024-05-11T23:01:43.351 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.351 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.351 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.351 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4110 2024-05-11T23:01:43.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.351 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457578+0000", 2024-05-11T23:01:43.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457578+0000", 2024-05-11T23:01:43.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457579+0000", 2024-05-11T23:01:43.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2200000000000004e-07 2024-05-11T23:01:43.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457583+0000", 2024-05-11T23:01:43.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9079999999999997e-06 2024-05-11T23:01:43.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457592+0000", 2024-05-11T23:01:43.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4290000000000007e-06 2024-05-11T23:01:43.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457599+0000", 2024-05-11T23:01:43.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3790000000000003e-06 2024-05-11T23:01:43.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457697+0000", 2024-05-11T23:01:43.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8492000000000005e-05 2024-05-11T23:01:43.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457768+0000", 2024-05-11T23:01:43.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0798999999999998e-05 2024-05-11T23:01:43.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:43.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457869+0000", 2024-05-11T23:01:43.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101118 2024-05-11T23:01:43.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.462402+0000", 2024-05-11T23:01:43.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045324400000000004 2024-05-11T23:01:43.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465088+0000", 2024-05-11T23:01:43.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026866260000000001 2024-05-11T23:01:43.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465117+0000", 2024-05-11T23:01:43.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8517000000000002e-05 2024-05-11T23:01:43.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465145+0000", 2024-05-11T23:01:43.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8520000000000001e-05 2024-05-11T23:01:43.356 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.356 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.356 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.356 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4111 3.2 e25/19 3:45191464:::rbd_data.111e89cdde7d.00000000000000a9:head v 25'50, mlcod=25'48)", 2024-05-11T23:01:43.356 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.457624+0000", 2024-05-11T23:01:43.356 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.8413911570000003, 2024-05-11T23:01:43.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0074051719999999998, 2024-05-11T23:01:43.356 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.356 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.356 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.356 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457624+0000", 2024-05-11T23:01:43.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457624+0000", 2024-05-11T23:01:43.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457624+0000", 2024-05-11T23:01:43.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6100000000000001e-07 2024-05-11T23:01:43.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457629+0000", 2024-05-11T23:01:43.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4580000000000003e-06 2024-05-11T23:01:43.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457636+0000", 2024-05-11T23:01:43.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5360000000000004e-06 2024-05-11T23:01:43.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457641+0000", 2024-05-11T23:01:43.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0760000000000002e-06 2024-05-11T23:01:43.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457689+0000", 2024-05-11T23:01:43.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8019999999999998e-05 2024-05-11T23:01:43.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457715+0000", 2024-05-11T23:01:43.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5647999999999999e-05 2024-05-11T23:01:43.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.464999+0000", 2024-05-11T23:01:43.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072837329999999997 2024-05-11T23:01:43.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465029+0000", 2024-05-11T23:01:43.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0239999999999998e-05 2024-05-11T23:01:43.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.360 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.360 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4109 3.17 e25/19 3:ec04b13d:::rbd_data.111e89cdde7d.0000000000000086:head v 25'22, mlcod=25'20)", 2024-05-11T23:01:43.360 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.457665+0000", 2024-05-11T23:01:43.360 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.8413498219999997, 2024-05-11T23:01:43.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.007399699, 2024-05-11T23:01:43.360 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.360 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.360 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.360 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457665+0000", 2024-05-11T23:01:43.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457665+0000", 2024-05-11T23:01:43.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457665+0000", 2024-05-11T23:01:43.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1199999999999999e-07 2024-05-11T23:01:43.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457670+0000", 2024-05-11T23:01:43.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:01:43.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457677+0000", 2024-05-11T23:01:43.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6959999999999999e-06 2024-05-11T23:01:43.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457682+0000", 2024-05-11T23:01:43.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7910000000000004e-06 2024-05-11T23:01:43.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457723+0000", 2024-05-11T23:01:43.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1607999999999999e-05 2024-05-11T23:01:43.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457740+0000", 2024-05-11T23:01:43.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6643e-05 2024-05-11T23:01:43.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465033+0000", 2024-05-11T23:01:43.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072929639999999999 2024-05-11T23:01:43.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465065+0000", 2024-05-11T23:01:43.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1801000000000003e-05 2024-05-11T23:01:43.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.364 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.364 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.364 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4158 3.1c 3:3e719808:::rbd_data.111e89cdde7d.0000000000000084:head [set-alloc-hint object_size 1048576 write_size 1048576,write 622592~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.364 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.498662+0000", 2024-05-11T23:01:43.364 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.8003523769999994, 2024-05-11T23:01:43.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0058631339999999999, 2024-05-11T23:01:43.364 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.364 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.364 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.365 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.365 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4158 2024-05-11T23:01:43.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.365 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498662+0000", 2024-05-11T23:01:43.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498662+0000", 2024-05-11T23:01:43.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498664+0000", 2024-05-11T23:01:43.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.886e-06 2024-05-11T23:01:43.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498676+0000", 2024-05-11T23:01:43.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1919999999999999e-05 2024-05-11T23:01:43.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498692+0000", 2024-05-11T23:01:43.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5576000000000001e-05 2024-05-11T23:01:43.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498701+0000", 2024-05-11T23:01:43.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0580000000000004e-06 2024-05-11T23:01:43.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498763+0000", 2024-05-11T23:01:43.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2181000000000007e-05 2024-05-11T23:01:43.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498839+0000", 2024-05-11T23:01:43.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6261999999999996e-05 2024-05-11T23:01:43.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:43.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498969+0000", 2024-05-11T23:01:43.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000129176 2024-05-11T23:01:43.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.501285+0000", 2024-05-11T23:01:43.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023169559999999998 2024-05-11T23:01:43.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504450+0000", 2024-05-11T23:01:43.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0031641920000000001 2024-05-11T23:01:43.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504493+0000", 2024-05-11T23:01:43.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3442999999999997e-05 2024-05-11T23:01:43.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504526+0000", 2024-05-11T23:01:43.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2484000000000003e-05 2024-05-11T23:01:43.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.369 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.370 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.370 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4159 3.1d 3:be073325:::rbd_data.111e89cdde7d.0000000000000085:head [set-alloc-hint object_size 1048576 write_size 1048576,write 622592~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.370 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.498840+0000", 2024-05-11T23:01:43.370 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.8001748580000001, 2024-05-11T23:01:43.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056856160000000001, 2024-05-11T23:01:43.370 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.370 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.370 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.370 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4159 2024-05-11T23:01:43.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.371 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498840+0000", 2024-05-11T23:01:43.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498840+0000", 2024-05-11T23:01:43.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498841+0000", 2024-05-11T23:01:43.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5199999999999995e-07 2024-05-11T23:01:43.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498854+0000", 2024-05-11T23:01:43.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3502e-05 2024-05-11T23:01:43.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498865+0000", 2024-05-11T23:01:43.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0394000000000001e-05 2024-05-11T23:01:43.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498873+0000", 2024-05-11T23:01:43.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8029999999999997e-06 2024-05-11T23:01:43.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498933+0000", 2024-05-11T23:01:43.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0955000000000002e-05 2024-05-11T23:01:43.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499008+0000", 2024-05-11T23:01:43.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4210999999999998e-05 2024-05-11T23:01:43.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:43.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499165+0000", 2024-05-11T23:01:43.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015752100000000001 2024-05-11T23:01:43.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.501518+0000", 2024-05-11T23:01:43.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023527180000000002 2024-05-11T23:01:43.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504432+0000", 2024-05-11T23:01:43.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029144700000000002 2024-05-11T23:01:43.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504485+0000", 2024-05-11T23:01:43.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2639999999999997e-05 2024-05-11T23:01:43.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504526+0000", 2024-05-11T23:01:43.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0550000000000003e-05 2024-05-11T23:01:43.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.375 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.375 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4249 3.1c 3:3e719808:::rbd_data.111e89cdde7d.0000000000000084:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.376 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598332+0000", 2024-05-11T23:01:43.376 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.7006830540000006, 2024-05-11T23:01:43.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0077041669999999996, 2024-05-11T23:01:43.376 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.376 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.376 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.376 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4249 2024-05-11T23:01:43.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.377 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598332+0000", 2024-05-11T23:01:43.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598332+0000", 2024-05-11T23:01:43.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598333+0000", 2024-05-11T23:01:43.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7499999999999998e-07 2024-05-11T23:01:43.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598337+0000", 2024-05-11T23:01:43.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6609999999999999e-06 2024-05-11T23:01:43.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598350+0000", 2024-05-11T23:01:43.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2999e-05 2024-05-11T23:01:43.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598358+0000", 2024-05-11T23:01:43.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9500000000000001e-06 2024-05-11T23:01:43.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598409+0000", 2024-05-11T23:01:43.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0192000000000003e-05 2024-05-11T23:01:43.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598489+0000", 2024-05-11T23:01:43.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0605000000000005e-05 2024-05-11T23:01:43.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:43.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598592+0000", 2024-05-11T23:01:43.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010261 2024-05-11T23:01:43.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.603246+0000", 2024-05-11T23:01:43.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0046541619999999999 2024-05-11T23:01:43.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605952+0000", 2024-05-11T23:01:43.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027065169999999999 2024-05-11T23:01:43.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605999+0000", 2024-05-11T23:01:43.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6283999999999999e-05 2024-05-11T23:01:43.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606036+0000", 2024-05-11T23:01:43.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7212000000000002e-05 2024-05-11T23:01:43.382 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.382 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.382 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.382 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4250 3.1d 3:be073325:::rbd_data.111e89cdde7d.0000000000000085:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.382 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598376+0000", 2024-05-11T23:01:43.382 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.7006384180000005, 2024-05-11T23:01:43.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0076523390000000002, 2024-05-11T23:01:43.382 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.382 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.383 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.383 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4250 2024-05-11T23:01:43.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.383 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598376+0000", 2024-05-11T23:01:43.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598376+0000", 2024-05-11T23:01:43.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598377+0000", 2024-05-11T23:01:43.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9200000000000001e-07 2024-05-11T23:01:43.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598380+0000", 2024-05-11T23:01:43.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8899999999999999e-06 2024-05-11T23:01:43.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598388+0000", 2024-05-11T23:01:43.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8210000000000002e-06 2024-05-11T23:01:43.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598396+0000", 2024-05-11T23:01:43.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1079999999999993e-06 2024-05-11T23:01:43.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598436+0000", 2024-05-11T23:01:43.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0219000000000002e-05 2024-05-11T23:01:43.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598514+0000", 2024-05-11T23:01:43.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7918999999999997e-05 2024-05-11T23:01:43.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:43.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598613+0000", 2024-05-11T23:01:43.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9335999999999999e-05 2024-05-11T23:01:43.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.603227+0000", 2024-05-11T23:01:43.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0046141799999999998 2024-05-11T23:01:43.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605932+0000", 2024-05-11T23:01:43.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002704438 2024-05-11T23:01:43.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605981+0000", 2024-05-11T23:01:43.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8806e-05 2024-05-11T23:01:43.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606029+0000", 2024-05-11T23:01:43.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8130000000000002e-05 2024-05-11T23:01:43.389 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.389 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.389 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.389 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4253 3.1d 3:ba7bd5e3:::rbd_data.111e89cdde7d.00000000000000e3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.389 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.600882+0000", 2024-05-11T23:01:43.389 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.6981329859999992, 2024-05-11T23:01:43.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054413350000000003, 2024-05-11T23:01:43.389 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.389 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.389 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.390 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.390 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.390 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.390 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4253 2024-05-11T23:01:43.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.390 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600882+0000", 2024-05-11T23:01:43.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600882+0000", 2024-05-11T23:01:43.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600883+0000", 2024-05-11T23:01:43.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5400000000000001e-07 2024-05-11T23:01:43.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600887+0000", 2024-05-11T23:01:43.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.092e-06 2024-05-11T23:01:43.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600897+0000", 2024-05-11T23:01:43.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0093e-05 2024-05-11T23:01:43.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600902+0000", 2024-05-11T23:01:43.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1399999999999999e-06 2024-05-11T23:01:43.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600951+0000", 2024-05-11T23:01:43.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9407000000000003e-05 2024-05-11T23:01:43.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.601034+0000", 2024-05-11T23:01:43.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3108000000000005e-05 2024-05-11T23:01:43.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:43.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.601137+0000", 2024-05-11T23:01:43.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010276999999999999 2024-05-11T23:01:43.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606041+0000", 2024-05-11T23:01:43.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049042649999999997 2024-05-11T23:01:43.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606242+0000", 2024-05-11T23:01:43.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020061300000000001 2024-05-11T23:01:43.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606287+0000", 2024-05-11T23:01:43.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4824000000000003e-05 2024-05-11T23:01:43.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606323+0000", 2024-05-11T23:01:43.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6368999999999997e-05 2024-05-11T23:01:43.395 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.395 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.395 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.395 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:6733 3.1c 3:3bac25fd:::rbd_data.111e65d1e80f.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-11T23:01:43.395 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:37.962181+0000", 2024-05-11T23:01:43.395 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.3368333970000004, 2024-05-11T23:01:43.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025925762000000002, 2024-05-11T23:01:43.396 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.396 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.396 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.396 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.396 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.397 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.397 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 6733 2024-05-11T23:01:43.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.397 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962181+0000", 2024-05-11T23:01:43.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962181+0000", 2024-05-11T23:01:43.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962182+0000", 2024-05-11T23:01:43.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2699999999999998e-07 2024-05-11T23:01:43.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962187+0000", 2024-05-11T23:01:43.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3050000000000003e-06 2024-05-11T23:01:43.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962197+0000", 2024-05-11T23:01:43.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0010000000000001e-05 2024-05-11T23:01:43.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962201+0000", 2024-05-11T23:01:43.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8709999999999996e-06 2024-05-11T23:01:43.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962276+0000", 2024-05-11T23:01:43.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5004000000000002e-05 2024-05-11T23:01:43.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962325+0000", 2024-05-11T23:01:43.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8093999999999997e-05 2024-05-11T23:01:43.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:43.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.966865+0000", 2024-05-11T23:01:43.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045404720000000003 2024-05-11T23:01:43.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.987960+0000", 2024-05-11T23:01:43.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021095442999999998 2024-05-11T23:01:43.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.988032+0000", 2024-05-11T23:01:43.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2013000000000003e-05 2024-05-11T23:01:43.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.988070+0000", 2024-05-11T23:01:43.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7981000000000001e-05 2024-05-11T23:01:43.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.988107+0000", 2024-05-11T23:01:43.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6742000000000002e-05 2024-05-11T23:01:43.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.403 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.403 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4382.0:6733 3.1c e28/19 ondisk, result = 0)", 2024-05-11T23:01:43.403 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:37.982065+0000", 2024-05-11T23:01:43.403 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.316949835, 2024-05-11T23:01:43.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060490049999999997, 2024-05-11T23:01:43.403 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.404 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.404 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:43.404 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.982065+0000", 2024-05-11T23:01:43.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.982065+0000", 2024-05-11T23:01:43.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.982066+0000", 2024-05-11T23:01:43.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.156e-06 2024-05-11T23:01:43.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.982071+0000", 2024-05-11T23:01:43.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:01:43.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.982083+0000", 2024-05-11T23:01:43.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1742e-05 2024-05-11T23:01:43.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.982108+0000", 2024-05-11T23:01:43.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5001999999999999e-05 2024-05-11T23:01:43.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.988021+0000", 2024-05-11T23:01:43.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059133149999999997 2024-05-11T23:01:43.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.988028+0000", 2024-05-11T23:01:43.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4649999999999997e-06 2024-05-11T23:01:43.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.988114+0000", 2024-05-11T23:01:43.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5538000000000007e-05 2024-05-11T23:01:43.407 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.407 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.407 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.407 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.407 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.407 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:43.409 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:01:43.409 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:43.408+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:43.509 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:43.510+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:43.515 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:43.515 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:43.516 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:01:43.516 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:43.525 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:01:43.610 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:43.610+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:43.629 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:43.629 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:01:43.630 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:01:43.630 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:43.638 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:01:43.711 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:43.711+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:43.786 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:43.786 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:01:43.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:01:43.786 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:01:43.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.786 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:1494 3.a 3:5237eba0:::rbd_data.111e50384849.0000000000000044:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.786 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.543327+0000", 2024-05-11T23:01:43.786 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.241570068, 2024-05-11T23:01:43.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026719579999999999, 2024-05-11T23:01:43.787 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.787 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.787 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:43.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.787 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1494 2024-05-11T23:01:43.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.787 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543327+0000", 2024-05-11T23:01:43.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543327+0000", 2024-05-11T23:01:43.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543328+0000", 2024-05-11T23:01:43.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1999999999999999e-07 2024-05-11T23:01:43.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543331+0000", 2024-05-11T23:01:43.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.32e-06 2024-05-11T23:01:43.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543350+0000", 2024-05-11T23:01:43.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8936e-05 2024-05-11T23:01:43.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543359+0000", 2024-05-11T23:01:43.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8060000000000001e-06 2024-05-11T23:01:43.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543939+0000", 2024-05-11T23:01:43.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00058049000000000002 2024-05-11T23:01:43.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.544050+0000", 2024-05-11T23:01:43.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011028599999999999 2024-05-11T23:01:43.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.545999+0000", 2024-05-11T23:01:43.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019495000000000001 2024-05-11T23:01:43.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.791 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.791 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:1666 3.a 3:5755b5e9:::rbd_data.111e50384849.0000000000000042:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.791 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.553359+0000", 2024-05-11T23:01:43.791 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.231538339, 2024-05-11T23:01:43.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0034024540000000001, 2024-05-11T23:01:43.791 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.791 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.792 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.792 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1666 2024-05-11T23:01:43.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.792 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553359+0000", 2024-05-11T23:01:43.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553359+0000", 2024-05-11T23:01:43.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553360+0000", 2024-05-11T23:01:43.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1800000000000005e-07 2024-05-11T23:01:43.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553369+0000", 2024-05-11T23:01:43.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7049999999999994e-06 2024-05-11T23:01:43.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553380+0000", 2024-05-11T23:01:43.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0285999999999999e-05 2024-05-11T23:01:43.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553387+0000", 2024-05-11T23:01:43.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3989999999999998e-06 2024-05-11T23:01:43.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553454+0000", 2024-05-11T23:01:43.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6581999999999994e-05 2024-05-11T23:01:43.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553543+0000", 2024-05-11T23:01:43.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9599999999999996e-05 2024-05-11T23:01:43.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:43.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553762+0000", 2024-05-11T23:01:43.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000218402 2024-05-11T23:01:43.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.554603+0000", 2024-05-11T23:01:43.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00084155000000000005 2024-05-11T23:01:43.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.556663+0000", 2024-05-11T23:01:43.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020600430000000001 2024-05-11T23:01:43.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.556717+0000", 2024-05-11T23:01:43.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3942e-05 2024-05-11T23:01:43.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.556761+0000", 2024-05-11T23:01:43.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4127e-05 2024-05-11T23:01:43.797 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.797 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.797 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.797 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:1667 3.a 3:5237eba0:::rbd_data.111e50384849.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.797 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.553412+0000", 2024-05-11T23:01:43.797 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.231485218, 2024-05-11T23:01:43.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036787489999999998, 2024-05-11T23:01:43.797 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.798 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.798 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.798 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1667 2024-05-11T23:01:43.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.798 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553412+0000", 2024-05-11T23:01:43.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553412+0000", 2024-05-11T23:01:43.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553413+0000", 2024-05-11T23:01:43.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8699999999999995e-07 2024-05-11T23:01:43.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553416+0000", 2024-05-11T23:01:43.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-05-11T23:01:43.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553424+0000", 2024-05-11T23:01:43.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.011e-06 2024-05-11T23:01:43.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553429+0000", 2024-05-11T23:01:43.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7400000000000001e-06 2024-05-11T23:01:43.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.554054+0000", 2024-05-11T23:01:43.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00062427700000000001 2024-05-11T23:01:43.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.554148+0000", 2024-05-11T23:01:43.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4145000000000001e-05 2024-05-11T23:01:43.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:43.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.554353+0000", 2024-05-11T23:01:43.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000204835 2024-05-11T23:01:43.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.556278+0000", 2024-05-11T23:01:43.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001925019 2024-05-11T23:01:43.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557014+0000", 2024-05-11T23:01:43.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00073628599999999995 2024-05-11T23:01:43.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557066+0000", 2024-05-11T23:01:43.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1836999999999999e-05 2024-05-11T23:01:43.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557091+0000", 2024-05-11T23:01:43.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4961999999999999e-05 2024-05-11T23:01:43.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.803 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.803 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:1854 3.a 3:527cad80:::rbd_data.111e50384849.0000000000000057:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.803 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.568320+0000", 2024-05-11T23:01:43.803 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.216577124000001, 2024-05-11T23:01:43.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033148499999999998, 2024-05-11T23:01:43.804 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.804 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.804 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:43.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.804 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1854 2024-05-11T23:01:43.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.804 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568320+0000", 2024-05-11T23:01:43.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568320+0000", 2024-05-11T23:01:43.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568321+0000", 2024-05-11T23:01:43.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9100000000000003e-07 2024-05-11T23:01:43.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568325+0000", 2024-05-11T23:01:43.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0570000000000003e-06 2024-05-11T23:01:43.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568335+0000", 2024-05-11T23:01:43.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.842e-06 2024-05-11T23:01:43.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568345+0000", 2024-05-11T23:01:43.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0766e-05 2024-05-11T23:01:43.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568411+0000", 2024-05-11T23:01:43.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5321999999999996e-05 2024-05-11T23:01:43.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568539+0000", 2024-05-11T23:01:43.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000128059 2024-05-11T23:01:43.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.571635+0000", 2024-05-11T23:01:43.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030961130000000002 2024-05-11T23:01:43.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.809 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.809 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "pg_update_log_missing(3.1e epoch 25/19 rep_tid 82 entries 25'46 (0'0) error 3:7e05a7a0:::rbd_data.111e50384849.0000000000000058:head by client.4382.0:1858 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'44)", 2024-05-11T23:01:43.809 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.569301+0000", 2024-05-11T23:01:43.809 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.215595878, 2024-05-11T23:01:43.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048630660000000001, 2024-05-11T23:01:43.809 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.809 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.810 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:43.810 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569301+0000", 2024-05-11T23:01:43.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569301+0000", 2024-05-11T23:01:43.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569302+0000", 2024-05-11T23:01:43.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0800000000000003e-07 2024-05-11T23:01:43.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569310+0000", 2024-05-11T23:01:43.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9370000000000004e-06 2024-05-11T23:01:43.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569329+0000", 2024-05-11T23:01:43.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8808999999999999e-05 2024-05-11T23:01:43.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569344+0000", 2024-05-11T23:01:43.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4789e-05 2024-05-11T23:01:43.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569405+0000", 2024-05-11T23:01:43.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0714000000000002e-05 2024-05-11T23:01:43.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574164+0000", 2024-05-11T23:01:43.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047599089999999997 2024-05-11T23:01:43.812 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.813 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.813 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.813 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:1902 3.1e e25/19 3:7e2d774f:::rbd_data.111e50384849.0000000000000050:head v 25'47, mlcod=25'44)", 2024-05-11T23:01:43.813 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.573037+0000", 2024-05-11T23:01:43.813 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.211860687, 2024-05-11T23:01:43.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042723099999999996, 2024-05-11T23:01:43.813 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.813 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.813 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.813 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573037+0000", 2024-05-11T23:01:43.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573037+0000", 2024-05-11T23:01:43.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573037+0000", 2024-05-11T23:01:43.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-05-11T23:01:43.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573438+0000", 2024-05-11T23:01:43.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00040049399999999999 2024-05-11T23:01:43.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573446+0000", 2024-05-11T23:01:43.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1890000000000007e-06 2024-05-11T23:01:43.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573451+0000", 2024-05-11T23:01:43.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1959999999999999e-06 2024-05-11T23:01:43.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573488+0000", 2024-05-11T23:01:43.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6467999999999997e-05 2024-05-11T23:01:43.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573507+0000", 2024-05-11T23:01:43.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9321e-05 2024-05-11T23:01:43.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577275+0000", 2024-05-11T23:01:43.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037682750000000002 2024-05-11T23:01:43.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577309+0000", 2024-05-11T23:01:43.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3744000000000001e-05 2024-05-11T23:01:43.817 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.817 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.817 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.817 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:1914 3.a 3:527cad80:::rbd_data.111e50384849.0000000000000057:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.817 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.573229+0000", 2024-05-11T23:01:43.817 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.211668415, 2024-05-11T23:01:43.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041158779999999999, 2024-05-11T23:01:43.817 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.817 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.817 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.818 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.818 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1914 2024-05-11T23:01:43.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.818 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573229+0000", 2024-05-11T23:01:43.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573229+0000", 2024-05-11T23:01:43.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573230+0000", 2024-05-11T23:01:43.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8800000000000002e-07 2024-05-11T23:01:43.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573234+0000", 2024-05-11T23:01:43.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6070000000000001e-06 2024-05-11T23:01:43.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573244+0000", 2024-05-11T23:01:43.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9389999999999992e-06 2024-05-11T23:01:43.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573249+0000", 2024-05-11T23:01:43.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:01:43.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573297+0000", 2024-05-11T23:01:43.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7636000000000003e-05 2024-05-11T23:01:43.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573373+0000", 2024-05-11T23:01:43.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6693000000000006e-05 2024-05-11T23:01:43.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:43.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573566+0000", 2024-05-11T23:01:43.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019307699999999999 2024-05-11T23:01:43.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.576944+0000", 2024-05-11T23:01:43.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033778900000000001 2024-05-11T23:01:43.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577244+0000", 2024-05-11T23:01:43.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030006400000000001 2024-05-11T23:01:43.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577306+0000", 2024-05-11T23:01:43.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1625999999999996e-05 2024-05-11T23:01:43.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577345+0000", 2024-05-11T23:01:43.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8773999999999998e-05 2024-05-11T23:01:43.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.823 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.823 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:1904 3.1e e25/19 3:7e2d774f:::rbd_data.111e50384849.0000000000000050:head v 25'49, mlcod=25'44)", 2024-05-11T23:01:43.823 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.574045+0000", 2024-05-11T23:01:43.823 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.210852395, 2024-05-11T23:01:43.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0058621020000000001, 2024-05-11T23:01:43.823 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.823 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.823 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.823 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574045+0000", 2024-05-11T23:01:43.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574045+0000", 2024-05-11T23:01:43.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574045+0000", 2024-05-11T23:01:43.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:01:43.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574049+0000", 2024-05-11T23:01:43.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5899999999999999e-06 2024-05-11T23:01:43.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574057+0000", 2024-05-11T23:01:43.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4859999999999997e-06 2024-05-11T23:01:43.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574062+0000", 2024-05-11T23:01:43.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6530000000000002e-06 2024-05-11T23:01:43.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574199+0000", 2024-05-11T23:01:43.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013656900000000001 2024-05-11T23:01:43.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574237+0000", 2024-05-11T23:01:43.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8055000000000001e-05 2024-05-11T23:01:43.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.579872+0000", 2024-05-11T23:01:43.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005635223 2024-05-11T23:01:43.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.579907+0000", 2024-05-11T23:01:43.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5027000000000002e-05 2024-05-11T23:01:43.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.828 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.828 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:1972 3.1e e25/19 3:7e05a7a0:::rbd_data.111e50384849.0000000000000058:head v 25'50, mlcod=25'46)", 2024-05-11T23:01:43.828 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.578576+0000", 2024-05-11T23:01:43.828 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.206320999000001, 2024-05-11T23:01:43.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038930829999999999, 2024-05-11T23:01:43.828 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.828 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.828 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.829 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578576+0000", 2024-05-11T23:01:43.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578576+0000", 2024-05-11T23:01:43.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578577+0000", 2024-05-11T23:01:43.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8999999999999996e-07 2024-05-11T23:01:43.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578582+0000", 2024-05-11T23:01:43.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5990000000000004e-06 2024-05-11T23:01:43.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578591+0000", 2024-05-11T23:01:43.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9690000000000006e-06 2024-05-11T23:01:43.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578598+0000", 2024-05-11T23:01:43.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2179999999999996e-06 2024-05-11T23:01:43.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578657+0000", 2024-05-11T23:01:43.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9001e-05 2024-05-11T23:01:43.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578679+0000", 2024-05-11T23:01:43.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2259000000000002e-05 2024-05-11T23:01:43.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.582439+0000", 2024-05-11T23:01:43.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037597960000000001 2024-05-11T23:01:43.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.582469+0000", 2024-05-11T23:01:43.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0550999999999999e-05 2024-05-11T23:01:43.832 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.832 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.832 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.832 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "pg_update_log_missing(3.1e epoch 25/19 rep_tid 90 entries 25'51 (0'0) error 3:7facaf60:::rbd_data.111e50384849.0000000000000061:head by client.4382.0:2005 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'47)", 2024-05-11T23:01:43.832 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.581892+0000", 2024-05-11T23:01:43.832 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.203005580999999, 2024-05-11T23:01:43.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042265779999999999, 2024-05-11T23:01:43.833 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.833 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.833 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:43.833 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581892+0000", 2024-05-11T23:01:43.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581892+0000", 2024-05-11T23:01:43.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581892+0000", 2024-05-11T23:01:43.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9299999999999998e-07 2024-05-11T23:01:43.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581898+0000", 2024-05-11T23:01:43.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0390000000000001e-06 2024-05-11T23:01:43.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581915+0000", 2024-05-11T23:01:43.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6418000000000001e-05 2024-05-11T23:01:43.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581929+0000", 2024-05-11T23:01:43.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4131000000000001e-05 2024-05-11T23:01:43.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581984+0000", 2024-05-11T23:01:43.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4821999999999998e-05 2024-05-11T23:01:43.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.586118+0000", 2024-05-11T23:01:43.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004134675 2024-05-11T23:01:43.836 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.836 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.836 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.836 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3301 3.10 3:0d09d413:::rbd_data.111e7a935520.0000000000000009:4 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.836 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.722658+0000", 2024-05-11T23:01:43.836 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.062238757999999, 2024-05-11T23:01:43.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026955619999999999, 2024-05-11T23:01:43.836 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.836 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.836 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:43.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.836 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3301 2024-05-11T23:01:43.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.837 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722658+0000", 2024-05-11T23:01:43.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722658+0000", 2024-05-11T23:01:43.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722659+0000", 2024-05-11T23:01:43.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5199999999999996e-07 2024-05-11T23:01:43.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722663+0000", 2024-05-11T23:01:43.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4309999999999998e-06 2024-05-11T23:01:43.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722672+0000", 2024-05-11T23:01:43.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1409999999999994e-06 2024-05-11T23:01:43.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722678+0000", 2024-05-11T23:01:43.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3239999999999999e-06 2024-05-11T23:01:43.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.725175+0000", 2024-05-11T23:01:43.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024966789999999999 2024-05-11T23:01:43.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.725258+0000", 2024-05-11T23:01:43.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3220999999999995e-05 2024-05-11T23:01:43.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.725354+0000", 2024-05-11T23:01:43.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6113999999999995e-05 2024-05-11T23:01:43.840 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.840 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.840 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.840 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3313 3.10 3:0d09d413:::rbd_data.111e7a935520.0000000000000009:4 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.840 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.722692+0000", 2024-05-11T23:01:43.840 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.062205710000001, 2024-05-11T23:01:43.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028682030000000002, 2024-05-11T23:01:43.840 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.840 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.840 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:43.840 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.841 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.841 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.841 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3313 2024-05-11T23:01:43.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.841 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722692+0000", 2024-05-11T23:01:43.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722692+0000", 2024-05-11T23:01:43.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722692+0000", 2024-05-11T23:01:43.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:01:43.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722694+0000", 2024-05-11T23:01:43.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4719999999999998e-06 2024-05-11T23:01:43.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722702+0000", 2024-05-11T23:01:43.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2470000000000002e-06 2024-05-11T23:01:43.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722707+0000", 2024-05-11T23:01:43.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2089999999999996e-06 2024-05-11T23:01:43.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.725383+0000", 2024-05-11T23:01:43.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026757270000000001 2024-05-11T23:01:43.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.725465+0000", 2024-05-11T23:01:43.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2226999999999997e-05 2024-05-11T23:01:43.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.725560+0000", 2024-05-11T23:01:43.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4877000000000007e-05 2024-05-11T23:01:43.844 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.844 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.844 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.844 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3326 3.10 3:0d09d413:::rbd_data.111e7a935520.0000000000000009:4 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.844 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.722752+0000", 2024-05-11T23:01:43.844 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.06214509, 2024-05-11T23:01:43.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030445110000000002, 2024-05-11T23:01:43.844 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.845 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.845 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:43.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.845 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3326 2024-05-11T23:01:43.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.845 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722752+0000", 2024-05-11T23:01:43.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722752+0000", 2024-05-11T23:01:43.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722753+0000", 2024-05-11T23:01:43.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9500000000000002e-07 2024-05-11T23:01:43.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722757+0000", 2024-05-11T23:01:43.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5770000000000004e-06 2024-05-11T23:01:43.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722765+0000", 2024-05-11T23:01:43.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0979999999999994e-06 2024-05-11T23:01:43.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722771+0000", 2024-05-11T23:01:43.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.643e-06 2024-05-11T23:01:43.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.725602+0000", 2024-05-11T23:01:43.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028311970000000001 2024-05-11T23:01:43.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.725693+0000", 2024-05-11T23:01:43.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0997999999999998e-05 2024-05-11T23:01:43.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.725797+0000", 2024-05-11T23:01:43.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103403 2024-05-11T23:01:43.848 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.848 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.848 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.848 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3471 3.a 3:50ba23c0:::rbd_data.111e89cdde7d.00000000000000b2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.848 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.894634+0000", 2024-05-11T23:01:43.848 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.8902632009999998, 2024-05-11T23:01:43.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053259079999999999, 2024-05-11T23:01:43.848 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.849 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.849 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.849 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.849 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.849 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.849 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3471 2024-05-11T23:01:43.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.849 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894634+0000", 2024-05-11T23:01:43.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894634+0000", 2024-05-11T23:01:43.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894636+0000", 2024-05-11T23:01:43.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.542e-06 2024-05-11T23:01:43.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894640+0000", 2024-05-11T23:01:43.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9439999999999998e-06 2024-05-11T23:01:43.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894651+0000", 2024-05-11T23:01:43.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1952e-05 2024-05-11T23:01:43.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894657+0000", 2024-05-11T23:01:43.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5280000000000003e-06 2024-05-11T23:01:43.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894727+0000", 2024-05-11T23:01:43.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0029000000000003e-05 2024-05-11T23:01:43.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894822+0000", 2024-05-11T23:01:43.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5319e-05 2024-05-11T23:01:43.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:43.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894892+0000", 2024-05-11T23:01:43.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9542999999999995e-05 2024-05-11T23:01:43.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.897034+0000", 2024-05-11T23:01:43.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021425139999999999 2024-05-11T23:01:43.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.899868+0000", 2024-05-11T23:01:43.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028339659999999998 2024-05-11T23:01:43.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.899918+0000", 2024-05-11T23:01:43.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9548000000000001e-05 2024-05-11T23:01:43.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.899960+0000", 2024-05-11T23:01:43.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2023000000000001e-05 2024-05-11T23:01:43.853 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.853 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.853 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.854 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3697 3.a 3:50ba23c0:::rbd_data.111e89cdde7d.00000000000000b2:head [set-alloc-hint object_size 1048576 write_size 1048576,write 573440~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.854 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.121225+0000", 2024-05-11T23:01:43.854 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.6636724530000002, 2024-05-11T23:01:43.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054297959999999998, 2024-05-11T23:01:43.854 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.854 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.854 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.854 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.854 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.854 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.854 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3697 2024-05-11T23:01:43.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.854 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121225+0000", 2024-05-11T23:01:43.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121225+0000", 2024-05-11T23:01:43.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121227+0000", 2024-05-11T23:01:43.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.776e-06 2024-05-11T23:01:43.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121236+0000", 2024-05-11T23:01:43.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7510000000000003e-06 2024-05-11T23:01:43.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121247+0000", 2024-05-11T23:01:43.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.115e-05 2024-05-11T23:01:43.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121253+0000", 2024-05-11T23:01:43.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.75e-06 2024-05-11T23:01:43.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121297+0000", 2024-05-11T23:01:43.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3930000000000001e-05 2024-05-11T23:01:43.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121350+0000", 2024-05-11T23:01:43.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2978e-05 2024-05-11T23:01:43.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:43.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121441+0000", 2024-05-11T23:01:43.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0662999999999996e-05 2024-05-11T23:01:43.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.123760+0000", 2024-05-11T23:01:43.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023193549999999999 2024-05-11T23:01:43.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.126594+0000", 2024-05-11T23:01:43.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028343410000000002 2024-05-11T23:01:43.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.126628+0000", 2024-05-11T23:01:43.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3621000000000002e-05 2024-05-11T23:01:43.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.126655+0000", 2024-05-11T23:01:43.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6480999999999999e-05 2024-05-11T23:01:43.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.859 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.860 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3849 3.a 3:50ba23c0:::rbd_data.111e89cdde7d.00000000000000b2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.860 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.243274+0000", 2024-05-11T23:01:43.860 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.5416234850000006, 2024-05-11T23:01:43.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038658270000000001, 2024-05-11T23:01:43.860 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.860 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.860 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.860 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.860 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.860 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.860 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3849 2024-05-11T23:01:43.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.860 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243274+0000", 2024-05-11T23:01:43.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243274+0000", 2024-05-11T23:01:43.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243275+0000", 2024-05-11T23:01:43.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3399999999999997e-07 2024-05-11T23:01:43.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243279+0000", 2024-05-11T23:01:43.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7779999999999998e-06 2024-05-11T23:01:43.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243294+0000", 2024-05-11T23:01:43.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4253000000000001e-05 2024-05-11T23:01:43.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243302+0000", 2024-05-11T23:01:43.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9240000000000007e-06 2024-05-11T23:01:43.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243364+0000", 2024-05-11T23:01:43.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2349999999999998e-05 2024-05-11T23:01:43.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243443+0000", 2024-05-11T23:01:43.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8877999999999999e-05 2024-05-11T23:01:43.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:43.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243543+0000", 2024-05-11T23:01:43.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010016199999999999 2024-05-11T23:01:43.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244177+0000", 2024-05-11T23:01:43.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00063432099999999997 2024-05-11T23:01:43.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.247082+0000", 2024-05-11T23:01:43.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029042 2024-05-11T23:01:43.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.247111+0000", 2024-05-11T23:01:43.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9839e-05 2024-05-11T23:01:43.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.247140+0000", 2024-05-11T23:01:43.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8187999999999998e-05 2024-05-11T23:01:43.865 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.865 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.865 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.865 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4161 3.a 3:50c2a80d:::rbd_data.111e89cdde7d.0000000000000087:head [set-alloc-hint object_size 1048576 write_size 1048576,write 614400~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.865 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.499080+0000", 2024-05-11T23:01:43.865 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.2858176179999994, 2024-05-11T23:01:43.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054594780000000002, 2024-05-11T23:01:43.865 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.865 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.866 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.866 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4161 2024-05-11T23:01:43.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.866 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499080+0000", 2024-05-11T23:01:43.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499080+0000", 2024-05-11T23:01:43.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499081+0000", 2024-05-11T23:01:43.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.733e-06 2024-05-11T23:01:43.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499096+0000", 2024-05-11T23:01:43.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4721e-05 2024-05-11T23:01:43.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499114+0000", 2024-05-11T23:01:43.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8238000000000001e-05 2024-05-11T23:01:43.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499124+0000", 2024-05-11T23:01:43.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0096e-05 2024-05-11T23:01:43.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499186+0000", 2024-05-11T23:01:43.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1124000000000006e-05 2024-05-11T23:01:43.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499323+0000", 2024-05-11T23:01:43.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013730599999999999 2024-05-11T23:01:43.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:43.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499458+0000", 2024-05-11T23:01:43.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013542599999999999 2024-05-11T23:01:43.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.501629+0000", 2024-05-11T23:01:43.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002170543 2024-05-11T23:01:43.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504476+0000", 2024-05-11T23:01:43.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028470879999999998 2024-05-11T23:01:43.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504508+0000", 2024-05-11T23:01:43.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2360000000000002e-05 2024-05-11T23:01:43.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504539+0000", 2024-05-11T23:01:43.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0843000000000003e-05 2024-05-11T23:01:43.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.871 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.871 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4252 3.a 3:50c2a80d:::rbd_data.111e89cdde7d.0000000000000087:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:43.871 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.600827+0000", 2024-05-11T23:01:43.871 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.1840699069999996, 2024-05-11T23:01:43.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054938859999999999, 2024-05-11T23:01:43.871 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.872 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.872 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.872 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4252 2024-05-11T23:01:43.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.872 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600827+0000", 2024-05-11T23:01:43.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600827+0000", 2024-05-11T23:01:43.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600829+0000", 2024-05-11T23:01:43.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2139999999999999e-06 2024-05-11T23:01:43.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600833+0000", 2024-05-11T23:01:43.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5580000000000002e-06 2024-05-11T23:01:43.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600844+0000", 2024-05-11T23:01:43.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0915e-05 2024-05-11T23:01:43.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600852+0000", 2024-05-11T23:01:43.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6260000000000003e-06 2024-05-11T23:01:43.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600924+0000", 2024-05-11T23:01:43.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1884000000000005e-05 2024-05-11T23:01:43.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.601012+0000", 2024-05-11T23:01:43.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8732000000000004e-05 2024-05-11T23:01:43.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:43.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.601137+0000", 2024-05-11T23:01:43.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012487500000000001 2024-05-11T23:01:43.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.603281+0000", 2024-05-11T23:01:43.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002144165 2024-05-11T23:01:43.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606216+0000", 2024-05-11T23:01:43.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029350790000000002 2024-05-11T23:01:43.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606277+0000", 2024-05-11T23:01:43.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0752999999999999e-05 2024-05-11T23:01:43.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606321+0000", 2024-05-11T23:01:43.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4085000000000002e-05 2024-05-11T23:01:43.877 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.877 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.877 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.877 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:10265 3.a 3:50e2f1fe:::rbd_data.111e520e7062.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 7667712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-05-11T23:01:43.877 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:43.006566+0000", 2024-05-11T23:01:43.877 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.77833104600000003, 2024-05-11T23:01:43.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002786359, 2024-05-11T23:01:43.877 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.877 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.877 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.877 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.878 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.878 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.878 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10265 2024-05-11T23:01:43.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.878 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.006566+0000", 2024-05-11T23:01:43.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.006566+0000", 2024-05-11T23:01:43.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.006568+0000", 2024-05-11T23:01:43.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7039999999999999e-06 2024-05-11T23:01:43.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.006654+0000", 2024-05-11T23:01:43.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5937000000000001e-05 2024-05-11T23:01:43.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.006670+0000", 2024-05-11T23:01:43.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6056e-05 2024-05-11T23:01:43.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.006679+0000", 2024-05-11T23:01:43.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6589999999999996e-06 2024-05-11T23:01:43.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.006735+0000", 2024-05-11T23:01:43.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6706000000000002e-05 2024-05-11T23:01:43.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.006854+0000", 2024-05-11T23:01:43.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011867299999999999 2024-05-11T23:01:43.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:43.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.007061+0000", 2024-05-11T23:01:43.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020734599999999999 2024-05-11T23:01:43.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.008553+0000", 2024-05-11T23:01:43.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001491968 2024-05-11T23:01:43.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.009245+0000", 2024-05-11T23:01:43.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00069169499999999998 2024-05-11T23:01:43.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.009308+0000", 2024-05-11T23:01:43.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2911999999999997e-05 2024-05-11T23:01:43.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.009353+0000", 2024-05-11T23:01:43.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4703000000000002e-05 2024-05-11T23:01:43.882 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.882 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.882 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.883 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:10456 3.10 3:0af30f4b:::rbd_data.111e520e7062.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-05-11T23:01:43.883 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:43.092742+0000", 2024-05-11T23:01:43.883 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.69215541300000005, 2024-05-11T23:01:43.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028432539999999999, 2024-05-11T23:01:43.883 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.883 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.883 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:43.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.883 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10456 2024-05-11T23:01:43.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.883 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.092742+0000", 2024-05-11T23:01:43.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.092742+0000", 2024-05-11T23:01:43.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.092743+0000", 2024-05-11T23:01:43.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3850000000000001e-06 2024-05-11T23:01:43.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.092754+0000", 2024-05-11T23:01:43.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1029000000000001e-05 2024-05-11T23:01:43.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.092771+0000", 2024-05-11T23:01:43.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6645000000000001e-05 2024-05-11T23:01:43.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:43.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.092779+0000", 2024-05-11T23:01:43.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3059999999999998e-06 2024-05-11T23:01:43.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:43.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.092836+0000", 2024-05-11T23:01:43.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6663000000000003e-05 2024-05-11T23:01:43.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:43.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.092929+0000", 2024-05-11T23:01:43.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2705000000000001e-05 2024-05-11T23:01:43.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:43.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.093040+0000", 2024-05-11T23:01:43.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011109599999999999 2024-05-11T23:01:43.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:43.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.093972+0000", 2024-05-11T23:01:43.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093239500000000003 2024-05-11T23:01:43.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:43.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.095490+0000", 2024-05-11T23:01:43.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015183989999999999 2024-05-11T23:01:43.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:43.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.095545+0000", 2024-05-11T23:01:43.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4703000000000001e-05 2024-05-11T23:01:43.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:43.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.095585+0000", 2024-05-11T23:01:43.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9928000000000001e-05 2024-05-11T23:01:43.887 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.887 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.887 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.888 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:43.888 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:43.888 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:43.888 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-11T23:01:43.890 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:01:43.890 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:43.889+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:43.990 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:43.990+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:43.997 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:43.997 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:01:43.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.997 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:10616 3.11 3:8832f216:::rbd_object_map.111e520e7062.000000000000000c:head [call rbd.object_map_load in=18b out=31b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-05-11T23:01:43.997 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:43.997310+0000", 2024-05-11T23:01:43.997 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00024092800000000001, 2024-05-11T23:01:43.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000242728, 2024-05-11T23:01:43.997 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:43.997 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:43.997 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:43.997 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:43.997 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:43.998 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:43.998 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10616 2024-05-11T23:01:43.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.998 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:43.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:43.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.997310+0000", 2024-05-11T23:01:43.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:43.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.997310+0000", 2024-05-11T23:01:43.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:43.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:43.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.997310+0000", 2024-05-11T23:01:43.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1999999999999999e-07 2024-05-11T23:01:43.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:43.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.997314+0000", 2024-05-11T23:01:43.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1540000000000004e-06 2024-05-11T23:01:43.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:43.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:43.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:43.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.997325+0000", 2024-05-11T23:01:43.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0499000000000001e-05 2024-05-11T23:01:44.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.997330+0000", 2024-05-11T23:01:44.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2700000000000004e-06 2024-05-11T23:01:44.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.997372+0000", 2024-05-11T23:01:44.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2061000000000003e-05 2024-05-11T23:01:44.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.997431+0000", 2024-05-11T23:01:44.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8962999999999998e-05 2024-05-11T23:01:44.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.997552+0000", 2024-05-11T23:01:44.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000121061 2024-05-11T23:01:44.002 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.002 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.002 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.002 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.003 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-11T23:01:44.003 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 1 2024-05-11T23:01:44.003 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:44.006 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:01:44.007 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-05-11T23:01:44.091 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:44.091+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:44.158 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:44.158 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:44.159 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:01:44.159 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:01:44.159 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:44.167 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:01:44.184 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:01:44.209 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:44.208+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:44.258 INFO:teuthology.orchestra.run.smithi005.stdout:1430 write 0x2157af thru 0x21c30d (0x6b5f bytes) 2024-05-11T23:01:44.266 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:44.266 INFO:teuthology.orchestra.run.smithi005.stdout:1432 read 0x5235976 thru 0x524370c (0xdd97 bytes) 2024-05-11T23:01:44.266 INFO:teuthology.orchestra.run.smithi005.stdout:1433 write 0xa50f03e thru 0xa519b48 (0xab0b bytes) 2024-05-11T23:01:44.286 INFO:teuthology.orchestra.run.smithi005.stdout:1435 trunc from 0xe82d7c2 to 0x2051f9 2024-05-11T23:01:44.293 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:44.293+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:44.297 INFO:teuthology.orchestra.run.smithi005.stdout:1436 trunc from 0x2051f9 to 0x16d3289 2024-05-11T23:01:44.312 INFO:teuthology.orchestra.run.smithi005.stdout:1437 trunc from 0x16d3289 to 0x957d87c 2024-05-11T23:01:44.318 INFO:teuthology.orchestra.run.smithi005.stdout:1438 trunc from 0x957d87c to 0xa203075 2024-05-11T23:01:44.319 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:44.319 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:01:44.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:01:44.319 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:01:44.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.319 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:01:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:01:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.154625959000001, 2024-05-11T23:01:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:01:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:01:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:44.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:01:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:01:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:01:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:01:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:01:44.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:01:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:01:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:01:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:01:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:01:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:01:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:01:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:01:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:01:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:01:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:01:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:01:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:01:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:01:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:01:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:01:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:01:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:01:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:44.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:01:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:01:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:01:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:01:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:01:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:01:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.796771+0000", 2024-05-11T23:01:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.521106720999999, 2024-05-11T23:01:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010000207000000001, 2024-05-11T23:01:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 139 2024-05-11T23:01:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.326 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796771+0000", 2024-05-11T23:01:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796771+0000", 2024-05-11T23:01:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796772+0000", 2024-05-11T23:01:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.46e-07 2024-05-11T23:01:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796777+0000", 2024-05-11T23:01:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0200000000000002e-06 2024-05-11T23:01:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796783+0000", 2024-05-11T23:01:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9630000000000003e-06 2024-05-11T23:01:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796787+0000", 2024-05-11T23:01:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9309999999999999e-06 2024-05-11T23:01:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.800763+0000", 2024-05-11T23:01:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039755750000000003 2024-05-11T23:01:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.800807+0000", 2024-05-11T23:01:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4070000000000003e-05 2024-05-11T23:01:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.800892+0000", 2024-05-11T23:01:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5402000000000007e-05 2024-05-11T23:01:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:44.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801760+0000", 2024-05-11T23:01:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00086746299999999996 2024-05-11T23:01:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806715+0000", 2024-05-11T23:01:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049554760000000003 2024-05-11T23:01:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806747+0000", 2024-05-11T23:01:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1606999999999997e-05 2024-05-11T23:01:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806771+0000", 2024-05-11T23:01:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4253999999999999e-05 2024-05-11T23:01:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.796795+0000", 2024-05-11T23:01:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.521082572000001, 2024-05-11T23:01:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010021855, 2024-05-11T23:01:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:44.331 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 140 2024-05-11T23:01:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796795+0000", 2024-05-11T23:01:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796795+0000", 2024-05-11T23:01:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:44.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796796+0000", 2024-05-11T23:01:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3499999999999996e-07 2024-05-11T23:01:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796852+0000", 2024-05-11T23:01:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5785000000000002e-05 2024-05-11T23:01:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796859+0000", 2024-05-11T23:01:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5560000000000002e-06 2024-05-11T23:01:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796864+0000", 2024-05-11T23:01:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0000000000000004e-06 2024-05-11T23:01:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801142+0000", 2024-05-11T23:01:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042779740000000004 2024-05-11T23:01:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801190+0000", 2024-05-11T23:01:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7466000000000003e-05 2024-05-11T23:01:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801272+0000", 2024-05-11T23:01:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2534999999999995e-05 2024-05-11T23:01:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.802308+0000", 2024-05-11T23:01:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010360129999999999 2024-05-11T23:01:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806778+0000", 2024-05-11T23:01:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044694699999999997 2024-05-11T23:01:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806801+0000", 2024-05-11T23:01:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2972999999999998e-05 2024-05-11T23:01:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806817+0000", 2024-05-11T23:01:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6548000000000001e-05 2024-05-11T23:01:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.796875+0000", 2024-05-11T23:01:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 25.521003039, 2024-05-11T23:01:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0099879219999999998, 2024-05-11T23:01:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:44.336 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 141 2024-05-11T23:01:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796875+0000", 2024-05-11T23:01:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796875+0000", 2024-05-11T23:01:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796875+0000", 2024-05-11T23:01:44.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7299999999999996e-07 2024-05-11T23:01:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796879+0000", 2024-05-11T23:01:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9929999999999997e-06 2024-05-11T23:01:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796885+0000", 2024-05-11T23:01:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0279999999999997e-06 2024-05-11T23:01:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796890+0000", 2024-05-11T23:01:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3279999999999999e-06 2024-05-11T23:01:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801781+0000", 2024-05-11T23:01:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048916599999999999 2024-05-11T23:01:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801824+0000", 2024-05-11T23:01:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3006999999999997e-05 2024-05-11T23:01:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801893+0000", 2024-05-11T23:01:44.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8940000000000001e-05 2024-05-11T23:01:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.802554+0000", 2024-05-11T23:01:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00066069900000000005 2024-05-11T23:01:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806820+0000", 2024-05-11T23:01:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004265975 2024-05-11T23:01:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806843+0000", 2024-05-11T23:01:44.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3467000000000001e-05 2024-05-11T23:01:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806863+0000", 2024-05-11T23:01:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9252000000000001e-05 2024-05-11T23:01:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:1966 3.7 e25/19 3:e5c0e1bc:::rbd_data.111e50384849.0000000000000051:head v 25'5, mlcod=24'1)", 2024-05-11T23:01:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.581545+0000", 2024-05-11T23:01:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.736332496999999, 2024-05-11T23:01:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0087119039999999995, 2024-05-11T23:01:44.341 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581545+0000", 2024-05-11T23:01:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581545+0000", 2024-05-11T23:01:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581546+0000", 2024-05-11T23:01:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7999999999999995e-07 2024-05-11T23:01:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581552+0000", 2024-05-11T23:01:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4049999999999997e-06 2024-05-11T23:01:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581562+0000", 2024-05-11T23:01:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0047999999999999e-05 2024-05-11T23:01:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581569+0000", 2024-05-11T23:01:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5729999999999996e-06 2024-05-11T23:01:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581631+0000", 2024-05-11T23:01:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2515999999999996e-05 2024-05-11T23:01:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581661+0000", 2024-05-11T23:01:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9247999999999998e-05 2024-05-11T23:01:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:44.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.590214+0000", 2024-05-11T23:01:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0085532009999999999 2024-05-11T23:01:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.590257+0000", 2024-05-11T23:01:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3333e-05 2024-05-11T23:01:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3420 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.837877+0000", 2024-05-11T23:01:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.480001168999999, 2024-05-11T23:01:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010185928, 2024-05-11T23:01:44.345 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3420 2024-05-11T23:01:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.837877+0000", 2024-05-11T23:01:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.837877+0000", 2024-05-11T23:01:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.837877+0000", 2024-05-11T23:01:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3099999999999998e-07 2024-05-11T23:01:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.837880+0000", 2024-05-11T23:01:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6419999999999999e-06 2024-05-11T23:01:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.837886+0000", 2024-05-11T23:01:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8579999999999996e-06 2024-05-11T23:01:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.837890+0000", 2024-05-11T23:01:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7129999999999996e-06 2024-05-11T23:01:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.841920+0000", 2024-05-11T23:01:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040296840000000004 2024-05-11T23:01:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.841997+0000", 2024-05-11T23:01:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6942999999999998e-05 2024-05-11T23:01:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.842284+0000", 2024-05-11T23:01:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028709100000000001 2024-05-11T23:01:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.845267+0000", 2024-05-11T23:01:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029831319999999999 2024-05-11T23:01:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.848002+0000", 2024-05-11T23:01:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027353540000000002 2024-05-11T23:01:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.848033+0000", 2024-05-11T23:01:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0528999999999999e-05 2024-05-11T23:01:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.848063+0000", 2024-05-11T23:01:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9550999999999999e-05 2024-05-11T23:01:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3519 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.948570+0000", 2024-05-11T23:01:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.369307581999999, 2024-05-11T23:01:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.009020073, 2024-05-11T23:01:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3519 2024-05-11T23:01:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.351 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948570+0000", 2024-05-11T23:01:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948570+0000", 2024-05-11T23:01:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948570+0000", 2024-05-11T23:01:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3900000000000001e-07 2024-05-11T23:01:44.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948572+0000", 2024-05-11T23:01:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8810000000000001e-06 2024-05-11T23:01:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948576+0000", 2024-05-11T23:01:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5930000000000002e-06 2024-05-11T23:01:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948578+0000", 2024-05-11T23:01:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5459999999999998e-06 2024-05-11T23:01:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.951234+0000", 2024-05-11T23:01:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026551349999999999 2024-05-11T23:01:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.951314+0000", 2024-05-11T23:01:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9937000000000004e-05 2024-05-11T23:01:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.951680+0000", 2024-05-11T23:01:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00036631599999999999 2024-05-11T23:01:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.954851+0000", 2024-05-11T23:01:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0031709070000000002 2024-05-11T23:01:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957260+0000", 2024-05-11T23:01:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002409436 2024-05-11T23:01:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957306+0000", 2024-05-11T23:01:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5494999999999997e-05 2024-05-11T23:01:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957590+0000", 2024-05-11T23:01:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028458800000000001 2024-05-11T23:01:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3520 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.948582+0000", 2024-05-11T23:01:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.369295957, 2024-05-11T23:01:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0091539389999999998, 2024-05-11T23:01:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:44.356 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3520 2024-05-11T23:01:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948582+0000", 2024-05-11T23:01:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948582+0000", 2024-05-11T23:01:44.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948582+0000", 2024-05-11T23:01:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3999999999999998e-07 2024-05-11T23:01:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948584+0000", 2024-05-11T23:01:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6080000000000001e-06 2024-05-11T23:01:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948587+0000", 2024-05-11T23:01:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0290000000000001e-06 2024-05-11T23:01:44.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948589+0000", 2024-05-11T23:01:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4590000000000001e-06 2024-05-11T23:01:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.952038+0000", 2024-05-11T23:01:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003448502 2024-05-11T23:01:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.952120+0000", 2024-05-11T23:01:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2115999999999998e-05 2024-05-11T23:01:44.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.952489+0000", 2024-05-11T23:01:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00036898600000000002 2024-05-11T23:01:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.955755+0000", 2024-05-11T23:01:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032666430000000001 2024-05-11T23:01:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957646+0000", 2024-05-11T23:01:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018909350000000001 2024-05-11T23:01:44.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957699+0000", 2024-05-11T23:01:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2231000000000002e-05 2024-05-11T23:01:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957736+0000", 2024-05-11T23:01:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7190000000000001e-05 2024-05-11T23:01:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3521 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.948592+0000", 2024-05-11T23:01:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.369285542, 2024-05-11T23:01:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00930396, 2024-05-11T23:01:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3521 2024-05-11T23:01:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948592+0000", 2024-05-11T23:01:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948592+0000", 2024-05-11T23:01:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948593+0000", 2024-05-11T23:01:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2999999999999999e-07 2024-05-11T23:01:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948594+0000", 2024-05-11T23:01:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.545e-06 2024-05-11T23:01:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948601+0000", 2024-05-11T23:01:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5350000000000001e-06 2024-05-11T23:01:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948603+0000", 2024-05-11T23:01:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8009999999999999e-06 2024-05-11T23:01:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.952822+0000", 2024-05-11T23:01:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042181509999999998 2024-05-11T23:01:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.952908+0000", 2024-05-11T23:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6292999999999995e-05 2024-05-11T23:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.953231+0000", 2024-05-11T23:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032354399999999999 2024-05-11T23:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.955760+0000", 2024-05-11T23:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0025289259999999999 2024-05-11T23:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957792+0000", 2024-05-11T23:01:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020321760000000001 2024-05-11T23:01:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957853+0000", 2024-05-11T23:01:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0541000000000002e-05 2024-05-11T23:01:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957896+0000", 2024-05-11T23:01:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3217999999999999e-05 2024-05-11T23:01:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3522 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.948609+0000", 2024-05-11T23:01:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.369268910000001, 2024-05-11T23:01:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0094139299999999992, 2024-05-11T23:01:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3522 2024-05-11T23:01:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948609+0000", 2024-05-11T23:01:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948609+0000", 2024-05-11T23:01:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948609+0000", 2024-05-11T23:01:44.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.36e-07 2024-05-11T23:01:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948611+0000", 2024-05-11T23:01:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9530000000000002e-06 2024-05-11T23:01:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948614+0000", 2024-05-11T23:01:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3859999999999999e-06 2024-05-11T23:01:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948617+0000", 2024-05-11T23:01:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5229999999999998e-06 2024-05-11T23:01:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.953596+0000", 2024-05-11T23:01:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049785840000000003 2024-05-11T23:01:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.953670+0000", 2024-05-11T23:01:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4146000000000005e-05 2024-05-11T23:01:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.953869+0000", 2024-05-11T23:01:44.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019907099999999999 2024-05-11T23:01:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.955764+0000", 2024-05-11T23:01:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018952280000000001 2024-05-11T23:01:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957939+0000", 2024-05-11T23:01:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002175424 2024-05-11T23:01:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957992+0000", 2024-05-11T23:01:44.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2373000000000002e-05 2024-05-11T23:01:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.958023+0000", 2024-05-11T23:01:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1006000000000001e-05 2024-05-11T23:01:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3645 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.057376+0000", 2024-05-11T23:01:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.260501501, 2024-05-11T23:01:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0090708760000000003, 2024-05-11T23:01:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:44.372 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3645 2024-05-11T23:01:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057376+0000", 2024-05-11T23:01:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:44.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057376+0000", 2024-05-11T23:01:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057377+0000", 2024-05-11T23:01:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.28e-07 2024-05-11T23:01:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057378+0000", 2024-05-11T23:01:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.711e-06 2024-05-11T23:01:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057382+0000", 2024-05-11T23:01:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.422e-06 2024-05-11T23:01:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057384+0000", 2024-05-11T23:01:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.503e-06 2024-05-11T23:01:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.062163+0000", 2024-05-11T23:01:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047783679999999999 2024-05-11T23:01:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.062264+0000", 2024-05-11T23:01:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010114499999999999 2024-05-11T23:01:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.062657+0000", 2024-05-11T23:01:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039347200000000002 2024-05-11T23:01:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.064523+0000", 2024-05-11T23:01:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001865678 2024-05-11T23:01:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.066347+0000", 2024-05-11T23:01:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018238810000000001 2024-05-11T23:01:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.066402+0000", 2024-05-11T23:01:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5315999999999997e-05 2024-05-11T23:01:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.066447+0000", 2024-05-11T23:01:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5151999999999997e-05 2024-05-11T23:01:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3646 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.057388+0000", 2024-05-11T23:01:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.260490293, 2024-05-11T23:01:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011967663999999999, 2024-05-11T23:01:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3646 2024-05-11T23:01:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.378 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057388+0000", 2024-05-11T23:01:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057388+0000", 2024-05-11T23:01:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057388+0000", 2024-05-11T23:01:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2399999999999999e-07 2024-05-11T23:01:44.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057389+0000", 2024-05-11T23:01:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.652e-06 2024-05-11T23:01:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057392+0000", 2024-05-11T23:01:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0649999999999999e-06 2024-05-11T23:01:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057395+0000", 2024-05-11T23:01:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4490000000000002e-06 2024-05-11T23:01:44.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063116+0000", 2024-05-11T23:01:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0057215540000000002 2024-05-11T23:01:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063206+0000", 2024-05-11T23:01:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9476999999999997e-05 2024-05-11T23:01:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063559+0000", 2024-05-11T23:01:44.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035349900000000001 2024-05-11T23:01:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.069086+0000", 2024-05-11T23:01:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0055262150000000001 2024-05-11T23:01:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.069282+0000", 2024-05-11T23:01:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019678299999999999 2024-05-11T23:01:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.069319+0000", 2024-05-11T23:01:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6112000000000003e-05 2024-05-11T23:01:44.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.069355+0000", 2024-05-11T23:01:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6634000000000002e-05 2024-05-11T23:01:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3670 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.088064+0000", 2024-05-11T23:01:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.229813734, 2024-05-11T23:01:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080673350000000001, 2024-05-11T23:01:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:44.383 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3670 2024-05-11T23:01:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.088064+0000", 2024-05-11T23:01:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.088064+0000", 2024-05-11T23:01:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.088064+0000", 2024-05-11T23:01:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0900000000000002e-07 2024-05-11T23:01:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.088067+0000", 2024-05-11T23:01:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5610000000000001e-06 2024-05-11T23:01:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.088087+0000", 2024-05-11T23:01:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0208e-05 2024-05-11T23:01:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.088093+0000", 2024-05-11T23:01:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0619999999999997e-06 2024-05-11T23:01:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.090650+0000", 2024-05-11T23:01:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0025564799999999999 2024-05-11T23:01:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.090735+0000", 2024-05-11T23:01:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4829999999999999e-05 2024-05-11T23:01:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.091092+0000", 2024-05-11T23:01:44.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000357771 2024-05-11T23:01:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.093704+0000", 2024-05-11T23:01:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026118579999999999 2024-05-11T23:01:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.096085+0000", 2024-05-11T23:01:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023805499999999999 2024-05-11T23:01:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:44.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.096109+0000", 2024-05-11T23:01:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4621e-05 2024-05-11T23:01:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.096131+0000", 2024-05-11T23:01:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1985e-05 2024-05-11T23:01:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3729 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.158110+0000", 2024-05-11T23:01:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.159768081999999, 2024-05-11T23:01:44.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0081205400000000007, 2024-05-11T23:01:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3729 2024-05-11T23:01:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158110+0000", 2024-05-11T23:01:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158110+0000", 2024-05-11T23:01:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158110+0000", 2024-05-11T23:01:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2099999999999997e-07 2024-05-11T23:01:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158113+0000", 2024-05-11T23:01:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0319999999999999e-06 2024-05-11T23:01:44.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158121+0000", 2024-05-11T23:01:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1200000000000002e-06 2024-05-11T23:01:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158126+0000", 2024-05-11T23:01:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8099999999999997e-06 2024-05-11T23:01:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.160866+0000", 2024-05-11T23:01:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027402799999999999 2024-05-11T23:01:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.160934+0000", 2024-05-11T23:01:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7203e-05 2024-05-11T23:01:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.161320+0000", 2024-05-11T23:01:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000385893 2024-05-11T23:01:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.165881+0000", 2024-05-11T23:01:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045617879999999998 2024-05-11T23:01:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.166145+0000", 2024-05-11T23:01:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026390400000000003 2024-05-11T23:01:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.166190+0000", 2024-05-11T23:01:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4234999999999999e-05 2024-05-11T23:01:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.166230+0000", 2024-05-11T23:01:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0754000000000003e-05 2024-05-11T23:01:44.393 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3730 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.158133+0000", 2024-05-11T23:01:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.159745274, 2024-05-11T23:01:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0081900059999999997, 2024-05-11T23:01:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:44.394 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3730 2024-05-11T23:01:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158133+0000", 2024-05-11T23:01:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158133+0000", 2024-05-11T23:01:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158133+0000", 2024-05-11T23:01:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6899999999999998e-07 2024-05-11T23:01:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158136+0000", 2024-05-11T23:01:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8660000000000002e-06 2024-05-11T23:01:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158142+0000", 2024-05-11T23:01:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8340000000000004e-06 2024-05-11T23:01:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158146+0000", 2024-05-11T23:01:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7570000000000004e-06 2024-05-11T23:01:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.161757+0000", 2024-05-11T23:01:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036105709999999999 2024-05-11T23:01:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.161843+0000", 2024-05-11T23:01:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5609e-05 2024-05-11T23:01:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:44.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.162146+0000", 2024-05-11T23:01:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030325799999999999 2024-05-11T23:01:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.165905+0000", 2024-05-11T23:01:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003759063 2024-05-11T23:01:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.166264+0000", 2024-05-11T23:01:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035860500000000001 2024-05-11T23:01:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:44.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.166295+0000", 2024-05-11T23:01:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1878000000000002e-05 2024-05-11T23:01:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.166323+0000", 2024-05-11T23:01:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7095999999999999e-05 2024-05-11T23:01:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3731 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.158153+0000", 2024-05-11T23:01:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.15972506, 2024-05-11T23:01:44.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0090137840000000004, 2024-05-11T23:01:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3731 2024-05-11T23:01:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158153+0000", 2024-05-11T23:01:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158153+0000", 2024-05-11T23:01:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158153+0000", 2024-05-11T23:01:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8799999999999998e-07 2024-05-11T23:01:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158156+0000", 2024-05-11T23:01:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6469999999999998e-06 2024-05-11T23:01:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158163+0000", 2024-05-11T23:01:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3250000000000002e-06 2024-05-11T23:01:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158168+0000", 2024-05-11T23:01:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8949999999999997e-06 2024-05-11T23:01:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.162490+0000", 2024-05-11T23:01:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00432176 2024-05-11T23:01:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.162571+0000", 2024-05-11T23:01:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1459999999999996e-05 2024-05-11T23:01:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.162939+0000", 2024-05-11T23:01:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00036788899999999999 2024-05-11T23:01:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.166338+0000", 2024-05-11T23:01:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00339895 2024-05-11T23:01:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.167091+0000", 2024-05-11T23:01:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00075337699999999995 2024-05-11T23:01:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.167139+0000", 2024-05-11T23:01:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7883000000000002e-05 2024-05-11T23:01:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.167167+0000", 2024-05-11T23:01:44.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7209999999999999e-05 2024-05-11T23:01:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3896 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.279281+0000", 2024-05-11T23:01:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.038596712, 2024-05-11T23:01:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080757069999999997, 2024-05-11T23:01:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:44.405 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3896 2024-05-11T23:01:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279281+0000", 2024-05-11T23:01:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279281+0000", 2024-05-11T23:01:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279281+0000", 2024-05-11T23:01:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3699999999999999e-07 2024-05-11T23:01:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279283+0000", 2024-05-11T23:01:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6729999999999999e-06 2024-05-11T23:01:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279286+0000", 2024-05-11T23:01:44.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2160000000000002e-06 2024-05-11T23:01:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279289+0000", 2024-05-11T23:01:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.123e-06 2024-05-11T23:01:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.283324+0000", 2024-05-11T23:01:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040346440000000004 2024-05-11T23:01:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.283428+0000", 2024-05-11T23:01:44.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010368499999999999 2024-05-11T23:01:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.283778+0000", 2024-05-11T23:01:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000350806 2024-05-11T23:01:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.287204+0000", 2024-05-11T23:01:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0034256460000000001 2024-05-11T23:01:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:44.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.287309+0000", 2024-05-11T23:01:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104514 2024-05-11T23:01:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.287335+0000", 2024-05-11T23:01:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6724999999999999e-05 2024-05-11T23:01:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.287357+0000", 2024-05-11T23:01:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1438e-05 2024-05-11T23:01:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3897 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.279295+0000", 2024-05-11T23:01:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 10.038582420999999, 2024-05-11T23:01:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089004529999999991, 2024-05-11T23:01:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3897 2024-05-11T23:01:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.411 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279295+0000", 2024-05-11T23:01:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279295+0000", 2024-05-11T23:01:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279296+0000", 2024-05-11T23:01:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4900000000000002e-07 2024-05-11T23:01:44.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279298+0000", 2024-05-11T23:01:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.091e-06 2024-05-11T23:01:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279301+0000", 2024-05-11T23:01:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3160000000000001e-06 2024-05-11T23:01:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279304+0000", 2024-05-11T23:01:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.542e-06 2024-05-11T23:01:44.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.284176+0000", 2024-05-11T23:01:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048726159999999998 2024-05-11T23:01:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.284260+0000", 2024-05-11T23:01:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4270000000000005e-05 2024-05-11T23:01:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.284652+0000", 2024-05-11T23:01:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039102499999999998 2024-05-11T23:01:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.287398+0000", 2024-05-11T23:01:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027467899999999998 2024-05-11T23:01:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.288099+0000", 2024-05-11T23:01:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070078100000000004 2024-05-11T23:01:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.288154+0000", 2024-05-11T23:01:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4706000000000001e-05 2024-05-11T23:01:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.288196+0000", 2024-05-11T23:01:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2067000000000002e-05 2024-05-11T23:01:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4185 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.526304+0000", 2024-05-11T23:01:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.7915734669999992, 2024-05-11T23:01:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080873190000000008, 2024-05-11T23:01:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:44.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4185 2024-05-11T23:01:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.526304+0000", 2024-05-11T23:01:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.526304+0000", 2024-05-11T23:01:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.526305+0000", 2024-05-11T23:01:44.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5600000000000002e-07 2024-05-11T23:01:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.526306+0000", 2024-05-11T23:01:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6500000000000001e-06 2024-05-11T23:01:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.526311+0000", 2024-05-11T23:01:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2570000000000001e-06 2024-05-11T23:01:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.526314+0000", 2024-05-11T23:01:44.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.597e-06 2024-05-11T23:01:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.529081+0000", 2024-05-11T23:01:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027672119999999998 2024-05-11T23:01:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.529160+0000", 2024-05-11T23:01:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9135999999999996e-05 2024-05-11T23:01:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.529395+0000", 2024-05-11T23:01:44.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000234737 2024-05-11T23:01:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.531442+0000", 2024-05-11T23:01:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020470470000000002 2024-05-11T23:01:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.534339+0000", 2024-05-11T23:01:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028972580000000002 2024-05-11T23:01:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.534369+0000", 2024-05-11T23:01:44.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9844000000000001e-05 2024-05-11T23:01:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.534392+0000", 2024-05-11T23:01:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2325000000000001e-05 2024-05-11T23:01:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4208 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.550445+0000", 2024-05-11T23:01:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.767432501, 2024-05-11T23:01:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089990360000000002, 2024-05-11T23:01:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:44.421 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4208 2024-05-11T23:01:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.550445+0000", 2024-05-11T23:01:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:44.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.550445+0000", 2024-05-11T23:01:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.550446+0000", 2024-05-11T23:01:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:01:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.550448+0000", 2024-05-11T23:01:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.588e-06 2024-05-11T23:01:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:44.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.550455+0000", 2024-05-11T23:01:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5200000000000003e-06 2024-05-11T23:01:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.550460+0000", 2024-05-11T23:01:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:01:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.551784+0000", 2024-05-11T23:01:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013239790000000001 2024-05-11T23:01:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.551832+0000", 2024-05-11T23:01:44.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8299e-05 2024-05-11T23:01:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552040+0000", 2024-05-11T23:01:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020784300000000001 2024-05-11T23:01:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.557025+0000", 2024-05-11T23:01:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004985349 2024-05-11T23:01:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559373+0000", 2024-05-11T23:01:44.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023476719999999999 2024-05-11T23:01:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559408+0000", 2024-05-11T23:01:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4928000000000002e-05 2024-05-11T23:01:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559444+0000", 2024-05-11T23:01:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6619000000000003e-05 2024-05-11T23:01:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:44.426 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:44.427 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:44.427 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:01:44.428 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:01:44.430 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:01:44.430 INFO:teuthology.orchestra.run.smithi005.stdout:1439 punch from 0x6d734ae to 0x6d79607, (0x6159 bytes) 2024-05-11T23:01:44.430 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:44.430 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:44.430 INFO:teuthology.orchestra.run.smithi005.stdout:1445 read 0x6f59aca thru 0x6f5f6ed (0x5c24 bytes) 2024-05-11T23:01:44.431 INFO:teuthology.orchestra.run.smithi005.stdout:1446 punch from 0x22728a6 to 0x228184d, (0xefa7 bytes) 2024-05-11T23:01:44.431 INFO:teuthology.orchestra.run.smithi005.stdout:1447 writesame 0xb46ebfa thru 0xb47777e data_size 0x1(0x8b85 bytes) 2024-05-11T23:01:44.431 INFO:teuthology.orchestra.run.smithi005.stdout:1448 writesame 0x40d0acb thru 0x40dd1bc data_size 0x37(0xc6f2 bytes) 2024-05-11T23:01:44.431 INFO:teuthology.orchestra.run.smithi005.stdout:1449 read 0xb0a5ef9 thru 0xb0af957 (0x9a5f bytes) 2024-05-11T23:01:44.431 INFO:teuthology.orchestra.run.smithi005.stdout:1450 write 0x32ba933 thru 0x32bae6f (0x53d bytes) 2024-05-11T23:01:44.431 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:01:44.431 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:01:44.432 INFO:teuthology.orchestra.run.smithi005.stdout:1451 write 0xd1577b7 thru 0xd16530a (0xdb54 bytes) 2024-05-11T23:01:44.432 INFO:teuthology.orchestra.run.smithi005.stdout:1452 read 0x37ed71b thru 0x37ef7e4 (0x20ca bytes) 2024-05-11T23:01:44.432 INFO:teuthology.orchestra.run.smithi005.stdout:1453 write 0xaf1a15c thru 0xaf2220b (0x80b0 bytes) 2024-05-11T23:01:44.432 INFO:teuthology.orchestra.run.smithi005.stdout:1454 write 0x2f6094e thru 0x2f6ae45 (0xa4f8 bytes) 2024-05-11T23:01:44.432 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:01:44.433 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:01:44.433 INFO:teuthology.orchestra.run.smithi005.stdout:1456 trunc from 0xd16530b to 0xb1bfda 2024-05-11T23:01:44.433 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:01:44.433 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:01:44.434 INFO:teuthology.orchestra.run.smithi005.stdout:1457 write 0xc200cf5 thru 0xc20e2bc (0xd5c8 bytes) 2024-05-11T23:01:44.434 INFO:teuthology.orchestra.run.smithi005.stdout:1458 read 0x9321a7 thru 0x93a558 (0x83b2 bytes) 2024-05-11T23:01:44.435 INFO:teuthology.orchestra.run.smithi005.stdout:1459 punch from 0x41d82b3 to 0x41df66c, (0x73b9 bytes) 2024-05-11T23:01:44.435 INFO:teuthology.orchestra.run.smithi005.stdout:1460 punch from 0x741358e to 0x7415237, (0x1ca9 bytes) 2024-05-11T23:01:44.435 INFO:teuthology.orchestra.run.smithi005.stdout:1461 writesame 0x7a1e5e6 thru 0x7a263b4 data_size 0x2b(0x7dcf bytes) 2024-05-11T23:01:44.435 INFO:teuthology.orchestra.run.smithi005.stdout:1462 write 0x34ff9c0 thru 0x350bc00 (0xc241 bytes) 2024-05-11T23:01:44.435 INFO:teuthology.orchestra.run.smithi005.stdout:1463 read 0x16ba37f thru 0x16c94be (0xf140 bytes) 2024-05-11T23:01:44.435 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:44.435 INFO:teuthology.orchestra.run.smithi005.stdout:1465 read 0x3b72489 thru 0x3b7fd92 (0xd90a bytes) 2024-05-11T23:01:44.435 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:01:44.435 INFO:teuthology.orchestra.run.smithi005.stdout:1466 writesame 0x247b0be thru 0x248af33 data_size 0x63(0xfe76 bytes) 2024-05-11T23:01:44.436 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:01:44.437 INFO:teuthology.orchestra.run.smithi005.stdout:1468 read 0x108e45 thru 0x10f7c6 (0x6982 bytes) 2024-05-11T23:01:44.437 INFO:teuthology.orchestra.run.smithi005.stdout:1469 punch from 0x3300c62 to 0x330d80a, (0xcba8 bytes) 2024-05-11T23:01:44.437 INFO:teuthology.orchestra.run.smithi005.stdout:1470 write 0xeb6959e thru 0xeb6cd8d (0x37f0 bytes) 2024-05-11T23:01:44.437 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:44.437 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:44.437 INFO:teuthology.orchestra.run.smithi005.stdout:1473 write 0xae05fe1 thru 0xae0ab74 (0x4b94 bytes) 2024-05-11T23:01:44.437 INFO:teuthology.orchestra.run.smithi005.stdout:1475 write 0x7a8a27e thru 0x7a8cd5c (0x2adf bytes) 2024-05-11T23:01:44.437 INFO:teuthology.orchestra.run.smithi005.stdout:1476 read 0xc13a5e6 thru 0xc13b2e6 (0xd01 bytes) 2024-05-11T23:01:44.437 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:44.437 INFO:teuthology.orchestra.run.smithi005.stdout:1478 write 0x248cca2 thru 0x2492a8c (0x5deb bytes) 2024-05-11T23:01:44.438 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:01:44.438 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:01:44.438 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:44.428+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:44.439 INFO:teuthology.orchestra.run.smithi005.stdout:1479 trunc from 0xeb6cd8e to 0x844ced3 2024-05-11T23:01:44.441 INFO:teuthology.orchestra.run.smithi005.stdout:1480 write 0x7638b22 thru 0x7644f6b (0xc44a bytes) 2024-05-11T23:01:44.443 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:01:44.444 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:01:44.445 INFO:teuthology.orchestra.run.smithi005.stdout:1481 write 0x1fe03b3 thru 0x1fee580 (0xe1ce bytes) 2024-05-11T23:01:44.448 INFO:teuthology.orchestra.run.smithi005.stdout:1482 read 0x311ae3c thru 0x3123eb9 (0x907e bytes) 2024-05-11T23:01:44.449 INFO:teuthology.orchestra.run.smithi005.stdout:1485 trunc from 0x844ced3 to 0x8b3be3f 2024-05-11T23:01:44.454 INFO:teuthology.orchestra.run.smithi005.stdout:1488 writesame 0x13821c8 thru 0x138ee03 data_size 0x6(0xcc3c bytes) 2024-05-11T23:01:44.464 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:01:44.464 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:01:44.469 INFO:teuthology.orchestra.run.smithi005.stdout:1489 writesame 0xaa18448 thru 0xaa271f1 data_size 0x1d(0xedaa bytes) 2024-05-11T23:01:44.469 INFO:teuthology.orchestra.run.smithi005.stdout:1490 writesame 0x812531d thru 0x812fa39 data_size 0x1(0xa71d bytes) 2024-05-11T23:01:44.491 INFO:teuthology.orchestra.run.smithi005.stdout:1492 read 0x67e4e32 thru 0x67e8d46 (0x3f15 bytes) 2024-05-11T23:01:44.492 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:44.499 INFO:teuthology.orchestra.run.smithi005.stdout:1494 write 0xc949712 thru 0xc95748c (0xdd7b bytes) 2024-05-11T23:01:44.502 INFO:teuthology.orchestra.run.smithi005.stdout:1496 read 0xc08ce6 thru 0xc1293d (0x9c58 bytes) 2024-05-11T23:01:44.506 INFO:teuthology.orchestra.run.smithi005.stdout:1497 trunc from 0xc95748d to 0x9181c52 2024-05-11T23:01:44.522 INFO:teuthology.orchestra.run.smithi005.stdout:1498 write 0x9f329f3 thru 0x9f372f4 (0x4902 bytes) 2024-05-11T23:01:44.525 INFO:teuthology.orchestra.run.smithi005.stdout:1499 write 0xcf2388 thru 0xcffe57 (0xdad0 bytes) 2024-05-11T23:01:44.528 INFO:teuthology.orchestra.run.smithi005.stdout:1500 punch from 0x8d32631 to 0x8d37e19, (0x57e8 bytes) 2024-05-11T23:01:44.529 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:44.528+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:44.531 INFO:teuthology.orchestra.run.smithi005.stdout:1501 trunc from 0x9f372f5 to 0xd29f978 2024-05-11T23:01:44.542 INFO:teuthology.orchestra.run.smithi005.stdout:1502 trunc from 0xd29f978 to 0x8478aa0 2024-05-11T23:01:44.552 INFO:teuthology.orchestra.run.smithi005.stdout:1504 writesame 0x837cf83 thru 0x837d8d5 data_size 0x7(0x953 bytes) 2024-05-11T23:01:44.554 INFO:teuthology.orchestra.run.smithi005.stdout:1505 read 0x25c9038 thru 0x25d2d8f (0x9d58 bytes) 2024-05-11T23:01:44.555 INFO:teuthology.orchestra.run.smithi005.stdout:1507 write 0x2262a0f thru 0x2265bf4 (0x31e6 bytes) 2024-05-11T23:01:44.564 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:44.564 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:44.565 INFO:teuthology.orchestra.run.smithi005.stdout:1510 writesame 0xa0cc9cd thru 0xa0d402c data_size 0x10(0x7660 bytes) 2024-05-11T23:01:44.565 INFO:teuthology.orchestra.run.smithi005.stdout:1511 punch from 0x72d5411 to 0x72dafcb, (0x5bba bytes) 2024-05-11T23:01:44.565 INFO:teuthology.orchestra.run.smithi005.stdout:1512 punch from 0x279f6dc to 0x27a9add, (0xa401 bytes) 2024-05-11T23:01:44.574 INFO:teuthology.orchestra.run.smithi005.stdout:1513 write 0xd36cd19 thru 0xd3743f9 (0x76e1 bytes) 2024-05-11T23:01:44.576 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:01:44.577 INFO:teuthology.orchestra.run.smithi005.stdout:1515 punch from 0xcaac7e8 to 0xcaaeaf2, (0x230a bytes) 2024-05-11T23:01:44.582 INFO:teuthology.orchestra.run.smithi005.stdout:1516 write 0xd50af98 thru 0xd51019c (0x5205 bytes) 2024-05-11T23:01:44.585 INFO:teuthology.orchestra.run.smithi005.stdout:1517 read 0x8c61bb7 thru 0x8c69de8 (0x8232 bytes) 2024-05-11T23:01:44.591 INFO:teuthology.orchestra.run.smithi005.stdout:1519 write 0xe23c8e9 thru 0xe24718d (0xa8a5 bytes) 2024-05-11T23:01:44.600 INFO:teuthology.orchestra.run.smithi005.stdout:1520 writesame 0xe99a24f thru 0xe9a6aeb data_size 0x13(0xc89d bytes) 2024-05-11T23:01:44.601 INFO:teuthology.orchestra.run.smithi005.stdout:1521 read 0x4b13d14 thru 0x4b22219 (0xe506 bytes) 2024-05-11T23:01:44.601 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:44.601 INFO:teuthology.orchestra.run.smithi005.stdout:1523 read 0xd8f726b thru 0xd8f88bb (0x1651 bytes) 2024-05-11T23:01:44.601 INFO:teuthology.orchestra.run.smithi005.stdout:1525 writesame 0xce5558f thru 0xce581b7 data_size 0x55(0x2c29 bytes) 2024-05-11T23:01:44.603 INFO:teuthology.orchestra.run.smithi005.stdout:1527 write 0x28dccff thru 0x28ec83e (0xfb40 bytes) 2024-05-11T23:01:44.607 INFO:teuthology.orchestra.run.smithi005.stdout:1529 read 0x49070f thru 0x497f8a (0x787c bytes) 2024-05-11T23:01:44.607 INFO:teuthology.orchestra.run.smithi005.stdout:1530 read 0x32186a8 thru 0x3223bbe (0xb517 bytes) 2024-05-11T23:01:44.607 INFO:teuthology.orchestra.run.smithi005.stdout:1531 writesame 0xafc252c thru 0xafd01fb data_size 0x4(0xdcd0 bytes) 2024-05-11T23:01:44.615 INFO:teuthology.orchestra.run.smithi005.stdout:1532 read 0x258e851 thru 0x2598543 (0x9cf3 bytes) 2024-05-11T23:01:44.615 INFO:teuthology.orchestra.run.smithi005.stdout:1533 writesame 0xb9ea75e thru 0xb9f2431 data_size 0x6(0x7cd4 bytes) 2024-05-11T23:01:44.620 INFO:teuthology.orchestra.run.smithi005.stdout:1536 read 0xc03b360 thru 0xc03b697 (0x338 bytes) 2024-05-11T23:01:44.623 INFO:teuthology.orchestra.run.smithi005.stdout:1537 trunc from 0xe9a6aec to 0xbd2f464 2024-05-11T23:01:44.629 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:44.628+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:44.633 INFO:teuthology.orchestra.run.smithi005.stdout:1538 write 0x7b630f0 thru 0x7b6b43c (0x834d bytes) 2024-05-11T23:01:44.636 INFO:teuthology.orchestra.run.smithi005.stdout:1539 write 0x6fe29af thru 0x6fef212 (0xc864 bytes) 2024-05-11T23:01:44.639 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:44.648 INFO:teuthology.orchestra.run.smithi005.stdout:1541 write 0xeabc240 thru 0xeac6526 (0xa2e7 bytes) 2024-05-11T23:01:44.653 INFO:teuthology.orchestra.run.smithi005.stdout:1542 punch from 0xa389d05 to 0xa38d963, (0x3c5e bytes) 2024-05-11T23:01:44.653 INFO:teuthology.orchestra.run.smithi005.stdout:1543 writesame 0xbf761c6 thru 0xbf7a030 data_size 0x13(0x3e6b bytes) 2024-05-11T23:01:44.655 INFO:teuthology.orchestra.run.smithi005.stdout:1545 write 0xa8424a6 thru 0xa845aa8 (0x3603 bytes) 2024-05-11T23:01:44.658 INFO:teuthology.orchestra.run.smithi005.stdout:1547 writesame 0x19562ff thru 0x195afd1 data_size 0x1(0x4cd3 bytes) 2024-05-11T23:01:44.671 INFO:teuthology.orchestra.run.smithi005.stdout:1548 write 0xc49cb93 thru 0xc49dd39 (0x11a7 bytes) 2024-05-11T23:01:44.674 INFO:teuthology.orchestra.run.smithi005.stdout:1549 writesame 0x2b0c4c5 thru 0x2b10c19 data_size 0x3(0x4755 bytes) 2024-05-11T23:01:44.678 INFO:teuthology.orchestra.run.smithi005.stdout:1551 write 0x8e77f76 thru 0x8e79d64 (0x1def bytes) 2024-05-11T23:01:44.696 INFO:teuthology.orchestra.run.smithi005.stdout:1552 trunc from 0xeac6527 to 0x30d3d16 2024-05-11T23:01:44.719 INFO:teuthology.orchestra.run.smithi005.stdout:1554 write 0x77c8e87 thru 0x77d3e97 (0xb011 bytes) 2024-05-11T23:01:44.719 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:44.719 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:44.722 INFO:teuthology.orchestra.run.smithi005.stdout:1555 read 0x6b6b30a thru 0x6b6f990 (0x4687 bytes) 2024-05-11T23:01:44.722 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:44.722 INFO:teuthology.orchestra.run.smithi005.stdout:1559 read 0x5f0b2e2 thru 0x5f1aa22 (0xf741 bytes) 2024-05-11T23:01:44.722 INFO:teuthology.orchestra.run.smithi005.stdout:1560 read 0x287d251 thru 0x288039f (0x314f bytes) 2024-05-11T23:01:44.722 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:44.729 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:44.729+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:44.738 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:01:44.738 INFO:teuthology.orchestra.run.smithi005.stdout:1562 write 0xedcd2e4 thru 0xedcdf2b (0xc48 bytes) 2024-05-11T23:01:44.741 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:44.741 INFO:teuthology.orchestra.run.smithi005.stdout:1564 punch from 0x8ff5265 to 0x90029ba, (0xd755 bytes) 2024-05-11T23:01:44.741 INFO:teuthology.orchestra.run.smithi005.stdout:1565 punch from 0x9929026 to 0x9930707, (0x76e1 bytes) 2024-05-11T23:01:44.741 INFO:teuthology.orchestra.run.smithi005.stdout:1566 writesame 0x900b80c thru 0x900bb49 data_size 0x5(0x33e bytes) 2024-05-11T23:01:44.743 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:44.743 INFO:teuthology.orchestra.run.smithi005.stdout:1570 punch from 0x7ef2008 to 0x7ef6460, (0x4458 bytes) 2024-05-11T23:01:44.743 INFO:teuthology.orchestra.run.smithi005.stdout:1572 write 0x7feaf64 thru 0x7feb960 (0x9fd bytes) 2024-05-11T23:01:44.746 INFO:teuthology.orchestra.run.smithi005.stdout:1573 punch from 0x4b6e9a6 to 0x4b7a919, (0xbf73 bytes) 2024-05-11T23:01:44.746 INFO:teuthology.orchestra.run.smithi005.stdout:1574 writesame 0x4296af7 thru 0x42a15c0 data_size 0x9(0xaaca bytes) 2024-05-11T23:01:44.750 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:44.750 INFO:teuthology.orchestra.run.smithi005.stdout:1577 punch from 0x683bc46 to 0x68489c7, (0xcd81 bytes) 2024-05-11T23:01:44.752 INFO:teuthology.orchestra.run.smithi005.stdout:1579 trunc from 0xedcdf2c to 0xc70d978 2024-05-11T23:01:44.767 INFO:teuthology.orchestra.run.smithi005.stdout:1580 trunc from 0xc70d978 to 0x9248dc8 2024-05-11T23:01:44.773 INFO:teuthology.orchestra.run.smithi005.stdout:1581 punch from 0x7e20c41 to 0x7e2cd3f, (0xc0fe bytes) 2024-05-11T23:01:44.782 INFO:teuthology.orchestra.run.smithi005.stdout:1582 write 0xc1adf04 thru 0xc1b7926 (0x9a23 bytes) 2024-05-11T23:01:44.785 INFO:teuthology.orchestra.run.smithi005.stdout:1583 read 0x3e363ba thru 0x3e3a700 (0x4347 bytes) 2024-05-11T23:01:44.792 INFO:teuthology.orchestra.run.smithi005.stdout:1584 write 0xdbd940c thru 0xdbe1a08 (0x85fd bytes) 2024-05-11T23:01:44.794 INFO:teuthology.orchestra.run.smithi005.stdout:1586 write 0xb5bc03b thru 0xb5ca494 (0xe45a bytes) 2024-05-11T23:01:44.798 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:01:44.798 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:01:44.798 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-11T23:01:44.798 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-11T23:01:44.799 INFO:teuthology.orchestra.run.smithi005.stdout:1587 write 0x2ca37ed thru 0x2cab503 (0x7d17 bytes) 2024-05-11T23:01:44.801 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:44.805 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-05-11T23:01:44.810 INFO:teuthology.orchestra.run.smithi005.stdout:1589 writesame 0xdd6c19b thru 0xdd6d02a data_size 0x8(0xe90 bytes) 2024-05-11T23:01:44.810 INFO:teuthology.orchestra.run.smithi005.stdout:1592 read 0x5b3a3fe thru 0x5b4025f (0x5e62 bytes) 2024-05-11T23:01:44.824 INFO:teuthology.orchestra.run.smithi005.stdout:1593 trunc from 0xdd6d02b to 0x1cc28d3 2024-05-11T23:01:44.859 INFO:teuthology.orchestra.run.smithi005.stdout:1594 write 0xb2b31ea thru 0xb2b3258 (0x6f bytes) 2024-05-11T23:01:44.859 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:44.859+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:44.862 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:44.862 INFO:teuthology.orchestra.run.smithi005.stdout:1597 write 0x2ae6aed thru 0x2aee505 (0x7a19 bytes) 2024-05-11T23:01:44.865 INFO:teuthology.orchestra.run.smithi005.stdout:1599 write 0x45515a7 thru 0x4554a4d (0x34a7 bytes) 2024-05-11T23:01:44.868 INFO:teuthology.orchestra.run.smithi005.stdout:1600 read 0x9f8e41e thru 0x9f97ab3 (0x9696 bytes) 2024-05-11T23:01:44.868 INFO:teuthology.orchestra.run.smithi005.stdout:1601 read 0x86ee50b thru 0x86f80bc (0x9bb2 bytes) 2024-05-11T23:01:44.868 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:44.868 INFO:teuthology.orchestra.run.smithi005.stdout:1606 read 0x325ee7a thru 0x32674eb (0x8672 bytes) 2024-05-11T23:01:44.868 INFO:teuthology.orchestra.run.smithi005.stdout:1608 write 0xa723ed7 thru 0xa72fe9a (0xbfc4 bytes) 2024-05-11T23:01:44.871 INFO:teuthology.orchestra.run.smithi005.stdout:1609 punch from 0x9e0fac1 to 0x9e15e62, (0x63a1 bytes) 2024-05-11T23:01:44.872 INFO:teuthology.orchestra.run.smithi005.stdout:1613 punch from 0x6799c7 to 0x6801b8, (0x67f1 bytes) 2024-05-11T23:01:44.872 INFO:teuthology.orchestra.run.smithi005.stdout:1614 read 0xa687f8c thru 0xa6936a4 (0xb719 bytes) 2024-05-11T23:01:44.872 INFO:teuthology.orchestra.run.smithi005.stdout:1619 write 0x6794c36 thru 0x679bec4 (0x728f bytes) 2024-05-11T23:01:44.874 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:44.874 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:44.881 INFO:teuthology.orchestra.run.smithi005.stdout:1620 writesame 0xbfc62f6 thru 0xbfcea9a data_size 0x19(0x87a5 bytes) 2024-05-11T23:01:44.881 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:44.881 INFO:teuthology.orchestra.run.smithi005.stdout:1622 read 0x804f95a thru 0x8055689 (0x5d30 bytes) 2024-05-11T23:01:44.881 INFO:teuthology.orchestra.run.smithi005.stdout:1625 write 0x99a9eb6 thru 0x99b1522 (0x766d bytes) 2024-05-11T23:01:44.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:01:44.889 INFO:teuthology.orchestra.run.smithi005.stdout:1626 trunc from 0xbfcea9b to 0x85dcc43 2024-05-11T23:01:44.903 INFO:teuthology.orchestra.run.smithi005.stdout:1627 trunc from 0x85dcc43 to 0xd55aa94 2024-05-11T23:01:44.909 INFO:teuthology.orchestra.run.smithi005.stdout:1628 write 0x57ad564 thru 0x57bce1a (0xf8b7 bytes) 2024-05-11T23:01:44.912 INFO:teuthology.orchestra.run.smithi005.stdout:1629 punch from 0xd2a2319 to 0xd2aefec, (0xccd3 bytes) 2024-05-11T23:01:44.912 INFO:teuthology.orchestra.run.smithi005.stdout:1631 writesame 0x90923c8 thru 0x909baf7 data_size 0x1(0x9730 bytes) 2024-05-11T23:01:44.931 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:44.931+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:44.932 INFO:teuthology.orchestra.run.smithi005.stdout:1633 writesame 0xc946865 thru 0xc94872d data_size 0x25(0x1ec9 bytes) 2024-05-11T23:01:44.936 INFO:teuthology.orchestra.run.smithi005.stdout:1635 punch from 0xa9bb8f8 to 0xa9bc73f, (0xe47 bytes) 2024-05-11T23:01:44.942 INFO:teuthology.orchestra.run.smithi005.stdout:1637 trunc from 0xd55aa94 to 0x8092631 2024-05-11T23:01:44.952 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:44.952 INFO:teuthology.orchestra.run.smithi005.stdout:1639 read 0x1ab8803 thru 0x1aba6b4 (0x1eb2 bytes) 2024-05-11T23:01:44.952 INFO:teuthology.orchestra.run.smithi005.stdout:1640 read 0x1f797da thru 0x1f856a9 (0xbed0 bytes) 2024-05-11T23:01:44.956 INFO:teuthology.orchestra.run.smithi005.stdout:1641 trunc from 0x8092631 to 0xb9cdfd1 2024-05-11T23:01:44.961 INFO:teuthology.orchestra.run.smithi005.stdout:1642 writesame 0x5a83322 thru 0x5a8f59c data_size 0x1(0xc27b bytes) 2024-05-11T23:01:44.984 INFO:teuthology.orchestra.run.smithi005.stdout:1643 punch from 0xa45c23c to 0xa465458, (0x921c bytes) 2024-05-11T23:01:44.984 INFO:teuthology.orchestra.run.smithi005.stdout:1644 read 0x6180728 thru 0x618d060 (0xc939 bytes) 2024-05-11T23:01:44.993 INFO:teuthology.orchestra.run.smithi005.stdout:1645 write 0xec9cf9e thru 0xeca8dac (0xbe0f bytes) 2024-05-11T23:01:44.996 INFO:teuthology.orchestra.run.smithi005.stdout:1646 write 0xe93a3a8 thru 0xe93cc56 (0x28af bytes) 2024-05-11T23:01:44.999 INFO:teuthology.orchestra.run.smithi005.stdout:1649 write 0x95081ea thru 0x950bf39 (0x3d50 bytes) 2024-05-11T23:01:45.003 INFO:teuthology.orchestra.run.smithi005.stdout:1650 write 0x89a2fa5 thru 0x89aa993 (0x79ef bytes) 2024-05-11T23:01:45.005 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:45.017 INFO:teuthology.orchestra.run.smithi005.stdout:1653 trunc from 0xeca8dad to 0x4501717 2024-05-11T23:01:45.031 INFO:teuthology.orchestra.run.smithi005.stdout:1656 trunc from 0x4501717 to 0x2248cdb 2024-05-11T23:01:45.032 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:45.031+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:45.041 INFO:teuthology.orchestra.run.smithi005.stdout:1657 trunc from 0x2248cdb to 0x2259c1f 2024-05-11T23:01:45.061 INFO:teuthology.orchestra.run.smithi005.stdout:1659 writesame 0xb77ddd7 thru 0xb780593 data_size 0x1(0x27bd bytes) 2024-05-11T23:01:45.062 INFO:teuthology.orchestra.run.smithi005.stdout:1660 write 0x5f0798 thru 0x5fa32f (0x9b98 bytes) 2024-05-11T23:01:45.065 INFO:teuthology.orchestra.run.smithi005.stdout:1661 read 0x3187ded thru 0x318db42 (0x5d56 bytes) 2024-05-11T23:01:45.065 INFO:teuthology.orchestra.run.smithi005.stdout:1662 read 0xa14739d thru 0xa152d3c (0xb9a0 bytes) 2024-05-11T23:01:45.065 INFO:teuthology.orchestra.run.smithi005.stdout:1663 read 0x35f24f1 thru 0x35f2793 (0x2a3 bytes) 2024-05-11T23:01:45.068 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:45.068 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:45.073 INFO:teuthology.orchestra.run.smithi005.stdout:1664 writesame 0xe01e67e thru 0xe02bd4e data_size 0x17(0xd6d1 bytes) 2024-05-11T23:01:45.073 INFO:teuthology.orchestra.run.smithi005.stdout:1665 write 0x8f63745 thru 0x8f68eed (0x57a9 bytes) 2024-05-11T23:01:45.077 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:45.077 INFO:teuthology.orchestra.run.smithi005.stdout:1670 read 0x4343a5d thru 0x434516b (0x170f bytes) 2024-05-11T23:01:45.077 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:45.077 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:45.077 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:45.077 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:45.077 INFO:teuthology.orchestra.run.smithi005.stdout:1676 write 0x7e3c7fc thru 0x7e43b4a (0x734f bytes) 2024-05-11T23:01:45.079 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:01:45.081 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:45.081 INFO:teuthology.orchestra.run.smithi005.stdout:1678 write 0xc39a79b thru 0xc3a1ff5 (0x785b bytes) 2024-05-11T23:01:45.084 INFO:teuthology.orchestra.run.smithi005.stdout:1679 read 0xa74f7a8 thru 0xa756155 (0x69ae bytes) 2024-05-11T23:01:45.094 INFO:teuthology.orchestra.run.smithi005.stdout:1681 trunc from 0xe02bd4f to 0x4d8b546 2024-05-11T23:01:45.113 INFO:teuthology.orchestra.run.smithi005.stdout:1682 write 0x7c60663 thru 0x7c6ca12 (0xc3b0 bytes) 2024-05-11T23:01:45.121 INFO:teuthology.orchestra.run.smithi005.stdout:1684 trunc from 0x7c6ca13 to 0xb6e3490 2024-05-11T23:01:45.131 INFO:teuthology.orchestra.run.smithi005.stdout:1685 trunc from 0xb6e3490 to 0x6e5da5f 2024-05-11T23:01:45.133 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:45.133+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:45.140 INFO:teuthology.orchestra.run.smithi005.stdout:1686 read 0x515355e thru 0x515c195 (0x8c38 bytes) 2024-05-11T23:01:45.141 INFO:teuthology.orchestra.run.smithi005.stdout:1688 punch from 0x6367d9b to 0x6369356, (0x15bb bytes) 2024-05-11T23:01:45.141 INFO:teuthology.orchestra.run.smithi005.stdout:1689 read 0x55191cb thru 0x552696f (0xd7a5 bytes) 2024-05-11T23:01:45.141 INFO:teuthology.orchestra.run.smithi005.stdout:1690 write 0x208e14f thru 0x209a4e7 (0xc399 bytes) 2024-05-11T23:01:45.143 INFO:teuthology.orchestra.run.smithi005.stdout:1692 writesame 0x4b9f3f3 thru 0x4bab9a1 data_size 0x3(0xc5af bytes) 2024-05-11T23:01:45.153 INFO:teuthology.orchestra.run.smithi005.stdout:1693 write 0x436c71e thru 0x436fb30 (0x3413 bytes) 2024-05-11T23:01:45.156 INFO:teuthology.orchestra.run.smithi005.stdout:1694 read 0x2ccc082 thru 0x2ccd226 (0x11a5 bytes) 2024-05-11T23:01:45.156 INFO:teuthology.orchestra.run.smithi005.stdout:1695 punch from 0x6302577 to 0x6302e48, (0x8d1 bytes) 2024-05-11T23:01:45.156 INFO:teuthology.orchestra.run.smithi005.stdout:1697 read 0x2889c90 thru 0x2895e74 (0xc1e5 bytes) 2024-05-11T23:01:45.167 INFO:teuthology.orchestra.run.smithi005.stdout:1698 write 0xc2ff0a5 thru 0xc30b32e (0xc28a bytes) 2024-05-11T23:01:45.172 INFO:teuthology.orchestra.run.smithi005.stdout:1700 writesame 0x7969c2d thru 0x7973450 data_size 0x7(0x9824 bytes) 2024-05-11T23:01:45.177 INFO:teuthology.orchestra.run.smithi005.stdout:1701 read 0x37f5f7a thru 0x3803413 (0xd49a bytes) 2024-05-11T23:01:45.177 INFO:teuthology.orchestra.run.smithi005.stdout:1702 read 0x93b0775 thru 0x93bfb97 (0xf423 bytes) 2024-05-11T23:01:45.177 INFO:teuthology.orchestra.run.smithi005.stdout:1703 read 0xc2f2a12 thru 0xc2f98b2 (0x6ea1 bytes) 2024-05-11T23:01:45.177 INFO:teuthology.orchestra.run.smithi005.stdout:1704 write 0x4653012 thru 0x46558e8 (0x28d7 bytes) 2024-05-11T23:01:45.180 INFO:teuthology.orchestra.run.smithi005.stdout:1706 write 0x6931524 thru 0x693152d (0xa bytes) 2024-05-11T23:01:45.190 INFO:teuthology.orchestra.run.smithi005.stdout:1707 trunc from 0xc30b32f to 0x5bf436c 2024-05-11T23:01:45.200 INFO:teuthology.orchestra.run.smithi005.stdout:1709 write 0x34bf74d thru 0x34c99dc (0xa290 bytes) 2024-05-11T23:01:45.209 INFO:teuthology.orchestra.run.smithi005.stdout:1710 write 0x5ff84a5 thru 0x5ffe024 (0x5b80 bytes) 2024-05-11T23:01:45.213 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:45.213 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:45.217 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:01:45.217 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:45.217 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 34, 2024-05-11T23:01:45.217 INFO:teuthology.orchestra.run.smithi005.stdout: "fsid": "34b20022-f11b-4169-a5f6-6386270a2f8f", 2024-05-11T23:01:45.217 INFO:teuthology.orchestra.run.smithi005.stdout: "created": "2024-05-11T23:01:05.977693+0000", 2024-05-11T23:01:45.217 INFO:teuthology.orchestra.run.smithi005.stdout: "modified": "2024-05-11T23:01:44.798434+0000", 2024-05-11T23:01:45.217 INFO:teuthology.orchestra.run.smithi005.stdout: "last_up_change": "2024-05-11T23:01:17.148916+0000", 2024-05-11T23:01:45.217 INFO:teuthology.orchestra.run.smithi005.stdout: "last_in_change": "2024-05-11T23:01:08.985275+0000", 2024-05-11T23:01:45.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-11T23:01:45.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_num": 5804032, 2024-05-11T23:01:45.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_set": [ 2024-05-11T23:01:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: "nodeep-scrub", 2024-05-11T23:01:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: "pglog_hardlimit", 2024-05-11T23:01:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: "purged_snapdirs", 2024-05-11T23:01:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_deletes", 2024-05-11T23:01:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: "sortbitwise" 2024-05-11T23:01:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:01:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_version": 5, 2024-05-11T23:01:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: "full_ratio": 0.94999998807907104, 2024-05-11T23:01:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-11T23:01:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-11T23:01:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_snapshot": "", 2024-05-11T23:01:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_max": 3, 2024-05-11T23:01:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd": 8, 2024-05-11T23:01:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: "require_min_compat_client": "luminous", 2024-05-11T23:01:45.219 INFO:teuthology.orchestra.run.smithi005.stdout: "min_compat_client": "jewel", 2024-05-11T23:01:45.219 INFO:teuthology.orchestra.run.smithi005.stdout: "require_osd_release": "squid", 2024-05-11T23:01:45.219 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_crimson": false, 2024-05-11T23:01:45.219 INFO:teuthology.orchestra.run.smithi005.stdout: "pools": [ 2024-05-11T23:01:45.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.219 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 1, 2024-05-11T23:01:45.219 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": ".mgr", 2024-05-11T23:01:45.219 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-05-11T23:01:15.187614+0000", 2024-05-11T23:01:45.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-11T23:01:45.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool", 2024-05-11T23:01:45.219 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-05-11T23:01:45.219 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-05-11T23:01:45.219 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-05-11T23:01:45.219 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-05-11T23:01:45.219 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-05-11T23:01:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-05-11T23:01:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-05-11T23:01:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-11T23:01:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-05-11T23:01:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-05-11T23:01:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 1, 2024-05-11T23:01:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 1, 2024-05-11T23:01:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 1, 2024-05-11T23:01:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 1, 2024-05-11T23:01:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 1, 2024-05-11T23:01:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-05-11T23:01:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-05-11T23:01:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-05-11T23:01:45.220 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-05-11T23:01:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-05-11T23:01:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-05-11T23:01:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-05-11T23:01:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "17", 2024-05-11T23:01:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-05-11T23:01:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-11T23:01:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-05-11T23:01:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-05-11T23:01:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-05-11T23:01:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 0, 2024-05-11T23:01:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 0, 2024-05-11T23:01:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-05-11T23:01:45.221 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-05-11T23:01:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-05-11T23:01:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-05-11T23:01:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-05-11T23:01:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-05-11T23:01:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-05-11T23:01:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-05-11T23:01:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-05-11T23:01:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-05-11T23:01:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-05-11T23:01:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-11T23:01:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-11T23:01:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-11T23:01:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-05-11T23:01:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-05-11T23:01:45.222 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-05-11T23:01:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-05-11T23:01:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-05-11T23:01:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-05-11T23:01:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-05-11T23:01:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-05-11T23:01:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-05-11T23:01:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-05-11T23:01:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-05-11T23:01:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-05-11T23:01:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-05-11T23:01:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-05-11T23:01:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-05-11T23:01:45.223 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-05-11T23:01:45.224 INFO:teuthology.orchestra.run.smithi005.stdout: "options": { 2024-05-11T23:01:45.224 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_max": 32, 2024-05-11T23:01:45.224 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_min": 1 2024-05-11T23:01:45.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.224 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-05-11T23:01:45.224 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": {} 2024-05-11T23:01:45.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.224 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-05-11T23:01:45.224 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-05-11T23:01:45.224 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 8, 2024-05-11T23:01:45.224 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 8, 2024-05-11T23:01:45.224 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 0.25, 2024-05-11T23:01:45.224 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2, 2024-05-11T23:01:45.224 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2, 2024-05-11T23:01:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 1, 2024-05-11T23:01:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 1, 2024-05-11T23:01:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 1 2024-05-11T23:01:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 2, 2024-05-11T23:01:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "rbd", 2024-05-11T23:01:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-05-11T23:01:17.689285+0000", 2024-05-11T23:01:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-11T23:01:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool", 2024-05-11T23:01:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-05-11T23:01:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-05-11T23:01:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-05-11T23:01:45.225 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-05-11T23:01:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-05-11T23:01:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-05-11T23:01:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-05-11T23:01:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-11T23:01:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-05-11T23:01:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-05-11T23:01:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 8, 2024-05-11T23:01:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 8, 2024-05-11T23:01:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 8, 2024-05-11T23:01:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 8, 2024-05-11T23:01:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 8, 2024-05-11T23:01:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-05-11T23:01:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-05-11T23:01:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-05-11T23:01:45.226 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-05-11T23:01:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-05-11T23:01:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-05-11T23:01:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-05-11T23:01:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "18", 2024-05-11T23:01:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-05-11T23:01:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-11T23:01:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-05-11T23:01:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-05-11T23:01:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-05-11T23:01:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 0, 2024-05-11T23:01:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 0, 2024-05-11T23:01:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-05-11T23:01:45.227 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-05-11T23:01:45.228 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-05-11T23:01:45.228 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-05-11T23:01:45.228 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-05-11T23:01:45.228 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-05-11T23:01:45.228 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-05-11T23:01:45.228 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-05-11T23:01:45.228 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-05-11T23:01:45.228 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-05-11T23:01:45.228 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-05-11T23:01:45.228 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-11T23:01:45.228 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-11T23:01:45.228 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-11T23:01:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-05-11T23:01:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-05-11T23:01:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-05-11T23:01:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-05-11T23:01:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-05-11T23:01:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-05-11T23:01:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-05-11T23:01:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-05-11T23:01:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-05-11T23:01:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-05-11T23:01:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-05-11T23:01:45.229 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-05-11T23:01:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-05-11T23:01:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-05-11T23:01:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-05-11T23:01:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-05-11T23:01:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-05-11T23:01:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-05-11T23:01:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-05-11T23:01:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-05-11T23:01:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-05-11T23:01:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 1.9900000095367432, 2024-05-11T23:01:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 1.9900000095367432, 2024-05-11T23:01:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 0.87999999523162842, 2024-05-11T23:01:45.230 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 1.75, 2024-05-11T23:01:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 1.75, 2024-05-11T23:01:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 1, 2024-05-11T23:01:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 1, 2024-05-11T23:01:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 1 2024-05-11T23:01:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 3, 2024-05-11T23:01:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "pool_client.0", 2024-05-11T23:01:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-05-11T23:01:25.743347+0000", 2024-05-11T23:01:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-05-11T23:01:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-11T23:01:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-05-11T23:01:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-05-11T23:01:45.231 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-05-11T23:01:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-05-11T23:01:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-05-11T23:01:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-05-11T23:01:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-05-11T23:01:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-11T23:01:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-05-11T23:01:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-05-11T23:01:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 32, 2024-05-11T23:01:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 32, 2024-05-11T23:01:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 32, 2024-05-11T23:01:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 32, 2024-05-11T23:01:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 32, 2024-05-11T23:01:45.232 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-05-11T23:01:45.233 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-05-11T23:01:45.233 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-05-11T23:01:45.233 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-05-11T23:01:45.233 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-05-11T23:01:45.233 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-05-11T23:01:45.233 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-05-11T23:01:45.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.233 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "33", 2024-05-11T23:01:45.233 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-05-11T23:01:45.233 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-11T23:01:45.233 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-05-11T23:01:45.233 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-05-11T23:01:45.233 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-05-11T23:01:45.233 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 12, 2024-05-11T23:01:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 33, 2024-05-11T23:01:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-05-11T23:01:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-05-11T23:01:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-05-11T23:01:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-05-11T23:01:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-05-11T23:01:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-05-11T23:01:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-05-11T23:01:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-05-11T23:01:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-05-11T23:01:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-05-11T23:01:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-05-11T23:01:45.234 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-11T23:01:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-11T23:01:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-11T23:01:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-05-11T23:01:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-05-11T23:01:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-05-11T23:01:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-05-11T23:01:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-05-11T23:01:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-05-11T23:01:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-05-11T23:01:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-05-11T23:01:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-05-11T23:01:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-05-11T23:01:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-05-11T23:01:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-05-11T23:01:45.235 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-05-11T23:01:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-05-11T23:01:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-05-11T23:01:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-05-11T23:01:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-05-11T23:01:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-05-11T23:01:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-05-11T23:01:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-05-11T23:01:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-05-11T23:01:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 1.5, 2024-05-11T23:01:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 1.5, 2024-05-11T23:01:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 1, 2024-05-11T23:01:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 1.5, 2024-05-11T23:01:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 1.5, 2024-05-11T23:01:45.236 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 1, 2024-05-11T23:01:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 1, 2024-05-11T23:01:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 1 2024-05-11T23:01:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:01:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: "osds": [ 2024-05-11T23:01:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-05-11T23:01:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "57ab1c70-9478-4a26-bb41-d753b4f3eb79", 2024-05-11T23:01:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:01:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:01:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 0.95001220703125, 2024-05-11T23:01:45.237 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:01:45.238 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:01:45.238 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:01:45.238 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 13, 2024-05-11T23:01:45.238 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 19, 2024-05-11T23:01:45.238 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:01:45.238 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:01:45.238 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:01:45.238 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.238 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-05-11T23:01:45.238 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:01:45.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-05-11T23:01:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:01:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:01:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6802", 2024-05-11T23:01:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:01:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6803", 2024-05-11T23:01:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:01:45.239 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:01:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6806", 2024-05-11T23:01:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:01:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6808", 2024-05-11T23:01:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:01:45.240 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:01:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6804", 2024-05-11T23:01:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:01:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6805", 2024-05-11T23:01:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:01:45.241 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6801/3452725643", 2024-05-11T23:01:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6803/3452725643", 2024-05-11T23:01:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6808/3452725643", 2024-05-11T23:01:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6805/3452725643", 2024-05-11T23:01:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:01:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:01:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:01:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-05-11T23:01:45.242 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "05ece110-4a99-46f0-b061-5980f7ef2a55", 2024-05-11T23:01:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:01:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:01:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 0.95001220703125, 2024-05-11T23:01:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:01:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:01:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:01:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 14, 2024-05-11T23:01:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 19, 2024-05-11T23:01:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:01:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:01:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:01:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.244 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6824", 2024-05-11T23:01:45.244 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:01:45.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.244 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6825", 2024-05-11T23:01:45.244 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:01:45.244 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.244 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:01:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6826", 2024-05-11T23:01:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:01:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6827", 2024-05-11T23:01:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:01:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.245 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:01:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6830", 2024-05-11T23:01:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:01:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6831", 2024-05-11T23:01:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:01:45.246 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.247 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.247 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:01:45.247 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.247 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.247 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6828", 2024-05-11T23:01:45.247 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:01:45.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.247 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.247 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6829", 2024-05-11T23:01:45.247 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:01:45.248 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.249 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.249 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6825/4028727138", 2024-05-11T23:01:45.249 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6827/4028727138", 2024-05-11T23:01:45.249 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6831/4028727138", 2024-05-11T23:01:45.249 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6829/4028727138", 2024-05-11T23:01:45.249 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:01:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:01:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:01:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-05-11T23:01:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5", 2024-05-11T23:01:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:01:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:01:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:01:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:01:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:01:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:01:45.250 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 13, 2024-05-11T23:01:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 16, 2024-05-11T23:01:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:01:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:01:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:01:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6807", 2024-05-11T23:01:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:01:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6809", 2024-05-11T23:01:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:01:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.251 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:01:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6810", 2024-05-11T23:01:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:01:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6811", 2024-05-11T23:01:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:01:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.252 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:01:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6814", 2024-05-11T23:01:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:01:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6815", 2024-05-11T23:01:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:01:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:01:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6812", 2024-05-11T23:01:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:01:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6813", 2024-05-11T23:01:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:01:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6809/2059959509", 2024-05-11T23:01:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6811/2059959509", 2024-05-11T23:01:45.254 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6815/2059959509", 2024-05-11T23:01:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6813/2059959509", 2024-05-11T23:01:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:01:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:01:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:01:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-05-11T23:01:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a", 2024-05-11T23:01:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:01:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:01:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 0.9000244140625, 2024-05-11T23:01:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:01:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:01:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:01:45.255 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 14, 2024-05-11T23:01:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 19, 2024-05-11T23:01:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:01:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:01:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:01:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6816", 2024-05-11T23:01:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:01:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6817", 2024-05-11T23:01:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:01:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:01:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6818", 2024-05-11T23:01:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:01:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6819", 2024-05-11T23:01:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:01:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:01:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6822", 2024-05-11T23:01:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:01:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.258 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6823", 2024-05-11T23:01:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:01:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:01:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6820", 2024-05-11T23:01:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:01:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6821", 2024-05-11T23:01:45.259 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:01:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6817/296930146", 2024-05-11T23:01:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6819/296930146", 2024-05-11T23:01:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6823/296930146", 2024-05-11T23:01:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6821/296930146", 2024-05-11T23:01:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:01:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:01:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:01:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-05-11T23:01:45.260 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "4e87e4f0-288e-4ee6-919e-9337925a5d5e", 2024-05-11T23:01:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:01:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:01:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:01:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:01:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:01:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:01:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 15, 2024-05-11T23:01:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 29, 2024-05-11T23:01:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:01:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:01:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:01:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6800", 2024-05-11T23:01:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:01:45.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6801", 2024-05-11T23:01:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:01:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:01:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6802", 2024-05-11T23:01:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:01:45.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6803", 2024-05-11T23:01:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:01:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:01:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6806", 2024-05-11T23:01:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:01:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6807", 2024-05-11T23:01:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:01:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:01:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6804", 2024-05-11T23:01:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:01:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6805", 2024-05-11T23:01:45.264 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:01:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.115:6801/495703077", 2024-05-11T23:01:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.115:6803/495703077", 2024-05-11T23:01:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.115:6807/495703077", 2024-05-11T23:01:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.115:6805/495703077", 2024-05-11T23:01:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:01:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:01:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:01:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-05-11T23:01:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5", 2024-05-11T23:01:45.265 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:01:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:01:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:01:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:01:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:01:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:01:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 15, 2024-05-11T23:01:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 19, 2024-05-11T23:01:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:01:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:01:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:01:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.266 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6811", 2024-05-11T23:01:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:01:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6813", 2024-05-11T23:01:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:01:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:01:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6815", 2024-05-11T23:01:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:01:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6817", 2024-05-11T23:01:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:01:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:01:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.268 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6822", 2024-05-11T23:01:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:01:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6823", 2024-05-11T23:01:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:01:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:01:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6819", 2024-05-11T23:01:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:01:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6821", 2024-05-11T23:01:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:01:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.115:6813/902573489", 2024-05-11T23:01:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.115:6817/902573489", 2024-05-11T23:01:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.115:6823/902573489", 2024-05-11T23:01:45.270 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.115:6821/902573489", 2024-05-11T23:01:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:01:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:01:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:01:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 6, 2024-05-11T23:01:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "edb686ec-e1a2-40d6-975a-e32e0e7760ff", 2024-05-11T23:01:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:01:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:01:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:01:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:01:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:01:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:01:45.271 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 15, 2024-05-11T23:01:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 19, 2024-05-11T23:01:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:01:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:01:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:01:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6824", 2024-05-11T23:01:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:01:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6825", 2024-05-11T23:01:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:01:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:01:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6826", 2024-05-11T23:01:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:01:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6827", 2024-05-11T23:01:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:01:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.273 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:01:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6830", 2024-05-11T23:01:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:01:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6831", 2024-05-11T23:01:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:01:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:01:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6828", 2024-05-11T23:01:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:01:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6829", 2024-05-11T23:01:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:01:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.275 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.115:6825/2749332272", 2024-05-11T23:01:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.115:6827/2749332272", 2024-05-11T23:01:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.115:6831/2749332272", 2024-05-11T23:01:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.115:6829/2749332272", 2024-05-11T23:01:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:01:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:01:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:01:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 7, 2024-05-11T23:01:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "7aa14dbb-c6cb-4540-b160-3fd346f6a046", 2024-05-11T23:01:45.276 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:01:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:01:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:01:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:01:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:01:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:01:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 15, 2024-05-11T23:01:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 19, 2024-05-11T23:01:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:01:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:01:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:01:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6808", 2024-05-11T23:01:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:01:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6809", 2024-05-11T23:01:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:01:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:01:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.278 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6810", 2024-05-11T23:01:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:01:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6812", 2024-05-11T23:01:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:01:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:01:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.279 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6818", 2024-05-11T23:01:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:01:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6820", 2024-05-11T23:01:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:01:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:01:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6814", 2024-05-11T23:01:45.280 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:01:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6816", 2024-05-11T23:01:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:01:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.115:6809/1043826337", 2024-05-11T23:01:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.115:6812/1043826337", 2024-05-11T23:01:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.115:6820/1043826337", 2024-05-11T23:01:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.115:6816/1043826337", 2024-05-11T23:01:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:01:45.281 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:01:45.282 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:01:45.282 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:45.282 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.282 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:01:45.282 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_xinfo": [ 2024-05-11T23:01:45.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.282 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-05-11T23:01:45.282 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:01:45.282 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:01:45.282 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:01:45.282 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:01:45.282 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:01:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.113232+0000", 2024-05-11T23:01:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:01:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-05-11T23:01:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:01:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:01:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:01:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:01:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:01:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.174679+0000", 2024-05-11T23:01:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:01:45.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-05-11T23:01:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:01:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:01:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:01:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:01:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:01:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.131118+0000", 2024-05-11T23:01:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:01:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-05-11T23:01:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:01:45.285 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:01:45.285 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:01:45.285 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:01:45.285 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:01:45.285 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.207668+0000", 2024-05-11T23:01:45.285 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:01:45.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.285 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-05-11T23:01:45.285 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:01:45.285 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:01:45.285 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:01:45.286 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:01:45.286 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:01:45.286 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.124542+0000", 2024-05-11T23:01:45.286 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:01:45.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.286 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-05-11T23:01:45.286 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:01:45.286 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:01:45.286 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:01:45.286 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:01:45.286 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:01:45.286 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.200832+0000", 2024-05-11T23:01:45.287 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:01:45.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.287 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 6, 2024-05-11T23:01:45.287 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:01:45.287 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:01:45.287 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:01:45.287 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:01:45.287 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:01:45.287 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.169560+0000", 2024-05-11T23:01:45.287 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:01:45.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 7, 2024-05-11T23:01:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:01:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:01:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:01:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:01:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:01:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.182465+0000", 2024-05-11T23:01:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:01:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:01:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap": [], 2024-05-11T23:01:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_items": [], 2024-05-11T23:01:45.288 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_primaries": [], 2024-05-11T23:01:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_temp": [], 2024-05-11T23:01:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_temp": [], 2024-05-11T23:01:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklist": {}, 2024-05-11T23:01:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "range_blocklist": {}, 2024-05-11T23:01:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profiles": { 2024-05-11T23:01:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "default": { 2024-05-11T23:01:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-failure-domain": "osd", 2024-05-11T23:01:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "k": "2", 2024-05-11T23:01:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "m": "1", 2024-05-11T23:01:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "plugin": "jerasure", 2024-05-11T23:01:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "technique": "reed_sol_van" 2024-05-11T23:01:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps_queue": [], 2024-05-11T23:01:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "new_removed_snaps": [], 2024-05-11T23:01:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: "new_purged_snaps": [], 2024-05-11T23:01:45.290 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_node_flags": {}, 2024-05-11T23:01:45.290 INFO:teuthology.orchestra.run.smithi005.stdout: "device_class_flags": {}, 2024-05-11T23:01:45.290 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode": { 2024-05-11T23:01:45.290 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_enabled": false, 2024-05-11T23:01:45.290 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_bucket_count": 0, 2024-05-11T23:01:45.290 INFO:teuthology.orchestra.run.smithi005.stdout: "degraded_stretch_mode": 0, 2024-05-11T23:01:45.290 INFO:teuthology.orchestra.run.smithi005.stdout: "recovering_stretch_mode": 0, 2024-05-11T23:01:45.290 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_bucket": 0 2024-05-11T23:01:45.290 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:45.290 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:45.291 INFO:teuthology.orchestra.run.smithi005.stdout:1711 write 0x9598ef4 thru 0x95a36e0 (0xa7ed bytes) 2024-05-11T23:01:45.292 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:01:45.292 INFO:teuthology.orchestra.run.smithi005.stdout:1712 write 0xacda8ac thru 0xace45b5 (0x9d0a bytes) 2024-05-11T23:01:45.294 INFO:tasks.thrashosds.thrasher:j is {'epoch': 34, 'fsid': '34b20022-f11b-4169-a5f6-6386270a2f8f', 'created': '2024-05-11T23:01:05.977693+0000', 'modified': '2024-05-11T23:01:44.798434+0000', 'last_up_change': '2024-05-11T23:01:17.148916+0000', 'last_in_change': '2024-05-11T23:01:08.985275+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-11T23:01:15.187614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-05-11T23:01:17.689285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-11T23:01:25.743347+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 12, 'snap_epoch': 33, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '57ab1c70-9478-4a26-bb41-d753b4f3eb79', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6800', 'nonce': 3452725643}, {'type': 'v1', 'addr': '172.21.15.5:6801', 'nonce': 3452725643}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6802', 'nonce': 3452725643}, {'type': 'v1', 'addr': '172.21.15.5:6803', 'nonce': 3452725643}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6806', 'nonce': 3452725643}, {'type': 'v1', 'addr': '172.21.15.5:6808', 'nonce': 3452725643}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6804', 'nonce': 3452725643}, {'type': 'v1', 'addr': '172.21.15.5:6805', 'nonce': 3452725643}]}, 'public_addr': '172.21.15.5:6801/3452725643', 'cluster_addr': '172.21.15.5:6803/3452725643', 'heartbeat_back_addr': '172.21.15.5:6808/3452725643', 'heartbeat_front_addr': '172.21.15.5:6805/3452725643', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '05ece110-4a99-46f0-b061-5980f7ef2a55', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6824', 'nonce': 4028727138}, {'type': 'v1', 'addr': '172.21.15.5:6825', 'nonce': 4028727138}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6826', 'nonce': 4028727138}, {'type': 'v1', 'addr': '172.21.15.5:6827', 'nonce': 4028727138}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6830', 'nonce': 4028727138}, {'type': 'v1', 'addr': '172.21.15.5:6831', 'nonce': 4028727138}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6828', 'nonce': 4028727138}, {'type': 'v1', 'addr': '172.21.15.5:6829', 'nonce': 4028727138}]}, 'public_addr': '172.21.15.5:6825/4028727138', 'cluster_addr': '172.21.15.5:6827/4028727138', 'heartbeat_back_addr': '172.21.15.5:6831/4028727138', 'heartbeat_front_addr': '172.21.15.5:6829/4028727138', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 16, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6807', 'nonce': 2059959509}, {'type': 'v1', 'addr': '172.21.15.5:6809', 'nonce': 2059959509}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6810', 'nonce': 2059959509}, {'type': 'v1', 'addr': '172.21.15.5:6811', 'nonce': 2059959509}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6814', 'nonce': 2059959509}, {'type': 'v1', 'addr': '172.21.15.5:6815', 'nonce': 2059959509}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6812', 'nonce': 2059959509}, {'type': 'v1', 'addr': '172.21.15.5:6813', 'nonce': 2059959509}]}, 'public_addr': '172.21.15.5:6809/2059959509', 'cluster_addr': '172.21.15.5:6811/2059959509', 'heartbeat_back_addr': '172.21.15.5:6815/2059959509', 'heartbeat_front_addr': '172.21.15.5:6813/2059959509', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a', 'up': 1, 'in': 1, 'weight': 0.9000244140625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6816', 'nonce': 296930146}, {'type': 'v1', 'addr': '172.21.15.5:6817', 'nonce': 296930146}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6818', 'nonce': 296930146}, {'type': 'v1', 'addr': '172.21.15.5:6819', 'nonce': 296930146}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6822', 'nonce': 296930146}, {'type': 'v1', 'addr': '172.21.15.5:6823', 'nonce': 296930146}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6820', 'nonce': 296930146}, {'type': 'v1', 'addr': '172.21.15.5:6821', 'nonce': 296930146}]}, 'public_addr': '172.21.15.5:6817/296930146', 'cluster_addr': '172.21.15.5:6819/296930146', 'heartbeat_back_addr': '172.21.15.5:6823/296930146', 'heartbeat_front_addr': '172.21.15.5:6821/296930146', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '4e87e4f0-288e-4ee6-919e-9337925a5d5e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 29, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 495703077}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 495703077}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 495703077}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 495703077}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 495703077}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 495703077}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 495703077}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 495703077}]}, 'public_addr': '172.21.15.115:6801/495703077', 'cluster_addr': '172.21.15.115:6803/495703077', 'heartbeat_back_addr': '172.21.15.115:6807/495703077', 'heartbeat_front_addr': '172.21.15.115:6805/495703077', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6811', 'nonce': 902573489}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 902573489}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6815', 'nonce': 902573489}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 902573489}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 902573489}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 902573489}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6819', 'nonce': 902573489}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 902573489}]}, 'public_addr': '172.21.15.115:6813/902573489', 'cluster_addr': '172.21.15.115:6817/902573489', 'heartbeat_back_addr': '172.21.15.115:6823/902573489', 'heartbeat_front_addr': '172.21.15.115:6821/902573489', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'edb686ec-e1a2-40d6-975a-e32e0e7760ff', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6824', 'nonce': 2749332272}, {'type': 'v1', 'addr': '172.21.15.115:6825', 'nonce': 2749332272}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 2749332272}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 2749332272}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 2749332272}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 2749332272}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 2749332272}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 2749332272}]}, 'public_addr': '172.21.15.115:6825/2749332272', 'cluster_addr': '172.21.15.115:6827/2749332272', 'heartbeat_back_addr': '172.21.15.115:6831/2749332272', 'heartbeat_front_addr': '172.21.15.115:6829/2749332272', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '7aa14dbb-c6cb-4540-b160-3fd346f6a046', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 1043826337}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 1043826337}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 1043826337}, {'type': 'v1', 'addr': '172.21.15.115:6812', 'nonce': 1043826337}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 1043826337}, {'type': 'v1', 'addr': '172.21.15.115:6820', 'nonce': 1043826337}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 1043826337}, {'type': 'v1', 'addr': '172.21.15.115:6816', 'nonce': 1043826337}]}, 'public_addr': '172.21.15.115:6809/1043826337', 'cluster_addr': '172.21.15.115:6812/1043826337', 'heartbeat_back_addr': '172.21.15.115:6820/1043826337', 'heartbeat_front_addr': '172.21.15.115:6816/1043826337', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.113232+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.174679+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.131118+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.207668+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.124542+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.200832+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.169560+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.182465+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-11T23:01:45.294 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-05-11T23:01:45.295 INFO:teuthology.orchestra.run.smithi005.stdout:1713 read 0x19599ad thru 0x195a7a7 (0xdfb bytes) 2024-05-11T23:01:45.295 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout:1717 write 0xbb3746d thru 0xbb40094 (0x8c28 bytes) 2024-05-11T23:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout:1718 write 0x162bc61 thru 0x1635bb4 (0x9f54 bytes) 2024-05-11T23:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout:1720 punch from 0x8f0f148 to 0x8f1ed44, (0xfbfc bytes) 2024-05-11T23:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout:1721 write 0x42864d9 thru 0x428ec72 (0x879a bytes) 2024-05-11T23:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout:1722 flatten 2024-05-11T23:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout:1723 read 0xb940945 thru 0xb94502a (0x46e6 bytes) 2024-05-11T23:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout:1724 write 0x341a438 thru 0x3422870 (0x8439 bytes) 2024-05-11T23:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout:1725 writesame 0x9b59488 thru 0x9b61e8c data_size 0x1(0x8a05 bytes) 2024-05-11T23:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout:1727 read 0x18aea5e thru 0x18be903 (0xfea6 bytes) 2024-05-11T23:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout:1729 punch from 0x2fe3215 to 0x2ff3050, (0xfe3b bytes) 2024-05-11T23:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout:1730 punch from 0x6118cea to 0x611a204, (0x151a bytes) 2024-05-11T23:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout:1734 read 0x8069cd1 thru 0x806cdc0 (0x30f0 bytes) 2024-05-11T23:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout:1736 write 0x3100466 thru 0x3103118 (0x2cb3 bytes) 2024-05-11T23:01:45.296 INFO:teuthology.orchestra.run.smithi005.stdout:1737 writesame 0x95daf66 thru 0x95e4f08 data_size 0x1(0x9fa3 bytes) 2024-05-11T23:01:45.297 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:45.291+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:45.309 INFO:teuthology.orchestra.run.smithi005.stdout:1738 trunc from 0xbb40095 to 0x26a524f 2024-05-11T23:01:45.320 INFO:teuthology.orchestra.run.smithi005.stdout:1742 clone 11 order 19 su 8192 sc 14 2024-05-11T23:01:45.392 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:45.392+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:45.420 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:45.420 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:45.429 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:01:45.494 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:45.494+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:45.579 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:45.580 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:45.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:01:45.633 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:45.633+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:45.695 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:45.695+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:45.697 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:45.698 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:45.706 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:01:45.796 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:45.796+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:45.861 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:45.861 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:45.871 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:01:45.897 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:45.897+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:45.904 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-05-11T23:01:45.998 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:45.998+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:46.027 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:46.027 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:46.036 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:01:46.099 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:46.100+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:46.164 INFO:teuthology.orchestra.run.smithi005.stdout:1743 trunc from 0x26a524f to 0xb479ad7 2024-05-11T23:01:46.173 INFO:teuthology.orchestra.run.smithi005.stdout:1744 trunc from 0xb479ad7 to 0x9cc6556 2024-05-11T23:01:46.188 INFO:teuthology.orchestra.run.smithi005.stdout:1745 write 0xd726fac thru 0xd735579 (0xe5ce bytes) 2024-05-11T23:01:46.196 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:46.197 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:46.198 INFO:teuthology.orchestra.run.smithi005.stdout:1747 read 0xc5aeb0a thru 0xc5aef00 (0x3f7 bytes) 2024-05-11T23:01:46.198 INFO:teuthology.orchestra.run.smithi005.stdout:1750 read 0xb4e93b thru 0xb5b6dc (0xcda2 bytes) 2024-05-11T23:01:46.198 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:46.198 INFO:teuthology.orchestra.run.smithi005.stdout:1752 punch from 0x3fc700d to 0x3fd4982, (0xd975 bytes) 2024-05-11T23:01:46.198 INFO:teuthology.orchestra.run.smithi005.stdout:1753 writesame 0xd49b54c thru 0xd4a3cbb data_size 0x58(0x8770 bytes) 2024-05-11T23:01:46.200 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:46.200+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:46.203 INFO:teuthology.orchestra.run.smithi005.stdout:1754 writesame 0x9f208e9 thru 0x9f29766 data_size 0x3d(0x8e7e bytes) 2024-05-11T23:01:46.208 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:01:46.210 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:46.210 INFO:teuthology.orchestra.run.smithi005.stdout:1756 read 0x2aff410 thru 0x2b0b33b (0xbf2c bytes) 2024-05-11T23:01:46.210 INFO:teuthology.orchestra.run.smithi005.stdout:1757 writesame 0x43f81d thru 0x4493b5 data_size 0x1(0x9b99 bytes) 2024-05-11T23:01:46.229 INFO:teuthology.orchestra.run.smithi005.stdout:1759 write 0x80d82cf thru 0x80e0319 (0x804b bytes) 2024-05-11T23:01:46.236 INFO:teuthology.orchestra.run.smithi005.stdout:1760 write 0x1f2420c thru 0x1f31d12 (0xdb07 bytes) 2024-05-11T23:01:46.252 INFO:teuthology.orchestra.run.smithi005.stdout:1761 write 0xc46faf9 thru 0xc47b8e7 (0xbdef bytes) 2024-05-11T23:01:46.260 INFO:teuthology.orchestra.run.smithi005.stdout:1762 punch from 0xd70dd9a to 0xd7137d9, (0x5a3f bytes) 2024-05-11T23:01:46.260 INFO:teuthology.orchestra.run.smithi005.stdout:1763 punch from 0x4889cbd to 0x4891260, (0x75a3 bytes) 2024-05-11T23:01:46.274 INFO:teuthology.orchestra.run.smithi005.stdout:1766 trunc from 0xd73557a to 0x2a44343 2024-05-11T23:01:46.289 INFO:teuthology.orchestra.run.smithi005.stdout:1767 punch from 0x1166fd4 to 0x116e28a, (0x72b6 bytes) 2024-05-11T23:01:46.289 INFO:teuthology.orchestra.run.smithi005.stdout:1768 punch from 0x2df40e to 0x2e8235, (0x8e27 bytes) 2024-05-11T23:01:46.289 INFO:teuthology.orchestra.run.smithi005.stdout:1769 read 0x9d0392 thru 0x9d7a4d (0x76bc bytes) 2024-05-11T23:01:46.290 INFO:teuthology.orchestra.run.smithi005.stdout:1770 punch from 0x1ec964f to 0x1ed7d4d, (0xe6fe bytes) 2024-05-11T23:01:46.301 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:46.301+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:46.303 INFO:teuthology.orchestra.run.smithi005.stdout:1773 trunc from 0x2a44343 to 0xd1f109f 2024-05-11T23:01:46.310 INFO:teuthology.orchestra.run.smithi005.stdout:1774 trunc from 0xd1f109f to 0xb1e7ec2 2024-05-11T23:01:46.325 INFO:teuthology.orchestra.run.smithi005.stdout:1775 writesame 0xe924c02 thru 0xe9345c4 data_size 0x1(0xf9c3 bytes) 2024-05-11T23:01:46.325 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:46.325 INFO:teuthology.orchestra.run.smithi005.stdout:1778 trunc from 0xe9345c5 to 0xe024ed8 2024-05-11T23:01:46.331 INFO:teuthology.orchestra.run.smithi005.stdout:1780 punch from 0x11eec2a to 0x11f8483, (0x9859 bytes) 2024-05-11T23:01:46.331 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:46.332 INFO:teuthology.orchestra.run.smithi005.stdout:1782 trunc from 0xe024ed8 to 0xd02b184 2024-05-11T23:01:46.337 INFO:teuthology.orchestra.run.smithi005.stdout:1783 read 0x37d7d20 thru 0x37e54a4 (0xd785 bytes) 2024-05-11T23:01:46.339 INFO:teuthology.orchestra.run.smithi005.stdout:1785 trunc from 0xd02b184 to 0xb49ab9e 2024-05-11T23:01:46.345 INFO:teuthology.orchestra.run.smithi005.stdout:1788 read 0x6226be0 thru 0x62280ff (0x1520 bytes) 2024-05-11T23:01:46.345 INFO:teuthology.orchestra.run.smithi005.stdout:1789 read 0x4a9ae8 thru 0x4ada07 (0x3f20 bytes) 2024-05-11T23:01:46.345 INFO:teuthology.orchestra.run.smithi005.stdout:1790 write 0x63ad76e thru 0x63b448c (0x6d1f bytes) 2024-05-11T23:01:46.352 INFO:teuthology.orchestra.run.smithi005.stdout:1791 read 0x1757df6 thru 0x176681e (0xea29 bytes) 2024-05-11T23:01:46.352 INFO:teuthology.orchestra.run.smithi005.stdout:1793 write 0x49c1064 thru 0x49c34c4 (0x2461 bytes) 2024-05-11T23:01:46.356 INFO:teuthology.orchestra.run.smithi005.stdout:1794 read 0x4cadba4 thru 0x4cb9aac (0xbf09 bytes) 2024-05-11T23:01:46.356 INFO:teuthology.orchestra.run.smithi005.stdout:1795 read 0x640d649 thru 0x6417c5f (0xa617 bytes) 2024-05-11T23:01:46.356 INFO:teuthology.orchestra.run.smithi005.stdout:1796 write 0x7af7312 thru 0x7b045a6 (0xd295 bytes) 2024-05-11T23:01:46.365 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:46.365 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:46.365 INFO:teuthology.orchestra.run.smithi005.stdout:1797 writesame 0xa851660 thru 0xa861532 data_size 0x5(0xfed3 bytes) 2024-05-11T23:01:46.378 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:01:46.390 INFO:teuthology.orchestra.run.smithi005.stdout:1798 trunc from 0xb49ab9e to 0xf3f3d1 2024-05-11T23:01:46.402 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:46.402+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:46.446 INFO:teuthology.orchestra.run.smithi005.stdout:1799 trunc from 0xf3f3d1 to 0xc7044a9 2024-05-11T23:01:46.452 INFO:teuthology.orchestra.run.smithi005.stdout:1800 read 0x445d603 thru 0x446314f (0x5b4d bytes) 2024-05-11T23:01:46.452 INFO:teuthology.orchestra.run.smithi005.stdout:1802 read 0x82aeb4a thru 0x82b729c (0x8753 bytes) 2024-05-11T23:01:46.452 INFO:teuthology.orchestra.run.smithi005.stdout:1803 read 0x4bbdd75 thru 0x4bcd2cb (0xf557 bytes) 2024-05-11T23:01:46.452 INFO:teuthology.orchestra.run.smithi005.stdout:1804 read 0x6c6648d thru 0x6c6971c (0x3290 bytes) 2024-05-11T23:01:46.452 INFO:teuthology.orchestra.run.smithi005.stdout:1805 read 0xa6e505c thru 0xa6eeaf3 (0x9a98 bytes) 2024-05-11T23:01:46.463 INFO:teuthology.orchestra.run.smithi005.stdout:1806 trunc from 0xc7044a9 to 0x2c7a332 2024-05-11T23:01:46.478 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:01:46.479 INFO:teuthology.orchestra.run.smithi005.stdout:1809 writesame 0x7096401 thru 0x709a555 data_size 0xf(0x4155 bytes) 2024-05-11T23:01:46.481 INFO:teuthology.orchestra.run.smithi005.stdout:1811 punch from 0x1ebeb48 to 0x1ec0bc7, (0x207f bytes) 2024-05-11T23:01:46.481 INFO:teuthology.orchestra.run.smithi005.stdout:1814 read 0xefe1bc thru 0xf087a4 (0xa5e9 bytes) 2024-05-11T23:01:46.481 INFO:teuthology.orchestra.run.smithi005.stdout:1815 punch from 0x4d0dc48 to 0x4d1104b, (0x3403 bytes) 2024-05-11T23:01:46.482 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:46.487 INFO:teuthology.orchestra.run.smithi005.stdout:1819 trunc from 0x709a556 to 0xd2155a1 2024-05-11T23:01:46.490 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:46.490 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:46.493 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:46.493 INFO:teuthology.orchestra.run.smithi005.stdout:1821 read 0x772f98d thru 0x7734c94 (0x5308 bytes) 2024-05-11T23:01:46.493 INFO:teuthology.orchestra.run.smithi005.stdout:1822 punch from 0xbe74e71 to 0xbe7d2d2, (0x8461 bytes) 2024-05-11T23:01:46.493 INFO:teuthology.orchestra.run.smithi005.stdout:1823 read 0x15a91c1 thru 0x15b7b84 (0xe9c4 bytes) 2024-05-11T23:01:46.494 INFO:teuthology.orchestra.run.smithi005.stdout:1824 write 0x39f7e48 thru 0x39fbc03 (0x3dbc bytes) 2024-05-11T23:01:46.498 INFO:teuthology.orchestra.run.smithi005.stdout:1825 write 0x675ce1f thru 0x67645c0 (0x77a2 bytes) 2024-05-11T23:01:46.499 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:01:46.504 INFO:teuthology.orchestra.run.smithi005.stdout:1826 write 0xc79ef8a thru 0xc7a92b7 (0xa32e bytes) 2024-05-11T23:01:46.511 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:46.511 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:46.511 INFO:teuthology.orchestra.run.smithi005.stdout:1831 write 0xcc0d5ec thru 0xcc1369e (0x60b3 bytes) 2024-05-11T23:01:46.517 INFO:teuthology.orchestra.run.smithi005.stdout:1832 writesame 0x223a3d0 thru 0x2242c10 data_size 0x15(0x8841 bytes) 2024-05-11T23:01:46.524 INFO:teuthology.orchestra.run.smithi005.stdout:1833 writesame 0x6cde8f6 thru 0x6cea7bb data_size 0x1(0xbec6 bytes) 2024-05-11T23:01:46.539 INFO:teuthology.orchestra.run.smithi005.stdout:1834 trunc from 0xd2155a1 to 0xa91e10f 2024-05-11T23:01:46.542 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:46.542+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:46.551 INFO:teuthology.orchestra.run.smithi005.stdout:1835 punch from 0x22a52a9 to 0x22a5342, (0x99 bytes) 2024-05-11T23:01:46.551 INFO:teuthology.orchestra.run.smithi005.stdout:1836 write 0x242a566 thru 0x24383bb (0xde56 bytes) 2024-05-11T23:01:46.561 INFO:teuthology.orchestra.run.smithi005.stdout:1838 trunc from 0xa91e10f to 0xb3a970f 2024-05-11T23:01:46.566 INFO:teuthology.orchestra.run.smithi005.stdout:1839 punch from 0x23e924d to 0x23eae54, (0x1c07 bytes) 2024-05-11T23:01:46.566 INFO:teuthology.orchestra.run.smithi005.stdout:1841 read 0x4d466fe thru 0x4d49407 (0x2d0a bytes) 2024-05-11T23:01:46.566 INFO:teuthology.orchestra.run.smithi005.stdout:1842 punch from 0x45df756 to 0x45ee3d4, (0xec7e bytes) 2024-05-11T23:01:46.577 INFO:teuthology.orchestra.run.smithi005.stdout:1843 trunc from 0xb3a970f to 0x1baf945 2024-05-11T23:01:46.598 INFO:teuthology.orchestra.run.smithi005.stdout:1844 writesame 0x2e1340d thru 0x2e1ff4a data_size 0x37(0xcb3e bytes) 2024-05-11T23:01:46.598 INFO:teuthology.orchestra.run.smithi005.stdout:1847 read 0x10f1266 thru 0x10f3105 (0x1ea0 bytes) 2024-05-11T23:01:46.604 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:46.604+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:46.609 INFO:teuthology.orchestra.run.smithi005.stdout:1848 trunc from 0x2e1ff4b to 0xc84bbf5 2024-05-11T23:01:46.614 INFO:teuthology.orchestra.run.smithi005.stdout:1849 read 0x28fc59c thru 0x290491e (0x8383 bytes) 2024-05-11T23:01:46.615 INFO:teuthology.orchestra.run.smithi005.stdout:1850 writesame 0x5dbbd07 thru 0x5dbd6bb data_size 0x1(0x19b5 bytes) 2024-05-11T23:01:46.615 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:46.615 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:46.620 INFO:teuthology.orchestra.run.smithi005.stdout:1851 punch from 0x4c9d61d to 0x4ca8283, (0xac66 bytes) 2024-05-11T23:01:46.621 INFO:teuthology.orchestra.run.smithi005.stdout:1853 trunc from 0xc84bbf5 to 0xd5a6ce7 2024-05-11T23:01:46.624 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:01:46.626 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:46.626 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:46.626 INFO:teuthology.orchestra.run.smithi005.stdout:1856 read 0x79a83c3 thru 0x79afc27 (0x7865 bytes) 2024-05-11T23:01:46.626 INFO:teuthology.orchestra.run.smithi005.stdout:1858 read 0x265d07 thru 0x2707c5 (0xaabf bytes) 2024-05-11T23:01:46.627 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:46.634 INFO:teuthology.orchestra.run.smithi005.stdout:1864 trunc from 0xd5a6ce7 to 0x686821d 2024-05-11T23:01:46.635 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:01:46.635 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:01:46.640 INFO:teuthology.orchestra.run.smithi005.stdout:1866 read 0x1bbe33a thru 0x1bcb596 (0xd25d bytes) 2024-05-11T23:01:46.650 INFO:teuthology.orchestra.run.smithi005.stdout:1867 write 0xb0d30ed thru 0xb0dfad1 (0xc9e5 bytes) 2024-05-11T23:01:46.657 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:01:46.657 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:01:46.659 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:46.659 INFO:teuthology.orchestra.run.smithi005.stdout:1871 read 0x4e52cb9 thru 0x4e60688 (0xd9d0 bytes) 2024-05-11T23:01:46.665 INFO:teuthology.orchestra.run.smithi005.stdout:1872 writesame 0xb8a72dc thru 0xb8b6bf1 data_size 0x1(0xf916 bytes) 2024-05-11T23:01:46.665 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:46.666 INFO:teuthology.orchestra.run.smithi005.stdout:1877 punch from 0x95822d5 to 0x958ff16, (0xdc41 bytes) 2024-05-11T23:01:46.666 INFO:teuthology.orchestra.run.smithi005.stdout:1879 punch from 0x194ff51 to 0x195e1f4, (0xe2a3 bytes) 2024-05-11T23:01:46.666 INFO:teuthology.orchestra.run.smithi005.stdout:1881 write 0x4f1e761 thru 0x4f2cef4 (0xe794 bytes) 2024-05-11T23:01:46.674 INFO:teuthology.orchestra.run.smithi005.stdout:1884 read 0x382100b thru 0x382ac54 (0x9c4a bytes) 2024-05-11T23:01:46.675 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:46.675 INFO:teuthology.orchestra.run.smithi005.stdout:1886 read 0x21202e4 thru 0x2124a18 (0x4735 bytes) 2024-05-11T23:01:46.675 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:46.675 INFO:teuthology.orchestra.run.smithi005.stdout:1888 read 0x5d7edb3 thru 0x5d83c53 (0x4ea1 bytes) 2024-05-11T23:01:46.675 INFO:teuthology.orchestra.run.smithi005.stdout:1889 write 0x51e2c1b thru 0x51e747e (0x4864 bytes) 2024-05-11T23:01:46.677 INFO:teuthology.orchestra.run.smithi005.stdout:1890 read 0x5c4f725 thru 0x5c5dd78 (0xe654 bytes) 2024-05-11T23:01:46.677 INFO:teuthology.orchestra.run.smithi005.stdout:1891 punch from 0xb641942 to 0xb6475df, (0x5c9d bytes) 2024-05-11T23:01:46.677 INFO:teuthology.orchestra.run.smithi005.stdout:1892 read 0x4f3ab0 thru 0x4fe7e2 (0xad33 bytes) 2024-05-11T23:01:46.678 INFO:teuthology.orchestra.run.smithi005.stdout:1893 punch from 0xaf93026 to 0xaf942e0, (0x12ba bytes) 2024-05-11T23:01:46.678 INFO:teuthology.orchestra.run.smithi005.stdout:1895 write 0x6cd9b04 thru 0x6ce6d1d (0xd21a bytes) 2024-05-11T23:01:46.681 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:01:46.681 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:01:46.688 INFO:teuthology.orchestra.run.smithi005.stdout:1896 punch from 0x587503a to 0x5875fc8, (0xf8e bytes) 2024-05-11T23:01:46.695 INFO:teuthology.orchestra.run.smithi005.stdout:1897 write 0xd30d487 thru 0xd30f294 (0x1e0e bytes) 2024-05-11T23:01:46.698 INFO:teuthology.orchestra.run.smithi005.stdout:1899 write 0x69fcd29 thru 0x6a04e5e (0x8136 bytes) 2024-05-11T23:01:46.701 INFO:teuthology.orchestra.run.smithi005.stdout:1900 writesame 0xb9bd45b thru 0xb9c091f data_size 0x13(0x34c5 bytes) 2024-05-11T23:01:46.704 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:01:46.704 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:01:46.706 INFO:teuthology.orchestra.run.smithi005.stdout:1901 read 0x86b4dcb thru 0x86ba0a7 (0x52dd bytes) 2024-05-11T23:01:46.707 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:46.707+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:46.720 INFO:teuthology.orchestra.run.smithi005.stdout:1902 trunc from 0xd30f295 to 0xa621e4 2024-05-11T23:01:46.724 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:01:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:01:46.756 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:01:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:01:46.765 INFO:teuthology.orchestra.run.smithi005.stdout:1905 punch from 0x777a28 to 0x779a43, (0x201b bytes) 2024-05-11T23:01:46.765 INFO:teuthology.orchestra.run.smithi005.stdout:1907 read 0x5b442f thru 0x5b4d35 (0x907 bytes) 2024-05-11T23:01:46.769 INFO:teuthology.orchestra.run.smithi005.stdout:1908 trunc from 0xa621e4 to 0x3d3d005 2024-05-11T23:01:46.776 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:01:46.776 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:01:46.784 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:46.784 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:01:46.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:01:46.784 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:01:46.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.784 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-05-11T23:01:46.784 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:29.463901+0000", 2024-05-11T23:01:46.784 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.318912178000001, 2024-05-11T23:01:46.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0077045129999999996, 2024-05-11T23:01:46.784 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:46.784 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:46.784 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:46.784 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:46.785 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:46.785 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:46.785 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 9 2024-05-11T23:01:46.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.785 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:46.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:46.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463901+0000", 2024-05-11T23:01:46.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:46.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463901+0000", 2024-05-11T23:01:46.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:46.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463902+0000", 2024-05-11T23:01:46.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.367e-06 2024-05-11T23:01:46.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:46.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463908+0000", 2024-05-11T23:01:46.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0789999999999997e-06 2024-05-11T23:01:46.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:46.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463921+0000", 2024-05-11T23:01:46.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2734e-05 2024-05-11T23:01:46.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:46.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463928+0000", 2024-05-11T23:01:46.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2429999999999999e-06 2024-05-11T23:01:46.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463973+0000", 2024-05-11T23:01:46.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4116999999999998e-05 2024-05-11T23:01:46.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op must wait for map", 2024-05-11T23:01:46.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.463977+0000", 2024-05-11T23:01:46.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3030000000000001e-06 2024-05-11T23:01:46.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.465607+0000", 2024-05-11T23:01:46.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016304119999999999 2024-05-11T23:01:46.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:46.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.465688+0000", 2024-05-11T23:01:46.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0957000000000004e-05 2024-05-11T23:01:46.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-05-11T23:01:46.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.465822+0000", 2024-05-11T23:01:46.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013396099999999999 2024-05-11T23:01:46.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:46.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.466978+0000", 2024-05-11T23:01:46.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001155664 2024-05-11T23:01:46.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:46.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.471543+0000", 2024-05-11T23:01:46.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045646870000000004 2024-05-11T23:01:46.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:46.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.471580+0000", 2024-05-11T23:01:46.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7398999999999999e-05 2024-05-11T23:01:46.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:46.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:29.471606+0000", 2024-05-11T23:01:46.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5590000000000001e-05 2024-05-11T23:01:46.790 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.790 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.790 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.790 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:2223 3.1a 3:5bdb11a3:::rbd_data.111e50384849.000000000000003e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 180224] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:46.790 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.608039+0000", 2024-05-11T23:01:46.790 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.174774531000001, 2024-05-11T23:01:46.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0078733559999999998, 2024-05-11T23:01:46.790 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:46.790 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:46.790 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:46.790 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:46.790 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:46.790 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:46.791 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 2223 2024-05-11T23:01:46.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.791 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:46.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:46.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608039+0000", 2024-05-11T23:01:46.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:46.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608039+0000", 2024-05-11T23:01:46.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:46.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608040+0000", 2024-05-11T23:01:46.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3900000000000004e-07 2024-05-11T23:01:46.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:46.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608044+0000", 2024-05-11T23:01:46.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0380000000000001e-06 2024-05-11T23:01:46.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:46.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608053+0000", 2024-05-11T23:01:46.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3449999999999995e-06 2024-05-11T23:01:46.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:46.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608058+0000", 2024-05-11T23:01:46.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2809999999999999e-06 2024-05-11T23:01:46.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608106+0000", 2024-05-11T23:01:46.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7552000000000001e-05 2024-05-11T23:01:46.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:46.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608191+0000", 2024-05-11T23:01:46.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4981999999999994e-05 2024-05-11T23:01:46.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-05-11T23:01:46.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.608307+0000", 2024-05-11T23:01:46.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011595 2024-05-11T23:01:46.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:46.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.609447+0000", 2024-05-11T23:01:46.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011405600000000001 2024-05-11T23:01:46.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:46.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.615854+0000", 2024-05-11T23:01:46.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0064062959999999997 2024-05-11T23:01:46.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:46.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.615887+0000", 2024-05-11T23:01:46.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3456e-05 2024-05-11T23:01:46.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:46.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.615912+0000", 2024-05-11T23:01:46.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5057000000000001e-05 2024-05-11T23:01:46.795 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.795 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.795 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.795 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4208 3.e e25/19 3:774edc63:::rbd_object_map.111e89cdde7d:head v 25'259, mlcod=25'255)", 2024-05-11T23:01:46.795 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.552088+0000", 2024-05-11T23:01:46.795 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.230724913, 2024-05-11T23:01:46.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071901839999999996, 2024-05-11T23:01:46.795 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:46.795 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:46.795 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:46.796 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:46.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:46.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552088+0000", 2024-05-11T23:01:46.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:46.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552088+0000", 2024-05-11T23:01:46.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:46.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552089+0000", 2024-05-11T23:01:46.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4900000000000001e-07 2024-05-11T23:01:46.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:46.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552093+0000", 2024-05-11T23:01:46.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7190000000000001e-06 2024-05-11T23:01:46.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:46.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552102+0000", 2024-05-11T23:01:46.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6319999999999997e-06 2024-05-11T23:01:46.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:46.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552107+0000", 2024-05-11T23:01:46.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3980000000000001e-06 2024-05-11T23:01:46.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552147+0000", 2024-05-11T23:01:46.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0423999999999998e-05 2024-05-11T23:01:46.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:46.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552160+0000", 2024-05-11T23:01:46.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3246e-05 2024-05-11T23:01:46.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:46.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559245+0000", 2024-05-11T23:01:46.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0070852620000000002 2024-05-11T23:01:46.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:46.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559278+0000", 2024-05-11T23:01:46.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3053999999999999e-05 2024-05-11T23:01:46.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.799 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.799 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4209 3.9 3:92dfc630:::rbd_data.111e89cdde7d.00000000000000ed:head [set-alloc-hint object_size 1048576 write_size 1048576,write 9891~6493] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:46.799 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.552603+0000", 2024-05-11T23:01:46.799 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.230210380999999, 2024-05-11T23:01:46.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0067329449999999997, 2024-05-11T23:01:46.799 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:46.799 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:46.799 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:46.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:46.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:46.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:46.800 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4209 2024-05-11T23:01:46.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.800 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:46.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:46.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552603+0000", 2024-05-11T23:01:46.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:46.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552603+0000", 2024-05-11T23:01:46.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:46.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552603+0000", 2024-05-11T23:01:46.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2799999999999996e-07 2024-05-11T23:01:46.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:46.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552610+0000", 2024-05-11T23:01:46.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.472e-06 2024-05-11T23:01:46.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:46.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552618+0000", 2024-05-11T23:01:46.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3299999999999999e-06 2024-05-11T23:01:46.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:46.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552623+0000", 2024-05-11T23:01:46.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8010000000000003e-06 2024-05-11T23:01:46.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552670+0000", 2024-05-11T23:01:46.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6686e-05 2024-05-11T23:01:46.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:46.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552782+0000", 2024-05-11T23:01:46.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000112389 2024-05-11T23:01:46.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:01:46.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552923+0000", 2024-05-11T23:01:46.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000141225 2024-05-11T23:01:46.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:46.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554063+0000", 2024-05-11T23:01:46.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011397130000000001 2024-05-11T23:01:46.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:46.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559270+0000", 2024-05-11T23:01:46.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052066589999999998 2024-05-11T23:01:46.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:46.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559306+0000", 2024-05-11T23:01:46.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6133000000000001e-05 2024-05-11T23:01:46.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:46.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559336+0000", 2024-05-11T23:01:46.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9909000000000001e-05 2024-05-11T23:01:46.804 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.804 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.804 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.804 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4210 3.3 3:c0d177bc:::rbd_data.111e89cdde7d.00000000000000ee:head [set-alloc-hint object_size 1048576 write_size 1048576,write 8192~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:46.804 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.554387+0000", 2024-05-11T23:01:46.804 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.228426553, 2024-05-11T23:01:46.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080170269999999995, 2024-05-11T23:01:46.805 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:46.805 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:46.805 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:46.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:46.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:46.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:46.805 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4210 2024-05-11T23:01:46.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.805 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:46.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:46.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554387+0000", 2024-05-11T23:01:46.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:46.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554387+0000", 2024-05-11T23:01:46.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:46.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554387+0000", 2024-05-11T23:01:46.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6199999999999997e-07 2024-05-11T23:01:46.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:46.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554395+0000", 2024-05-11T23:01:46.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8720000000000002e-06 2024-05-11T23:01:46.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:46.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554405+0000", 2024-05-11T23:01:46.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5889999999999992e-06 2024-05-11T23:01:46.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:46.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554410+0000", 2024-05-11T23:01:46.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0980000000000001e-06 2024-05-11T23:01:46.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554458+0000", 2024-05-11T23:01:46.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8189999999999998e-05 2024-05-11T23:01:46.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:46.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554533+0000", 2024-05-11T23:01:46.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4900000000000005e-05 2024-05-11T23:01:46.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:46.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.554615+0000", 2024-05-11T23:01:46.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2363999999999999e-05 2024-05-11T23:01:46.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:46.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.555352+0000", 2024-05-11T23:01:46.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00073687199999999996 2024-05-11T23:01:46.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:46.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.562311+0000", 2024-05-11T23:01:46.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0069588610000000002 2024-05-11T23:01:46.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:46.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.562363+0000", 2024-05-11T23:01:46.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2058999999999997e-05 2024-05-11T23:01:46.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:46.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.562404+0000", 2024-05-11T23:01:46.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0460000000000002e-05 2024-05-11T23:01:46.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.809 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.810 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4245 3.3 3:c52a1aee:::rbd_data.111e89cdde7d.0000000000000080:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:46.810 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598236+0000", 2024-05-11T23:01:46.810 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.184577644999999, 2024-05-11T23:01:46.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0075491519999999999, 2024-05-11T23:01:46.810 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:46.810 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:46.810 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:46.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:46.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:46.810 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:46.810 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4245 2024-05-11T23:01:46.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.810 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:46.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:46.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598236+0000", 2024-05-11T23:01:46.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:46.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598236+0000", 2024-05-11T23:01:46.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:46.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598236+0000", 2024-05-11T23:01:46.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9899999999999999e-07 2024-05-11T23:01:46.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:46.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598240+0000", 2024-05-11T23:01:46.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5200000000000002e-06 2024-05-11T23:01:46.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:46.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598250+0000", 2024-05-11T23:01:46.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.764e-06 2024-05-11T23:01:46.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:46.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598257+0000", 2024-05-11T23:01:46.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2640000000000002e-06 2024-05-11T23:01:46.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598309+0000", 2024-05-11T23:01:46.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2410999999999997e-05 2024-05-11T23:01:46.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:46.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598388+0000", 2024-05-11T23:01:46.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9120000000000001e-05 2024-05-11T23:01:46.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:46.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598490+0000", 2024-05-11T23:01:46.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101285 2024-05-11T23:01:46.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:46.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.599363+0000", 2024-05-11T23:01:46.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00087332100000000004 2024-05-11T23:01:46.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:46.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605664+0000", 2024-05-11T23:01:46.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063005889999999997 2024-05-11T23:01:46.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:46.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605744+0000", 2024-05-11T23:01:46.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0507999999999996e-05 2024-05-11T23:01:46.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:46.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605785+0000", 2024-05-11T23:01:46.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0571000000000002e-05 2024-05-11T23:01:46.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.815 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.815 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4246 3.3 3:c5bb5d23:::rbd_data.111e89cdde7d.0000000000000081:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:46.815 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598285+0000", 2024-05-11T23:01:46.815 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.184528374999999, 2024-05-11T23:01:46.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0075911599999999996, 2024-05-11T23:01:46.815 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:46.815 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:46.815 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:46.815 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:46.815 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:46.815 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:46.815 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4246 2024-05-11T23:01:46.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.816 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:46.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:46.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598285+0000", 2024-05-11T23:01:46.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:46.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598285+0000", 2024-05-11T23:01:46.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:46.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598285+0000", 2024-05-11T23:01:46.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4700000000000001e-07 2024-05-11T23:01:46.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:46.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598288+0000", 2024-05-11T23:01:46.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0120000000000001e-06 2024-05-11T23:01:46.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:46.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598296+0000", 2024-05-11T23:01:46.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7000000000000008e-06 2024-05-11T23:01:46.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:46.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598302+0000", 2024-05-11T23:01:46.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4790000000000004e-06 2024-05-11T23:01:46.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598716+0000", 2024-05-11T23:01:46.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00041448 2024-05-11T23:01:46.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:46.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598784+0000", 2024-05-11T23:01:46.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7689999999999997e-05 2024-05-11T23:01:46.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:46.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598881+0000", 2024-05-11T23:01:46.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7163000000000005e-05 2024-05-11T23:01:46.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:46.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.599467+0000", 2024-05-11T23:01:46.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00058593100000000004 2024-05-11T23:01:46.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:46.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605792+0000", 2024-05-11T23:01:46.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063255739999999996 2024-05-11T23:01:46.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:46.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605832+0000", 2024-05-11T23:01:46.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9301000000000002e-05 2024-05-11T23:01:46.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:46.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605876+0000", 2024-05-11T23:01:46.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4283000000000003e-05 2024-05-11T23:01:46.820 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.820 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.820 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.820 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4247 3.1 e25/19 3:80c18ebf:::rbd_data.111e89cdde7d.0000000000000082:head v 25'53, mlcod=25'51)", 2024-05-11T23:01:46.820 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598556+0000", 2024-05-11T23:01:46.820 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.184256912, 2024-05-11T23:01:46.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072492870000000001, 2024-05-11T23:01:46.820 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:46.820 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:46.820 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:46.820 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:46.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:46.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598556+0000", 2024-05-11T23:01:46.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:46.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598556+0000", 2024-05-11T23:01:46.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:46.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598557+0000", 2024-05-11T23:01:46.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-07 2024-05-11T23:01:46.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:46.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598562+0000", 2024-05-11T23:01:46.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6999999999999999e-06 2024-05-11T23:01:46.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:46.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598571+0000", 2024-05-11T23:01:46.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5629999999999998e-06 2024-05-11T23:01:46.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:46.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598585+0000", 2024-05-11T23:01:46.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4066000000000001e-05 2024-05-11T23:01:46.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598632+0000", 2024-05-11T23:01:46.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6471000000000003e-05 2024-05-11T23:01:46.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:46.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598650+0000", 2024-05-11T23:01:46.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8593999999999998e-05 2024-05-11T23:01:46.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:46.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605754+0000", 2024-05-11T23:01:46.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071034740000000002 2024-05-11T23:01:46.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:46.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605806+0000", 2024-05-11T23:01:46.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1820999999999997e-05 2024-05-11T23:01:46.824 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.824 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.824 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.824 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4248 3.f e25/19 3:f1322f7b:::rbd_data.111e89cdde7d.0000000000000083:head v 25'15, mlcod=25'13)", 2024-05-11T23:01:46.824 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598632+0000", 2024-05-11T23:01:46.824 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.184180999000001, 2024-05-11T23:01:46.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071071980000000003, 2024-05-11T23:01:46.824 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:46.824 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:46.824 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:46.824 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:46.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:46.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598632+0000", 2024-05-11T23:01:46.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:46.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598632+0000", 2024-05-11T23:01:46.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:46.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598633+0000", 2024-05-11T23:01:46.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1600000000000001e-07 2024-05-11T23:01:46.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:46.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598638+0000", 2024-05-11T23:01:46.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5040000000000002e-06 2024-05-11T23:01:46.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:46.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598657+0000", 2024-05-11T23:01:46.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8195000000000001e-05 2024-05-11T23:01:46.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:46.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598664+0000", 2024-05-11T23:01:46.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0729999999999999e-06 2024-05-11T23:01:46.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598714+0000", 2024-05-11T23:01:46.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0766000000000001e-05 2024-05-11T23:01:46.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:46.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598734+0000", 2024-05-11T23:01:46.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9544000000000001e-05 2024-05-11T23:01:46.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:46.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605676+0000", 2024-05-11T23:01:46.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0069418090000000002 2024-05-11T23:01:46.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:46.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605739+0000", 2024-05-11T23:01:46.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3590999999999995e-05 2024-05-11T23:01:46.827 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.828 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.828 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4250 3.1d e25/19 3:be073325:::rbd_data.111e89cdde7d.0000000000000085:head v 25'18, mlcod=25'16)", 2024-05-11T23:01:46.828 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598711+0000", 2024-05-11T23:01:46.828 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.184102393, 2024-05-11T23:01:46.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0070702610000000004, 2024-05-11T23:01:46.828 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:46.828 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:46.828 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:46.828 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:46.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:46.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598711+0000", 2024-05-11T23:01:46.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:46.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598711+0000", 2024-05-11T23:01:46.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:46.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598711+0000", 2024-05-11T23:01:46.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.68e-07 2024-05-11T23:01:46.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:46.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598717+0000", 2024-05-11T23:01:46.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4009999999999996e-06 2024-05-11T23:01:46.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:46.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598726+0000", 2024-05-11T23:01:46.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1400000000000006e-06 2024-05-11T23:01:46.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:46.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598732+0000", 2024-05-11T23:01:46.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4479999999999999e-06 2024-05-11T23:01:46.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598777+0000", 2024-05-11T23:01:46.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.473e-05 2024-05-11T23:01:46.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:46.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598799+0000", 2024-05-11T23:01:46.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1651999999999999e-05 2024-05-11T23:01:46.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:46.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605692+0000", 2024-05-11T23:01:46.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0068933680000000004 2024-05-11T23:01:46.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:46.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605781+0000", 2024-05-11T23:01:46.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8954000000000002e-05 2024-05-11T23:01:46.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.831 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.832 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4249 3.1c e25/19 3:3e719808:::rbd_data.111e89cdde7d.0000000000000084:head v 25'28, mlcod=25'26)", 2024-05-11T23:01:46.832 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598858+0000", 2024-05-11T23:01:46.832 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.183955157, 2024-05-11T23:01:46.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.007008294, 2024-05-11T23:01:46.832 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:46.832 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:46.832 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:46.832 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:46.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:46.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598858+0000", 2024-05-11T23:01:46.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:46.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598858+0000", 2024-05-11T23:01:46.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:46.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598859+0000", 2024-05-11T23:01:46.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5200000000000002e-07 2024-05-11T23:01:46.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:46.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598862+0000", 2024-05-11T23:01:46.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5489999999999998e-06 2024-05-11T23:01:46.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:46.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598868+0000", 2024-05-11T23:01:46.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2770000000000002e-06 2024-05-11T23:01:46.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:46.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598872+0000", 2024-05-11T23:01:46.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9650000000000002e-06 2024-05-11T23:01:46.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598912+0000", 2024-05-11T23:01:46.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0064999999999997e-05 2024-05-11T23:01:46.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:46.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598930+0000", 2024-05-11T23:01:46.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8014999999999999e-05 2024-05-11T23:01:46.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:46.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605703+0000", 2024-05-11T23:01:46.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0067726169999999999 2024-05-11T23:01:46.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:46.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605866+0000", 2024-05-11T23:01:46.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016335399999999999 2024-05-11T23:01:46.835 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.835 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.835 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.836 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:6502 3.14 3:284c4678:::rbd_data.111e65d1e80f.0000000000000016:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-11T23:01:46.836 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:37.616415+0000", 2024-05-11T23:01:46.836 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.1663978749999995, 2024-05-11T23:01:46.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022765519000000001, 2024-05-11T23:01:46.836 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:46.836 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:46.836 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:46.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:46.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:46.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:46.836 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 6502 2024-05-11T23:01:46.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.836 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:46.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:46.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616415+0000", 2024-05-11T23:01:46.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:46.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616415+0000", 2024-05-11T23:01:46.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:46.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616417+0000", 2024-05-11T23:01:46.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8899999999999999e-06 2024-05-11T23:01:46.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:46.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616427+0000", 2024-05-11T23:01:46.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7759999999999992e-06 2024-05-11T23:01:46.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:46.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616454+0000", 2024-05-11T23:01:46.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6596000000000001e-05 2024-05-11T23:01:46.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:46.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616467+0000", 2024-05-11T23:01:46.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3114e-05 2024-05-11T23:01:46.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616557+0000", 2024-05-11T23:01:46.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9823999999999999e-05 2024-05-11T23:01:46.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:46.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.616600+0000", 2024-05-11T23:01:46.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3822999999999997e-05 2024-05-11T23:01:46.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:01:46.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.620723+0000", 2024-05-11T23:01:46.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041222430000000003 2024-05-11T23:01:46.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:46.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.639100+0000", 2024-05-11T23:01:46.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.018377853999999999 2024-05-11T23:01:46.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:46.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.639113+0000", 2024-05-11T23:01:46.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2198000000000001e-05 2024-05-11T23:01:46.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:46.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.639147+0000", 2024-05-11T23:01:46.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4344000000000002e-05 2024-05-11T23:01:46.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:46.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.639181+0000", 2024-05-11T23:01:46.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3856999999999998e-05 2024-05-11T23:01:46.840 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.841 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.841 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:7526 3.14 3:298c29ea:::rbd_data.111e65d1e80f.0000000000000002:8 [sparse-read 4587520~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-05-11T23:01:46.841 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:38.991662+0000", 2024-05-11T23:01:46.841 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 7.7911516269999996, 2024-05-11T23:01:46.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0069244689999999999, 2024-05-11T23:01:46.841 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:46.841 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:46.841 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:46.841 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:46.841 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:46.841 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:46.841 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 7526 2024-05-11T23:01:46.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.842 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:46.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:46.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991662+0000", 2024-05-11T23:01:46.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:46.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991662+0000", 2024-05-11T23:01:46.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:46.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991662+0000", 2024-05-11T23:01:46.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:01:46.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:46.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991665+0000", 2024-05-11T23:01:46.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9579999999999999e-06 2024-05-11T23:01:46.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:46.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991668+0000", 2024-05-11T23:01:46.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.01e-06 2024-05-11T23:01:46.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:46.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.991670+0000", 2024-05-11T23:01:46.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5440000000000001e-06 2024-05-11T23:01:46.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998388+0000", 2024-05-11T23:01:46.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0067174000000000001 2024-05-11T23:01:46.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:46.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998479+0000", 2024-05-11T23:01:46.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1629e-05 2024-05-11T23:01:46.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:46.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:38.998586+0000", 2024-05-11T23:01:46.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000106708 2024-05-11T23:01:46.844 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.844 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.845 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.845 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:10072 3.1a 3:5fbbf0ff:::rbd_data.111e1e89614a.0000000000000266:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-05-11T23:01:46.845 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:42.385140+0000", 2024-05-11T23:01:46.845 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.397673063, 2024-05-11T23:01:46.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.016594922000000002, 2024-05-11T23:01:46.845 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:46.845 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:46.845 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:46.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:46.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:46.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:46.845 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10072 2024-05-11T23:01:46.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.846 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:46.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:46.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.385140+0000", 2024-05-11T23:01:46.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:46.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.385140+0000", 2024-05-11T23:01:46.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:46.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.385141+0000", 2024-05-11T23:01:46.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1790000000000001e-06 2024-05-11T23:01:46.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:46.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.385146+0000", 2024-05-11T23:01:46.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4440000000000001e-06 2024-05-11T23:01:46.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:46.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.385156+0000", 2024-05-11T23:01:46.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0456e-05 2024-05-11T23:01:46.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:46.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.385162+0000", 2024-05-11T23:01:46.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4569999999999996e-06 2024-05-11T23:01:46.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.385209+0000", 2024-05-11T23:01:46.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7759999999999997e-05 2024-05-11T23:01:46.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:46.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.385293+0000", 2024-05-11T23:01:46.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3059000000000006e-05 2024-05-11T23:01:46.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-05-11T23:01:46.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.388259+0000", 2024-05-11T23:01:46.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029662019999999998 2024-05-11T23:01:46.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:46.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.401614+0000", 2024-05-11T23:01:46.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.01335504 2024-05-11T23:01:46.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:46.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.401631+0000", 2024-05-11T23:01:46.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6867999999999998e-05 2024-05-11T23:01:46.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:46.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.401687+0000", 2024-05-11T23:01:46.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6648999999999999e-05 2024-05-11T23:01:46.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:46.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.401735+0000", 2024-05-11T23:01:46.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7808e-05 2024-05-11T23:01:46.850 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.850 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.850 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.850 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:10254 3.1b 3:db760c2b:::rbd_object_map.111e520e7062:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-05-11T23:01:46.850 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:42.999530+0000", 2024-05-11T23:01:46.850 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.7832829000000001, 2024-05-11T23:01:46.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0067974280000000003, 2024-05-11T23:01:46.850 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:46.850 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:46.850 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:46.850 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:46.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:46.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:46.851 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10254 2024-05-11T23:01:46.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.851 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:46.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:46.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.999530+0000", 2024-05-11T23:01:46.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:46.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.999530+0000", 2024-05-11T23:01:46.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:46.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.999531+0000", 2024-05-11T23:01:46.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7700000000000005e-07 2024-05-11T23:01:46.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:46.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.999533+0000", 2024-05-11T23:01:46.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4880000000000001e-06 2024-05-11T23:01:46.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:46.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.999541+0000", 2024-05-11T23:01:46.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7649999999999993e-06 2024-05-11T23:01:46.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:46.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:42.999544+0000", 2024-05-11T23:01:46.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3830000000000001e-06 2024-05-11T23:01:46.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.001454+0000", 2024-05-11T23:01:46.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019092899999999999 2024-05-11T23:01:46.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:46.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.001531+0000", 2024-05-11T23:01:46.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7442999999999997e-05 2024-05-11T23:01:46.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:01:46.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.001780+0000", 2024-05-11T23:01:46.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000249032 2024-05-11T23:01:46.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:46.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.003554+0000", 2024-05-11T23:01:46.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017735120000000001 2024-05-11T23:01:46.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:46.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.006278+0000", 2024-05-11T23:01:46.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027238219999999999 2024-05-11T23:01:46.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:46.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.006306+0000", 2024-05-11T23:01:46.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8249e-05 2024-05-11T23:01:46.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:46.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.006328+0000", 2024-05-11T23:01:46.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1967000000000002e-05 2024-05-11T23:01:46.855 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.855 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.855 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.855 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:10262 3.1b 3:db760c2b:::rbd_object_map.111e520e7062:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-05-11T23:01:46.855 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:43.001649+0000", 2024-05-11T23:01:46.855 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.7811641599999999, 2024-05-11T23:01:46.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.007106645, 2024-05-11T23:01:46.855 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:46.856 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:46.856 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:46.856 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:46.856 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:46.856 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:46.856 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10262 2024-05-11T23:01:46.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.856 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:46.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:46.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.001649+0000", 2024-05-11T23:01:46.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:46.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.001649+0000", 2024-05-11T23:01:46.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:46.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.001650+0000", 2024-05-11T23:01:46.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5400000000000001e-07 2024-05-11T23:01:46.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:46.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.001653+0000", 2024-05-11T23:01:46.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2200000000000001e-06 2024-05-11T23:01:46.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:46.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.001661+0000", 2024-05-11T23:01:46.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9170000000000006e-06 2024-05-11T23:01:46.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:46.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.001667+0000", 2024-05-11T23:01:46.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0560000000000001e-06 2024-05-11T23:01:46.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.005473+0000", 2024-05-11T23:01:46.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038060960000000001 2024-05-11T23:01:46.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:46.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.005568+0000", 2024-05-11T23:01:46.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5056999999999995e-05 2024-05-11T23:01:46.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:01:46.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.005942+0000", 2024-05-11T23:01:46.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00037382000000000002 2024-05-11T23:01:46.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:46.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.008480+0000", 2024-05-11T23:01:46.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002538701 2024-05-11T23:01:46.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:46.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.008658+0000", 2024-05-11T23:01:46.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017736800000000001 2024-05-11T23:01:46.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:46.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.008718+0000", 2024-05-11T23:01:46.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0579999999999999e-05 2024-05-11T23:01:46.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:46.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.008756+0000", 2024-05-11T23:01:46.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7276e-05 2024-05-11T23:01:46.860 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.860 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.860 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.860 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:10449 3.1b 3:db760c2b:::rbd_object_map.111e520e7062:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-05-11T23:01:46.860 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:43.088543+0000", 2024-05-11T23:01:46.861 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.6942707219999997, 2024-05-11T23:01:46.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0068862380000000003, 2024-05-11T23:01:46.861 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:46.861 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:46.861 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:46.861 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:46.861 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:46.861 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:46.861 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10449 2024-05-11T23:01:46.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.861 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:46.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:46.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.088543+0000", 2024-05-11T23:01:46.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:46.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.088543+0000", 2024-05-11T23:01:46.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:46.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.088543+0000", 2024-05-11T23:01:46.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8299999999999997e-07 2024-05-11T23:01:46.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:46.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.088546+0000", 2024-05-11T23:01:46.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7800000000000001e-06 2024-05-11T23:01:46.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:46.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.088552+0000", 2024-05-11T23:01:46.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9939999999999997e-06 2024-05-11T23:01:46.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:46.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.088557+0000", 2024-05-11T23:01:46.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2650000000000004e-06 2024-05-11T23:01:46.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.091632+0000", 2024-05-11T23:01:46.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030754329999999998 2024-05-11T23:01:46.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:46.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.091713+0000", 2024-05-11T23:01:46.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0863000000000002e-05 2024-05-11T23:01:46.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:01:46.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.092009+0000", 2024-05-11T23:01:46.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00029613300000000002 2024-05-11T23:01:46.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:46.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.094456+0000", 2024-05-11T23:01:46.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002446274 2024-05-11T23:01:46.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:46.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.095338+0000", 2024-05-11T23:01:46.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00088177500000000001 2024-05-11T23:01:46.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:46.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.095387+0000", 2024-05-11T23:01:46.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9589999999999998e-05 2024-05-11T23:01:46.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:46.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.095429+0000", 2024-05-11T23:01:46.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1647999999999998e-05 2024-05-11T23:01:46.865 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.865 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.865 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.866 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:10450 3.1b 3:db760c2b:::rbd_object_map.111e520e7062:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-05-11T23:01:46.866 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:43.089050+0000", 2024-05-11T23:01:46.866 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.693762768, 2024-05-11T23:01:46.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0068035709999999996, 2024-05-11T23:01:46.866 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:46.866 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:46.866 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:46.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:46.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:46.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:46.866 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10450 2024-05-11T23:01:46.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.866 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:46.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:46.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.089050+0000", 2024-05-11T23:01:46.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:46.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.089050+0000", 2024-05-11T23:01:46.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:46.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.089051+0000", 2024-05-11T23:01:46.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.034e-06 2024-05-11T23:01:46.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:46.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.089057+0000", 2024-05-11T23:01:46.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0930000000000002e-06 2024-05-11T23:01:46.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:46.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.089068+0000", 2024-05-11T23:01:46.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0981999999999999e-05 2024-05-11T23:01:46.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:46.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.089075+0000", 2024-05-11T23:01:46.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9750000000000001e-06 2024-05-11T23:01:46.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.092690+0000", 2024-05-11T23:01:46.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003615488 2024-05-11T23:01:46.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:46.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.092772+0000", 2024-05-11T23:01:46.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1678e-05 2024-05-11T23:01:46.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:01:46.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.093177+0000", 2024-05-11T23:01:46.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00040514400000000002 2024-05-11T23:01:46.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:46.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.095555+0000", 2024-05-11T23:01:46.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023785939999999999 2024-05-11T23:01:46.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:46.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.095764+0000", 2024-05-11T23:01:46.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000208254 2024-05-11T23:01:46.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:46.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.095817+0000", 2024-05-11T23:01:46.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3495000000000002e-05 2024-05-11T23:01:46.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:46.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.095854+0000", 2024-05-11T23:01:46.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6834e-05 2024-05-11T23:01:46.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.871 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.871 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:10451 3.1b 3:db760c2b:::rbd_object_map.111e520e7062:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-05-11T23:01:46.871 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:43.089766+0000", 2024-05-11T23:01:46.871 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.693047193, 2024-05-11T23:01:46.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0088571580000000004, 2024-05-11T23:01:46.871 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:46.871 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:46.871 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:46.871 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:46.871 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:46.871 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:46.871 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10451 2024-05-11T23:01:46.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.872 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:46.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:46.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.089766+0000", 2024-05-11T23:01:46.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:46.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.089766+0000", 2024-05-11T23:01:46.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:46.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.089767+0000", 2024-05-11T23:01:46.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1099999999999995e-07 2024-05-11T23:01:46.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:46.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.089770+0000", 2024-05-11T23:01:46.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.23e-06 2024-05-11T23:01:46.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:46.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.089777+0000", 2024-05-11T23:01:46.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6379999999999998e-06 2024-05-11T23:01:46.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:46.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.089781+0000", 2024-05-11T23:01:46.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1030000000000004e-06 2024-05-11T23:01:46.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.093572+0000", 2024-05-11T23:01:46.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037911849999999999 2024-05-11T23:01:46.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:46.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.093659+0000", 2024-05-11T23:01:46.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7194999999999994e-05 2024-05-11T23:01:46.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:01:46.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.094045+0000", 2024-05-11T23:01:46.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00038649399999999998 2024-05-11T23:01:46.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:46.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.098347+0000", 2024-05-11T23:01:46.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0043016790000000001 2024-05-11T23:01:46.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:46.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.098556+0000", 2024-05-11T23:01:46.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020835999999999999 2024-05-11T23:01:46.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:46.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.098595+0000", 2024-05-11T23:01:46.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9396e-05 2024-05-11T23:01:46.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:46.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.098623+0000", 2024-05-11T23:01:46.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8266999999999999e-05 2024-05-11T23:01:46.876 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.876 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.876 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.876 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:10454 3.1b 3:db760c2b:::rbd_object_map.111e520e7062:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-05-11T23:01:46.876 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:43.090270+0000", 2024-05-11T23:01:46.876 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.6925428010000001, 2024-05-11T23:01:46.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0085006690000000006, 2024-05-11T23:01:46.876 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:46.876 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:46.876 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:46.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:46.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:46.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:46.877 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10454 2024-05-11T23:01:46.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.877 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:46.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:46.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.090270+0000", 2024-05-11T23:01:46.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:46.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.090270+0000", 2024-05-11T23:01:46.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:46.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:46.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.090271+0000", 2024-05-11T23:01:46.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3200000000000004e-07 2024-05-11T23:01:46.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:46.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.090276+0000", 2024-05-11T23:01:46.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5539999999999999e-06 2024-05-11T23:01:46.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:46.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.090284+0000", 2024-05-11T23:01:46.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6340000000000007e-06 2024-05-11T23:01:46.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:46.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.090291+0000", 2024-05-11T23:01:46.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4679999999999997e-06 2024-05-11T23:01:46.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:46.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.094488+0000", 2024-05-11T23:01:46.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041974200000000003 2024-05-11T23:01:46.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:46.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.094573+0000", 2024-05-11T23:01:46.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4414999999999997e-05 2024-05-11T23:01:46.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:01:46.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.094962+0000", 2024-05-11T23:01:46.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000389205 2024-05-11T23:01:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.098370+0000", 2024-05-11T23:01:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0034080680000000002 2024-05-11T23:01:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.098671+0000", 2024-05-11T23:01:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030054799999999998 2024-05-11T23:01:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.098728+0000", 2024-05-11T23:01:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7713000000000001e-05 2024-05-11T23:01:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:46.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:46.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.098771+0000", 2024-05-11T23:01:46.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2812000000000003e-05 2024-05-11T23:01:46.881 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.881 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.881 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.881 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:46.881 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:46.881 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:46.881 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-11T23:01:46.885 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:01:46.886 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:46.882+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:46.886 INFO:teuthology.orchestra.run.smithi005.stdout:1909 writesame 0xda6fd4b thru 0xda77fee data_size 0x12(0x82a4 bytes) 2024-05-11T23:01:46.886 INFO:teuthology.orchestra.run.smithi005.stdout:1911 punch from 0x50873f6 to 0x50931d6, (0xbde0 bytes) 2024-05-11T23:01:46.886 INFO:teuthology.orchestra.run.smithi005.stdout:1912 read 0x12d5d90 thru 0x12dd943 (0x7bb4 bytes) 2024-05-11T23:01:46.886 INFO:teuthology.orchestra.run.smithi005.stdout:1913 punch from 0x1e4100b to 0x1e4f023, (0xe018 bytes) 2024-05-11T23:01:46.886 INFO:teuthology.orchestra.run.smithi005.stdout:1915 write 0x10bba82 thru 0x10bbd57 (0x2d6 bytes) 2024-05-11T23:01:46.886 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:46.886 INFO:teuthology.orchestra.run.smithi005.stdout:1917 write 0xd20528d thru 0xd208883 (0x35f7 bytes) 2024-05-11T23:01:46.887 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:01:46.887 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:01:46.887 INFO:teuthology.orchestra.run.smithi005.stdout:1918 trunc from 0xda77fef to 0x9109abe 2024-05-11T23:01:46.889 INFO:teuthology.orchestra.run.smithi005.stdout:1920 trunc from 0x9109abe to 0x3b378ec 2024-05-11T23:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout:1922 trunc from 0x3b378ec to 0xcd9274a 2024-05-11T23:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout:1923 write 0x9503a0f thru 0x9506d1d (0x330f bytes) 2024-05-11T23:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout:1931 punch from 0x77f2725 to 0x77f6f98, (0x4873 bytes) 2024-05-11T23:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout:1933 trunc from 0xcd9274a to 0x59f93e6 2024-05-11T23:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout:1934 punch from 0x2f3e47f to 0x2f42c89, (0x480a bytes) 2024-05-11T23:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout:1935 punch from 0x28b075c to 0x28be5e7, (0xde8b bytes) 2024-05-11T23:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout:1937 read 0x35d73db thru 0x35dd1a3 (0x5dc9 bytes) 2024-05-11T23:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout:1938 read 0x51caf08 thru 0x51d9894 (0xe98d bytes) 2024-05-11T23:01:46.890 INFO:teuthology.orchestra.run.smithi005.stdout:1940 read 0x3ba607b thru 0x3bb602e (0xffb4 bytes) 2024-05-11T23:01:46.891 INFO:teuthology.orchestra.run.smithi005.stdout:1942 writesame 0x6917405 thru 0x6922ad1 data_size 0x13(0xb6cd bytes) 2024-05-11T23:01:46.891 INFO:teuthology.orchestra.run.smithi005.stdout:1943 read 0x28045f5 thru 0x28081c5 (0x3bd1 bytes) 2024-05-11T23:01:46.891 INFO:teuthology.orchestra.run.smithi005.stdout:1946 read 0x2cf03e8 thru 0x2cf67f2 (0x640b bytes) 2024-05-11T23:01:46.891 INFO:teuthology.orchestra.run.smithi005.stdout:1947 write 0x5eff448 thru 0x5f09121 (0x9cda bytes) 2024-05-11T23:01:46.891 INFO:teuthology.orchestra.run.smithi005.stdout:1950 read 0x2efe427 thru 0x2f01c53 (0x382d bytes) 2024-05-11T23:01:46.891 INFO:teuthology.orchestra.run.smithi005.stdout:1951 write 0x69bde6f thru 0x69cbc31 (0xddc3 bytes) 2024-05-11T23:01:46.892 INFO:teuthology.orchestra.run.smithi005.stdout:1952 clone 12 order 20 su 4096 sc 3 2024-05-11T23:01:46.983 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:46.983+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:47.032 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:47.033 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:47.033 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:01:47.033 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:47.041 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:01:47.084 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:47.084+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:47.185 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:47.185+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:47.186 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:47.186 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:47.186 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:01:47.186 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:01:47.186 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:47.195 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:01:47.286 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:47.286+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:47.301 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:47.301 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:01:47.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:01:47.301 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:01:47.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.301 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:1499 3.c 3:30f012b2:::rbd_data.111e50384849.0000000000000049:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.543409+0000", 2024-05-11T23:01:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.757120445, 2024-05-11T23:01:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0075691669999999999, 2024-05-11T23:01:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1499 2024-05-11T23:01:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543409+0000", 2024-05-11T23:01:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543409+0000", 2024-05-11T23:01:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543410+0000", 2024-05-11T23:01:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6700000000000003e-07 2024-05-11T23:01:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543413+0000", 2024-05-11T23:01:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.574e-06 2024-05-11T23:01:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543422+0000", 2024-05-11T23:01:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6049999999999998e-06 2024-05-11T23:01:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543428+0000", 2024-05-11T23:01:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6460000000000004e-06 2024-05-11T23:01:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.544103+0000", 2024-05-11T23:01:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00067411900000000004 2024-05-11T23:01:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.544216+0000", 2024-05-11T23:01:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011385800000000001 2024-05-11T23:01:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.550978+0000", 2024-05-11T23:01:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006761698 2024-05-11T23:01:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:1681 3.c 3:30f012b2:::rbd_data.111e50384849.0000000000000049:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.557367+0000", 2024-05-11T23:01:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.743162570999999, 2024-05-11T23:01:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056449150000000003, 2024-05-11T23:01:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1681 2024-05-11T23:01:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557367+0000", 2024-05-11T23:01:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557367+0000", 2024-05-11T23:01:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557367+0000", 2024-05-11T23:01:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0200000000000003e-07 2024-05-11T23:01:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557370+0000", 2024-05-11T23:01:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5799999999999999e-06 2024-05-11T23:01:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557376+0000", 2024-05-11T23:01:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.319e-06 2024-05-11T23:01:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557381+0000", 2024-05-11T23:01:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6999999999999999e-06 2024-05-11T23:01:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.558331+0000", 2024-05-11T23:01:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00094994600000000002 2024-05-11T23:01:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.558380+0000", 2024-05-11T23:01:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9675000000000001e-05 2024-05-11T23:01:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:01:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.558482+0000", 2024-05-11T23:01:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101438 2024-05-11T23:01:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.562440+0000", 2024-05-11T23:01:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039585209999999996 2024-05-11T23:01:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.562852+0000", 2024-05-11T23:01:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00041132499999999998 2024-05-11T23:01:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.562971+0000", 2024-05-11T23:01:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011910500000000001 2024-05-11T23:01:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.563012+0000", 2024-05-11T23:01:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0904e-05 2024-05-11T23:01:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:1853 3.17 e25/19 3:ef0f2dd1:::rbd_object_map.111e50384849.0000000000000005:head v 25'5, mlcod=25'2)", 2024-05-11T23:01:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.568898+0000", 2024-05-11T23:01:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.731631287999999, 2024-05-11T23:01:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005500095, 2024-05-11T23:01:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568898+0000", 2024-05-11T23:01:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568898+0000", 2024-05-11T23:01:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568899+0000", 2024-05-11T23:01:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5000000000000003e-07 2024-05-11T23:01:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568904+0000", 2024-05-11T23:01:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4580000000000001e-06 2024-05-11T23:01:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568915+0000", 2024-05-11T23:01:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1088e-05 2024-05-11T23:01:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568922+0000", 2024-05-11T23:01:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6900000000000003e-06 2024-05-11T23:01:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568977+0000", 2024-05-11T23:01:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5299e-05 2024-05-11T23:01:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568995+0000", 2024-05-11T23:01:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8173999999999999e-05 2024-05-11T23:01:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574370+0000", 2024-05-11T23:01:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005374864 2024-05-11T23:01:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574398+0000", 2024-05-11T23:01:47.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7971999999999999e-05 2024-05-11T23:01:47.315 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.315 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.315 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.315 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "pg_update_log_missing(3.1f epoch 25/19 rep_tid 142 entries 25'64 (0'0) error 3:fc8394ee:::rbd_data.111e50384849.000000000000005d:head by client.4382.0:1942 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'61)", 2024-05-11T23:01:47.315 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.577276+0000", 2024-05-11T23:01:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.723253756, 2024-05-11T23:01:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0085889989999999999, 2024-05-11T23:01:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577276+0000", 2024-05-11T23:01:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577276+0000", 2024-05-11T23:01:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577276+0000", 2024-05-11T23:01:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5299999999999999e-07 2024-05-11T23:01:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577281+0000", 2024-05-11T23:01:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3489999999999999e-06 2024-05-11T23:01:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577292+0000", 2024-05-11T23:01:47.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0813e-05 2024-05-11T23:01:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577300+0000", 2024-05-11T23:01:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5209999999999997e-06 2024-05-11T23:01:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577377+0000", 2024-05-11T23:01:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7197e-05 2024-05-11T23:01:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.585865+0000", 2024-05-11T23:01:47.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0084876659999999996 2024-05-11T23:01:47.319 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.319 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.319 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.319 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3456 3.16 3:68b574d8:::rbd_data.111e89cdde7d.000000000000003c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.319 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.888719+0000", 2024-05-11T23:01:47.319 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.411810666999999, 2024-05-11T23:01:47.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0057862640000000002, 2024-05-11T23:01:47.319 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.319 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.319 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.319 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.319 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.319 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.320 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3456 2024-05-11T23:01:47.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.320 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888719+0000", 2024-05-11T23:01:47.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888719+0000", 2024-05-11T23:01:47.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888720+0000", 2024-05-11T23:01:47.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8100000000000001e-07 2024-05-11T23:01:47.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888724+0000", 2024-05-11T23:01:47.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3769999999999998e-06 2024-05-11T23:01:47.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888736+0000", 2024-05-11T23:01:47.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2237e-05 2024-05-11T23:01:47.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888743+0000", 2024-05-11T23:01:47.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6000000000000003e-06 2024-05-11T23:01:47.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888795+0000", 2024-05-11T23:01:47.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2206999999999997e-05 2024-05-11T23:01:47.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888873+0000", 2024-05-11T23:01:47.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8388999999999998e-05 2024-05-11T23:01:47.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:01:47.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888956+0000", 2024-05-11T23:01:47.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2183999999999998e-05 2024-05-11T23:01:47.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.891425+0000", 2024-05-11T23:01:47.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002469691 2024-05-11T23:01:47.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894453+0000", 2024-05-11T23:01:47.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030273259999999999 2024-05-11T23:01:47.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894481+0000", 2024-05-11T23:01:47.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8683999999999998e-05 2024-05-11T23:01:47.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894505+0000", 2024-05-11T23:01:47.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3688000000000001e-05 2024-05-11T23:01:47.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.324 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.324 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3457 3.c 3:353bdd7c:::rbd_data.111e89cdde7d.0000000000000050:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.324 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.888792+0000", 2024-05-11T23:01:47.324 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.411737576, 2024-05-11T23:01:47.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054713679999999999, 2024-05-11T23:01:47.324 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.324 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.325 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.325 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3457 2024-05-11T23:01:47.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.325 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888792+0000", 2024-05-11T23:01:47.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888792+0000", 2024-05-11T23:01:47.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888792+0000", 2024-05-11T23:01:47.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5300000000000004e-07 2024-05-11T23:01:47.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888795+0000", 2024-05-11T23:01:47.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0750000000000002e-06 2024-05-11T23:01:47.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888805+0000", 2024-05-11T23:01:47.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4790000000000002e-06 2024-05-11T23:01:47.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888811+0000", 2024-05-11T23:01:47.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5459999999999999e-06 2024-05-11T23:01:47.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888857+0000", 2024-05-11T23:01:47.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6007000000000002e-05 2024-05-11T23:01:47.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.888906+0000", 2024-05-11T23:01:47.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9496999999999997e-05 2024-05-11T23:01:47.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:01:47.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.889004+0000", 2024-05-11T23:01:47.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8270999999999995e-05 2024-05-11T23:01:47.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.889869+0000", 2024-05-11T23:01:47.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00086435500000000003 2024-05-11T23:01:47.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894165+0000", 2024-05-11T23:01:47.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042960259999999997 2024-05-11T23:01:47.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894215+0000", 2024-05-11T23:01:47.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9876999999999997e-05 2024-05-11T23:01:47.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894263+0000", 2024-05-11T23:01:47.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8581999999999997e-05 2024-05-11T23:01:47.329 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.329 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.329 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.329 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3852 3.1d 3:bde7e891:::rbd_data.111e89cdde7d.00000000000000b9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.330 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.243857+0000", 2024-05-11T23:01:47.330 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.056672454999999, 2024-05-11T23:01:47.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061537099999999997, 2024-05-11T23:01:47.330 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.330 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.330 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.330 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.330 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.330 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.330 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3852 2024-05-11T23:01:47.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.330 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243857+0000", 2024-05-11T23:01:47.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243857+0000", 2024-05-11T23:01:47.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243858+0000", 2024-05-11T23:01:47.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9400000000000005e-07 2024-05-11T23:01:47.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243863+0000", 2024-05-11T23:01:47.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.237e-06 2024-05-11T23:01:47.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243877+0000", 2024-05-11T23:01:47.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3754000000000001e-05 2024-05-11T23:01:47.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243884+0000", 2024-05-11T23:01:47.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4340000000000003e-06 2024-05-11T23:01:47.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243943+0000", 2024-05-11T23:01:47.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8705999999999997e-05 2024-05-11T23:01:47.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244022+0000", 2024-05-11T23:01:47.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9559000000000002e-05 2024-05-11T23:01:47.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:47.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244121+0000", 2024-05-11T23:01:47.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8227000000000007e-05 2024-05-11T23:01:47.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.246909+0000", 2024-05-11T23:01:47.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002788198 2024-05-11T23:01:47.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.249917+0000", 2024-05-11T23:01:47.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030083839999999998 2024-05-11T23:01:47.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.249968+0000", 2024-05-11T23:01:47.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.113e-05 2024-05-11T23:01:47.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.250011+0000", 2024-05-11T23:01:47.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2386999999999999e-05 2024-05-11T23:01:47.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.335 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.335 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:3855 3.17 e25/19 3:eb11d2b0:::rbd_data.111e89cdde7d.00000000000000bc:head v 25'17, mlcod=25'15)", 2024-05-11T23:01:47.335 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.244299+0000", 2024-05-11T23:01:47.335 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.056230174, 2024-05-11T23:01:47.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054219239999999998, 2024-05-11T23:01:47.335 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.335 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.335 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.335 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244299+0000", 2024-05-11T23:01:47.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244299+0000", 2024-05-11T23:01:47.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244300+0000", 2024-05-11T23:01:47.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1799999999999995e-07 2024-05-11T23:01:47.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244305+0000", 2024-05-11T23:01:47.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4009999999999996e-06 2024-05-11T23:01:47.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244314+0000", 2024-05-11T23:01:47.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7770000000000008e-06 2024-05-11T23:01:47.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244323+0000", 2024-05-11T23:01:47.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9959999999999999e-06 2024-05-11T23:01:47.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244383+0000", 2024-05-11T23:01:47.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9867000000000001e-05 2024-05-11T23:01:47.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244403+0000", 2024-05-11T23:01:47.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9885999999999999e-05 2024-05-11T23:01:47.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.249691+0000", 2024-05-11T23:01:47.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052880150000000001 2024-05-11T23:01:47.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.249721+0000", 2024-05-11T23:01:47.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0463999999999998e-05 2024-05-11T23:01:47.339 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.339 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.339 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.339 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4107 3.1c 3:3e719808:::rbd_data.111e89cdde7d.0000000000000084:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.339 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.457074+0000", 2024-05-11T23:01:47.339 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.843455630999999, 2024-05-11T23:01:47.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080074340000000008, 2024-05-11T23:01:47.339 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.339 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.339 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.340 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4107 2024-05-11T23:01:47.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.340 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457074+0000", 2024-05-11T23:01:47.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457074+0000", 2024-05-11T23:01:47.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457075+0000", 2024-05-11T23:01:47.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9899999999999999e-07 2024-05-11T23:01:47.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457079+0000", 2024-05-11T23:01:47.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.194e-06 2024-05-11T23:01:47.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457090+0000", 2024-05-11T23:01:47.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.154e-05 2024-05-11T23:01:47.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457096+0000", 2024-05-11T23:01:47.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3890000000000004e-06 2024-05-11T23:01:47.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457145+0000", 2024-05-11T23:01:47.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9245e-05 2024-05-11T23:01:47.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457196+0000", 2024-05-11T23:01:47.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1084000000000001e-05 2024-05-11T23:01:47.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:47.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457269+0000", 2024-05-11T23:01:47.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2712000000000004e-05 2024-05-11T23:01:47.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.462365+0000", 2024-05-11T23:01:47.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050963850000000002 2024-05-11T23:01:47.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.464998+0000", 2024-05-11T23:01:47.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026330799999999999 2024-05-11T23:01:47.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465047+0000", 2024-05-11T23:01:47.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8754000000000001e-05 2024-05-11T23:01:47.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465081+0000", 2024-05-11T23:01:47.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4251999999999997e-05 2024-05-11T23:01:47.344 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.344 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.344 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.344 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4108 3.1d 3:be073325:::rbd_data.111e89cdde7d.0000000000000085:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.344 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.457110+0000", 2024-05-11T23:01:47.344 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.843419208, 2024-05-11T23:01:47.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079569480000000001, 2024-05-11T23:01:47.344 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.345 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.345 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.345 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4108 2024-05-11T23:01:47.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.345 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457110+0000", 2024-05-11T23:01:47.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457110+0000", 2024-05-11T23:01:47.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457111+0000", 2024-05-11T23:01:47.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6699999999999999e-07 2024-05-11T23:01:47.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457113+0000", 2024-05-11T23:01:47.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6220000000000001e-06 2024-05-11T23:01:47.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457120+0000", 2024-05-11T23:01:47.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8909999999999998e-06 2024-05-11T23:01:47.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457124+0000", 2024-05-11T23:01:47.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6660000000000001e-06 2024-05-11T23:01:47.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457169+0000", 2024-05-11T23:01:47.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5439999999999999e-05 2024-05-11T23:01:47.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457256+0000", 2024-05-11T23:01:47.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7131000000000003e-05 2024-05-11T23:01:47.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:47.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457357+0000", 2024-05-11T23:01:47.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100655 2024-05-11T23:01:47.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.462328+0000", 2024-05-11T23:01:47.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049712630000000001 2024-05-11T23:01:47.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.464981+0000", 2024-05-11T23:01:47.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026531160000000001 2024-05-11T23:01:47.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465030+0000", 2024-05-11T23:01:47.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8659999999999998e-05 2024-05-11T23:01:47.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465067+0000", 2024-05-11T23:01:47.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7036999999999998e-05 2024-05-11T23:01:47.349 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.349 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.349 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.350 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4110 3.1c 3:3e4e9183:::rbd_data.111e89cdde7d.00000000000000a8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.350 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.457578+0000", 2024-05-11T23:01:47.350 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.842950979999999, 2024-05-11T23:01:47.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00756685, 2024-05-11T23:01:47.350 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.350 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.350 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.350 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.350 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.350 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.350 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4110 2024-05-11T23:01:47.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.350 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457578+0000", 2024-05-11T23:01:47.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457578+0000", 2024-05-11T23:01:47.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457579+0000", 2024-05-11T23:01:47.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2200000000000004e-07 2024-05-11T23:01:47.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457583+0000", 2024-05-11T23:01:47.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9079999999999997e-06 2024-05-11T23:01:47.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457592+0000", 2024-05-11T23:01:47.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4290000000000007e-06 2024-05-11T23:01:47.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457599+0000", 2024-05-11T23:01:47.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3790000000000003e-06 2024-05-11T23:01:47.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457697+0000", 2024-05-11T23:01:47.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8492000000000005e-05 2024-05-11T23:01:47.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457768+0000", 2024-05-11T23:01:47.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0798999999999998e-05 2024-05-11T23:01:47.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:47.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457869+0000", 2024-05-11T23:01:47.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101118 2024-05-11T23:01:47.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.462402+0000", 2024-05-11T23:01:47.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045324400000000004 2024-05-11T23:01:47.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465088+0000", 2024-05-11T23:01:47.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026866260000000001 2024-05-11T23:01:47.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465117+0000", 2024-05-11T23:01:47.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8517000000000002e-05 2024-05-11T23:01:47.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465145+0000", 2024-05-11T23:01:47.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8520000000000001e-05 2024-05-11T23:01:47.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.355 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.355 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4111 3.2 e25/19 3:45191464:::rbd_data.111e89cdde7d.00000000000000a9:head v 25'50, mlcod=25'48)", 2024-05-11T23:01:47.355 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.457624+0000", 2024-05-11T23:01:47.355 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.842905669, 2024-05-11T23:01:47.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0074051719999999998, 2024-05-11T23:01:47.355 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.355 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.355 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.355 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457624+0000", 2024-05-11T23:01:47.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457624+0000", 2024-05-11T23:01:47.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457624+0000", 2024-05-11T23:01:47.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6100000000000001e-07 2024-05-11T23:01:47.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457629+0000", 2024-05-11T23:01:47.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4580000000000003e-06 2024-05-11T23:01:47.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457636+0000", 2024-05-11T23:01:47.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5360000000000004e-06 2024-05-11T23:01:47.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457641+0000", 2024-05-11T23:01:47.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0760000000000002e-06 2024-05-11T23:01:47.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457689+0000", 2024-05-11T23:01:47.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8019999999999998e-05 2024-05-11T23:01:47.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457715+0000", 2024-05-11T23:01:47.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5647999999999999e-05 2024-05-11T23:01:47.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.464999+0000", 2024-05-11T23:01:47.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072837329999999997 2024-05-11T23:01:47.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465029+0000", 2024-05-11T23:01:47.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0239999999999998e-05 2024-05-11T23:01:47.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.359 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.359 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:4109 3.17 e25/19 3:ec04b13d:::rbd_data.111e89cdde7d.0000000000000086:head v 25'22, mlcod=25'20)", 2024-05-11T23:01:47.359 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.457665+0000", 2024-05-11T23:01:47.359 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.842864334, 2024-05-11T23:01:47.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.007399699, 2024-05-11T23:01:47.359 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.359 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.359 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.359 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457665+0000", 2024-05-11T23:01:47.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457665+0000", 2024-05-11T23:01:47.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457665+0000", 2024-05-11T23:01:47.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1199999999999999e-07 2024-05-11T23:01:47.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457670+0000", 2024-05-11T23:01:47.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:01:47.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457677+0000", 2024-05-11T23:01:47.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6959999999999999e-06 2024-05-11T23:01:47.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457682+0000", 2024-05-11T23:01:47.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7910000000000004e-06 2024-05-11T23:01:47.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457723+0000", 2024-05-11T23:01:47.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1607999999999999e-05 2024-05-11T23:01:47.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.457740+0000", 2024-05-11T23:01:47.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6643e-05 2024-05-11T23:01:47.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465033+0000", 2024-05-11T23:01:47.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072929639999999999 2024-05-11T23:01:47.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.465065+0000", 2024-05-11T23:01:47.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1801000000000003e-05 2024-05-11T23:01:47.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.363 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.363 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4158 3.1c 3:3e719808:::rbd_data.111e89cdde7d.0000000000000084:head [set-alloc-hint object_size 1048576 write_size 1048576,write 622592~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.363 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.498662+0000", 2024-05-11T23:01:47.363 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.801866888999999, 2024-05-11T23:01:47.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0058631339999999999, 2024-05-11T23:01:47.363 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.363 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.363 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.363 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.363 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.363 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.364 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4158 2024-05-11T23:01:47.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.364 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498662+0000", 2024-05-11T23:01:47.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498662+0000", 2024-05-11T23:01:47.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498664+0000", 2024-05-11T23:01:47.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.886e-06 2024-05-11T23:01:47.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498676+0000", 2024-05-11T23:01:47.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1919999999999999e-05 2024-05-11T23:01:47.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498692+0000", 2024-05-11T23:01:47.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5576000000000001e-05 2024-05-11T23:01:47.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498701+0000", 2024-05-11T23:01:47.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0580000000000004e-06 2024-05-11T23:01:47.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498763+0000", 2024-05-11T23:01:47.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2181000000000007e-05 2024-05-11T23:01:47.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498839+0000", 2024-05-11T23:01:47.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6261999999999996e-05 2024-05-11T23:01:47.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:47.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498969+0000", 2024-05-11T23:01:47.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000129176 2024-05-11T23:01:47.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.501285+0000", 2024-05-11T23:01:47.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023169559999999998 2024-05-11T23:01:47.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504450+0000", 2024-05-11T23:01:47.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0031641920000000001 2024-05-11T23:01:47.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504493+0000", 2024-05-11T23:01:47.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3442999999999997e-05 2024-05-11T23:01:47.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504526+0000", 2024-05-11T23:01:47.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2484000000000003e-05 2024-05-11T23:01:47.368 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.368 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.368 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.368 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4159 3.1d 3:be073325:::rbd_data.111e89cdde7d.0000000000000085:head [set-alloc-hint object_size 1048576 write_size 1048576,write 622592~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.368 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.498840+0000", 2024-05-11T23:01:47.368 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.80168937, 2024-05-11T23:01:47.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056856160000000001, 2024-05-11T23:01:47.368 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.368 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.369 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.369 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.369 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.369 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.369 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4159 2024-05-11T23:01:47.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.369 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498840+0000", 2024-05-11T23:01:47.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498840+0000", 2024-05-11T23:01:47.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498841+0000", 2024-05-11T23:01:47.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5199999999999995e-07 2024-05-11T23:01:47.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498854+0000", 2024-05-11T23:01:47.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3502e-05 2024-05-11T23:01:47.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498865+0000", 2024-05-11T23:01:47.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0394000000000001e-05 2024-05-11T23:01:47.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498873+0000", 2024-05-11T23:01:47.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8029999999999997e-06 2024-05-11T23:01:47.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.498933+0000", 2024-05-11T23:01:47.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0955000000000002e-05 2024-05-11T23:01:47.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499008+0000", 2024-05-11T23:01:47.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4210999999999998e-05 2024-05-11T23:01:47.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:47.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499165+0000", 2024-05-11T23:01:47.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015752100000000001 2024-05-11T23:01:47.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.501518+0000", 2024-05-11T23:01:47.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023527180000000002 2024-05-11T23:01:47.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504432+0000", 2024-05-11T23:01:47.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029144700000000002 2024-05-11T23:01:47.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504485+0000", 2024-05-11T23:01:47.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2639999999999997e-05 2024-05-11T23:01:47.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504526+0000", 2024-05-11T23:01:47.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0550000000000003e-05 2024-05-11T23:01:47.373 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.373 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.373 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.373 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4249 3.1c 3:3e719808:::rbd_data.111e89cdde7d.0000000000000084:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.373 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598332+0000", 2024-05-11T23:01:47.374 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.702197566000001, 2024-05-11T23:01:47.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0077041669999999996, 2024-05-11T23:01:47.374 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.374 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.374 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.374 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4249 2024-05-11T23:01:47.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.374 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598332+0000", 2024-05-11T23:01:47.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598332+0000", 2024-05-11T23:01:47.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598333+0000", 2024-05-11T23:01:47.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7499999999999998e-07 2024-05-11T23:01:47.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598337+0000", 2024-05-11T23:01:47.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6609999999999999e-06 2024-05-11T23:01:47.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598350+0000", 2024-05-11T23:01:47.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2999e-05 2024-05-11T23:01:47.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598358+0000", 2024-05-11T23:01:47.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9500000000000001e-06 2024-05-11T23:01:47.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598409+0000", 2024-05-11T23:01:47.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0192000000000003e-05 2024-05-11T23:01:47.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598489+0000", 2024-05-11T23:01:47.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0605000000000005e-05 2024-05-11T23:01:47.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:47.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598592+0000", 2024-05-11T23:01:47.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010261 2024-05-11T23:01:47.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.603246+0000", 2024-05-11T23:01:47.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0046541619999999999 2024-05-11T23:01:47.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605952+0000", 2024-05-11T23:01:47.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027065169999999999 2024-05-11T23:01:47.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605999+0000", 2024-05-11T23:01:47.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6283999999999999e-05 2024-05-11T23:01:47.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606036+0000", 2024-05-11T23:01:47.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7212000000000002e-05 2024-05-11T23:01:47.378 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.378 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.379 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.379 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4250 3.1d 3:be073325:::rbd_data.111e89cdde7d.0000000000000085:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.379 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.598376+0000", 2024-05-11T23:01:47.379 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.70215293, 2024-05-11T23:01:47.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0076523390000000002, 2024-05-11T23:01:47.379 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.379 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.379 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.379 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.379 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.379 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.379 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4250 2024-05-11T23:01:47.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.379 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598376+0000", 2024-05-11T23:01:47.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598376+0000", 2024-05-11T23:01:47.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598377+0000", 2024-05-11T23:01:47.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9200000000000001e-07 2024-05-11T23:01:47.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598380+0000", 2024-05-11T23:01:47.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8899999999999999e-06 2024-05-11T23:01:47.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598388+0000", 2024-05-11T23:01:47.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8210000000000002e-06 2024-05-11T23:01:47.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598396+0000", 2024-05-11T23:01:47.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1079999999999993e-06 2024-05-11T23:01:47.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598436+0000", 2024-05-11T23:01:47.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0219000000000002e-05 2024-05-11T23:01:47.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598514+0000", 2024-05-11T23:01:47.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7918999999999997e-05 2024-05-11T23:01:47.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:47.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.598613+0000", 2024-05-11T23:01:47.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9335999999999999e-05 2024-05-11T23:01:47.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.603227+0000", 2024-05-11T23:01:47.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0046141799999999998 2024-05-11T23:01:47.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605932+0000", 2024-05-11T23:01:47.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002704438 2024-05-11T23:01:47.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.605981+0000", 2024-05-11T23:01:47.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8806e-05 2024-05-11T23:01:47.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606029+0000", 2024-05-11T23:01:47.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8130000000000002e-05 2024-05-11T23:01:47.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.384 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.384 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4253 3.1d 3:ba7bd5e3:::rbd_data.111e89cdde7d.00000000000000e3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.384 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.600882+0000", 2024-05-11T23:01:47.384 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.699647498000001, 2024-05-11T23:01:47.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054413350000000003, 2024-05-11T23:01:47.384 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.384 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.384 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.385 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4253 2024-05-11T23:01:47.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.385 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600882+0000", 2024-05-11T23:01:47.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600882+0000", 2024-05-11T23:01:47.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600883+0000", 2024-05-11T23:01:47.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5400000000000001e-07 2024-05-11T23:01:47.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600887+0000", 2024-05-11T23:01:47.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.092e-06 2024-05-11T23:01:47.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600897+0000", 2024-05-11T23:01:47.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0093e-05 2024-05-11T23:01:47.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600902+0000", 2024-05-11T23:01:47.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1399999999999999e-06 2024-05-11T23:01:47.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600951+0000", 2024-05-11T23:01:47.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9407000000000003e-05 2024-05-11T23:01:47.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.601034+0000", 2024-05-11T23:01:47.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3108000000000005e-05 2024-05-11T23:01:47.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:47.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.601137+0000", 2024-05-11T23:01:47.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010276999999999999 2024-05-11T23:01:47.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606041+0000", 2024-05-11T23:01:47.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049042649999999997 2024-05-11T23:01:47.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606242+0000", 2024-05-11T23:01:47.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020061300000000001 2024-05-11T23:01:47.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606287+0000", 2024-05-11T23:01:47.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4824000000000003e-05 2024-05-11T23:01:47.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606323+0000", 2024-05-11T23:01:47.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6368999999999997e-05 2024-05-11T23:01:47.389 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.389 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.389 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.389 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:6733 3.1c 3:3bac25fd:::rbd_data.111e65d1e80f.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-11T23:01:47.389 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:37.962181+0000", 2024-05-11T23:01:47.389 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3383479089999994, 2024-05-11T23:01:47.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025925762000000002, 2024-05-11T23:01:47.390 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.390 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.390 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.390 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.390 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.390 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.390 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 6733 2024-05-11T23:01:47.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.390 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962181+0000", 2024-05-11T23:01:47.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962181+0000", 2024-05-11T23:01:47.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962182+0000", 2024-05-11T23:01:47.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2699999999999998e-07 2024-05-11T23:01:47.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962187+0000", 2024-05-11T23:01:47.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3050000000000003e-06 2024-05-11T23:01:47.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962197+0000", 2024-05-11T23:01:47.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0010000000000001e-05 2024-05-11T23:01:47.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962201+0000", 2024-05-11T23:01:47.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8709999999999996e-06 2024-05-11T23:01:47.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962276+0000", 2024-05-11T23:01:47.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5004000000000002e-05 2024-05-11T23:01:47.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.962325+0000", 2024-05-11T23:01:47.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8093999999999997e-05 2024-05-11T23:01:47.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:47.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.966865+0000", 2024-05-11T23:01:47.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045404720000000003 2024-05-11T23:01:47.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.987960+0000", 2024-05-11T23:01:47.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.021095442999999998 2024-05-11T23:01:47.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.988032+0000", 2024-05-11T23:01:47.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2013000000000003e-05 2024-05-11T23:01:47.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.988070+0000", 2024-05-11T23:01:47.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7981000000000001e-05 2024-05-11T23:01:47.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.988107+0000", 2024-05-11T23:01:47.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6742000000000002e-05 2024-05-11T23:01:47.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.394 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.395 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4382.0:6733 3.1c e28/19 ondisk, result = 0)", 2024-05-11T23:01:47.395 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:37.982065+0000", 2024-05-11T23:01:47.395 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3184643470000008, 2024-05-11T23:01:47.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060490049999999997, 2024-05-11T23:01:47.395 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.395 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.395 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:47.395 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.982065+0000", 2024-05-11T23:01:47.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.982065+0000", 2024-05-11T23:01:47.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.982066+0000", 2024-05-11T23:01:47.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.156e-06 2024-05-11T23:01:47.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.982071+0000", 2024-05-11T23:01:47.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:01:47.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.982083+0000", 2024-05-11T23:01:47.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1742e-05 2024-05-11T23:01:47.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.982108+0000", 2024-05-11T23:01:47.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5001999999999999e-05 2024-05-11T23:01:47.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.988021+0000", 2024-05-11T23:01:47.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059133149999999997 2024-05-11T23:01:47.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.988028+0000", 2024-05-11T23:01:47.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4649999999999997e-06 2024-05-11T23:01:47.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:37.988114+0000", 2024-05-11T23:01:47.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5538000000000007e-05 2024-05-11T23:01:47.398 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.398 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.398 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.398 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.398 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.398 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:47.400 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:01:47.400 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:47.400+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:47.500 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:47.500+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:47.504 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:47.505 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:47.505 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:01:47.505 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:47.513 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:01:47.601 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:47.601+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:47.659 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:47.659 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:47.659 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:01:47.659 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:01:47.659 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:47.669 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:01:47.710 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:47.710+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:47.803 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:47.803+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:47.807 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-05-11T23:01:47.818 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:47.818 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:01:47.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:01:47.819 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:01:47.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.819 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:1494 3.a 3:5237eba0:::rbd_data.111e50384849.0000000000000044:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.819 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.543327+0000", 2024-05-11T23:01:47.819 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.274280986000001, 2024-05-11T23:01:47.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026719579999999999, 2024-05-11T23:01:47.819 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.819 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.819 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:47.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.819 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1494 2024-05-11T23:01:47.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.819 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543327+0000", 2024-05-11T23:01:47.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543327+0000", 2024-05-11T23:01:47.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543328+0000", 2024-05-11T23:01:47.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1999999999999999e-07 2024-05-11T23:01:47.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543331+0000", 2024-05-11T23:01:47.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.32e-06 2024-05-11T23:01:47.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543350+0000", 2024-05-11T23:01:47.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8936e-05 2024-05-11T23:01:47.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543359+0000", 2024-05-11T23:01:47.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8060000000000001e-06 2024-05-11T23:01:47.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.543939+0000", 2024-05-11T23:01:47.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00058049000000000002 2024-05-11T23:01:47.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.544050+0000", 2024-05-11T23:01:47.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011028599999999999 2024-05-11T23:01:47.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.545999+0000", 2024-05-11T23:01:47.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0019495000000000001 2024-05-11T23:01:47.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.822 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.823 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:1666 3.a 3:5755b5e9:::rbd_data.111e50384849.0000000000000042:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.823 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.553359+0000", 2024-05-11T23:01:47.823 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.264249256999999, 2024-05-11T23:01:47.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0034024540000000001, 2024-05-11T23:01:47.823 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.823 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.823 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.823 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.823 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.823 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.823 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1666 2024-05-11T23:01:47.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.823 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553359+0000", 2024-05-11T23:01:47.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553359+0000", 2024-05-11T23:01:47.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553360+0000", 2024-05-11T23:01:47.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1800000000000005e-07 2024-05-11T23:01:47.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553369+0000", 2024-05-11T23:01:47.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7049999999999994e-06 2024-05-11T23:01:47.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553380+0000", 2024-05-11T23:01:47.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0285999999999999e-05 2024-05-11T23:01:47.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553387+0000", 2024-05-11T23:01:47.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3989999999999998e-06 2024-05-11T23:01:47.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553454+0000", 2024-05-11T23:01:47.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6581999999999994e-05 2024-05-11T23:01:47.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553543+0000", 2024-05-11T23:01:47.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9599999999999996e-05 2024-05-11T23:01:47.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:47.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553762+0000", 2024-05-11T23:01:47.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000218402 2024-05-11T23:01:47.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.554603+0000", 2024-05-11T23:01:47.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00084155000000000005 2024-05-11T23:01:47.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.556663+0000", 2024-05-11T23:01:47.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020600430000000001 2024-05-11T23:01:47.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.556717+0000", 2024-05-11T23:01:47.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3942e-05 2024-05-11T23:01:47.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.556761+0000", 2024-05-11T23:01:47.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4127e-05 2024-05-11T23:01:47.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.828 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.828 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:1667 3.a 3:5237eba0:::rbd_data.111e50384849.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.828 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.553412+0000", 2024-05-11T23:01:47.828 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.264196136000001, 2024-05-11T23:01:47.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036787489999999998, 2024-05-11T23:01:47.828 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.829 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.829 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.829 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1667 2024-05-11T23:01:47.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.829 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553412+0000", 2024-05-11T23:01:47.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553412+0000", 2024-05-11T23:01:47.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553413+0000", 2024-05-11T23:01:47.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8699999999999995e-07 2024-05-11T23:01:47.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553416+0000", 2024-05-11T23:01:47.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-05-11T23:01:47.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553424+0000", 2024-05-11T23:01:47.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.011e-06 2024-05-11T23:01:47.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553429+0000", 2024-05-11T23:01:47.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7400000000000001e-06 2024-05-11T23:01:47.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.554054+0000", 2024-05-11T23:01:47.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00062427700000000001 2024-05-11T23:01:47.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.554148+0000", 2024-05-11T23:01:47.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4145000000000001e-05 2024-05-11T23:01:47.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:47.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.554353+0000", 2024-05-11T23:01:47.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000204835 2024-05-11T23:01:47.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.556278+0000", 2024-05-11T23:01:47.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001925019 2024-05-11T23:01:47.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557014+0000", 2024-05-11T23:01:47.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00073628599999999995 2024-05-11T23:01:47.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557066+0000", 2024-05-11T23:01:47.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1836999999999999e-05 2024-05-11T23:01:47.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557091+0000", 2024-05-11T23:01:47.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4961999999999999e-05 2024-05-11T23:01:47.834 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.834 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.834 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.834 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:1854 3.a 3:527cad80:::rbd_data.111e50384849.0000000000000057:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.834 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.568320+0000", 2024-05-11T23:01:47.834 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.249288042, 2024-05-11T23:01:47.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033148499999999998, 2024-05-11T23:01:47.834 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.834 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.834 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:47.834 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.834 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.835 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.835 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1854 2024-05-11T23:01:47.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.835 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568320+0000", 2024-05-11T23:01:47.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568320+0000", 2024-05-11T23:01:47.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568321+0000", 2024-05-11T23:01:47.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9100000000000003e-07 2024-05-11T23:01:47.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568325+0000", 2024-05-11T23:01:47.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0570000000000003e-06 2024-05-11T23:01:47.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568335+0000", 2024-05-11T23:01:47.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.842e-06 2024-05-11T23:01:47.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568345+0000", 2024-05-11T23:01:47.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0766e-05 2024-05-11T23:01:47.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568411+0000", 2024-05-11T23:01:47.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5321999999999996e-05 2024-05-11T23:01:47.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.568539+0000", 2024-05-11T23:01:47.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000128059 2024-05-11T23:01:47.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.571635+0000", 2024-05-11T23:01:47.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030961130000000002 2024-05-11T23:01:47.838 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.838 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.838 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.838 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "pg_update_log_missing(3.1e epoch 25/19 rep_tid 82 entries 25'46 (0'0) error 3:7e05a7a0:::rbd_data.111e50384849.0000000000000058:head by client.4382.0:1858 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'44)", 2024-05-11T23:01:47.838 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.569301+0000", 2024-05-11T23:01:47.838 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.248306796, 2024-05-11T23:01:47.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048630660000000001, 2024-05-11T23:01:47.839 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.839 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.839 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:47.839 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569301+0000", 2024-05-11T23:01:47.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569301+0000", 2024-05-11T23:01:47.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569302+0000", 2024-05-11T23:01:47.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0800000000000003e-07 2024-05-11T23:01:47.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569310+0000", 2024-05-11T23:01:47.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9370000000000004e-06 2024-05-11T23:01:47.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569329+0000", 2024-05-11T23:01:47.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8808999999999999e-05 2024-05-11T23:01:47.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569344+0000", 2024-05-11T23:01:47.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4789e-05 2024-05-11T23:01:47.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569405+0000", 2024-05-11T23:01:47.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0714000000000002e-05 2024-05-11T23:01:47.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574164+0000", 2024-05-11T23:01:47.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047599089999999997 2024-05-11T23:01:47.842 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.842 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.842 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.842 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:1902 3.1e e25/19 3:7e2d774f:::rbd_data.111e50384849.0000000000000050:head v 25'47, mlcod=25'44)", 2024-05-11T23:01:47.842 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.573037+0000", 2024-05-11T23:01:47.842 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.244571605000001, 2024-05-11T23:01:47.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042723099999999996, 2024-05-11T23:01:47.842 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.842 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.842 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.843 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573037+0000", 2024-05-11T23:01:47.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573037+0000", 2024-05-11T23:01:47.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573037+0000", 2024-05-11T23:01:47.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-05-11T23:01:47.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573438+0000", 2024-05-11T23:01:47.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00040049399999999999 2024-05-11T23:01:47.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573446+0000", 2024-05-11T23:01:47.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1890000000000007e-06 2024-05-11T23:01:47.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573451+0000", 2024-05-11T23:01:47.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1959999999999999e-06 2024-05-11T23:01:47.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573488+0000", 2024-05-11T23:01:47.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6467999999999997e-05 2024-05-11T23:01:47.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573507+0000", 2024-05-11T23:01:47.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9321e-05 2024-05-11T23:01:47.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577275+0000", 2024-05-11T23:01:47.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037682750000000002 2024-05-11T23:01:47.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577309+0000", 2024-05-11T23:01:47.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3744000000000001e-05 2024-05-11T23:01:47.846 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.846 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.846 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.846 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:1914 3.a 3:527cad80:::rbd_data.111e50384849.0000000000000057:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.846 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.573229+0000", 2024-05-11T23:01:47.846 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.244379332999999, 2024-05-11T23:01:47.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041158779999999999, 2024-05-11T23:01:47.847 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.847 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.847 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.847 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.847 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.847 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.847 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1914 2024-05-11T23:01:47.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.847 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573229+0000", 2024-05-11T23:01:47.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573229+0000", 2024-05-11T23:01:47.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573230+0000", 2024-05-11T23:01:47.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8800000000000002e-07 2024-05-11T23:01:47.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573234+0000", 2024-05-11T23:01:47.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6070000000000001e-06 2024-05-11T23:01:47.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573244+0000", 2024-05-11T23:01:47.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9389999999999992e-06 2024-05-11T23:01:47.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573249+0000", 2024-05-11T23:01:47.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:01:47.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573297+0000", 2024-05-11T23:01:47.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7636000000000003e-05 2024-05-11T23:01:47.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573373+0000", 2024-05-11T23:01:47.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6693000000000006e-05 2024-05-11T23:01:47.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:47.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573566+0000", 2024-05-11T23:01:47.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019307699999999999 2024-05-11T23:01:47.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.576944+0000", 2024-05-11T23:01:47.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033778900000000001 2024-05-11T23:01:47.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577244+0000", 2024-05-11T23:01:47.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030006400000000001 2024-05-11T23:01:47.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577306+0000", 2024-05-11T23:01:47.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1625999999999996e-05 2024-05-11T23:01:47.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577345+0000", 2024-05-11T23:01:47.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8773999999999998e-05 2024-05-11T23:01:47.852 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.852 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.852 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.852 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:1904 3.1e e25/19 3:7e2d774f:::rbd_data.111e50384849.0000000000000050:head v 25'49, mlcod=25'44)", 2024-05-11T23:01:47.852 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.574045+0000", 2024-05-11T23:01:47.852 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.243563312999999, 2024-05-11T23:01:47.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0058621020000000001, 2024-05-11T23:01:47.852 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.852 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.852 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.852 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574045+0000", 2024-05-11T23:01:47.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574045+0000", 2024-05-11T23:01:47.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574045+0000", 2024-05-11T23:01:47.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:01:47.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574049+0000", 2024-05-11T23:01:47.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5899999999999999e-06 2024-05-11T23:01:47.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574057+0000", 2024-05-11T23:01:47.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4859999999999997e-06 2024-05-11T23:01:47.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574062+0000", 2024-05-11T23:01:47.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6530000000000002e-06 2024-05-11T23:01:47.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574199+0000", 2024-05-11T23:01:47.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013656900000000001 2024-05-11T23:01:47.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574237+0000", 2024-05-11T23:01:47.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8055000000000001e-05 2024-05-11T23:01:47.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.579872+0000", 2024-05-11T23:01:47.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005635223 2024-05-11T23:01:47.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.579907+0000", 2024-05-11T23:01:47.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5027000000000002e-05 2024-05-11T23:01:47.856 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.856 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.856 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.856 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:1972 3.1e e25/19 3:7e05a7a0:::rbd_data.111e50384849.0000000000000058:head v 25'50, mlcod=25'46)", 2024-05-11T23:01:47.856 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.578576+0000", 2024-05-11T23:01:47.856 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.239031917, 2024-05-11T23:01:47.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038930829999999999, 2024-05-11T23:01:47.856 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.856 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.856 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.856 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578576+0000", 2024-05-11T23:01:47.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578576+0000", 2024-05-11T23:01:47.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578577+0000", 2024-05-11T23:01:47.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8999999999999996e-07 2024-05-11T23:01:47.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578582+0000", 2024-05-11T23:01:47.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5990000000000004e-06 2024-05-11T23:01:47.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578591+0000", 2024-05-11T23:01:47.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9690000000000006e-06 2024-05-11T23:01:47.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578598+0000", 2024-05-11T23:01:47.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2179999999999996e-06 2024-05-11T23:01:47.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578657+0000", 2024-05-11T23:01:47.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9001e-05 2024-05-11T23:01:47.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578679+0000", 2024-05-11T23:01:47.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2259000000000002e-05 2024-05-11T23:01:47.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.582439+0000", 2024-05-11T23:01:47.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037597960000000001 2024-05-11T23:01:47.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.582469+0000", 2024-05-11T23:01:47.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0550999999999999e-05 2024-05-11T23:01:47.860 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.860 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.860 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.860 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "pg_update_log_missing(3.1e epoch 25/19 rep_tid 90 entries 25'51 (0'0) error 3:7facaf60:::rbd_data.111e50384849.0000000000000061:head by client.4382.0:2005 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'47)", 2024-05-11T23:01:47.860 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.581892+0000", 2024-05-11T23:01:47.860 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.235716499, 2024-05-11T23:01:47.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042265779999999999, 2024-05-11T23:01:47.860 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.860 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.860 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:47.860 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581892+0000", 2024-05-11T23:01:47.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581892+0000", 2024-05-11T23:01:47.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581892+0000", 2024-05-11T23:01:47.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9299999999999998e-07 2024-05-11T23:01:47.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581898+0000", 2024-05-11T23:01:47.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0390000000000001e-06 2024-05-11T23:01:47.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581915+0000", 2024-05-11T23:01:47.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6418000000000001e-05 2024-05-11T23:01:47.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581929+0000", 2024-05-11T23:01:47.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4131000000000001e-05 2024-05-11T23:01:47.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581984+0000", 2024-05-11T23:01:47.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4821999999999998e-05 2024-05-11T23:01:47.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.586118+0000", 2024-05-11T23:01:47.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004134675 2024-05-11T23:01:47.863 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.863 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.863 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.863 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3301 3.10 3:0d09d413:::rbd_data.111e7a935520.0000000000000009:4 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.863 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.722658+0000", 2024-05-11T23:01:47.863 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.094949676000001, 2024-05-11T23:01:47.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026955619999999999, 2024-05-11T23:01:47.863 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.863 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.864 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:47.864 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.864 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.864 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.864 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3301 2024-05-11T23:01:47.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.864 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722658+0000", 2024-05-11T23:01:47.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722658+0000", 2024-05-11T23:01:47.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722659+0000", 2024-05-11T23:01:47.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5199999999999996e-07 2024-05-11T23:01:47.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722663+0000", 2024-05-11T23:01:47.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4309999999999998e-06 2024-05-11T23:01:47.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722672+0000", 2024-05-11T23:01:47.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1409999999999994e-06 2024-05-11T23:01:47.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722678+0000", 2024-05-11T23:01:47.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3239999999999999e-06 2024-05-11T23:01:47.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.725175+0000", 2024-05-11T23:01:47.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024966789999999999 2024-05-11T23:01:47.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.725258+0000", 2024-05-11T23:01:47.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3220999999999995e-05 2024-05-11T23:01:47.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.725354+0000", 2024-05-11T23:01:47.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6113999999999995e-05 2024-05-11T23:01:47.867 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.867 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.867 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.867 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3313 3.10 3:0d09d413:::rbd_data.111e7a935520.0000000000000009:4 [read 581632~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.867 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.722692+0000", 2024-05-11T23:01:47.867 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.094916628, 2024-05-11T23:01:47.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028682030000000002, 2024-05-11T23:01:47.868 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.868 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.868 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:47.868 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.868 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.868 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.868 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3313 2024-05-11T23:01:47.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.868 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722692+0000", 2024-05-11T23:01:47.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722692+0000", 2024-05-11T23:01:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722692+0000", 2024-05-11T23:01:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:01:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722694+0000", 2024-05-11T23:01:47.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4719999999999998e-06 2024-05-11T23:01:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722702+0000", 2024-05-11T23:01:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2470000000000002e-06 2024-05-11T23:01:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722707+0000", 2024-05-11T23:01:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2089999999999996e-06 2024-05-11T23:01:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.725383+0000", 2024-05-11T23:01:47.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026757270000000001 2024-05-11T23:01:47.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.725465+0000", 2024-05-11T23:01:47.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2226999999999997e-05 2024-05-11T23:01:47.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.725560+0000", 2024-05-11T23:01:47.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4877000000000007e-05 2024-05-11T23:01:47.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.871 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.872 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.872 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3326 3.10 3:0d09d413:::rbd_data.111e7a935520.0000000000000009:4 [read 647168~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.872 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.722752+0000", 2024-05-11T23:01:47.872 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.094856008000001, 2024-05-11T23:01:47.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030445110000000002, 2024-05-11T23:01:47.872 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.872 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.872 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:47.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.873 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3326 2024-05-11T23:01:47.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.873 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722752+0000", 2024-05-11T23:01:47.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722752+0000", 2024-05-11T23:01:47.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722753+0000", 2024-05-11T23:01:47.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9500000000000002e-07 2024-05-11T23:01:47.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722757+0000", 2024-05-11T23:01:47.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5770000000000004e-06 2024-05-11T23:01:47.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722765+0000", 2024-05-11T23:01:47.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0979999999999994e-06 2024-05-11T23:01:47.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.722771+0000", 2024-05-11T23:01:47.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.643e-06 2024-05-11T23:01:47.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.725602+0000", 2024-05-11T23:01:47.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028311970000000001 2024-05-11T23:01:47.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.725693+0000", 2024-05-11T23:01:47.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0997999999999998e-05 2024-05-11T23:01:47.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.725797+0000", 2024-05-11T23:01:47.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103403 2024-05-11T23:01:47.876 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.876 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.876 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.876 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3471 3.a 3:50ba23c0:::rbd_data.111e89cdde7d.00000000000000b2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.876 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.894634+0000", 2024-05-11T23:01:47.876 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.922974118999999, 2024-05-11T23:01:47.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053259079999999999, 2024-05-11T23:01:47.876 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.876 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.876 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.877 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.877 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.877 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.877 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3471 2024-05-11T23:01:47.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.877 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894634+0000", 2024-05-11T23:01:47.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894634+0000", 2024-05-11T23:01:47.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894636+0000", 2024-05-11T23:01:47.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.542e-06 2024-05-11T23:01:47.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894640+0000", 2024-05-11T23:01:47.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9439999999999998e-06 2024-05-11T23:01:47.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894651+0000", 2024-05-11T23:01:47.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1952e-05 2024-05-11T23:01:47.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894657+0000", 2024-05-11T23:01:47.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5280000000000003e-06 2024-05-11T23:01:47.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894727+0000", 2024-05-11T23:01:47.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0029000000000003e-05 2024-05-11T23:01:47.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894822+0000", 2024-05-11T23:01:47.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5319e-05 2024-05-11T23:01:47.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:47.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894892+0000", 2024-05-11T23:01:47.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9542999999999995e-05 2024-05-11T23:01:47.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.897034+0000", 2024-05-11T23:01:47.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021425139999999999 2024-05-11T23:01:47.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.899868+0000", 2024-05-11T23:01:47.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028339659999999998 2024-05-11T23:01:47.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.899918+0000", 2024-05-11T23:01:47.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9548000000000001e-05 2024-05-11T23:01:47.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.899960+0000", 2024-05-11T23:01:47.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2023000000000001e-05 2024-05-11T23:01:47.881 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.881 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.881 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.881 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3697 3.a 3:50ba23c0:::rbd_data.111e89cdde7d.00000000000000b2:head [set-alloc-hint object_size 1048576 write_size 1048576,write 573440~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.882 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.121225+0000", 2024-05-11T23:01:47.882 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.696383371, 2024-05-11T23:01:47.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054297959999999998, 2024-05-11T23:01:47.882 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.882 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.882 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.882 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.882 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.882 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.882 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3697 2024-05-11T23:01:47.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.882 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121225+0000", 2024-05-11T23:01:47.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121225+0000", 2024-05-11T23:01:47.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121227+0000", 2024-05-11T23:01:47.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.776e-06 2024-05-11T23:01:47.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121236+0000", 2024-05-11T23:01:47.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7510000000000003e-06 2024-05-11T23:01:47.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121247+0000", 2024-05-11T23:01:47.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.115e-05 2024-05-11T23:01:47.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121253+0000", 2024-05-11T23:01:47.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.75e-06 2024-05-11T23:01:47.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121297+0000", 2024-05-11T23:01:47.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3930000000000001e-05 2024-05-11T23:01:47.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121350+0000", 2024-05-11T23:01:47.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2978e-05 2024-05-11T23:01:47.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:47.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121441+0000", 2024-05-11T23:01:47.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0662999999999996e-05 2024-05-11T23:01:47.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.123760+0000", 2024-05-11T23:01:47.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023193549999999999 2024-05-11T23:01:47.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.126594+0000", 2024-05-11T23:01:47.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028343410000000002 2024-05-11T23:01:47.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.126628+0000", 2024-05-11T23:01:47.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3621000000000002e-05 2024-05-11T23:01:47.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.126655+0000", 2024-05-11T23:01:47.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6480999999999999e-05 2024-05-11T23:01:47.887 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.887 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.887 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.887 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3849 3.a 3:50ba23c0:::rbd_data.111e89cdde7d.00000000000000b2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.887 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.243274+0000", 2024-05-11T23:01:47.887 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.574334403, 2024-05-11T23:01:47.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038658270000000001, 2024-05-11T23:01:47.887 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.887 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.887 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.887 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.888 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3849 2024-05-11T23:01:47.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.888 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243274+0000", 2024-05-11T23:01:47.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243274+0000", 2024-05-11T23:01:47.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243275+0000", 2024-05-11T23:01:47.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3399999999999997e-07 2024-05-11T23:01:47.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243279+0000", 2024-05-11T23:01:47.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7779999999999998e-06 2024-05-11T23:01:47.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243294+0000", 2024-05-11T23:01:47.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4253000000000001e-05 2024-05-11T23:01:47.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243302+0000", 2024-05-11T23:01:47.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9240000000000007e-06 2024-05-11T23:01:47.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243364+0000", 2024-05-11T23:01:47.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2349999999999998e-05 2024-05-11T23:01:47.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243443+0000", 2024-05-11T23:01:47.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8877999999999999e-05 2024-05-11T23:01:47.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:47.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243543+0000", 2024-05-11T23:01:47.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010016199999999999 2024-05-11T23:01:47.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244177+0000", 2024-05-11T23:01:47.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00063432099999999997 2024-05-11T23:01:47.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.247082+0000", 2024-05-11T23:01:47.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029042 2024-05-11T23:01:47.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.247111+0000", 2024-05-11T23:01:47.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9839e-05 2024-05-11T23:01:47.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.247140+0000", 2024-05-11T23:01:47.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8187999999999998e-05 2024-05-11T23:01:47.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.892 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.892 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4161 3.a 3:50c2a80d:::rbd_data.111e89cdde7d.0000000000000087:head [set-alloc-hint object_size 1048576 write_size 1048576,write 614400~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.892 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.499080+0000", 2024-05-11T23:01:47.892 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.318528536000001, 2024-05-11T23:01:47.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054594780000000002, 2024-05-11T23:01:47.892 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.893 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.893 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.893 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4161 2024-05-11T23:01:47.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.893 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499080+0000", 2024-05-11T23:01:47.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499080+0000", 2024-05-11T23:01:47.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499081+0000", 2024-05-11T23:01:47.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.733e-06 2024-05-11T23:01:47.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499096+0000", 2024-05-11T23:01:47.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4721e-05 2024-05-11T23:01:47.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499114+0000", 2024-05-11T23:01:47.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8238000000000001e-05 2024-05-11T23:01:47.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499124+0000", 2024-05-11T23:01:47.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0096e-05 2024-05-11T23:01:47.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499186+0000", 2024-05-11T23:01:47.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1124000000000006e-05 2024-05-11T23:01:47.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499323+0000", 2024-05-11T23:01:47.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013730599999999999 2024-05-11T23:01:47.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:47.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499458+0000", 2024-05-11T23:01:47.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013542599999999999 2024-05-11T23:01:47.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.501629+0000", 2024-05-11T23:01:47.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002170543 2024-05-11T23:01:47.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504476+0000", 2024-05-11T23:01:47.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028470879999999998 2024-05-11T23:01:47.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504508+0000", 2024-05-11T23:01:47.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2360000000000002e-05 2024-05-11T23:01:47.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504539+0000", 2024-05-11T23:01:47.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0843000000000003e-05 2024-05-11T23:01:47.897 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.897 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.897 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.898 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4252 3.a 3:50c2a80d:::rbd_data.111e89cdde7d.0000000000000087:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:47.898 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.600827+0000", 2024-05-11T23:01:47.898 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.216780825000001, 2024-05-11T23:01:47.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054938859999999999, 2024-05-11T23:01:47.898 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.898 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.898 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.898 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4252 2024-05-11T23:01:47.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.898 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600827+0000", 2024-05-11T23:01:47.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600827+0000", 2024-05-11T23:01:47.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600829+0000", 2024-05-11T23:01:47.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2139999999999999e-06 2024-05-11T23:01:47.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600833+0000", 2024-05-11T23:01:47.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5580000000000002e-06 2024-05-11T23:01:47.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600844+0000", 2024-05-11T23:01:47.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0915e-05 2024-05-11T23:01:47.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600852+0000", 2024-05-11T23:01:47.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6260000000000003e-06 2024-05-11T23:01:47.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600924+0000", 2024-05-11T23:01:47.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1884000000000005e-05 2024-05-11T23:01:47.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.601012+0000", 2024-05-11T23:01:47.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8732000000000004e-05 2024-05-11T23:01:47.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:47.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.601137+0000", 2024-05-11T23:01:47.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012487500000000001 2024-05-11T23:01:47.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.603281+0000", 2024-05-11T23:01:47.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002144165 2024-05-11T23:01:47.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606216+0000", 2024-05-11T23:01:47.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029350790000000002 2024-05-11T23:01:47.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606277+0000", 2024-05-11T23:01:47.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0752999999999999e-05 2024-05-11T23:01:47.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606321+0000", 2024-05-11T23:01:47.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4085000000000002e-05 2024-05-11T23:01:47.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.903 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.903 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:10265 3.a 3:50e2f1fe:::rbd_data.111e520e7062.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 7667712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-05-11T23:01:47.903 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:43.006566+0000", 2024-05-11T23:01:47.903 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.8110419640000002, 2024-05-11T23:01:47.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002786359, 2024-05-11T23:01:47.903 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.903 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.903 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.903 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10265 2024-05-11T23:01:47.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.903 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.006566+0000", 2024-05-11T23:01:47.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.006566+0000", 2024-05-11T23:01:47.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.006568+0000", 2024-05-11T23:01:47.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7039999999999999e-06 2024-05-11T23:01:47.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.006654+0000", 2024-05-11T23:01:47.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5937000000000001e-05 2024-05-11T23:01:47.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.006670+0000", 2024-05-11T23:01:47.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6056e-05 2024-05-11T23:01:47.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.006679+0000", 2024-05-11T23:01:47.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6589999999999996e-06 2024-05-11T23:01:47.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.006735+0000", 2024-05-11T23:01:47.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6706000000000002e-05 2024-05-11T23:01:47.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.006854+0000", 2024-05-11T23:01:47.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011867299999999999 2024-05-11T23:01:47.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:47.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.007061+0000", 2024-05-11T23:01:47.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020734599999999999 2024-05-11T23:01:47.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.008553+0000", 2024-05-11T23:01:47.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001491968 2024-05-11T23:01:47.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.009245+0000", 2024-05-11T23:01:47.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00069169499999999998 2024-05-11T23:01:47.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.009308+0000", 2024-05-11T23:01:47.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2911999999999997e-05 2024-05-11T23:01:47.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.009353+0000", 2024-05-11T23:01:47.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4703000000000002e-05 2024-05-11T23:01:47.910 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.910 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.911 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:10456 3.10 3:0af30f4b:::rbd_data.111e520e7062.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 2359296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-05-11T23:01:47.911 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:43.092742+0000", 2024-05-11T23:01:47.911 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.7248663310000003, 2024-05-11T23:01:47.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028432539999999999, 2024-05-11T23:01:47.911 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:47.911 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:47.911 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:47.911 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:47.911 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:47.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:47.912 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 10456 2024-05-11T23:01:47.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.912 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:47.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:47.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.092742+0000", 2024-05-11T23:01:47.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:47.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.092742+0000", 2024-05-11T23:01:47.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:47.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:47.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.092743+0000", 2024-05-11T23:01:47.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3850000000000001e-06 2024-05-11T23:01:47.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:47.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.092754+0000", 2024-05-11T23:01:47.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1029000000000001e-05 2024-05-11T23:01:47.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:47.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.092771+0000", 2024-05-11T23:01:47.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6645000000000001e-05 2024-05-11T23:01:47.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:47.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.092779+0000", 2024-05-11T23:01:47.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3059999999999998e-06 2024-05-11T23:01:47.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:47.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.092836+0000", 2024-05-11T23:01:47.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6663000000000003e-05 2024-05-11T23:01:47.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:47.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.092929+0000", 2024-05-11T23:01:47.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2705000000000001e-05 2024-05-11T23:01:47.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:47.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.093040+0000", 2024-05-11T23:01:47.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011109599999999999 2024-05-11T23:01:47.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:47.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.093972+0000", 2024-05-11T23:01:47.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093239500000000003 2024-05-11T23:01:47.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:47.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.095490+0000", 2024-05-11T23:01:47.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015183989999999999 2024-05-11T23:01:47.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:47.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.095545+0000", 2024-05-11T23:01:47.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4703000000000001e-05 2024-05-11T23:01:47.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:47.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:47.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:47.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:43.095585+0000", 2024-05-11T23:01:47.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9928000000000001e-05 2024-05-11T23:01:47.916 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.916 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.916 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.916 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:47.917 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:47.917 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:47.919 INFO:teuthology.orchestra.run.smithi005.stdout:flattening image image_client.0-clone11 2024-05-11T23:01:47.919 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:01:47.919 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:47.918+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:47.984 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-05-11T23:01:48.019 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:48.019+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:48.032 INFO:teuthology.orchestra.run.smithi005.stdout:1953 punch from 0x602121f to 0x602303f, (0x1e20 bytes) 2024-05-11T23:01:48.035 INFO:teuthology.orchestra.run.smithi005.stdout:1954 read 0x1072a75 thru 0x1082945 (0xfed1 bytes) 2024-05-11T23:01:48.036 INFO:teuthology.orchestra.run.smithi005.stdout:1956 punch from 0x92d0d2 to 0x9338ee, (0x681c bytes) 2024-05-11T23:01:48.043 INFO:teuthology.orchestra.run.smithi005.stdout:1957 write 0x8a4339f thru 0x8a50946 (0xd5a8 bytes) 2024-05-11T23:01:48.047 INFO:teuthology.orchestra.run.smithi005.stdout:1959 punch from 0x80fc7c to 0x815763, (0x5ae7 bytes) 2024-05-11T23:01:48.047 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:48.047 INFO:teuthology.orchestra.run.smithi005.stdout:1961 writesame 0x1ce45ae thru 0x1ce5593 data_size 0x25(0xfe6 bytes) 2024-05-11T23:01:48.051 INFO:teuthology.orchestra.run.smithi005.stdout:1962 punch from 0x583deaa to 0x5842422, (0x4578 bytes) 2024-05-11T23:01:48.051 INFO:teuthology.orchestra.run.smithi005.stdout:1963 write 0x2521cef thru 0x252ef96 (0xd2a8 bytes) 2024-05-11T23:01:48.063 INFO:teuthology.orchestra.run.smithi005.stdout:1964 write 0x971039e thru 0x9718770 (0x83d3 bytes) 2024-05-11T23:01:48.070 INFO:teuthology.orchestra.run.smithi005.stdout:1966 trunc from 0x9718771 to 0x6e420f2 2024-05-11T23:01:48.070 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:48.071 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:48.071 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:01:48.071 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:48.081 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:01:48.081 INFO:teuthology.orchestra.run.smithi005.stdout:1967 read 0x23c81e1 thru 0x23cdf9d (0x5dbd bytes) 2024-05-11T23:01:48.089 INFO:teuthology.orchestra.run.smithi005.stdout:1969 write 0x97610a2 thru 0x976d009 (0xbf68 bytes) 2024-05-11T23:01:48.097 INFO:teuthology.orchestra.run.smithi005.stdout:1970 writesame 0x555318a thru 0x555b43a data_size 0x1(0x82b1 bytes) 2024-05-11T23:01:48.107 INFO:teuthology.orchestra.run.smithi005.stdout:1971 write 0x54a0125 thru 0x54ab788 (0xb664 bytes) 2024-05-11T23:01:48.108 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:48.108 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:48.108 INFO:teuthology.orchestra.run.smithi005.stdout:1975 writesame 0x432bf93 thru 0x432ea34 data_size 0x33(0x2aa2 bytes) 2024-05-11T23:01:48.113 INFO:teuthology.orchestra.run.smithi005.stdout:1977 read 0x5046065 thru 0x504e931 (0x88cd bytes) 2024-05-11T23:01:48.114 INFO:teuthology.orchestra.run.smithi005.stdout:1978 write 0x834236 thru 0x8403dd (0xc1a8 bytes) 2024-05-11T23:01:48.120 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:48.120+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:48.133 INFO:teuthology.orchestra.run.smithi005.stdout:1981 punch from 0xacccb1 to 0xace01e, (0x136d bytes) 2024-05-11T23:01:48.139 INFO:teuthology.orchestra.run.smithi005.stdout:1982 trunc from 0x976d00a to 0xe923d17 2024-05-11T23:01:48.159 INFO:teuthology.orchestra.run.smithi005.stdout:1983 trunc from 0xe923d17 to 0x2344f2b 2024-05-11T23:01:48.174 INFO:teuthology.orchestra.run.smithi005.stdout:1986 read 0x187ce4 thru 0x19569a (0xd9b7 bytes) 2024-05-11T23:01:48.189 INFO:teuthology.orchestra.run.smithi005.stdout:1987 write 0xa6e9a8b thru 0xa6f1f79 (0x84ef bytes) 2024-05-11T23:01:48.193 INFO:teuthology.orchestra.run.smithi005.stdout:1989 writesame 0x1245e0c thru 0x124ab1f data_size 0x2(0x4d14 bytes) 2024-05-11T23:01:48.201 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:48.202 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:48.202 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:01:48.202 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:01:48.202 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:48.202 INFO:teuthology.orchestra.run.smithi005.stdout:1990 write 0x344f9c9 thru 0x345f987 (0xffbf bytes) 2024-05-11T23:01:48.205 INFO:teuthology.orchestra.run.smithi005.stdout:1991 punch from 0x3afd6d5 to 0x3b01653, (0x3f7e bytes) 2024-05-11T23:01:48.206 INFO:teuthology.orchestra.run.smithi005.stdout:1992 read 0x58d9d9 thru 0x592546 (0x4b6e bytes) 2024-05-11T23:01:48.206 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:48.206 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:48.207 INFO:teuthology.orchestra.run.smithi005.stdout:1997 read 0x5d1d867 thru 0x5d1f087 (0x1821 bytes) 2024-05-11T23:01:48.207 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:48.207 INFO:teuthology.orchestra.run.smithi005.stdout:2002 read 0x85aa864 thru 0x85b6cdb (0xc478 bytes) 2024-05-11T23:01:48.207 INFO:teuthology.orchestra.run.smithi005.stdout:2004 punch from 0x5b54846 to 0x5b5b39d, (0x6b57 bytes) 2024-05-11T23:01:48.207 INFO:teuthology.orchestra.run.smithi005.stdout:2005 write 0x2ad1a99 thru 0x2adb2d2 (0x983a bytes) 2024-05-11T23:01:48.210 INFO:teuthology.orchestra.run.smithi005.stdout:2006 write 0x54afb5a thru 0x54ba50b (0xa9b2 bytes) 2024-05-11T23:01:48.211 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:01:48.220 INFO:teuthology.orchestra.run.smithi005.stdout:2007 read 0x772b76e thru 0x7731937 (0x61ca bytes) 2024-05-11T23:01:48.220 INFO:teuthology.orchestra.run.smithi005.stdout:2009 read 0x988e00c thru 0x98934ea (0x54df bytes) 2024-05-11T23:01:48.220 INFO:teuthology.orchestra.run.smithi005.stdout:2012 punch from 0x82432df to 0x8246ef0, (0x3c11 bytes) 2024-05-11T23:01:48.220 INFO:teuthology.orchestra.run.smithi005.stdout:2013 read 0x662c676 thru 0x6630c5f (0x45ea bytes) 2024-05-11T23:01:48.220 INFO:teuthology.orchestra.run.smithi005.stdout:2014 write 0x6b2c561 thru 0x6b2ecf3 (0x2793 bytes) 2024-05-11T23:01:48.225 INFO:teuthology.orchestra.run.smithi005.stdout:2015 read 0x725c093 thru 0x72635a3 (0x7511 bytes) 2024-05-11T23:01:48.225 INFO:teuthology.orchestra.run.smithi005.stdout:2016 writesame 0x4e764a8 thru 0x4e78eff data_size 0x5(0x2a58 bytes) 2024-05-11T23:01:48.239 INFO:teuthology.orchestra.run.smithi005.stdout:2018 writesame 0xe165e08 thru 0xe16ed23 data_size 0x2b(0x8f1c bytes) 2024-05-11T23:01:48.239 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:48.245 INFO:teuthology.orchestra.run.smithi005.stdout:2020 writesame 0xe5d4cfc thru 0xe5df005 data_size 0x29(0xa30a bytes) 2024-05-11T23:01:48.245 INFO:teuthology.orchestra.run.smithi005.stdout:2021 read 0x50cdccf thru 0x50d2dad (0x50df bytes) 2024-05-11T23:01:48.245 INFO:teuthology.orchestra.run.smithi005.stdout:2022 writesame 0x1ce7eb5 thru 0x1cf14ff data_size 0x39(0x964b bytes) 2024-05-11T23:01:48.249 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:48.249 INFO:teuthology.orchestra.run.smithi005.stdout:2025 write 0xe0d2627 thru 0xe0da72d (0x8107 bytes) 2024-05-11T23:01:48.255 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:48.255 INFO:teuthology.orchestra.run.smithi005.stdout:2030 write 0x45cb0cc thru 0x45d227c (0x71b1 bytes) 2024-05-11T23:01:48.255 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:48.255+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:48.259 INFO:teuthology.orchestra.run.smithi005.stdout:2031 write 0xab38098 thru 0xab3d482 (0x53eb bytes) 2024-05-11T23:01:48.264 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:48.264 INFO:teuthology.orchestra.run.smithi005.stdout:2035 writesame 0x819c734 thru 0x819fbfd data_size 0x2(0x34ca bytes) 2024-05-11T23:01:48.269 INFO:teuthology.orchestra.run.smithi005.stdout:2036 clone 13 order 23 su 1048576 sc 10 2024-05-11T23:01:48.323 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:48.322+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:48.324 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:48.325 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:01:48.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:01:48.325 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:01:48.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.325 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:01:48.325 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:01:48.325 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 31.159581099, 2024-05-11T23:01:48.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:01:48.325 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:48.325 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:48.325 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:48.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:48.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:48.325 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:01:48.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.326 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:48.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:48.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:01:48.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:48.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:01:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:01:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:01:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:01:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:01:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:48.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:01:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:01:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:01:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:01:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:01:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:01:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:01:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:01:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:01:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:01:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:01:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:01:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:01:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:01:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:01:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:01:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:01:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:01:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:01:48.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:01:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:01:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:01:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:01:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.331 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.796771+0000", 2024-05-11T23:01:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.526061860999999, 2024-05-11T23:01:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010000207000000001, 2024-05-11T23:01:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 139 2024-05-11T23:01:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:48.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796771+0000", 2024-05-11T23:01:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796771+0000", 2024-05-11T23:01:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796772+0000", 2024-05-11T23:01:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.46e-07 2024-05-11T23:01:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796777+0000", 2024-05-11T23:01:48.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0200000000000002e-06 2024-05-11T23:01:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796783+0000", 2024-05-11T23:01:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9630000000000003e-06 2024-05-11T23:01:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796787+0000", 2024-05-11T23:01:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9309999999999999e-06 2024-05-11T23:01:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.800763+0000", 2024-05-11T23:01:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039755750000000003 2024-05-11T23:01:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.800807+0000", 2024-05-11T23:01:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4070000000000003e-05 2024-05-11T23:01:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.800892+0000", 2024-05-11T23:01:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5402000000000007e-05 2024-05-11T23:01:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801760+0000", 2024-05-11T23:01:48.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00086746299999999996 2024-05-11T23:01:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806715+0000", 2024-05-11T23:01:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049554760000000003 2024-05-11T23:01:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806747+0000", 2024-05-11T23:01:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1606999999999997e-05 2024-05-11T23:01:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806771+0000", 2024-05-11T23:01:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4253999999999999e-05 2024-05-11T23:01:48.336 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.796795+0000", 2024-05-11T23:01:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.526037712000001, 2024-05-11T23:01:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010021855, 2024-05-11T23:01:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:48.337 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 140 2024-05-11T23:01:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796795+0000", 2024-05-11T23:01:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796795+0000", 2024-05-11T23:01:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796796+0000", 2024-05-11T23:01:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3499999999999996e-07 2024-05-11T23:01:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796852+0000", 2024-05-11T23:01:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5785000000000002e-05 2024-05-11T23:01:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796859+0000", 2024-05-11T23:01:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5560000000000002e-06 2024-05-11T23:01:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796864+0000", 2024-05-11T23:01:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0000000000000004e-06 2024-05-11T23:01:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801142+0000", 2024-05-11T23:01:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042779740000000004 2024-05-11T23:01:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801190+0000", 2024-05-11T23:01:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7466000000000003e-05 2024-05-11T23:01:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:48.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801272+0000", 2024-05-11T23:01:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2534999999999995e-05 2024-05-11T23:01:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.802308+0000", 2024-05-11T23:01:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0010360129999999999 2024-05-11T23:01:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806778+0000", 2024-05-11T23:01:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044694699999999997 2024-05-11T23:01:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:48.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806801+0000", 2024-05-11T23:01:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2972999999999998e-05 2024-05-11T23:01:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806817+0000", 2024-05-11T23:01:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6548000000000001e-05 2024-05-11T23:01:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-11T23:01:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:18.796875+0000", 2024-05-11T23:01:48.342 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 29.525958179, 2024-05-11T23:01:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0099879219999999998, 2024-05-11T23:01:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 141 2024-05-11T23:01:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796875+0000", 2024-05-11T23:01:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796875+0000", 2024-05-11T23:01:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796875+0000", 2024-05-11T23:01:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7299999999999996e-07 2024-05-11T23:01:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796879+0000", 2024-05-11T23:01:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9929999999999997e-06 2024-05-11T23:01:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796885+0000", 2024-05-11T23:01:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0279999999999997e-06 2024-05-11T23:01:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.796890+0000", 2024-05-11T23:01:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3279999999999999e-06 2024-05-11T23:01:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801781+0000", 2024-05-11T23:01:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048916599999999999 2024-05-11T23:01:48.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801824+0000", 2024-05-11T23:01:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3006999999999997e-05 2024-05-11T23:01:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.801893+0000", 2024-05-11T23:01:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8940000000000001e-05 2024-05-11T23:01:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.802554+0000", 2024-05-11T23:01:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00066069900000000005 2024-05-11T23:01:48.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806820+0000", 2024-05-11T23:01:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004265975 2024-05-11T23:01:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806843+0000", 2024-05-11T23:01:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3467000000000001e-05 2024-05-11T23:01:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.806863+0000", 2024-05-11T23:01:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9252000000000001e-05 2024-05-11T23:01:48.347 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:1966 3.7 e25/19 3:e5c0e1bc:::rbd_data.111e50384849.0000000000000051:head v 25'5, mlcod=24'1)", 2024-05-11T23:01:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.581545+0000", 2024-05-11T23:01:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.741287636999999, 2024-05-11T23:01:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0087119039999999995, 2024-05-11T23:01:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:48.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581545+0000", 2024-05-11T23:01:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581545+0000", 2024-05-11T23:01:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581546+0000", 2024-05-11T23:01:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7999999999999995e-07 2024-05-11T23:01:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581552+0000", 2024-05-11T23:01:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4049999999999997e-06 2024-05-11T23:01:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581562+0000", 2024-05-11T23:01:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0047999999999999e-05 2024-05-11T23:01:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581569+0000", 2024-05-11T23:01:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5729999999999996e-06 2024-05-11T23:01:48.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581631+0000", 2024-05-11T23:01:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2515999999999996e-05 2024-05-11T23:01:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581661+0000", 2024-05-11T23:01:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9247999999999998e-05 2024-05-11T23:01:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.590214+0000", 2024-05-11T23:01:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0085532009999999999 2024-05-11T23:01:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.590257+0000", 2024-05-11T23:01:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3333e-05 2024-05-11T23:01:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3420 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.837877+0000", 2024-05-11T23:01:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.484956308999999, 2024-05-11T23:01:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.010185928, 2024-05-11T23:01:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:48.352 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3420 2024-05-11T23:01:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.837877+0000", 2024-05-11T23:01:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:48.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.837877+0000", 2024-05-11T23:01:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.837877+0000", 2024-05-11T23:01:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3099999999999998e-07 2024-05-11T23:01:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.837880+0000", 2024-05-11T23:01:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6419999999999999e-06 2024-05-11T23:01:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.837886+0000", 2024-05-11T23:01:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8579999999999996e-06 2024-05-11T23:01:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.837890+0000", 2024-05-11T23:01:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7129999999999996e-06 2024-05-11T23:01:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.841920+0000", 2024-05-11T23:01:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040296840000000004 2024-05-11T23:01:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.841997+0000", 2024-05-11T23:01:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6942999999999998e-05 2024-05-11T23:01:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.842284+0000", 2024-05-11T23:01:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028709100000000001 2024-05-11T23:01:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.845267+0000", 2024-05-11T23:01:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029831319999999999 2024-05-11T23:01:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.848002+0000", 2024-05-11T23:01:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027353540000000002 2024-05-11T23:01:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.848033+0000", 2024-05-11T23:01:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0528999999999999e-05 2024-05-11T23:01:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.848063+0000", 2024-05-11T23:01:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9550999999999999e-05 2024-05-11T23:01:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3519 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.948570+0000", 2024-05-11T23:01:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.374262721999999, 2024-05-11T23:01:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.009020073, 2024-05-11T23:01:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.358 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3519 2024-05-11T23:01:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948570+0000", 2024-05-11T23:01:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948570+0000", 2024-05-11T23:01:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948570+0000", 2024-05-11T23:01:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3900000000000001e-07 2024-05-11T23:01:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948572+0000", 2024-05-11T23:01:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8810000000000001e-06 2024-05-11T23:01:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948576+0000", 2024-05-11T23:01:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5930000000000002e-06 2024-05-11T23:01:48.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948578+0000", 2024-05-11T23:01:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5459999999999998e-06 2024-05-11T23:01:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.951234+0000", 2024-05-11T23:01:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026551349999999999 2024-05-11T23:01:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.951314+0000", 2024-05-11T23:01:48.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9937000000000004e-05 2024-05-11T23:01:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.951680+0000", 2024-05-11T23:01:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00036631599999999999 2024-05-11T23:01:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.954851+0000", 2024-05-11T23:01:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0031709070000000002 2024-05-11T23:01:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957260+0000", 2024-05-11T23:01:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002409436 2024-05-11T23:01:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957306+0000", 2024-05-11T23:01:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5494999999999997e-05 2024-05-11T23:01:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957590+0000", 2024-05-11T23:01:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00028458800000000001 2024-05-11T23:01:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3520 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.948582+0000", 2024-05-11T23:01:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.374251097, 2024-05-11T23:01:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0091539389999999998, 2024-05-11T23:01:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3520 2024-05-11T23:01:48.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948582+0000", 2024-05-11T23:01:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948582+0000", 2024-05-11T23:01:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948582+0000", 2024-05-11T23:01:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3999999999999998e-07 2024-05-11T23:01:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948584+0000", 2024-05-11T23:01:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6080000000000001e-06 2024-05-11T23:01:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:48.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948587+0000", 2024-05-11T23:01:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0290000000000001e-06 2024-05-11T23:01:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948589+0000", 2024-05-11T23:01:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4590000000000001e-06 2024-05-11T23:01:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.952038+0000", 2024-05-11T23:01:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003448502 2024-05-11T23:01:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:48.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.952120+0000", 2024-05-11T23:01:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2115999999999998e-05 2024-05-11T23:01:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.952489+0000", 2024-05-11T23:01:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00036898600000000002 2024-05-11T23:01:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.955755+0000", 2024-05-11T23:01:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032666430000000001 2024-05-11T23:01:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957646+0000", 2024-05-11T23:01:48.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018909350000000001 2024-05-11T23:01:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957699+0000", 2024-05-11T23:01:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2231000000000002e-05 2024-05-11T23:01:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957736+0000", 2024-05-11T23:01:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7190000000000001e-05 2024-05-11T23:01:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3521 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.948592+0000", 2024-05-11T23:01:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.374240682, 2024-05-11T23:01:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00930396, 2024-05-11T23:01:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3521 2024-05-11T23:01:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.370 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948592+0000", 2024-05-11T23:01:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948592+0000", 2024-05-11T23:01:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948593+0000", 2024-05-11T23:01:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2999999999999999e-07 2024-05-11T23:01:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948594+0000", 2024-05-11T23:01:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.545e-06 2024-05-11T23:01:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948601+0000", 2024-05-11T23:01:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5350000000000001e-06 2024-05-11T23:01:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948603+0000", 2024-05-11T23:01:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8009999999999999e-06 2024-05-11T23:01:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.952822+0000", 2024-05-11T23:01:48.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042181509999999998 2024-05-11T23:01:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.952908+0000", 2024-05-11T23:01:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6292999999999995e-05 2024-05-11T23:01:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.953231+0000", 2024-05-11T23:01:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00032354399999999999 2024-05-11T23:01:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.955760+0000", 2024-05-11T23:01:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0025289259999999999 2024-05-11T23:01:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957792+0000", 2024-05-11T23:01:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020321760000000001 2024-05-11T23:01:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957853+0000", 2024-05-11T23:01:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0541000000000002e-05 2024-05-11T23:01:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957896+0000", 2024-05-11T23:01:48.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3217999999999999e-05 2024-05-11T23:01:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3522 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.948609+0000", 2024-05-11T23:01:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.37422405, 2024-05-11T23:01:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0094139299999999992, 2024-05-11T23:01:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3522 2024-05-11T23:01:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.375 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948609+0000", 2024-05-11T23:01:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948609+0000", 2024-05-11T23:01:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948609+0000", 2024-05-11T23:01:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.36e-07 2024-05-11T23:01:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948611+0000", 2024-05-11T23:01:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9530000000000002e-06 2024-05-11T23:01:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948614+0000", 2024-05-11T23:01:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3859999999999999e-06 2024-05-11T23:01:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.948617+0000", 2024-05-11T23:01:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5229999999999998e-06 2024-05-11T23:01:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.953596+0000", 2024-05-11T23:01:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049785840000000003 2024-05-11T23:01:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.953670+0000", 2024-05-11T23:01:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4146000000000005e-05 2024-05-11T23:01:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.953869+0000", 2024-05-11T23:01:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019907099999999999 2024-05-11T23:01:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.955764+0000", 2024-05-11T23:01:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018952280000000001 2024-05-11T23:01:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957939+0000", 2024-05-11T23:01:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002175424 2024-05-11T23:01:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.957992+0000", 2024-05-11T23:01:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2373000000000002e-05 2024-05-11T23:01:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:48.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.958023+0000", 2024-05-11T23:01:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1006000000000001e-05 2024-05-11T23:01:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3645 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.057376+0000", 2024-05-11T23:01:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.265456641, 2024-05-11T23:01:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0090708760000000003, 2024-05-11T23:01:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3645 2024-05-11T23:01:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057376+0000", 2024-05-11T23:01:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057376+0000", 2024-05-11T23:01:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057377+0000", 2024-05-11T23:01:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.28e-07 2024-05-11T23:01:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057378+0000", 2024-05-11T23:01:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.711e-06 2024-05-11T23:01:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057382+0000", 2024-05-11T23:01:48.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.422e-06 2024-05-11T23:01:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057384+0000", 2024-05-11T23:01:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.503e-06 2024-05-11T23:01:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.062163+0000", 2024-05-11T23:01:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047783679999999999 2024-05-11T23:01:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.062264+0000", 2024-05-11T23:01:48.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010114499999999999 2024-05-11T23:01:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.062657+0000", 2024-05-11T23:01:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039347200000000002 2024-05-11T23:01:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.064523+0000", 2024-05-11T23:01:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001865678 2024-05-11T23:01:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.066347+0000", 2024-05-11T23:01:48.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018238810000000001 2024-05-11T23:01:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.066402+0000", 2024-05-11T23:01:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5315999999999997e-05 2024-05-11T23:01:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.066447+0000", 2024-05-11T23:01:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5151999999999997e-05 2024-05-11T23:01:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3646 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.057388+0000", 2024-05-11T23:01:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.265445433, 2024-05-11T23:01:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.011967663999999999, 2024-05-11T23:01:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3646 2024-05-11T23:01:48.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057388+0000", 2024-05-11T23:01:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057388+0000", 2024-05-11T23:01:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057388+0000", 2024-05-11T23:01:48.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2399999999999999e-07 2024-05-11T23:01:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057389+0000", 2024-05-11T23:01:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.652e-06 2024-05-11T23:01:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057392+0000", 2024-05-11T23:01:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0649999999999999e-06 2024-05-11T23:01:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.057395+0000", 2024-05-11T23:01:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4490000000000002e-06 2024-05-11T23:01:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063116+0000", 2024-05-11T23:01:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0057215540000000002 2024-05-11T23:01:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063206+0000", 2024-05-11T23:01:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9476999999999997e-05 2024-05-11T23:01:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:48.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.063559+0000", 2024-05-11T23:01:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035349900000000001 2024-05-11T23:01:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.069086+0000", 2024-05-11T23:01:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0055262150000000001 2024-05-11T23:01:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.069282+0000", 2024-05-11T23:01:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019678299999999999 2024-05-11T23:01:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.069319+0000", 2024-05-11T23:01:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6112000000000003e-05 2024-05-11T23:01:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.069355+0000", 2024-05-11T23:01:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6634000000000002e-05 2024-05-11T23:01:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.391 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3670 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.088064+0000", 2024-05-11T23:01:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.234768874, 2024-05-11T23:01:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080673350000000001, 2024-05-11T23:01:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3670 2024-05-11T23:01:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.088064+0000", 2024-05-11T23:01:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.088064+0000", 2024-05-11T23:01:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.088064+0000", 2024-05-11T23:01:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0900000000000002e-07 2024-05-11T23:01:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.088067+0000", 2024-05-11T23:01:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5610000000000001e-06 2024-05-11T23:01:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.088087+0000", 2024-05-11T23:01:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0208e-05 2024-05-11T23:01:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.088093+0000", 2024-05-11T23:01:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0619999999999997e-06 2024-05-11T23:01:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.090650+0000", 2024-05-11T23:01:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0025564799999999999 2024-05-11T23:01:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.090735+0000", 2024-05-11T23:01:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4829999999999999e-05 2024-05-11T23:01:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.091092+0000", 2024-05-11T23:01:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000357771 2024-05-11T23:01:48.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.093704+0000", 2024-05-11T23:01:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026118579999999999 2024-05-11T23:01:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.096085+0000", 2024-05-11T23:01:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023805499999999999 2024-05-11T23:01:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.096109+0000", 2024-05-11T23:01:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4621e-05 2024-05-11T23:01:48.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.096131+0000", 2024-05-11T23:01:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1985e-05 2024-05-11T23:01:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3729 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.158110+0000", 2024-05-11T23:01:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.164723221999999, 2024-05-11T23:01:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0081205400000000007, 2024-05-11T23:01:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:48.397 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3729 2024-05-11T23:01:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158110+0000", 2024-05-11T23:01:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158110+0000", 2024-05-11T23:01:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158110+0000", 2024-05-11T23:01:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2099999999999997e-07 2024-05-11T23:01:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158113+0000", 2024-05-11T23:01:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0319999999999999e-06 2024-05-11T23:01:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158121+0000", 2024-05-11T23:01:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1200000000000002e-06 2024-05-11T23:01:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158126+0000", 2024-05-11T23:01:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8099999999999997e-06 2024-05-11T23:01:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.160866+0000", 2024-05-11T23:01:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027402799999999999 2024-05-11T23:01:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.160934+0000", 2024-05-11T23:01:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7203e-05 2024-05-11T23:01:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.161320+0000", 2024-05-11T23:01:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000385893 2024-05-11T23:01:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.165881+0000", 2024-05-11T23:01:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045617879999999998 2024-05-11T23:01:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.166145+0000", 2024-05-11T23:01:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00026390400000000003 2024-05-11T23:01:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.166190+0000", 2024-05-11T23:01:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4234999999999999e-05 2024-05-11T23:01:48.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.166230+0000", 2024-05-11T23:01:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0754000000000003e-05 2024-05-11T23:01:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3730 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.158133+0000", 2024-05-11T23:01:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.164700414, 2024-05-11T23:01:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0081900059999999997, 2024-05-11T23:01:48.403 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3730 2024-05-11T23:01:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158133+0000", 2024-05-11T23:01:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158133+0000", 2024-05-11T23:01:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158133+0000", 2024-05-11T23:01:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6899999999999998e-07 2024-05-11T23:01:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158136+0000", 2024-05-11T23:01:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8660000000000002e-06 2024-05-11T23:01:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:48.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158142+0000", 2024-05-11T23:01:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8340000000000004e-06 2024-05-11T23:01:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158146+0000", 2024-05-11T23:01:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7570000000000004e-06 2024-05-11T23:01:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.161757+0000", 2024-05-11T23:01:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036105709999999999 2024-05-11T23:01:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:48.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.161843+0000", 2024-05-11T23:01:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5609e-05 2024-05-11T23:01:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.162146+0000", 2024-05-11T23:01:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030325799999999999 2024-05-11T23:01:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.165905+0000", 2024-05-11T23:01:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003759063 2024-05-11T23:01:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.166264+0000", 2024-05-11T23:01:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035860500000000001 2024-05-11T23:01:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.166295+0000", 2024-05-11T23:01:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1878000000000002e-05 2024-05-11T23:01:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.166323+0000", 2024-05-11T23:01:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7095999999999999e-05 2024-05-11T23:01:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.408 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3731 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.158153+0000", 2024-05-11T23:01:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.164680199999999, 2024-05-11T23:01:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0090137840000000004, 2024-05-11T23:01:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3731 2024-05-11T23:01:48.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158153+0000", 2024-05-11T23:01:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158153+0000", 2024-05-11T23:01:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158153+0000", 2024-05-11T23:01:48.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8799999999999998e-07 2024-05-11T23:01:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158156+0000", 2024-05-11T23:01:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6469999999999998e-06 2024-05-11T23:01:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158163+0000", 2024-05-11T23:01:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3250000000000002e-06 2024-05-11T23:01:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.158168+0000", 2024-05-11T23:01:48.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8949999999999997e-06 2024-05-11T23:01:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.162490+0000", 2024-05-11T23:01:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00432176 2024-05-11T23:01:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.162571+0000", 2024-05-11T23:01:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1459999999999996e-05 2024-05-11T23:01:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.162939+0000", 2024-05-11T23:01:48.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00036788899999999999 2024-05-11T23:01:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.166338+0000", 2024-05-11T23:01:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00339895 2024-05-11T23:01:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.167091+0000", 2024-05-11T23:01:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00075337699999999995 2024-05-11T23:01:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.167139+0000", 2024-05-11T23:01:48.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7883000000000002e-05 2024-05-11T23:01:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.167167+0000", 2024-05-11T23:01:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7209999999999999e-05 2024-05-11T23:01:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3896 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.279281+0000", 2024-05-11T23:01:48.414 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.043551852, 2024-05-11T23:01:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080757069999999997, 2024-05-11T23:01:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3896 2024-05-11T23:01:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279281+0000", 2024-05-11T23:01:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279281+0000", 2024-05-11T23:01:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279281+0000", 2024-05-11T23:01:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3699999999999999e-07 2024-05-11T23:01:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:48.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279283+0000", 2024-05-11T23:01:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6729999999999999e-06 2024-05-11T23:01:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279286+0000", 2024-05-11T23:01:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2160000000000002e-06 2024-05-11T23:01:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279289+0000", 2024-05-11T23:01:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.123e-06 2024-05-11T23:01:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.283324+0000", 2024-05-11T23:01:48.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040346440000000004 2024-05-11T23:01:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.283428+0000", 2024-05-11T23:01:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010368499999999999 2024-05-11T23:01:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.283778+0000", 2024-05-11T23:01:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000350806 2024-05-11T23:01:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.287204+0000", 2024-05-11T23:01:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0034256460000000001 2024-05-11T23:01:48.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.287309+0000", 2024-05-11T23:01:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104514 2024-05-11T23:01:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.287335+0000", 2024-05-11T23:01:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6724999999999999e-05 2024-05-11T23:01:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:48.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.287357+0000", 2024-05-11T23:01:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1438e-05 2024-05-11T23:01:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3897 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.279295+0000", 2024-05-11T23:01:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.043537561000001, 2024-05-11T23:01:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089004529999999991, 2024-05-11T23:01:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:48.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3897 2024-05-11T23:01:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279295+0000", 2024-05-11T23:01:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279295+0000", 2024-05-11T23:01:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279296+0000", 2024-05-11T23:01:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4900000000000002e-07 2024-05-11T23:01:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279298+0000", 2024-05-11T23:01:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.091e-06 2024-05-11T23:01:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279301+0000", 2024-05-11T23:01:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3160000000000001e-06 2024-05-11T23:01:48.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.279304+0000", 2024-05-11T23:01:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.542e-06 2024-05-11T23:01:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.284176+0000", 2024-05-11T23:01:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048726159999999998 2024-05-11T23:01:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.284260+0000", 2024-05-11T23:01:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4270000000000005e-05 2024-05-11T23:01:48.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.284652+0000", 2024-05-11T23:01:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00039102499999999998 2024-05-11T23:01:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.287398+0000", 2024-05-11T23:01:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027467899999999998 2024-05-11T23:01:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.288099+0000", 2024-05-11T23:01:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00070078100000000004 2024-05-11T23:01:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.288154+0000", 2024-05-11T23:01:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4706000000000001e-05 2024-05-11T23:01:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.288196+0000", 2024-05-11T23:01:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2067000000000002e-05 2024-05-11T23:01:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4185 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:48.425 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.526304+0000", 2024-05-11T23:01:48.426 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.796528607000001, 2024-05-11T23:01:48.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080873190000000008, 2024-05-11T23:01:48.426 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:48.426 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:48.426 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.426 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:48.426 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:48.426 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.426 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4185 2024-05-11T23:01:48.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.426 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:48.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:48.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.526304+0000", 2024-05-11T23:01:48.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:48.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.526304+0000", 2024-05-11T23:01:48.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:48.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.526305+0000", 2024-05-11T23:01:48.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5600000000000002e-07 2024-05-11T23:01:48.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:48.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.526306+0000", 2024-05-11T23:01:48.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6500000000000001e-06 2024-05-11T23:01:48.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:48.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.526311+0000", 2024-05-11T23:01:48.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2570000000000001e-06 2024-05-11T23:01:48.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.526314+0000", 2024-05-11T23:01:48.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.597e-06 2024-05-11T23:01:48.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.529081+0000", 2024-05-11T23:01:48.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027672119999999998 2024-05-11T23:01:48.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:48.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.529160+0000", 2024-05-11T23:01:48.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9135999999999996e-05 2024-05-11T23:01:48.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:48.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.529395+0000", 2024-05-11T23:01:48.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000234737 2024-05-11T23:01:48.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:48.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.531442+0000", 2024-05-11T23:01:48.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020470470000000002 2024-05-11T23:01:48.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.534339+0000", 2024-05-11T23:01:48.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028972580000000002 2024-05-11T23:01:48.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:48.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.534369+0000", 2024-05-11T23:01:48.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9844000000000001e-05 2024-05-11T23:01:48.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:48.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.534392+0000", 2024-05-11T23:01:48.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2325000000000001e-05 2024-05-11T23:01:48.431 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.431 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.431 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.431 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4208 3.e 3:774edc63:::rbd_object_map.111e89cdde7d:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:48.431 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.550445+0000", 2024-05-11T23:01:48.431 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.772387641, 2024-05-11T23:01:48.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089990360000000002, 2024-05-11T23:01:48.431 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:48.431 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:48.431 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.432 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:48.432 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:48.432 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.432 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4208 2024-05-11T23:01:48.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.432 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:48.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:48.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.550445+0000", 2024-05-11T23:01:48.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:48.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.550445+0000", 2024-05-11T23:01:48.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:48.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:48.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.550446+0000", 2024-05-11T23:01:48.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:01:48.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:48.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.550448+0000", 2024-05-11T23:01:48.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.588e-06 2024-05-11T23:01:48.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:48.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.550455+0000", 2024-05-11T23:01:48.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5200000000000003e-06 2024-05-11T23:01:48.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.550460+0000", 2024-05-11T23:01:48.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:01:48.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:48.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.551784+0000", 2024-05-11T23:01:48.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013239790000000001 2024-05-11T23:01:48.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:48.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.551832+0000", 2024-05-11T23:01:48.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8299e-05 2024-05-11T23:01:48.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:48.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.552040+0000", 2024-05-11T23:01:48.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00020784300000000001 2024-05-11T23:01:48.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:48.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.557025+0000", 2024-05-11T23:01:48.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004985349 2024-05-11T23:01:48.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559373+0000", 2024-05-11T23:01:48.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023476719999999999 2024-05-11T23:01:48.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:48.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559408+0000", 2024-05-11T23:01:48.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4928000000000002e-05 2024-05-11T23:01:48.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:48.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:48.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:48.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.559444+0000", 2024-05-11T23:01:48.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6619000000000003e-05 2024-05-11T23:01:48.436 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.436 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.437 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.437 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:48.437 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:48.437 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:48.438 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:48.438+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:48.439 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:01:48.539 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:48.540+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:48.549 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:48.549 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:48.549 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:01:48.549 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:48.558 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:01:48.640 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:48.640+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:48.668 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:48.669 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:48.669 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:01:48.669 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:01:48.669 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:48.677 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:01:48.741 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:48.742+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:48.831 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:48.832 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:01:48.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:01:48.832 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:01:48.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.832 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:1942 3.1f 3:fc8394ee:::rbd_data.111e50384849.000000000000005d:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:48.832 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:33.575284+0000", 2024-05-11T23:01:48.832 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 15.255079176000001, 2024-05-11T23:01:48.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.010861317000000001, 2024-05-11T23:01:48.832 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:48.832 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:48.832 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:48.832 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:48.832 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:48.832 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.833 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 1942 2024-05-11T23:01:48.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.833 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:48.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:48.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.575284+0000", 2024-05-11T23:01:48.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:48.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.575284+0000", 2024-05-11T23:01:48.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:48.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.575284+0000", 2024-05-11T23:01:48.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4000000000000001e-07 2024-05-11T23:01:48.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:48.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.575297+0000", 2024-05-11T23:01:48.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2849e-05 2024-05-11T23:01:48.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:48.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.575306+0000", 2024-05-11T23:01:48.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2599999999999994e-06 2024-05-11T23:01:48.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.575320+0000", 2024-05-11T23:01:48.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3097e-05 2024-05-11T23:01:48.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:48.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.575395+0000", 2024-05-11T23:01:48.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5730999999999997e-05 2024-05-11T23:01:48.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:48.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.575553+0000", 2024-05-11T23:01:48.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000157267 2024-05-11T23:01:48.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:48.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:33.586145+0000", 2024-05-11T23:01:48.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.010592473 2024-05-11T23:01:48.836 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.836 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.836 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.836 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3786 3.12 3:4c7124e9:::rbd_data.111e89cdde7d.0000000000000002:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:48.836 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:34.190628+0000", 2024-05-11T23:01:48.836 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.639734697, 2024-05-11T23:01:48.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.010472848, 2024-05-11T23:01:48.836 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:48.836 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:48.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:48.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:48.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.837 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3786 2024-05-11T23:01:48.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.837 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:48.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:48.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.190628+0000", 2024-05-11T23:01:48.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:48.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.190628+0000", 2024-05-11T23:01:48.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:48.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.190629+0000", 2024-05-11T23:01:48.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0400000000000002e-07 2024-05-11T23:01:48.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:48.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.190631+0000", 2024-05-11T23:01:48.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.21e-06 2024-05-11T23:01:48.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:48.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.190635+0000", 2024-05-11T23:01:48.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2969999999999997e-06 2024-05-11T23:01:48.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.190638+0000", 2024-05-11T23:01:48.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8760000000000001e-06 2024-05-11T23:01:48.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:48.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.190681+0000", 2024-05-11T23:01:48.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2960999999999997e-05 2024-05-11T23:01:48.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:48.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.190725+0000", 2024-05-11T23:01:48.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4159999999999997e-05 2024-05-11T23:01:48.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 7", 2024-05-11T23:01:48.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.192115+0000", 2024-05-11T23:01:48.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001389665 2024-05-11T23:01:48.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.201039+0000", 2024-05-11T23:01:48.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0089246589999999997 2024-05-11T23:01:48.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:48.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.201047+0000", 2024-05-11T23:01:48.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9310000000000008e-06 2024-05-11T23:01:48.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:48.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.201077+0000", 2024-05-11T23:01:48.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9807000000000001e-05 2024-05-11T23:01:48.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:48.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.201101+0000", 2024-05-11T23:01:48.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3878000000000001e-05 2024-05-11T23:01:48.842 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.842 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.842 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.842 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:3789 3.12 3:4c322c18:::rbd_data.111e89cdde7d.0000000000000007:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:48.842 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:34.190647+0000", 2024-05-11T23:01:48.842 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.639715812, 2024-05-11T23:01:48.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.010570378, 2024-05-11T23:01:48.842 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:48.842 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:48.842 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.842 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:48.843 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:48.843 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.843 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 3789 2024-05-11T23:01:48.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.843 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:48.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:48.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.190647+0000", 2024-05-11T23:01:48.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:48.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.190647+0000", 2024-05-11T23:01:48.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:48.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.190647+0000", 2024-05-11T23:01:48.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:01:48.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:48.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.190650+0000", 2024-05-11T23:01:48.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8320000000000002e-06 2024-05-11T23:01:48.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:48.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.190657+0000", 2024-05-11T23:01:48.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6080000000000001e-06 2024-05-11T23:01:48.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.190662+0000", 2024-05-11T23:01:48.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2460000000000003e-06 2024-05-11T23:01:48.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:48.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.197488+0000", 2024-05-11T23:01:48.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0068258149999999998 2024-05-11T23:01:48.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:48.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.197545+0000", 2024-05-11T23:01:48.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6953999999999997e-05 2024-05-11T23:01:48.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 7", 2024-05-11T23:01:48.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.198110+0000", 2024-05-11T23:01:48.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00056499499999999999 2024-05-11T23:01:48.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:48.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.201111+0000", 2024-05-11T23:01:48.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0030015049999999998 2024-05-11T23:01:48.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.201152+0000", 2024-05-11T23:01:48.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0901e-05 2024-05-11T23:01:48.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:48.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.201191+0000", 2024-05-11T23:01:48.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8405999999999998e-05 2024-05-11T23:01:48.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:48.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.201217+0000", 2024-05-11T23:01:48.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6767e-05 2024-05-11T23:01:48.848 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.848 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.848 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.848 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:4109 3.17 3:ec04b13d:::rbd_data.111e89cdde7d.0000000000000086:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:48.848 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:34.457100+0000", 2024-05-11T23:01:48.848 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.373262491, 2024-05-11T23:01:48.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082070949999999993, 2024-05-11T23:01:48.848 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:48.848 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:48.848 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.849 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:48.849 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:48.849 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.849 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4109 2024-05-11T23:01:48.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.849 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:48.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:48.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.457100+0000", 2024-05-11T23:01:48.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:48.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.457100+0000", 2024-05-11T23:01:48.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:48.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.457101+0000", 2024-05-11T23:01:48.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0100000000000004e-07 2024-05-11T23:01:48.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:48.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.457105+0000", 2024-05-11T23:01:48.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3799999999999998e-06 2024-05-11T23:01:48.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:48.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.457113+0000", 2024-05-11T23:01:48.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:01:48.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.457117+0000", 2024-05-11T23:01:48.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3710000000000002e-06 2024-05-11T23:01:48.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:48.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.457164+0000", 2024-05-11T23:01:48.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7329e-05 2024-05-11T23:01:48.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:48.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.457236+0000", 2024-05-11T23:01:48.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1385999999999997e-05 2024-05-11T23:01:48.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 5", 2024-05-11T23:01:48.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.457356+0000", 2024-05-11T23:01:48.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011996 2024-05-11T23:01:48.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:48.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.457781+0000", 2024-05-11T23:01:48.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00042509299999999998 2024-05-11T23:01:48.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.465207+0000", 2024-05-11T23:01:48.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0074259349999999998 2024-05-11T23:01:48.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:48.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.465264+0000", 2024-05-11T23:01:48.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7501999999999999e-05 2024-05-11T23:01:48.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:48.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.465308+0000", 2024-05-11T23:01:48.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3210000000000001e-05 2024-05-11T23:01:48.853 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.853 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.853 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.854 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:4247 3.1 3:80c18ebf:::rbd_data.111e89cdde7d.0000000000000082:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:48.854 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:34.598290+0000", 2024-05-11T23:01:48.854 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.232073232999999, 2024-05-11T23:01:48.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0078326279999999995, 2024-05-11T23:01:48.854 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:48.854 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:48.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.854 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:48.854 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:48.854 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.854 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 4247 2024-05-11T23:01:48.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.854 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:48.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:48.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.598290+0000", 2024-05-11T23:01:48.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:48.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.598290+0000", 2024-05-11T23:01:48.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:48.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.598291+0000", 2024-05-11T23:01:48.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6700000000000002e-07 2024-05-11T23:01:48.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:48.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.598294+0000", 2024-05-11T23:01:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8330000000000004e-06 2024-05-11T23:01:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.598305+0000", 2024-05-11T23:01:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0756e-05 2024-05-11T23:01:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.598313+0000", 2024-05-11T23:01:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2699999999999999e-06 2024-05-11T23:01:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.598362+0000", 2024-05-11T23:01:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9515999999999998e-05 2024-05-11T23:01:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.598415+0000", 2024-05-11T23:01:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2889000000000001e-05 2024-05-11T23:01:48.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:48.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.598477+0000", 2024-05-11T23:01:48.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2551999999999993e-05 2024-05-11T23:01:48.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.598827+0000", 2024-05-11T23:01:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034993099999999999 2024-05-11T23:01:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.606007+0000", 2024-05-11T23:01:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0071796070000000002 2024-05-11T23:01:48.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.606071+0000", 2024-05-11T23:01:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3713999999999993e-05 2024-05-11T23:01:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:48.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:34.606122+0000", 2024-05-11T23:01:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1592999999999999e-05 2024-05-11T23:01:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:9700 3.17 3:ec02f975:::rbd_object_map.111e1e89614a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-05-11T23:01:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:42.051783+0000", 2024-05-11T23:01:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.7785800790000001, 2024-05-11T23:01:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0076343180000000002, 2024-05-11T23:01:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9700 2024-05-11T23:01:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.051783+0000", 2024-05-11T23:01:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.051783+0000", 2024-05-11T23:01:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.051784+0000", 2024-05-11T23:01:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6099999999999999e-07 2024-05-11T23:01:48.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.051788+0000", 2024-05-11T23:01:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1080000000000003e-06 2024-05-11T23:01:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.051796+0000", 2024-05-11T23:01:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3860000000000007e-06 2024-05-11T23:01:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.051807+0000", 2024-05-11T23:01:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0608000000000001e-05 2024-05-11T23:01:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.058089+0000", 2024-05-11T23:01:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.006281727 2024-05-11T23:01:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:48.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.058152+0000", 2024-05-11T23:01:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3212000000000004e-05 2024-05-11T23:01:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 5", 2024-05-11T23:01:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.058498+0000", 2024-05-11T23:01:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000346072 2024-05-11T23:01:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.059163+0000", 2024-05-11T23:01:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000665034 2024-05-11T23:01:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.059318+0000", 2024-05-11T23:01:48.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000155093 2024-05-11T23:01:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.059375+0000", 2024-05-11T23:01:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7349999999999998e-05 2024-05-11T23:01:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.059417+0000", 2024-05-11T23:01:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1866999999999997e-05 2024-05-11T23:01:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.868 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:9778 3.1f 3:fe2a61f7:::rbd_data.111e1e89614a.00000000000000ed:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-05-11T23:01:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:42.136365+0000", 2024-05-11T23:01:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.6939979809999999, 2024-05-11T23:01:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011932175999999999, 2024-05-11T23:01:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 9778 2024-05-11T23:01:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:48.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.136365+0000", 2024-05-11T23:01:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.136365+0000", 2024-05-11T23:01:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.136366+0000", 2024-05-11T23:01:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.09e-07 2024-05-11T23:01:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:48.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.136370+0000", 2024-05-11T23:01:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8820000000000003e-06 2024-05-11T23:01:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.136379+0000", 2024-05-11T23:01:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4709999999999996e-06 2024-05-11T23:01:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.136384+0000", 2024-05-11T23:01:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7690000000000004e-06 2024-05-11T23:01:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.136425+0000", 2024-05-11T23:01:48.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.083e-05 2024-05-11T23:01:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.136489+0000", 2024-05-11T23:01:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3924000000000007e-05 2024-05-11T23:01:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 5", 2024-05-11T23:01:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.138957+0000", 2024-05-11T23:01:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0024688760000000001 2024-05-11T23:01:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.148162+0000", 2024-05-11T23:01:48.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0092049149999999993 2024-05-11T23:01:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.148182+0000", 2024-05-11T23:01:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9262e-05 2024-05-11T23:01:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.148240+0000", 2024-05-11T23:01:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8726e-05 2024-05-11T23:01:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:42.148297+0000", 2024-05-11T23:01:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6712000000000002e-05 2024-05-11T23:01:48.873 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:10310 3.17 3:ef7daa64:::rbd_data.111e520e7062.0000000000000010:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-05-11T23:01:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:43.036132+0000", 2024-05-11T23:01:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.7942308640000002, 2024-05-11T23:01:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012010761, 2024-05-11T23:01:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.874 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 10310 2024-05-11T23:01:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:43.036132+0000", 2024-05-11T23:01:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:43.036132+0000", 2024-05-11T23:01:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:43.036133+0000", 2024-05-11T23:01:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6000000000000003e-07 2024-05-11T23:01:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:43.036137+0000", 2024-05-11T23:01:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1459999999999998e-06 2024-05-11T23:01:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:43.036147+0000", 2024-05-11T23:01:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0502999999999999e-05 2024-05-11T23:01:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:43.036153+0000", 2024-05-11T23:01:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4269999999999999e-06 2024-05-11T23:01:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:43.036198+0000", 2024-05-11T23:01:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5566999999999999e-05 2024-05-11T23:01:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:43.036252+0000", 2024-05-11T23:01:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3389e-05 2024-05-11T23:01:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 5", 2024-05-11T23:01:48.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:43.038335+0000", 2024-05-11T23:01:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002083414 2024-05-11T23:01:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:43.048047+0000", 2024-05-11T23:01:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0097114230000000003 2024-05-11T23:01:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:43.048081+0000", 2024-05-11T23:01:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4128999999999998e-05 2024-05-11T23:01:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:43.048114+0000", 2024-05-11T23:01:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3562000000000002e-05 2024-05-11T23:01:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:43.048143+0000", 2024-05-11T23:01:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8441000000000001e-05 2024-05-11T23:01:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:13425 3.6 3:65e706f8:::rbd_object_map.111e70525772:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-05-11T23:01:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:46.651857+0000", 2024-05-11T23:01:48.879 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.1785060120000002, 2024-05-11T23:01:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0074524999999999999, 2024-05-11T23:01:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 13425 2024-05-11T23:01:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.651857+0000", 2024-05-11T23:01:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.651857+0000", 2024-05-11T23:01:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.651857+0000", 2024-05-11T23:01:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-05-11T23:01:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.651859+0000", 2024-05-11T23:01:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6050000000000001e-06 2024-05-11T23:01:48.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.651863+0000", 2024-05-11T23:01:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3839999999999999e-06 2024-05-11T23:01:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.651866+0000", 2024-05-11T23:01:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6919999999999998e-06 2024-05-11T23:01:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.657545+0000", 2024-05-11T23:01:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.005679501 2024-05-11T23:01:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.657627+0000", 2024-05-11T23:01:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1815000000000002e-05 2024-05-11T23:01:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-05-11T23:01:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.657993+0000", 2024-05-11T23:01:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00036615200000000001 2024-05-11T23:01:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.658683+0000", 2024-05-11T23:01:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00068947099999999999 2024-05-11T23:01:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.659220+0000", 2024-05-11T23:01:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00053738099999999997 2024-05-11T23:01:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.659273+0000", 2024-05-11T23:01:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3214999999999998e-05 2024-05-11T23:01:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.659309+0000", 2024-05-11T23:01:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6069999999999999e-05 2024-05-11T23:01:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:13476 3.6 3:65e706f8:::rbd_object_map.111e70525772:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-05-11T23:01:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:46.679577+0000", 2024-05-11T23:01:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.150785859, 2024-05-11T23:01:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0076087280000000004, 2024-05-11T23:01:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 13476 2024-05-11T23:01:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:48.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.679577+0000", 2024-05-11T23:01:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.679577+0000", 2024-05-11T23:01:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.679577+0000", 2024-05-11T23:01:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.28e-07 2024-05-11T23:01:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.679579+0000", 2024-05-11T23:01:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6619999999999999e-06 2024-05-11T23:01:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.679582+0000", 2024-05-11T23:01:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4120000000000001e-06 2024-05-11T23:01:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.679585+0000", 2024-05-11T23:01:48.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4899999999999999e-06 2024-05-11T23:01:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.684453+0000", 2024-05-11T23:01:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0048678139999999998 2024-05-11T23:01:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.684539+0000", 2024-05-11T23:01:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6823999999999994e-05 2024-05-11T23:01:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-05-11T23:01:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.684866+0000", 2024-05-11T23:01:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032632100000000002 2024-05-11T23:01:48.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.686388+0000", 2024-05-11T23:01:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015218810000000001 2024-05-11T23:01:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.687101+0000", 2024-05-11T23:01:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00071315300000000001 2024-05-11T23:01:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.687146+0000", 2024-05-11T23:01:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4781999999999999e-05 2024-05-11T23:01:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.687186+0000", 2024-05-11T23:01:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0160999999999997e-05 2024-05-11T23:01:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:13477 3.6 3:65e706f8:::rbd_object_map.111e70525772:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-05-11T23:01:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:46.679588+0000", 2024-05-11T23:01:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.1507748470000001, 2024-05-11T23:01:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0077096409999999997, 2024-05-11T23:01:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:48.890 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 13477 2024-05-11T23:01:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.679588+0000", 2024-05-11T23:01:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.679588+0000", 2024-05-11T23:01:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.679588+0000", 2024-05-11T23:01:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:01:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.679590+0000", 2024-05-11T23:01:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4789999999999999e-06 2024-05-11T23:01:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.679593+0000", 2024-05-11T23:01:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1889999999999999e-06 2024-05-11T23:01:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.679595+0000", 2024-05-11T23:01:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3290000000000001e-06 2024-05-11T23:01:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.685373+0000", 2024-05-11T23:01:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0057773399999999997 2024-05-11T23:01:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.685451+0000", 2024-05-11T23:01:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8263000000000006e-05 2024-05-11T23:01:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-05-11T23:01:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.685792+0000", 2024-05-11T23:01:48.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00034079999999999999 2024-05-11T23:01:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.687200+0000", 2024-05-11T23:01:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001408053 2024-05-11T23:01:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.687233+0000", 2024-05-11T23:01:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2938999999999998e-05 2024-05-11T23:01:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.687278+0000", 2024-05-11T23:01:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5652999999999998e-05 2024-05-11T23:01:48.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.687298+0000", 2024-05-11T23:01:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9386999999999999e-05 2024-05-11T23:01:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:13478 3.6 3:65e706f8:::rbd_object_map.111e70525772:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-05-11T23:01:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:46.679598+0000", 2024-05-11T23:01:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.1507644180000001, 2024-05-11T23:01:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0081028530000000001, 2024-05-11T23:01:48.895 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 13478 2024-05-11T23:01:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.679598+0000", 2024-05-11T23:01:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.679598+0000", 2024-05-11T23:01:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.679599+0000", 2024-05-11T23:01:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.28e-07 2024-05-11T23:01:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.679600+0000", 2024-05-11T23:01:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.545e-06 2024-05-11T23:01:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.679603+0000", 2024-05-11T23:01:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.14e-06 2024-05-11T23:01:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.679606+0000", 2024-05-11T23:01:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.413e-06 2024-05-11T23:01:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.686451+0000", 2024-05-11T23:01:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0068453730000000001 2024-05-11T23:01:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.686509+0000", 2024-05-11T23:01:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8298000000000003e-05 2024-05-11T23:01:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-05-11T23:01:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.686748+0000", 2024-05-11T23:01:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000238617 2024-05-11T23:01:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.687427+0000", 2024-05-11T23:01:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000678854 2024-05-11T23:01:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.687633+0000", 2024-05-11T23:01:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000206207 2024-05-11T23:01:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.687663+0000", 2024-05-11T23:01:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9691000000000001e-05 2024-05-11T23:01:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.687701+0000", 2024-05-11T23:01:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8486999999999999e-05 2024-05-11T23:01:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.900 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:13842 3.6 3:65e706f8:::rbd_object_map.111e70525772:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-05-11T23:01:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:46.884484+0000", 2024-05-11T23:01:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.9458794080000001, 2024-05-11T23:01:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0077482430000000001, 2024-05-11T23:01:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 13842 2024-05-11T23:01:48.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.884484+0000", 2024-05-11T23:01:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.884484+0000", 2024-05-11T23:01:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.884484+0000", 2024-05-11T23:01:48.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3200000000000001e-07 2024-05-11T23:01:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.884486+0000", 2024-05-11T23:01:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.809e-06 2024-05-11T23:01:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.884489+0000", 2024-05-11T23:01:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8380000000000004e-06 2024-05-11T23:01:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.884492+0000", 2024-05-11T23:01:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.435e-06 2024-05-11T23:01:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.890786+0000", 2024-05-11T23:01:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0062938079999999997 2024-05-11T23:01:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.890861+0000", 2024-05-11T23:01:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5356000000000002e-05 2024-05-11T23:01:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 1", 2024-05-11T23:01:48.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.891162+0000", 2024-05-11T23:01:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00030139799999999998 2024-05-11T23:01:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.891862+0000", 2024-05-11T23:01:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00069939600000000002 2024-05-11T23:01:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.892163+0000", 2024-05-11T23:01:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00030077599999999999 2024-05-11T23:01:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.892192+0000", 2024-05-11T23:01:48.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9433000000000001e-05 2024-05-11T23:01:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:46.892232+0000", 2024-05-11T23:01:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9762000000000003e-05 2024-05-11T23:01:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:14637 3.17 3:ebe2ea54:::rbd_data.111e70525772.0000000000000013:e [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-05-11T23:01:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:48.120106+0000", 2024-05-11T23:01:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.71025728499999996, 2024-05-11T23:01:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0075558129999999998, 2024-05-11T23:01:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:48.906 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 14637 2024-05-11T23:01:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120106+0000", 2024-05-11T23:01:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120106+0000", 2024-05-11T23:01:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120106+0000", 2024-05-11T23:01:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.03e-07 2024-05-11T23:01:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120108+0000", 2024-05-11T23:01:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6929999999999999e-06 2024-05-11T23:01:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120113+0000", 2024-05-11T23:01:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2399999999999998e-06 2024-05-11T23:01:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120116+0000", 2024-05-11T23:01:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8289999999999999e-06 2024-05-11T23:01:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.127467+0000", 2024-05-11T23:01:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0073509150000000004 2024-05-11T23:01:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.127557+0000", 2024-05-11T23:01:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0138999999999999e-05 2024-05-11T23:01:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.127661+0000", 2024-05-11T23:01:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000104694 2024-05-11T23:01:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:14676 3.17 3:ec6e9d67:::rbd_data.111e70525772.000000000000001a:e [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-05-11T23:01:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:48.120119+0000", 2024-05-11T23:01:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.71024373799999996, 2024-05-11T23:01:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0077305869999999997, 2024-05-11T23:01:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:48.910 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 14676 2024-05-11T23:01:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120119+0000", 2024-05-11T23:01:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120119+0000", 2024-05-11T23:01:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:48.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120119+0000", 2024-05-11T23:01:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-05-11T23:01:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120121+0000", 2024-05-11T23:01:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.198e-06 2024-05-11T23:01:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120125+0000", 2024-05-11T23:01:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2799999999999997e-06 2024-05-11T23:01:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120128+0000", 2024-05-11T23:01:48.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6170000000000001e-06 2024-05-11T23:01:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.127698+0000", 2024-05-11T23:01:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0075703250000000001 2024-05-11T23:01:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.127771+0000", 2024-05-11T23:01:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3110999999999998e-05 2024-05-11T23:01:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.127850+0000", 2024-05-11T23:01:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8826999999999996e-05 2024-05-11T23:01:48.913 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:14677 3.17 3:ec6e9d67:::rbd_data.111e70525772.000000000000001a:e [read 28672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-05-11T23:01:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:48.120131+0000", 2024-05-11T23:01:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.71023217999999999, 2024-05-11T23:01:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0078601890000000001, 2024-05-11T23:01:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 14677 2024-05-11T23:01:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.914 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120131+0000", 2024-05-11T23:01:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120131+0000", 2024-05-11T23:01:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120131+0000", 2024-05-11T23:01:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.28e-07 2024-05-11T23:01:48.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120132+0000", 2024-05-11T23:01:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.139e-06 2024-05-11T23:01:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120135+0000", 2024-05-11T23:01:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2320000000000001e-06 2024-05-11T23:01:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120138+0000", 2024-05-11T23:01:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5189999999999999e-06 2024-05-11T23:01:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.127878+0000", 2024-05-11T23:01:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0077401950000000001 2024-05-11T23:01:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.127927+0000", 2024-05-11T23:01:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9125999999999997e-05 2024-05-11T23:01:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.127991+0000", 2024-05-11T23:01:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3750000000000005e-05 2024-05-11T23:01:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.917 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:14678 3.17 3:ec6e9d67:::rbd_data.111e70525772.000000000000001a:e [read 53248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-05-11T23:01:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:48.120142+0000", 2024-05-11T23:01:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.71022107400000001, 2024-05-11T23:01:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0079734630000000001, 2024-05-11T23:01:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.918 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 14678 2024-05-11T23:01:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120142+0000", 2024-05-11T23:01:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120142+0000", 2024-05-11T23:01:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120142+0000", 2024-05-11T23:01:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1199999999999999e-07 2024-05-11T23:01:48.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120144+0000", 2024-05-11T23:01:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7600000000000001e-06 2024-05-11T23:01:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120149+0000", 2024-05-11T23:01:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5500000000000002e-06 2024-05-11T23:01:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120152+0000", 2024-05-11T23:01:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7089999999999998e-06 2024-05-11T23:01:48.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.128012+0000", 2024-05-11T23:01:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0078601000000000001 2024-05-11T23:01:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.128054+0000", 2024-05-11T23:01:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2030000000000002e-05 2024-05-11T23:01:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.128115+0000", 2024-05-11T23:01:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1002000000000003e-05 2024-05-11T23:01:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.921 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:14679 3.17 3:ec6e9d67:::rbd_data.111e70525772.000000000000001a:e [read 77824~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-05-11T23:01:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:48.120156+0000", 2024-05-11T23:01:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.71020719499999996, 2024-05-11T23:01:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0080821380000000009, 2024-05-11T23:01:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.922 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 14679 2024-05-11T23:01:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120156+0000", 2024-05-11T23:01:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120156+0000", 2024-05-11T23:01:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120156+0000", 2024-05-11T23:01:48.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-05-11T23:01:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120158+0000", 2024-05-11T23:01:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.669e-06 2024-05-11T23:01:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120163+0000", 2024-05-11T23:01:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8320000000000002e-06 2024-05-11T23:01:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120168+0000", 2024-05-11T23:01:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5680000000000001e-06 2024-05-11T23:01:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.128138+0000", 2024-05-11T23:01:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0079700810000000004 2024-05-11T23:01:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.128183+0000", 2024-05-11T23:01:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4703999999999998e-05 2024-05-11T23:01:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.128238+0000", 2024-05-11T23:01:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5059999999999998e-05 2024-05-11T23:01:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:14680 3.17 3:ec6e9d67:::rbd_data.111e70525772.000000000000001a:e [read 102400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-05-11T23:01:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:48.120176+0000", 2024-05-11T23:01:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.710187232, 2024-05-11T23:01:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0081752609999999996, 2024-05-11T23:01:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:48.926 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 14680 2024-05-11T23:01:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120176+0000", 2024-05-11T23:01:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120176+0000", 2024-05-11T23:01:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120176+0000", 2024-05-11T23:01:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9200000000000002e-07 2024-05-11T23:01:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120183+0000", 2024-05-11T23:01:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2570000000000001e-06 2024-05-11T23:01:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120189+0000", 2024-05-11T23:01:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3630000000000002e-06 2024-05-11T23:01:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120191+0000", 2024-05-11T23:01:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8609999999999998e-06 2024-05-11T23:01:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.128257+0000", 2024-05-11T23:01:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0080660249999999992 2024-05-11T23:01:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.128295+0000", 2024-05-11T23:01:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7067999999999998e-05 2024-05-11T23:01:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.128351+0000", 2024-05-11T23:01:48.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6394999999999998e-05 2024-05-11T23:01:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:14681 3.17 3:ec6e9d67:::rbd_data.111e70525772.000000000000001a:e [read 126976~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-05-11T23:01:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:48.120196+0000", 2024-05-11T23:01:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.71016688500000003, 2024-05-11T23:01:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082829510000000002, 2024-05-11T23:01:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:48.930 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 14681 2024-05-11T23:01:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120196+0000", 2024-05-11T23:01:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120196+0000", 2024-05-11T23:01:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120196+0000", 2024-05-11T23:01:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2999999999999999e-07 2024-05-11T23:01:48.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120198+0000", 2024-05-11T23:01:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3149999999999999e-06 2024-05-11T23:01:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120202+0000", 2024-05-11T23:01:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3460000000000004e-06 2024-05-11T23:01:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.120208+0000", 2024-05-11T23:01:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9179999999999999e-06 2024-05-11T23:01:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:48.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.128365+0000", 2024-05-11T23:01:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0081576110000000004 2024-05-11T23:01:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.128421+0000", 2024-05-11T23:01:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5312999999999997e-05 2024-05-11T23:01:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:48.128479+0000", 2024-05-11T23:01:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8217999999999997e-05 2024-05-11T23:01:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:48.933 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:48.934 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:01:48.936 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:01:48.936 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone12 from 0xe5df006 (overlap 0x2344f2b) to 0x1fbe0d9 2024-05-11T23:01:48.936 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:48.935+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:48.962 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-05-11T23:01:49.036 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:49.035+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:49.097 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:49.098 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:49.098 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:01:49.098 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:49.107 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:01:49.111 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:01:49.111 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:01:49.116 INFO:teuthology.orchestra.run.smithi005.stdout:2037 write 0xa3adb86 thru 0xa3b934d (0xb7c8 bytes) 2024-05-11T23:01:49.124 INFO:teuthology.orchestra.run.smithi005.stdout:2038 writesame 0xb914a72 thru 0xb91557d data_size 0x4(0xb0c bytes) 2024-05-11T23:01:49.128 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:49.128 INFO:teuthology.orchestra.run.smithi005.stdout:2041 punch from 0x3fe98e6 to 0x3ff9487, (0xfba1 bytes) 2024-05-11T23:01:49.128 INFO:teuthology.orchestra.run.smithi005.stdout:2042 clone 14 order 20 su 4096 sc 8 2024-05-11T23:01:49.132 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:01:49.132 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:01:49.136 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:49.136+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:49.150 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:01:49.150 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:01:49.167 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:01:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:01:49.185 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:01:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:01:49.203 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:01:49.203 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:01:49.220 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:49.220 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:49.221 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:01:49.221 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:01:49.229 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:01:49.238 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:01:49.238 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:01:49.239 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:49.238+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:49.337 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:49.337+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:49.347 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:49.347 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:49.355 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:01:49.438 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:49.438+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:49.504 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:49.505 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:49.513 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:01:49.557 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:49.557+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:49.623 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:49.623 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:49.631 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:01:49.673 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:49.673+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:49.742 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:49.742+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:49.808 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:49.808 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:49.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:01:49.842 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:49.842+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:49.887 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone13 from 0xe5df006 (overlap 0xe5df006) to 0x72ef803 2024-05-11T23:01:49.920 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-11T23:01:49.933 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:49.933 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:49.943 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:49.943+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:49.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:01:50.038 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-05-11T23:01:50.044 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:50.044+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:50.136 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:50.136 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:50.145 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:01:50.146 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:50.145+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:50.246 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:50.246+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:50.294 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:01:50.295 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:01:50.295 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-05-11T23:01:50.295 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-11T23:01:50.298 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:50.298 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:50.308 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:01:50.323 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:50.323 INFO:teuthology.orchestra.run.smithi005.stdout:2046 read 0x96ffa83 thru 0x9706c30 (0x71ae bytes) 2024-05-11T23:01:50.323 INFO:teuthology.orchestra.run.smithi005.stdout:2047 punch from 0x4874495 to 0x487a19b, (0x5d06 bytes) 2024-05-11T23:01:50.326 INFO:teuthology.orchestra.run.smithi005.stdout:2049 read 0x70d7816 thru 0x70e610f (0xe8fa bytes) 2024-05-11T23:01:50.327 INFO:teuthology.orchestra.run.smithi005.stdout:2050 write 0xc7d3968 thru 0xc7e06e3 (0xcd7c bytes) 2024-05-11T23:01:50.349 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:50.347+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:50.390 INFO:teuthology.orchestra.run.smithi005.stdout:2052 read 0xad4b211 thru 0xad52b67 (0x7957 bytes) 2024-05-11T23:01:50.391 INFO:teuthology.orchestra.run.smithi005.stdout:2053 writesame 0x78e7196 thru 0x78ea0e7 data_size 0x9(0x2f52 bytes) 2024-05-11T23:01:50.416 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:50.416 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:50.420 INFO:teuthology.orchestra.run.smithi005.stdout:2058 trunc from 0xe5df006 to 0xa7a6de7 2024-05-11T23:01:50.428 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:50.429 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:50.442 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:01:50.485 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:50.485+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:50.519 INFO:teuthology.orchestra.run.smithi005.stdout:2059 writesame 0xd4fb344 thru 0xd505402 data_size 0x57(0xa0bf bytes) 2024-05-11T23:01:50.519 INFO:teuthology.orchestra.run.smithi005.stdout:2060 punch from 0xbf4fd38 to 0xbf5d085, (0xd34d bytes) 2024-05-11T23:01:50.519 INFO:teuthology.orchestra.run.smithi005.stdout:2062 write 0xab71251 thru 0xab77492 (0x6242 bytes) 2024-05-11T23:01:50.527 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:50.533 INFO:teuthology.orchestra.run.smithi005.stdout:2066 trunc from 0xd505403 to 0x810fa4c 2024-05-11T23:01:50.550 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:50.550+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:50.572 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:50.572 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:50.585 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:01:50.610 INFO:teuthology.orchestra.run.smithi005.stdout:2067 write 0xac45253 thru 0xac47c12 (0x29c0 bytes) 2024-05-11T23:01:50.616 INFO:teuthology.orchestra.run.smithi005.stdout:2069 writesame 0xa3796e9 thru 0xa37f6d1 data_size 0x1(0x5fe9 bytes) 2024-05-11T23:01:50.626 INFO:teuthology.orchestra.run.smithi005.stdout:2070 punch from 0x5f99c70 to 0x5fa4ca9, (0xb039 bytes) 2024-05-11T23:01:50.626 INFO:teuthology.orchestra.run.smithi005.stdout:2071 writesame 0xa508978 thru 0xa5140aa data_size 0x51(0xb733 bytes) 2024-05-11T23:01:50.629 INFO:teuthology.orchestra.run.smithi005.stdout:2072 writesame 0x230e18d thru 0x2316901 data_size 0x3(0x8775 bytes) 2024-05-11T23:01:50.669 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:50.669+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:50.693 INFO:teuthology.orchestra.run.smithi005.stdout:2075 writesame 0x41bdee7 thru 0x41c343e data_size 0x4(0x5558 bytes) 2024-05-11T23:01:50.741 INFO:teuthology.orchestra.run.smithi005.stdout:2076 write 0x87edc62 thru 0x87f7710 (0x9aaf bytes) 2024-05-11T23:01:50.745 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:50.745 INFO:teuthology.orchestra.run.smithi005.stdout:2079 writesame 0x6b406e7 thru 0x6b41e8f data_size 0x1(0x17a9 bytes) 2024-05-11T23:01:50.751 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:50.750+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:50.758 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:50.758 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:01:50.765 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 38, 2024-05-11T23:01:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "fsid": "34b20022-f11b-4169-a5f6-6386270a2f8f", 2024-05-11T23:01:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "created": "2024-05-11T23:01:05.977693+0000", 2024-05-11T23:01:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "modified": "2024-05-11T23:01:49.813789+0000", 2024-05-11T23:01:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "last_up_change": "2024-05-11T23:01:17.148916+0000", 2024-05-11T23:01:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "last_in_change": "2024-05-11T23:01:08.985275+0000", 2024-05-11T23:01:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-11T23:01:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_num": 5799936, 2024-05-11T23:01:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_set": [ 2024-05-11T23:01:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "pglog_hardlimit", 2024-05-11T23:01:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "purged_snapdirs", 2024-05-11T23:01:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_deletes", 2024-05-11T23:01:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "sortbitwise" 2024-05-11T23:01:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:01:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_version": 5, 2024-05-11T23:01:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "full_ratio": 0.94999998807907104, 2024-05-11T23:01:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-11T23:01:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-11T23:01:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_snapshot": "", 2024-05-11T23:01:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_max": 3, 2024-05-11T23:01:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd": 8, 2024-05-11T23:01:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "require_min_compat_client": "luminous", 2024-05-11T23:01:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "min_compat_client": "jewel", 2024-05-11T23:01:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "require_osd_release": "squid", 2024-05-11T23:01:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_crimson": false, 2024-05-11T23:01:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "pools": [ 2024-05-11T23:01:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 1, 2024-05-11T23:01:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": ".mgr", 2024-05-11T23:01:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-05-11T23:01:15.187614+0000", 2024-05-11T23:01:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-11T23:01:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool", 2024-05-11T23:01:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-05-11T23:01:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-05-11T23:01:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-05-11T23:01:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-05-11T23:01:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-05-11T23:01:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-05-11T23:01:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-05-11T23:01:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-11T23:01:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-05-11T23:01:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-05-11T23:01:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 1, 2024-05-11T23:01:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 1, 2024-05-11T23:01:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 1, 2024-05-11T23:01:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 1, 2024-05-11T23:01:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 1, 2024-05-11T23:01:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-05-11T23:01:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-05-11T23:01:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-05-11T23:01:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-05-11T23:01:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-05-11T23:01:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-05-11T23:01:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-05-11T23:01:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "17", 2024-05-11T23:01:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-05-11T23:01:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-11T23:01:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-05-11T23:01:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-05-11T23:01:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-05-11T23:01:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 0, 2024-05-11T23:01:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 0, 2024-05-11T23:01:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-05-11T23:01:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-05-11T23:01:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-05-11T23:01:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-05-11T23:01:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-05-11T23:01:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-05-11T23:01:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-05-11T23:01:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-05-11T23:01:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-05-11T23:01:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-05-11T23:01:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-05-11T23:01:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-11T23:01:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-11T23:01:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-11T23:01:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-05-11T23:01:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-05-11T23:01:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-05-11T23:01:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-05-11T23:01:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-05-11T23:01:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-05-11T23:01:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-05-11T23:01:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-05-11T23:01:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-05-11T23:01:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-05-11T23:01:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-05-11T23:01:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-05-11T23:01:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-05-11T23:01:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-05-11T23:01:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-05-11T23:01:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-05-11T23:01:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "options": { 2024-05-11T23:01:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_max": 32, 2024-05-11T23:01:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_min": 1 2024-05-11T23:01:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-05-11T23:01:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": {} 2024-05-11T23:01:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-05-11T23:01:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-05-11T23:01:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 8, 2024-05-11T23:01:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 8, 2024-05-11T23:01:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 0.25, 2024-05-11T23:01:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2, 2024-05-11T23:01:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2, 2024-05-11T23:01:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 1, 2024-05-11T23:01:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 1, 2024-05-11T23:01:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 1 2024-05-11T23:01:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 2, 2024-05-11T23:01:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "rbd", 2024-05-11T23:01:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-05-11T23:01:17.689285+0000", 2024-05-11T23:01:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-11T23:01:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool", 2024-05-11T23:01:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-05-11T23:01:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-05-11T23:01:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-05-11T23:01:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-05-11T23:01:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-05-11T23:01:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-05-11T23:01:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-05-11T23:01:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-11T23:01:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-05-11T23:01:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-05-11T23:01:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 8, 2024-05-11T23:01:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 8, 2024-05-11T23:01:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 8, 2024-05-11T23:01:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 8, 2024-05-11T23:01:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 8, 2024-05-11T23:01:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-05-11T23:01:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-05-11T23:01:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-05-11T23:01:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-05-11T23:01:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-05-11T23:01:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-05-11T23:01:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-05-11T23:01:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "18", 2024-05-11T23:01:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-05-11T23:01:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-11T23:01:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-05-11T23:01:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-05-11T23:01:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-05-11T23:01:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 0, 2024-05-11T23:01:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 0, 2024-05-11T23:01:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-05-11T23:01:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-05-11T23:01:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-05-11T23:01:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-05-11T23:01:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-05-11T23:01:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-05-11T23:01:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-05-11T23:01:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-05-11T23:01:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-05-11T23:01:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-05-11T23:01:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-05-11T23:01:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-11T23:01:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-11T23:01:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-11T23:01:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-05-11T23:01:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-05-11T23:01:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-05-11T23:01:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-05-11T23:01:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-05-11T23:01:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-05-11T23:01:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-05-11T23:01:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-05-11T23:01:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-05-11T23:01:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-05-11T23:01:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-05-11T23:01:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-05-11T23:01:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-05-11T23:01:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-05-11T23:01:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-05-11T23:01:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-05-11T23:01:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-05-11T23:01:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-05-11T23:01:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-05-11T23:01:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-05-11T23:01:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-05-11T23:01:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 1.9900000095367432, 2024-05-11T23:01:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 1.9900000095367432, 2024-05-11T23:01:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 0.87999999523162842, 2024-05-11T23:01:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 1.75, 2024-05-11T23:01:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 1.75, 2024-05-11T23:01:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 1, 2024-05-11T23:01:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 1, 2024-05-11T23:01:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 1 2024-05-11T23:01:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 3, 2024-05-11T23:01:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "pool_client.0", 2024-05-11T23:01:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-05-11T23:01:25.743347+0000", 2024-05-11T23:01:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-05-11T23:01:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-11T23:01:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-05-11T23:01:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-05-11T23:01:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-05-11T23:01:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-05-11T23:01:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-05-11T23:01:50.780 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-05-11T23:01:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-05-11T23:01:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-11T23:01:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-05-11T23:01:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-05-11T23:01:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 32, 2024-05-11T23:01:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 32, 2024-05-11T23:01:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 32, 2024-05-11T23:01:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 32, 2024-05-11T23:01:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 32, 2024-05-11T23:01:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-05-11T23:01:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-05-11T23:01:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-05-11T23:01:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-05-11T23:01:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-05-11T23:01:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-05-11T23:01:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-05-11T23:01:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "38", 2024-05-11T23:01:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-05-11T23:01:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-11T23:01:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-05-11T23:01:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-05-11T23:01:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-05-11T23:01:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 16, 2024-05-11T23:01:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 38, 2024-05-11T23:01:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-05-11T23:01:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-05-11T23:01:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-05-11T23:01:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-05-11T23:01:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-05-11T23:01:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-05-11T23:01:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-05-11T23:01:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-05-11T23:01:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-05-11T23:01:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-05-11T23:01:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-05-11T23:01:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-11T23:01:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-11T23:01:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-11T23:01:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-05-11T23:01:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-05-11T23:01:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-05-11T23:01:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-05-11T23:01:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-05-11T23:01:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-05-11T23:01:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-05-11T23:01:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-05-11T23:01:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-05-11T23:01:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-05-11T23:01:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-05-11T23:01:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-05-11T23:01:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-05-11T23:01:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-05-11T23:01:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-05-11T23:01:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-05-11T23:01:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-05-11T23:01:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-05-11T23:01:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-05-11T23:01:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-05-11T23:01:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-05-11T23:01:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 1.5, 2024-05-11T23:01:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 1.5, 2024-05-11T23:01:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 1, 2024-05-11T23:01:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 1.5, 2024-05-11T23:01:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 1.5, 2024-05-11T23:01:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 1, 2024-05-11T23:01:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 1, 2024-05-11T23:01:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 1 2024-05-11T23:01:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:01:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "osds": [ 2024-05-11T23:01:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-05-11T23:01:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "57ab1c70-9478-4a26-bb41-d753b4f3eb79", 2024-05-11T23:01:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:01:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:01:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 0.95001220703125, 2024-05-11T23:01:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:01:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:01:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:01:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 13, 2024-05-11T23:01:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 19, 2024-05-11T23:01:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:01:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:01:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:01:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-05-11T23:01:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:01:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-05-11T23:01:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:01:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:01:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6802", 2024-05-11T23:01:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:01:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6803", 2024-05-11T23:01:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:01:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:01:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6806", 2024-05-11T23:01:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:01:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6808", 2024-05-11T23:01:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:01:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:01:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6804", 2024-05-11T23:01:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:01:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6805", 2024-05-11T23:01:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:01:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6801/3452725643", 2024-05-11T23:01:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6803/3452725643", 2024-05-11T23:01:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6808/3452725643", 2024-05-11T23:01:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6805/3452725643", 2024-05-11T23:01:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:01:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:01:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:01:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-05-11T23:01:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "05ece110-4a99-46f0-b061-5980f7ef2a55", 2024-05-11T23:01:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:01:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:01:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 0.95001220703125, 2024-05-11T23:01:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:01:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:01:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:01:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 14, 2024-05-11T23:01:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 19, 2024-05-11T23:01:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:01:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:01:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:01:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6824", 2024-05-11T23:01:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:01:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6825", 2024-05-11T23:01:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:01:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:01:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6826", 2024-05-11T23:01:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:01:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6827", 2024-05-11T23:01:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:01:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:01:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6830", 2024-05-11T23:01:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:01:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6831", 2024-05-11T23:01:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:01:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:01:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6828", 2024-05-11T23:01:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:01:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6829", 2024-05-11T23:01:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:01:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6825/4028727138", 2024-05-11T23:01:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6827/4028727138", 2024-05-11T23:01:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6831/4028727138", 2024-05-11T23:01:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6829/4028727138", 2024-05-11T23:01:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:01:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:01:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:01:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-05-11T23:01:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5", 2024-05-11T23:01:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:01:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:01:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:01:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:01:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:01:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:01:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 13, 2024-05-11T23:01:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 16, 2024-05-11T23:01:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:01:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:01:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:01:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6807", 2024-05-11T23:01:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:01:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6809", 2024-05-11T23:01:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:01:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:01:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6810", 2024-05-11T23:01:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:01:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6811", 2024-05-11T23:01:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:01:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:01:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6814", 2024-05-11T23:01:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:01:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6815", 2024-05-11T23:01:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:01:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:01:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6812", 2024-05-11T23:01:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:01:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6813", 2024-05-11T23:01:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:01:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6809/2059959509", 2024-05-11T23:01:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6811/2059959509", 2024-05-11T23:01:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6815/2059959509", 2024-05-11T23:01:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6813/2059959509", 2024-05-11T23:01:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:01:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:01:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:01:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-05-11T23:01:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a", 2024-05-11T23:01:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:01:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:01:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 0.9000244140625, 2024-05-11T23:01:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:01:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:01:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:01:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 14, 2024-05-11T23:01:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 19, 2024-05-11T23:01:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:01:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:01:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:01:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6816", 2024-05-11T23:01:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:01:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6817", 2024-05-11T23:01:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:01:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:01:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6818", 2024-05-11T23:01:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:01:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6819", 2024-05-11T23:01:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:01:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:01:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6822", 2024-05-11T23:01:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:01:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6823", 2024-05-11T23:01:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:01:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:01:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6820", 2024-05-11T23:01:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:01:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6821", 2024-05-11T23:01:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:01:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6817/296930146", 2024-05-11T23:01:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6819/296930146", 2024-05-11T23:01:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6823/296930146", 2024-05-11T23:01:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6821/296930146", 2024-05-11T23:01:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:01:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:01:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:01:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-05-11T23:01:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "4e87e4f0-288e-4ee6-919e-9337925a5d5e", 2024-05-11T23:01:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:01:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:01:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:01:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:01:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:01:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:01:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 15, 2024-05-11T23:01:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 29, 2024-05-11T23:01:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:01:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:01:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:01:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6800", 2024-05-11T23:01:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:01:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6801", 2024-05-11T23:01:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:01:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:01:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6802", 2024-05-11T23:01:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:01:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6803", 2024-05-11T23:01:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:01:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:01:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6806", 2024-05-11T23:01:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:01:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6807", 2024-05-11T23:01:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:01:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:01:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6804", 2024-05-11T23:01:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:01:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6805", 2024-05-11T23:01:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:01:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.115:6801/495703077", 2024-05-11T23:01:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.115:6803/495703077", 2024-05-11T23:01:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.115:6807/495703077", 2024-05-11T23:01:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.115:6805/495703077", 2024-05-11T23:01:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:01:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:01:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:01:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-05-11T23:01:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5", 2024-05-11T23:01:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:01:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:01:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:01:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:01:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:01:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:01:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 15, 2024-05-11T23:01:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 19, 2024-05-11T23:01:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:01:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:01:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:01:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6811", 2024-05-11T23:01:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:01:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6813", 2024-05-11T23:01:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:01:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:01:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6815", 2024-05-11T23:01:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:01:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6817", 2024-05-11T23:01:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:01:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:01:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6822", 2024-05-11T23:01:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:01:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6823", 2024-05-11T23:01:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:01:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:01:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6819", 2024-05-11T23:01:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:01:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6821", 2024-05-11T23:01:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:01:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.115:6813/902573489", 2024-05-11T23:01:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.115:6817/902573489", 2024-05-11T23:01:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.115:6823/902573489", 2024-05-11T23:01:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.115:6821/902573489", 2024-05-11T23:01:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:01:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:01:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:01:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 6, 2024-05-11T23:01:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "edb686ec-e1a2-40d6-975a-e32e0e7760ff", 2024-05-11T23:01:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:01:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:01:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:01:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:01:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:01:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:01:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 15, 2024-05-11T23:01:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 19, 2024-05-11T23:01:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:01:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:01:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:01:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6824", 2024-05-11T23:01:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:01:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6825", 2024-05-11T23:01:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:01:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:01:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6826", 2024-05-11T23:01:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:01:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6827", 2024-05-11T23:01:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:01:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:01:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6830", 2024-05-11T23:01:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:01:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6831", 2024-05-11T23:01:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:01:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:01:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6828", 2024-05-11T23:01:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:01:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6829", 2024-05-11T23:01:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:01:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.115:6825/2749332272", 2024-05-11T23:01:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.115:6827/2749332272", 2024-05-11T23:01:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.115:6831/2749332272", 2024-05-11T23:01:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.115:6829/2749332272", 2024-05-11T23:01:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:01:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:01:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:01:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 7, 2024-05-11T23:01:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "7aa14dbb-c6cb-4540-b160-3fd346f6a046", 2024-05-11T23:01:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:01:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:01:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:01:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:01:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:01:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:01:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 15, 2024-05-11T23:01:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 19, 2024-05-11T23:01:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:01:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:01:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:01:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6808", 2024-05-11T23:01:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:01:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6809", 2024-05-11T23:01:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:01:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:01:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6810", 2024-05-11T23:01:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:01:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6812", 2024-05-11T23:01:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:01:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:01:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6818", 2024-05-11T23:01:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:01:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6820", 2024-05-11T23:01:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:01:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:01:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:01:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:01:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6814", 2024-05-11T23:01:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:01:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:01:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6816", 2024-05-11T23:01:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:01:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.115:6809/1043826337", 2024-05-11T23:01:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.115:6812/1043826337", 2024-05-11T23:01:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.115:6820/1043826337", 2024-05-11T23:01:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.115:6816/1043826337", 2024-05-11T23:01:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:01:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:01:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:01:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:01:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_xinfo": [ 2024-05-11T23:01:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-05-11T23:01:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:01:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:01:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:01:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:01:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:01:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.113232+0000", 2024-05-11T23:01:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:01:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-05-11T23:01:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:01:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:01:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:01:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:01:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:01:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.174679+0000", 2024-05-11T23:01:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:01:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-05-11T23:01:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:01:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:01:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:01:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:01:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:01:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.131118+0000", 2024-05-11T23:01:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:01:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-05-11T23:01:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:01:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:01:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:01:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:01:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:01:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.207668+0000", 2024-05-11T23:01:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:01:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-05-11T23:01:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:01:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:01:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:01:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:01:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:01:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.124542+0000", 2024-05-11T23:01:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:01:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-05-11T23:01:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:01:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:01:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:01:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:01:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:01:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.200832+0000", 2024-05-11T23:01:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:01:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 6, 2024-05-11T23:01:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:01:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:01:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:01:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:01:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:01:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.169560+0000", 2024-05-11T23:01:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:01:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 7, 2024-05-11T23:01:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:01:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:01:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:01:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:01:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:01:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.182465+0000", 2024-05-11T23:01:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:01:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:01:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap": [], 2024-05-11T23:01:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_items": [], 2024-05-11T23:01:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_primaries": [], 2024-05-11T23:01:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_temp": [], 2024-05-11T23:01:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_temp": [], 2024-05-11T23:01:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklist": {}, 2024-05-11T23:01:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "range_blocklist": {}, 2024-05-11T23:01:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profiles": { 2024-05-11T23:01:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "default": { 2024-05-11T23:01:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-failure-domain": "osd", 2024-05-11T23:01:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "k": "2", 2024-05-11T23:01:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "m": "1", 2024-05-11T23:01:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "plugin": "jerasure", 2024-05-11T23:01:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "technique": "reed_sol_van" 2024-05-11T23:01:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps_queue": [], 2024-05-11T23:01:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "new_removed_snaps": [], 2024-05-11T23:01:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "new_purged_snaps": [], 2024-05-11T23:01:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_node_flags": {}, 2024-05-11T23:01:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "device_class_flags": {}, 2024-05-11T23:01:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode": { 2024-05-11T23:01:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_enabled": false, 2024-05-11T23:01:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_bucket_count": 0, 2024-05-11T23:01:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "degraded_stretch_mode": 0, 2024-05-11T23:01:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "recovering_stretch_mode": 0, 2024-05-11T23:01:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_bucket": 0 2024-05-11T23:01:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:50.836 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:50.837 INFO:teuthology.orchestra.run.smithi005.stdout:2080 punch from 0x54296ea to 0x54329b7, (0x92cd bytes) 2024-05-11T23:01:50.837 INFO:teuthology.orchestra.run.smithi005.stdout:2081 writesame 0xa936b7c thru 0xa93c7b9 data_size 0x1(0x5c3e bytes) 2024-05-11T23:01:50.837 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:50.837 INFO:teuthology.orchestra.run.smithi005.stdout:2083 write 0x9de31df thru 0x9de687d (0x369f bytes) 2024-05-11T23:01:50.839 INFO:tasks.thrashosds.thrasher:j is {'epoch': 38, 'fsid': '34b20022-f11b-4169-a5f6-6386270a2f8f', 'created': '2024-05-11T23:01:05.977693+0000', 'modified': '2024-05-11T23:01:49.813789+0000', 'last_up_change': '2024-05-11T23:01:17.148916+0000', 'last_in_change': '2024-05-11T23:01:08.985275+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-11T23:01:15.187614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-05-11T23:01:17.689285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-11T23:01:25.743347+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 16, 'snap_epoch': 38, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '57ab1c70-9478-4a26-bb41-d753b4f3eb79', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6800', 'nonce': 3452725643}, {'type': 'v1', 'addr': '172.21.15.5:6801', 'nonce': 3452725643}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6802', 'nonce': 3452725643}, {'type': 'v1', 'addr': '172.21.15.5:6803', 'nonce': 3452725643}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6806', 'nonce': 3452725643}, {'type': 'v1', 'addr': '172.21.15.5:6808', 'nonce': 3452725643}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6804', 'nonce': 3452725643}, {'type': 'v1', 'addr': '172.21.15.5:6805', 'nonce': 3452725643}]}, 'public_addr': '172.21.15.5:6801/3452725643', 'cluster_addr': '172.21.15.5:6803/3452725643', 'heartbeat_back_addr': '172.21.15.5:6808/3452725643', 'heartbeat_front_addr': '172.21.15.5:6805/3452725643', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '05ece110-4a99-46f0-b061-5980f7ef2a55', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6824', 'nonce': 4028727138}, {'type': 'v1', 'addr': '172.21.15.5:6825', 'nonce': 4028727138}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6826', 'nonce': 4028727138}, {'type': 'v1', 'addr': '172.21.15.5:6827', 'nonce': 4028727138}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6830', 'nonce': 4028727138}, {'type': 'v1', 'addr': '172.21.15.5:6831', 'nonce': 4028727138}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6828', 'nonce': 4028727138}, {'type': 'v1', 'addr': '172.21.15.5:6829', 'nonce': 4028727138}]}, 'public_addr': '172.21.15.5:6825/4028727138', 'cluster_addr': '172.21.15.5:6827/4028727138', 'heartbeat_back_addr': '172.21.15.5:6831/4028727138', 'heartbeat_front_addr': '172.21.15.5:6829/4028727138', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 16, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6807', 'nonce': 2059959509}, {'type': 'v1', 'addr': '172.21.15.5:6809', 'nonce': 2059959509}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6810', 'nonce': 2059959509}, {'type': 'v1', 'addr': '172.21.15.5:6811', 'nonce': 2059959509}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6814', 'nonce': 2059959509}, {'type': 'v1', 'addr': '172.21.15.5:6815', 'nonce': 2059959509}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6812', 'nonce': 2059959509}, {'type': 'v1', 'addr': '172.21.15.5:6813', 'nonce': 2059959509}]}, 'public_addr': '172.21.15.5:6809/2059959509', 'cluster_addr': '172.21.15.5:6811/2059959509', 'heartbeat_back_addr': '172.21.15.5:6815/2059959509', 'heartbeat_front_addr': '172.21.15.5:6813/2059959509', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a', 'up': 1, 'in': 1, 'weight': 0.9000244140625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6816', 'nonce': 296930146}, {'type': 'v1', 'addr': '172.21.15.5:6817', 'nonce': 296930146}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6818', 'nonce': 296930146}, {'type': 'v1', 'addr': '172.21.15.5:6819', 'nonce': 296930146}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6822', 'nonce': 296930146}, {'type': 'v1', 'addr': '172.21.15.5:6823', 'nonce': 296930146}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6820', 'nonce': 296930146}, {'type': 'v1', 'addr': '172.21.15.5:6821', 'nonce': 296930146}]}, 'public_addr': '172.21.15.5:6817/296930146', 'cluster_addr': '172.21.15.5:6819/296930146', 'heartbeat_back_addr': '172.21.15.5:6823/296930146', 'heartbeat_front_addr': '172.21.15.5:6821/296930146', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '4e87e4f0-288e-4ee6-919e-9337925a5d5e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 29, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 495703077}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 495703077}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 495703077}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 495703077}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 495703077}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 495703077}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 495703077}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 495703077}]}, 'public_addr': '172.21.15.115:6801/495703077', 'cluster_addr': '172.21.15.115:6803/495703077', 'heartbeat_back_addr': '172.21.15.115:6807/495703077', 'heartbeat_front_addr': '172.21.15.115:6805/495703077', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6811', 'nonce': 902573489}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 902573489}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6815', 'nonce': 902573489}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 902573489}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 902573489}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 902573489}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6819', 'nonce': 902573489}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 902573489}]}, 'public_addr': '172.21.15.115:6813/902573489', 'cluster_addr': '172.21.15.115:6817/902573489', 'heartbeat_back_addr': '172.21.15.115:6823/902573489', 'heartbeat_front_addr': '172.21.15.115:6821/902573489', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'edb686ec-e1a2-40d6-975a-e32e0e7760ff', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6824', 'nonce': 2749332272}, {'type': 'v1', 'addr': '172.21.15.115:6825', 'nonce': 2749332272}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 2749332272}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 2749332272}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 2749332272}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 2749332272}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 2749332272}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 2749332272}]}, 'public_addr': '172.21.15.115:6825/2749332272', 'cluster_addr': '172.21.15.115:6827/2749332272', 'heartbeat_back_addr': '172.21.15.115:6831/2749332272', 'heartbeat_front_addr': '172.21.15.115:6829/2749332272', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '7aa14dbb-c6cb-4540-b160-3fd346f6a046', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 1043826337}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 1043826337}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 1043826337}, {'type': 'v1', 'addr': '172.21.15.115:6812', 'nonce': 1043826337}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 1043826337}, {'type': 'v1', 'addr': '172.21.15.115:6820', 'nonce': 1043826337}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 1043826337}, {'type': 'v1', 'addr': '172.21.15.115:6816', 'nonce': 1043826337}]}, 'public_addr': '172.21.15.115:6809/1043826337', 'cluster_addr': '172.21.15.115:6812/1043826337', 'heartbeat_back_addr': '172.21.15.115:6820/1043826337', 'heartbeat_front_addr': '172.21.15.115:6816/1043826337', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.113232+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.174679+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.131118+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.207668+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.124542+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.200832+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.169560+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.182465+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-11T23:01:50.839 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T23:01:50.840 INFO:teuthology.orchestra.run.smithi005.stdout:2085 trunc from 0xac47c13 to 0xd3157bf 2024-05-11T23:01:50.840 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:50.840 INFO:teuthology.orchestra.run.smithi005.stdout:2087 write 0xe6bcc8c thru 0xe6c0d7b (0x40f0 bytes) 2024-05-11T23:01:50.840 INFO:teuthology.orchestra.run.smithi005.stdout:2088 read 0x3ad1f86 thru 0x3ae11f2 (0xf26d bytes) 2024-05-11T23:01:50.840 INFO:teuthology.orchestra.run.smithi005.stdout:2089 punch from 0xda71e93 to 0xda7f83c, (0xd9a9 bytes) 2024-05-11T23:01:50.840 INFO:teuthology.orchestra.run.smithi005.stdout:2090 read 0x8ff214f thru 0x90012b2 (0xf164 bytes) 2024-05-11T23:01:50.840 INFO:teuthology.orchestra.run.smithi005.stdout:2091 trunc from 0xe6c0d7c to 0x8baca63 2024-05-11T23:01:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-05-11T23:01:50.841 INFO:teuthology.orchestra.run.smithi005.stdout:2092 trunc from 0x8baca63 to 0x2fe5bd8 2024-05-11T23:01:50.842 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:01:50.883 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:50.883+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:50.914 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:50.914 INFO:teuthology.orchestra.run.smithi005.stdout:2095 read 0x2c7493d thru 0x2c81d3e (0xd402 bytes) 2024-05-11T23:01:50.915 INFO:teuthology.orchestra.run.smithi005.stdout:2097 read 0x11d9382 thru 0x11e409c (0xad1b bytes) 2024-05-11T23:01:50.915 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:50.915 INFO:teuthology.orchestra.run.smithi005.stdout:2099 read 0x2e9a883 thru 0x2e9eedc (0x465a bytes) 2024-05-11T23:01:50.916 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:50.916 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:50.916 INFO:teuthology.orchestra.run.smithi005.stdout:2102 read 0xad9a8f thru 0xadb6c4 (0x1c36 bytes) 2024-05-11T23:01:50.924 INFO:teuthology.orchestra.run.smithi005.stdout:2104 writesame 0x5422148 thru 0x542720d data_size 0x31(0x50c6 bytes) 2024-05-11T23:01:50.933 INFO:teuthology.orchestra.run.smithi005.stdout:2105 trunc from 0x542720e to 0xcb34770 2024-05-11T23:01:50.946 INFO:teuthology.orchestra.run.smithi005.stdout:2106 writesame 0xe8562ec thru 0xe85c2fd data_size 0x3(0x6012 bytes) 2024-05-11T23:01:50.953 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:50.953+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:50.956 INFO:teuthology.orchestra.run.smithi005.stdout:2107 trunc from 0xe85c2fe to 0x525dfc9 2024-05-11T23:01:50.963 INFO:teuthology.orchestra.run.smithi005.stdout:2108 punch from 0xfe708f to 0xff6cbd, (0xfc2e bytes) 2024-05-11T23:01:50.963 INFO:teuthology.orchestra.run.smithi005.stdout:2109 read 0x3e1dfde thru 0x3e2d117 (0xf13a bytes) 2024-05-11T23:01:50.963 INFO:teuthology.orchestra.run.smithi005.stdout:2110 punch from 0xa33749 to 0xa38bb8, (0x546f bytes) 2024-05-11T23:01:50.963 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:50.969 INFO:teuthology.orchestra.run.smithi005.stdout:2112 trunc from 0x525dfc9 to 0xaa2cc6d 2024-05-11T23:01:50.974 INFO:teuthology.orchestra.run.smithi005.stdout:2113 writesame 0x874e8a5 thru 0x8756eaf data_size 0x1(0x860b bytes) 2024-05-11T23:01:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:2117 read 0x89276ae thru 0x89306d1 (0x9024 bytes) 2024-05-11T23:01:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:2118 write 0x9d33e42 thru 0x9d37f6c (0x412b bytes) 2024-05-11T23:01:50.999 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:51.000 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:51.002 INFO:teuthology.orchestra.run.smithi005.stdout:2119 trunc from 0xaa2cc6d to 0x344687f 2024-05-11T23:01:51.010 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:01:51.014 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:51.015 INFO:teuthology.orchestra.run.smithi005.stdout:2123 read 0x2598db8 thru 0x25a5376 (0xc5bf bytes) 2024-05-11T23:01:51.015 INFO:teuthology.orchestra.run.smithi005.stdout:2124 clone 15 order 20 su 8192 sc 9 2024-05-11T23:01:51.053 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:51.053+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:51.154 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:51.154+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:51.156 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:51.156 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:51.165 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:01:51.207 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:01:51.208 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-11T23:01:51.219 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2024-05-11T23:01:51.142508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":292917412,"num_objects":351,"num_object_clones":60,"num_object_copies":702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":351,"num_whiteouts":84,"num_read":24584,"num_read_kb":62751,"num_write":5107,"num_write_kb":23015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4138,"ondisk_log_size":4138,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":284060,"kb_used_data":9680,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754690660,"statfs":{"total":773094113280,"available":772803235840,"internally_reserved":0,"allocated":9912320,"data_stored":12554728,"data_compressed":538994,"data_compressed_allocated":921600,"data_compressed_original":6254592,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":43105745,"num_objects":87,"num_object_clones":13,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":9,"num_read":5025,"num_read_kb":8592,"num_write":1081,"num_write_kb":4122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000549"},"pg_stats":[{"pgid":"3.1f","version":"38'155","reported_seq":724,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-11T23:01:50.092930+0000","last_change":"2024-05-11T23:01:29.462665+0000","last_active":"2024-05-11T23:01:50.092930+0000","last_peered":"2024-05-11T23:01:50.092930+0000","last_clean":"2024-05-11T23:01:50.092930+0000","last_became_active":"2024-05-11T23:01:27.459623+0000","last_became_peered":"2024-05-11T23:01:27.459623+0000","last_unstale":"2024-05-11T23:01:50.092930+0000","last_undegraded":"2024-05-11T23:01:50.092930+0000","last_fullsized":"2024-05-11T23:01:50.092930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:39:28.478553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096814299999999998,"stat_sum":{"num_bytes":12907882,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":7,"num_read":1474,"num_read_kb":2203,"num_write":170,"num_write_kb":562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"34'161","reported_seq":1135,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-11T23:01:45.811773+0000","last_change":"2024-05-11T23:01:29.583286+0000","last_active":"2024-05-11T23:01:45.811773+0000","last_peered":"2024-05-11T23:01:45.811773+0000","last_clean":"2024-05-11T23:01:45.811773+0000","last_became_active":"2024-05-11T23:01:27.459124+0000","last_became_peered":"2024-05-11T23:01:27.459124+0000","last_unstale":"2024-05-11T23:01:45.811773+0000","last_undegraded":"2024-05-11T23:01:45.811773+0000","last_fullsized":"2024-05-11T23:01:45.811773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:28:35.348731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062673000000000004,"stat_sum":{"num_bytes":24755599,"num_objects":15,"num_object_clones":5,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":1503,"num_read_kb":7474,"num_write":145,"num_write_kb":1345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"36'64","reported_seq":129,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:48.108055+0000","last_change":"2024-05-11T23:01:29.474888+0000","last_active":"2024-05-11T23:01:48.108055+0000","last_peered":"2024-05-11T23:01:48.108055+0000","last_clean":"2024-05-11T23:01:48.108055+0000","last_became_active":"2024-05-11T23:01:27.458054+0000","last_became_peered":"2024-05-11T23:01:27.458054+0000","last_unstale":"2024-05-11T23:01:48.108055+0000","last_undegraded":"2024-05-11T23:01:48.108055+0000","last_fullsized":"2024-05-11T23:01:48.108055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:51:47.321226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001000658,"stat_sum":{"num_bytes":476758,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":4,"num_write":62,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"36'111","reported_seq":455,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:48.128816+0000","last_change":"2024-05-11T23:01:29.474685+0000","last_active":"2024-05-11T23:01:48.128816+0000","last_peered":"2024-05-11T23:01:48.128816+0000","last_clean":"2024-05-11T23:01:48.128816+0000","last_became_active":"2024-05-11T23:01:27.463341+0000","last_became_peered":"2024-05-11T23:01:27.463341+0000","last_unstale":"2024-05-11T23:01:48.128816+0000","last_undegraded":"2024-05-11T23:01:48.128816+0000","last_fullsized":"2024-05-11T23:01:48.128816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:20:49.671055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091161300000000003,"stat_sum":{"num_bytes":4765525,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":346,"num_read_kb":825,"num_write":107,"num_write_kb":661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"36'119","reported_seq":464,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:48.074777+0000","last_change":"2024-05-11T23:01:39.800279+0000","last_active":"2024-05-11T23:01:48.074777+0000","last_peered":"2024-05-11T23:01:48.074777+0000","last_clean":"2024-05-11T23:01:48.074777+0000","last_became_active":"2024-05-11T23:01:39.787165+0000","last_became_peered":"2024-05-11T23:01:39.787165+0000","last_unstale":"2024-05-11T23:01:48.074777+0000","last_undegraded":"2024-05-11T23:01:48.074777+0000","last_fullsized":"2024-05-11T23:01:48.074777+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:03:20.224092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065062200000000001,"stat_sum":{"num_bytes":6436610,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":493,"num_read_kb":3438,"num_write":147,"num_write_kb":641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"36'52","reported_seq":689,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:48.125213+0000","last_change":"2024-05-11T23:01:29.466255+0000","last_active":"2024-05-11T23:01:48.125213+0000","last_peered":"2024-05-11T23:01:48.125213+0000","last_clean":"2024-05-11T23:01:48.125213+0000","last_became_active":"2024-05-11T23:01:27.460541+0000","last_became_peered":"2024-05-11T23:01:27.460541+0000","last_unstale":"2024-05-11T23:01:48.125213+0000","last_undegraded":"2024-05-11T23:01:48.125213+0000","last_fullsized":"2024-05-11T23:01:48.125213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:33:46.176515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087678099999999998,"stat_sum":{"num_bytes":13741079,"num_objects":16,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":1,"num_read":591,"num_read_kb":1034,"num_write":53,"num_write_kb":457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"34'247","reported_seq":806,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-11T23:01:45.815126+0000","last_change":"2024-05-11T23:01:29.467984+0000","last_active":"2024-05-11T23:01:45.815126+0000","last_peered":"2024-05-11T23:01:45.815126+0000","last_clean":"2024-05-11T23:01:45.815126+0000","last_became_active":"2024-05-11T23:01:27.458905+0000","last_became_peered":"2024-05-11T23:01:27.458905+0000","last_unstale":"2024-05-11T23:01:45.815126+0000","last_undegraded":"2024-05-11T23:01:45.815126+0000","last_fullsized":"2024-05-11T23:01:45.815126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:37:44.237031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000433275,"stat_sum":{"num_bytes":13252385,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":1788,"num_read_kb":4326,"num_write":345,"num_write_kb":842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"34'51","reported_seq":237,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-11T23:01:45.811845+0000","last_change":"2024-05-11T23:01:29.583577+0000","last_active":"2024-05-11T23:01:45.811845+0000","last_peered":"2024-05-11T23:01:45.811845+0000","last_clean":"2024-05-11T23:01:45.811845+0000","last_became_active":"2024-05-11T23:01:27.458382+0000","last_became_peered":"2024-05-11T23:01:27.458382+0000","last_unstale":"2024-05-11T23:01:45.811845+0000","last_undegraded":"2024-05-11T23:01:45.811845+0000","last_fullsized":"2024-05-11T23:01:45.811845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:23:29.145380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079167699999999996,"stat_sum":{"num_bytes":7136660,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":148,"num_read_kb":263,"num_write":50,"num_write_kb":352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"37'156","reported_seq":540,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-11T23:01:50.099862+0000","last_change":"2024-05-11T23:01:29.462988+0000","last_active":"2024-05-11T23:01:50.099862+0000","last_peered":"2024-05-11T23:01:50.099862+0000","last_clean":"2024-05-11T23:01:50.099862+0000","last_became_active":"2024-05-11T23:01:27.460088+0000","last_became_peered":"2024-05-11T23:01:27.460088+0000","last_unstale":"2024-05-11T23:01:50.099862+0000","last_undegraded":"2024-05-11T23:01:50.099862+0000","last_fullsized":"2024-05-11T23:01:50.099862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:49:45.905027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045912,"stat_sum":{"num_bytes":4256462,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":823,"num_read_kb":1315,"num_write":225,"num_write_kb":660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"36'94","reported_seq":186,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:48.121722+0000","last_change":"2024-05-11T23:01:29.474960+0000","last_active":"2024-05-11T23:01:48.121722+0000","last_peered":"2024-05-11T23:01:48.121722+0000","last_clean":"2024-05-11T23:01:48.121722+0000","last_became_active":"2024-05-11T23:01:27.463623+0000","last_became_peered":"2024-05-11T23:01:27.463623+0000","last_unstale":"2024-05-11T23:01:48.121722+0000","last_undegraded":"2024-05-11T23:01:48.121722+0000","last_fullsized":"2024-05-11T23:01:48.121722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:59:43.949420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001047748,"stat_sum":{"num_bytes":4156390,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":258,"num_read_kb":310,"num_write":129,"num_write_kb":702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"36'71","reported_seq":202,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:48.095357+0000","last_change":"2024-05-11T23:01:29.864029+0000","last_active":"2024-05-11T23:01:48.095357+0000","last_peered":"2024-05-11T23:01:48.095357+0000","last_clean":"2024-05-11T23:01:48.095357+0000","last_became_active":"2024-05-11T23:01:27.465476+0000","last_became_peered":"2024-05-11T23:01:27.465476+0000","last_unstale":"2024-05-11T23:01:48.095357+0000","last_undegraded":"2024-05-11T23:01:48.095357+0000","last_fullsized":"2024-05-11T23:01:48.095357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:47:48.426394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039810900000000001,"stat_sum":{"num_bytes":5070390,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":78,"num_read_kb":80,"num_write":67,"num_write_kb":871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"36'111","reported_seq":366,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:48.123491+0000","last_change":"2024-05-11T23:01:29.466206+0000","last_active":"2024-05-11T23:01:48.123491+0000","last_peered":"2024-05-11T23:01:48.123491+0000","last_clean":"2024-05-11T23:01:48.123491+0000","last_became_active":"2024-05-11T23:01:27.465952+0000","last_became_peered":"2024-05-11T23:01:27.465952+0000","last_unstale":"2024-05-11T23:01:48.123491+0000","last_undegraded":"2024-05-11T23:01:48.123491+0000","last_fullsized":"2024-05-11T23:01:48.123491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:17:29.326085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066166100000000002,"stat_sum":{"num_bytes":11950514,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":3,"num_read":275,"num_read_kb":551,"num_write":121,"num_write_kb":998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":46,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-11T23:01:45.847270+0000","last_change":"2024-05-11T23:01:19.220007+0000","last_active":"2024-05-11T23:01:45.847270+0000","last_peered":"2024-05-11T23:01:45.847270+0000","last_clean":"2024-05-11T23:01:45.847270+0000","last_became_active":"2024-05-11T23:01:19.219681+0000","last_became_peered":"2024-05-11T23:01:19.219681+0000","last_unstale":"2024-05-11T23:01:45.847270+0000","last_undegraded":"2024-05-11T23:01:45.847270+0000","last_fullsized":"2024-05-11T23:01:45.847270+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:49:14.757899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"38'329","reported_seq":802,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-11T23:01:50.075101+0000","last_change":"2024-05-11T23:01:29.462499+0000","last_active":"2024-05-11T23:01:50.075101+0000","last_peered":"2024-05-11T23:01:50.075101+0000","last_clean":"2024-05-11T23:01:50.075101+0000","last_became_active":"2024-05-11T23:01:27.458177+0000","last_became_peered":"2024-05-11T23:01:27.458177+0000","last_unstale":"2024-05-11T23:01:50.075101+0000","last_undegraded":"2024-05-11T23:01:50.075101+0000","last_fullsized":"2024-05-11T23:01:50.075101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:45:24.910965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099453400000000004,"stat_sum":{"num_bytes":5560964,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":2450,"num_read_kb":2428,"num_write":525,"num_write_kb":764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":46,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-11T23:01:45.815226+0000","last_change":"2024-05-11T23:01:19.221663+0000","last_active":"2024-05-11T23:01:45.815226+0000","last_peered":"2024-05-11T23:01:45.815226+0000","last_clean":"2024-05-11T23:01:45.815226+0000","last_became_active":"2024-05-11T23:01:19.221440+0000","last_became_peered":"2024-05-11T23:01:19.221440+0000","last_unstale":"2024-05-11T23:01:45.815226+0000","last_undegraded":"2024-05-11T23:01:45.815226+0000","last_fullsized":"2024-05-11T23:01:45.815226+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:50:51.464824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"34'108","reported_seq":632,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-11T23:01:45.811784+0000","last_change":"2024-05-11T23:01:29.583359+0000","last_active":"2024-05-11T23:01:45.811784+0000","last_peered":"2024-05-11T23:01:45.811784+0000","last_clean":"2024-05-11T23:01:45.811784+0000","last_became_active":"2024-05-11T23:01:27.467650+0000","last_became_peered":"2024-05-11T23:01:27.467650+0000","last_unstale":"2024-05-11T23:01:45.811784+0000","last_undegraded":"2024-05-11T23:01:45.811784+0000","last_fullsized":"2024-05-11T23:01:45.811784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:23:31.264575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058786099999999996,"stat_sum":{"num_bytes":7055637,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":1275,"num_read_kb":1240,"num_write":107,"num_write_kb":831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:47.810418+0000","last_change":"2024-05-11T23:01:19.216700+0000","last_active":"2024-05-11T23:01:47.810418+0000","last_peered":"2024-05-11T23:01:47.810418+0000","last_clean":"2024-05-11T23:01:47.810418+0000","last_became_active":"2024-05-11T23:01:19.216413+0000","last_became_peered":"2024-05-11T23:01:19.216413+0000","last_unstale":"2024-05-11T23:01:47.810418+0000","last_undegraded":"2024-05-11T23:01:47.810418+0000","last_fullsized":"2024-05-11T23:01:47.810418+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:38.639409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"34'46","reported_seq":396,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-11T23:01:45.815128+0000","last_change":"2024-05-11T23:01:29.468176+0000","last_active":"2024-05-11T23:01:45.815128+0000","last_peered":"2024-05-11T23:01:45.815128+0000","last_clean":"2024-05-11T23:01:45.815128+0000","last_became_active":"2024-05-11T23:01:27.460866+0000","last_became_peered":"2024-05-11T23:01:27.460866+0000","last_unstale":"2024-05-11T23:01:45.815128+0000","last_undegraded":"2024-05-11T23:01:45.815128+0000","last_fullsized":"2024-05-11T23:01:45.815128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:22:55.245081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050028300000000002,"stat_sum":{"num_bytes":20103029,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":318,"num_read_kb":3176,"num_write":44,"num_write_kb":636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-11T23:01:45.815212+0000","last_change":"2024-05-11T23:01:19.220232+0000","last_active":"2024-05-11T23:01:45.815212+0000","last_peered":"2024-05-11T23:01:45.815212+0000","last_clean":"2024-05-11T23:01:45.815212+0000","last_became_active":"2024-05-11T23:01:19.219917+0000","last_became_peered":"2024-05-11T23:01:19.219917+0000","last_unstale":"2024-05-11T23:01:45.815212+0000","last_undegraded":"2024-05-11T23:01:45.815212+0000","last_fullsized":"2024-05-11T23:01:45.815212+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:44:02.868252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"35'45","reported_seq":258,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:47.884555+0000","last_change":"2024-05-11T23:01:29.475154+0000","last_active":"2024-05-11T23:01:47.884555+0000","last_peered":"2024-05-11T23:01:47.884555+0000","last_clean":"2024-05-11T23:01:47.884555+0000","last_became_active":"2024-05-11T23:01:27.458787+0000","last_became_peered":"2024-05-11T23:01:27.458787+0000","last_unstale":"2024-05-11T23:01:47.884555+0000","last_undegraded":"2024-05-11T23:01:47.884555+0000","last_fullsized":"2024-05-11T23:01:47.884555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:55:02.617088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012391380000000001,"stat_sum":{"num_bytes":1373539,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":166,"num_read_kb":1138,"num_write":46,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"35'181","reported_seq":1076,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:47.814451+0000","last_change":"2024-05-11T23:01:29.465808+0000","last_active":"2024-05-11T23:01:47.814451+0000","last_peered":"2024-05-11T23:01:47.814451+0000","last_clean":"2024-05-11T23:01:47.814451+0000","last_became_active":"2024-05-11T23:01:27.457710+0000","last_became_peered":"2024-05-11T23:01:27.457710+0000","last_unstale":"2024-05-11T23:01:47.814451+0000","last_undegraded":"2024-05-11T23:01:47.814451+0000","last_fullsized":"2024-05-11T23:01:47.814451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:53:34.116910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038468999999999998,"stat_sum":{"num_bytes":5023914,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":1898,"num_read_kb":4610,"num_write":176,"num_write_kb":617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":48,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:47.824428+0000","last_change":"2024-05-11T23:01:19.221118+0000","last_active":"2024-05-11T23:01:47.824428+0000","last_peered":"2024-05-11T23:01:47.824428+0000","last_clean":"2024-05-11T23:01:47.824428+0000","last_became_active":"2024-05-11T23:01:19.220855+0000","last_became_peered":"2024-05-11T23:01:19.220855+0000","last_unstale":"2024-05-11T23:01:47.824428+0000","last_undegraded":"2024-05-11T23:01:47.824428+0000","last_fullsized":"2024-05-11T23:01:47.824428+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:01:56.076963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"35'89","reported_seq":484,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-11T23:01:45.905131+0000","last_change":"2024-05-11T23:01:29.467921+0000","last_active":"2024-05-11T23:01:45.905131+0000","last_peered":"2024-05-11T23:01:45.905131+0000","last_clean":"2024-05-11T23:01:45.905131+0000","last_became_active":"2024-05-11T23:01:27.459593+0000","last_became_peered":"2024-05-11T23:01:27.459593+0000","last_unstale":"2024-05-11T23:01:45.905131+0000","last_undegraded":"2024-05-11T23:01:45.905131+0000","last_fullsized":"2024-05-11T23:01:45.905131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:03:52.041471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047314299999999998,"stat_sum":{"num_bytes":1318944,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":1017,"num_read_kb":895,"num_write":83,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-11T23:01:49.819546+0000","last_change":"2024-05-11T23:01:19.578258+0000","last_active":"2024-05-11T23:01:49.819546+0000","last_peered":"2024-05-11T23:01:49.819546+0000","last_clean":"2024-05-11T23:01:49.819546+0000","last_became_active":"2024-05-11T23:01:19.577973+0000","last_became_peered":"2024-05-11T23:01:19.577973+0000","last_unstale":"2024-05-11T23:01:49.819546+0000","last_undegraded":"2024-05-11T23:01:49.819546+0000","last_fullsized":"2024-05-11T23:01:49.819546+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:21:37.965501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"37'88","reported_seq":412,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-11T23:01:49.819838+0000","last_change":"2024-05-11T23:01:29.462388+0000","last_active":"2024-05-11T23:01:49.819838+0000","last_peered":"2024-05-11T23:01:49.819838+0000","last_clean":"2024-05-11T23:01:49.819838+0000","last_became_active":"2024-05-11T23:01:27.460113+0000","last_became_peered":"2024-05-11T23:01:27.460113+0000","last_unstale":"2024-05-11T23:01:49.819838+0000","last_undegraded":"2024-05-11T23:01:49.819838+0000","last_fullsized":"2024-05-11T23:01:49.819838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:50:08.796219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062926200000000001,"stat_sum":{"num_bytes":10979355,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":372,"num_read_kb":3279,"num_write":91,"num_write_kb":366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":48,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:47.810095+0000","last_change":"2024-05-11T23:01:19.219755+0000","last_active":"2024-05-11T23:01:47.810095+0000","last_peered":"2024-05-11T23:01:47.810095+0000","last_clean":"2024-05-11T23:01:47.810095+0000","last_became_active":"2024-05-11T23:01:19.219471+0000","last_became_peered":"2024-05-11T23:01:19.219471+0000","last_unstale":"2024-05-11T23:01:47.810095+0000","last_undegraded":"2024-05-11T23:01:47.810095+0000","last_fullsized":"2024-05-11T23:01:47.810095+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:39:05.284213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"34'124","reported_seq":526,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-11T23:01:45.811668+0000","last_change":"2024-05-11T23:01:29.583061+0000","last_active":"2024-05-11T23:01:45.811668+0000","last_peered":"2024-05-11T23:01:45.811668+0000","last_clean":"2024-05-11T23:01:45.811668+0000","last_became_active":"2024-05-11T23:01:27.461785+0000","last_became_peered":"2024-05-11T23:01:27.461785+0000","last_unstale":"2024-05-11T23:01:45.811668+0000","last_undegraded":"2024-05-11T23:01:45.811668+0000","last_fullsized":"2024-05-11T23:01:45.811668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:36:32.150690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000407852,"stat_sum":{"num_bytes":21404870,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":3,"num_read":619,"num_read_kb":4068,"num_write":152,"num_write_kb":440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":48,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:47.824454+0000","last_change":"2024-05-11T23:01:19.219128+0000","last_active":"2024-05-11T23:01:47.824454+0000","last_peered":"2024-05-11T23:01:47.824454+0000","last_clean":"2024-05-11T23:01:47.824454+0000","last_became_active":"2024-05-11T23:01:19.218846+0000","last_became_peered":"2024-05-11T23:01:19.218846+0000","last_unstale":"2024-05-11T23:01:47.824454+0000","last_undegraded":"2024-05-11T23:01:47.824454+0000","last_fullsized":"2024-05-11T23:01:47.824454+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:41:09.111000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":200,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:47.810104+0000","last_change":"2024-05-11T23:01:18.164452+0000","last_active":"2024-05-11T23:01:47.810104+0000","last_peered":"2024-05-11T23:01:47.810104+0000","last_clean":"2024-05-11T23:01:47.810104+0000","last_became_active":"2024-05-11T23:01:18.163431+0000","last_became_peered":"2024-05-11T23:01:18.163431+0000","last_unstale":"2024-05-11T23:01:47.810104+0000","last_undegraded":"2024-05-11T23:01:47.810104+0000","last_fullsized":"2024-05-11T23:01:47.810104+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_clean_scrub_stamp":"2024-05-11T23:01:16.147564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:40:59.974639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"35'247","reported_seq":679,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-11T23:01:45.814811+0000","last_change":"2024-05-11T23:01:29.583563+0000","last_active":"2024-05-11T23:01:45.814811+0000","last_peered":"2024-05-11T23:01:45.814811+0000","last_clean":"2024-05-11T23:01:45.814811+0000","last_became_active":"2024-05-11T23:01:27.462464+0000","last_became_peered":"2024-05-11T23:01:27.462464+0000","last_unstale":"2024-05-11T23:01:45.814811+0000","last_undegraded":"2024-05-11T23:01:45.814811+0000","last_fullsized":"2024-05-11T23:01:45.814811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:40:27.706519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061400699999999996,"stat_sum":{"num_bytes":15732814,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":1339,"num_read_kb":4330,"num_write":366,"num_write_kb":868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"36'54","reported_seq":136,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:48.120252+0000","last_change":"2024-05-11T23:01:29.465998+0000","last_active":"2024-05-11T23:01:48.120252+0000","last_peered":"2024-05-11T23:01:48.120252+0000","last_clean":"2024-05-11T23:01:48.120252+0000","last_became_active":"2024-05-11T23:01:27.461824+0000","last_became_peered":"2024-05-11T23:01:27.461824+0000","last_unstale":"2024-05-11T23:01:48.120252+0000","last_undegraded":"2024-05-11T23:01:48.120252+0000","last_fullsized":"2024-05-11T23:01:48.120252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:33:14.716359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055969500000000003,"stat_sum":{"num_bytes":8159276,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":35,"num_read_kb":130,"num_write":53,"num_write_kb":474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"35'92","reported_seq":238,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-11T23:01:46.798278+0000","last_change":"2024-05-11T23:01:29.461288+0000","last_active":"2024-05-11T23:01:46.798278+0000","last_peered":"2024-05-11T23:01:46.798278+0000","last_clean":"2024-05-11T23:01:46.798278+0000","last_became_active":"2024-05-11T23:01:27.460156+0000","last_became_peered":"2024-05-11T23:01:27.460156+0000","last_unstale":"2024-05-11T23:01:46.798278+0000","last_undegraded":"2024-05-11T23:01:46.798278+0000","last_fullsized":"2024-05-11T23:01:46.798278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:15:01.866511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030522600000000002,"stat_sum":{"num_bytes":6801003,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":121,"num_read_kb":240,"num_write":91,"num_write_kb":619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"33'69","reported_seq":364,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-11T23:01:45.811736+0000","last_change":"2024-05-11T23:01:29.583060+0000","last_active":"2024-05-11T23:01:45.811736+0000","last_peered":"2024-05-11T23:01:45.811736+0000","last_clean":"2024-05-11T23:01:45.811736+0000","last_became_active":"2024-05-11T23:01:27.457211+0000","last_became_peered":"2024-05-11T23:01:27.457211+0000","last_unstale":"2024-05-11T23:01:45.811736+0000","last_undegraded":"2024-05-11T23:01:45.811736+0000","last_fullsized":"2024-05-11T23:01:45.811736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:51:06.285581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042056999999999999,"stat_sum":{"num_bytes":6239729,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":277,"num_read_kb":221,"num_write":65,"num_write_kb":449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"36'78","reported_seq":162,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:48.125653+0000","last_change":"2024-05-11T23:01:29.474665+0000","last_active":"2024-05-11T23:01:48.125653+0000","last_peered":"2024-05-11T23:01:48.125653+0000","last_clean":"2024-05-11T23:01:48.125653+0000","last_became_active":"2024-05-11T23:01:27.458514+0000","last_became_peered":"2024-05-11T23:01:27.458514+0000","last_unstale":"2024-05-11T23:01:48.125653+0000","last_undegraded":"2024-05-11T23:01:48.125653+0000","last_fullsized":"2024-05-11T23:01:48.125653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:12:20.474048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084388499999999995,"stat_sum":{"num_bytes":4542359,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":42,"num_read_kb":81,"num_write":74,"num_write_kb":1167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"36'63","reported_seq":396,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:48.123323+0000","last_change":"2024-05-11T23:01:29.864324+0000","last_active":"2024-05-11T23:01:48.123323+0000","last_peered":"2024-05-11T23:01:48.123323+0000","last_clean":"2024-05-11T23:01:48.123323+0000","last_became_active":"2024-05-11T23:01:27.465570+0000","last_became_peered":"2024-05-11T23:01:27.465570+0000","last_unstale":"2024-05-11T23:01:48.123323+0000","last_undegraded":"2024-05-11T23:01:48.123323+0000","last_fullsized":"2024-05-11T23:01:48.123323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:37:41.192539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062217299999999995,"stat_sum":{"num_bytes":7617334,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":304,"num_read_kb":3199,"num_write":57,"num_write_kb":430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"36'378","reported_seq":1174,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:48.003801+0000","last_change":"2024-05-11T23:01:29.864587+0000","last_active":"2024-05-11T23:01:48.003801+0000","last_peered":"2024-05-11T23:01:48.003801+0000","last_clean":"2024-05-11T23:01:48.003801+0000","last_became_active":"2024-05-11T23:01:27.466572+0000","last_became_peered":"2024-05-11T23:01:27.466572+0000","last_unstale":"2024-05-11T23:01:48.003801+0000","last_undegraded":"2024-05-11T23:01:48.003801+0000","last_fullsized":"2024-05-11T23:01:48.003801+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:14:36.964408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087175200000000005,"stat_sum":{"num_bytes":11397937,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":2900,"num_read_kb":5303,"num_write":583,"num_write_kb":979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"35'55","reported_seq":143,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:48.125193+0000","last_change":"2024-05-11T23:01:29.864577+0000","last_active":"2024-05-11T23:01:48.125193+0000","last_peered":"2024-05-11T23:01:48.125193+0000","last_clean":"2024-05-11T23:01:48.125193+0000","last_became_active":"2024-05-11T23:01:27.466778+0000","last_became_peered":"2024-05-11T23:01:27.466778+0000","last_unstale":"2024-05-11T23:01:48.125193+0000","last_undegraded":"2024-05-11T23:01:48.125193+0000","last_fullsized":"2024-05-11T23:01:48.125193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:27:49.376482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080844100000000002,"stat_sum":{"num_bytes":3207710,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":5,"num_read":46,"num_read_kb":190,"num_write":50,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"35'63","reported_seq":624,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-05-11T23:01:46.560204+0000","last_change":"2024-05-11T23:01:29.461328+0000","last_active":"2024-05-11T23:01:46.560204+0000","last_peered":"2024-05-11T23:01:46.560204+0000","last_clean":"2024-05-11T23:01:46.560204+0000","last_became_active":"2024-05-11T23:01:27.458611+0000","last_became_peered":"2024-05-11T23:01:27.458611+0000","last_unstale":"2024-05-11T23:01:46.560204+0000","last_undegraded":"2024-05-11T23:01:46.560204+0000","last_fullsized":"2024-05-11T23:01:46.560204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:16:54.726453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033171200000000001,"stat_sum":{"num_bytes":11752417,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":1058,"num_read_kb":3721,"num_write":62,"num_write_kb":209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"36'105","reported_seq":480,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:48.125111+0000","last_change":"2024-05-11T23:01:29.864048+0000","last_active":"2024-05-11T23:01:48.125111+0000","last_peered":"2024-05-11T23:01:48.125111+0000","last_clean":"2024-05-11T23:01:48.125111+0000","last_became_active":"2024-05-11T23:01:27.465260+0000","last_became_peered":"2024-05-11T23:01:27.465260+0000","last_unstale":"2024-05-11T23:01:48.125111+0000","last_undegraded":"2024-05-11T23:01:48.125111+0000","last_fullsized":"2024-05-11T23:01:48.125111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T09:21:15.211122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044654299999999998,"stat_sum":{"num_bytes":11856993,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":947,"num_read_kb":1011,"num_write":123,"num_write_kb":428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"38'344","reported_seq":534,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-05-11T23:01:49.902587+0000","last_change":"2024-05-11T23:01:29.462751+0000","last_active":"2024-05-11T23:01:49.902587+0000","last_peered":"2024-05-11T23:01:49.902587+0000","last_clean":"2024-05-11T23:01:49.902587+0000","last_became_active":"2024-05-11T23:01:27.459137+0000","last_became_peered":"2024-05-11T23:01:27.459137+0000","last_unstale":"2024-05-11T23:01:49.902587+0000","last_undegraded":"2024-05-11T23:01:49.902587+0000","last_fullsized":"2024-05-11T23:01:49.902587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:23:35.326685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010326510000000001,"stat_sum":{"num_bytes":18205576,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":1448,"num_read_kb":1470,"num_write":513,"num_write_kb":1131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"36'71","reported_seq":161,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-05-11T23:01:47.978521+0000","last_change":"2024-05-11T23:01:29.864290+0000","last_active":"2024-05-11T23:01:47.978521+0000","last_peered":"2024-05-11T23:01:47.978521+0000","last_clean":"2024-05-11T23:01:47.978521+0000","last_became_active":"2024-05-11T23:01:27.465953+0000","last_became_peered":"2024-05-11T23:01:27.465953+0000","last_unstale":"2024-05-11T23:01:47.978521+0000","last_undegraded":"2024-05-11T23:01:47.978521+0000","last_fullsized":"2024-05-11T23:01:47.978521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:14:19.477173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065432399999999996,"stat_sum":{"num_bytes":5087390,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":50,"num_read_kb":98,"num_write":66,"num_write_kb":598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":292327044,"num_objects":349,"num_object_clones":60,"num_object_copies":698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":349,"num_whiteouts":84,"num_read":24468,"num_read_kb":62651,"num_write":4948,"num_write_kb":20599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6975488,"data_stored":11028544,"data_compressed":572931,"data_compressed_allocated":970752,"data_compressed_original":5947392,"omap_allocated":0,"internal_metadata":0},"log_size":4021,"ondisk_log_size":4021,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65020,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36972,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334868,"statfs":{"total":96636764160,"available":96598904832,"internally_reserved":0,"allocated":1675264,"data_stored":2275702,"data_compressed":72691,"data_compressed_allocated":143360,"data_compressed_original":1122304,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1306624,"data_stored":2044729,"data_compressed":161663,"data_compressed_allocated":225280,"data_compressed_original":1306624,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1626112,"data_stored":1543316,"data_compressed":19493,"data_compressed_allocated":81920,"data_compressed_original":405504,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":819200,"data_stored":821417,"data_compressed":32161,"data_compressed_allocated":53248,"data_compressed_original":348160,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":1445888,"data_stored":2292417,"data_compressed":151513,"data_compressed_allocated":204800,"data_compressed_original":1384448,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36096,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335744,"statfs":{"total":96636764160,"available":96599801856,"internally_reserved":0,"allocated":778240,"data_stored":903261,"data_compressed":22397,"data_compressed_allocated":53248,"data_compressed_original":471040,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35940,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335900,"statfs":{"total":96636764160,"available":96599961600,"internally_reserved":0,"allocated":614400,"data_stored":600965,"data_compressed":11757,"data_compressed_allocated":28672,"data_compressed_original":245760,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41108,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330732,"statfs":{"total":96636764160,"available":96594669568,"internally_reserved":0,"allocated":1646592,"data_stored":2072921,"data_compressed":67319,"data_compressed_allocated":131072,"data_compressed_original":970752,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1163264,"data_stored":1305120,"data_compressed":34809,"data_compressed_allocated":86016,"data_compressed_original":397312,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":750343,"data_compressed":22397,"data_compressed_allocated":53248,"data_compressed_original":471040,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":429643,"data_compressed":11757,"data_compressed_allocated":28672,"data_compressed_original":245760,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1077248,"data_stored":2139428,"data_compressed":151513,"data_compressed_allocated":204800,"data_compressed_original":1384448,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1376256,"data_stored":1653167,"data_compressed":44132,"data_compressed_allocated":118784,"data_compressed_original":593920,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":913408,"data_stored":1879518,"data_compressed":161663,"data_compressed_allocated":225280,"data_compressed_original":1306624,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":438272,"data_stored":662317,"data_compressed":32161,"data_compressed_allocated":53248,"data_compressed_original":348160,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1380352,"data_stored":2209008,"data_compressed":114499,"data_compressed_allocated":200704,"data_compressed_original":1200128,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T23:01:51.222 INFO:tasks.thrashosds.thrasher:Setting 3.11 to [5, 1, 6, 0] 2024-05-11T23:01:51.222 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.11', '5', '1', '6', '0'] 2024-05-11T23:01:51.222 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.11 5 1 6 0 2024-05-11T23:01:51.250 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:01:51.255 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:51.255+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:51.269 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:51.270 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:51.278 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:01:51.356 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:51.356+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:51.388 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:01:51.389 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:01:51.407 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:01:51.407 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:01:51.423 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:01:51.424 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:01:51.424 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:51.425 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:51.433 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:01:51.441 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:01:51.442 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:01:51.457 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:51.457+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:51.459 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:01:51.459 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:01:51.477 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:01:51.477 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:01:51.494 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:01:51.495 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:01:51.512 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:01:51.512 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:01:51.540 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:51.540 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:51.541 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:01:51.541 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:01:51.541 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:51.549 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:01:51.593 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:51.594+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:51.656 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:51.656 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:01:51.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:01:51.656 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:01:51.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.657 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:1666 3.a 3:5755b5e9:::rbd_data.111e50384849.0000000000000042:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:51.657 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.553359+0000", 2024-05-11T23:01:51.657 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.10196526, 2024-05-11T23:01:51.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0034024540000000001, 2024-05-11T23:01:51.657 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.657 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.657 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:51.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.657 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1666 2024-05-11T23:01:51.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.657 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553359+0000", 2024-05-11T23:01:51.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553359+0000", 2024-05-11T23:01:51.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553360+0000", 2024-05-11T23:01:51.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1800000000000005e-07 2024-05-11T23:01:51.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553369+0000", 2024-05-11T23:01:51.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7049999999999994e-06 2024-05-11T23:01:51.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553380+0000", 2024-05-11T23:01:51.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0285999999999999e-05 2024-05-11T23:01:51.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553387+0000", 2024-05-11T23:01:51.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3989999999999998e-06 2024-05-11T23:01:51.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:51.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553454+0000", 2024-05-11T23:01:51.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6581999999999994e-05 2024-05-11T23:01:51.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:51.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553543+0000", 2024-05-11T23:01:51.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9599999999999996e-05 2024-05-11T23:01:51.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:51.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553762+0000", 2024-05-11T23:01:51.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000218402 2024-05-11T23:01:51.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:51.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.554603+0000", 2024-05-11T23:01:51.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00084155000000000005 2024-05-11T23:01:51.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:51.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.556663+0000", 2024-05-11T23:01:51.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020600430000000001 2024-05-11T23:01:51.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:51.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.556717+0000", 2024-05-11T23:01:51.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3942e-05 2024-05-11T23:01:51.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:51.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.556761+0000", 2024-05-11T23:01:51.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4127e-05 2024-05-11T23:01:51.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.662 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.662 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:1667 3.a 3:5237eba0:::rbd_data.111e50384849.0000000000000044:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:51.662 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.553412+0000", 2024-05-11T23:01:51.663 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.101912139, 2024-05-11T23:01:51.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036787489999999998, 2024-05-11T23:01:51.663 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.663 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.663 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:51.663 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.663 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.663 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.663 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1667 2024-05-11T23:01:51.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.663 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553412+0000", 2024-05-11T23:01:51.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553412+0000", 2024-05-11T23:01:51.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553413+0000", 2024-05-11T23:01:51.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8699999999999995e-07 2024-05-11T23:01:51.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553416+0000", 2024-05-11T23:01:51.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.05e-06 2024-05-11T23:01:51.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553424+0000", 2024-05-11T23:01:51.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.011e-06 2024-05-11T23:01:51.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.553429+0000", 2024-05-11T23:01:51.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7400000000000001e-06 2024-05-11T23:01:51.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:51.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.554054+0000", 2024-05-11T23:01:51.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00062427700000000001 2024-05-11T23:01:51.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:51.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.554148+0000", 2024-05-11T23:01:51.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4145000000000001e-05 2024-05-11T23:01:51.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:51.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.554353+0000", 2024-05-11T23:01:51.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000204835 2024-05-11T23:01:51.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:51.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.556278+0000", 2024-05-11T23:01:51.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001925019 2024-05-11T23:01:51.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:51.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557014+0000", 2024-05-11T23:01:51.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00073628599999999995 2024-05-11T23:01:51.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:51.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557066+0000", 2024-05-11T23:01:51.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1836999999999999e-05 2024-05-11T23:01:51.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:51.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.557091+0000", 2024-05-11T23:01:51.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4961999999999999e-05 2024-05-11T23:01:51.668 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.668 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.668 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.668 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "pg_update_log_missing(3.1e epoch 25/19 rep_tid 82 entries 25'46 (0'0) error 3:7e05a7a0:::rbd_data.111e50384849.0000000000000058:head by client.4382.0:1858 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'44)", 2024-05-11T23:01:51.668 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.569301+0000", 2024-05-11T23:01:51.668 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.086022798999998, 2024-05-11T23:01:51.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048630660000000001, 2024-05-11T23:01:51.668 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.669 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.669 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:51.669 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569301+0000", 2024-05-11T23:01:51.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569301+0000", 2024-05-11T23:01:51.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569302+0000", 2024-05-11T23:01:51.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0800000000000003e-07 2024-05-11T23:01:51.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569310+0000", 2024-05-11T23:01:51.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9370000000000004e-06 2024-05-11T23:01:51.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569329+0000", 2024-05-11T23:01:51.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8808999999999999e-05 2024-05-11T23:01:51.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569344+0000", 2024-05-11T23:01:51.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4789e-05 2024-05-11T23:01:51.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:51.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.569405+0000", 2024-05-11T23:01:51.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0714000000000002e-05 2024-05-11T23:01:51.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:51.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574164+0000", 2024-05-11T23:01:51.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047599089999999997 2024-05-11T23:01:51.672 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.672 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.672 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.672 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:1902 3.1e e25/19 3:7e2d774f:::rbd_data.111e50384849.0000000000000050:head v 25'47, mlcod=25'44)", 2024-05-11T23:01:51.672 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.573037+0000", 2024-05-11T23:01:51.672 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.082287608000001, 2024-05-11T23:01:51.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042723099999999996, 2024-05-11T23:01:51.672 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.672 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.672 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:51.673 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573037+0000", 2024-05-11T23:01:51.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573037+0000", 2024-05-11T23:01:51.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573037+0000", 2024-05-11T23:01:51.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2300000000000001e-07 2024-05-11T23:01:51.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573438+0000", 2024-05-11T23:01:51.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00040049399999999999 2024-05-11T23:01:51.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573446+0000", 2024-05-11T23:01:51.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1890000000000007e-06 2024-05-11T23:01:51.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573451+0000", 2024-05-11T23:01:51.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1959999999999999e-06 2024-05-11T23:01:51.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:51.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573488+0000", 2024-05-11T23:01:51.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6467999999999997e-05 2024-05-11T23:01:51.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:51.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573507+0000", 2024-05-11T23:01:51.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9321e-05 2024-05-11T23:01:51.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:51.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577275+0000", 2024-05-11T23:01:51.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037682750000000002 2024-05-11T23:01:51.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:51.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577309+0000", 2024-05-11T23:01:51.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3744000000000001e-05 2024-05-11T23:01:51.676 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.676 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.676 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.676 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:1914 3.a 3:527cad80:::rbd_data.111e50384849.0000000000000057:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:51.676 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.573229+0000", 2024-05-11T23:01:51.676 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.082095335999998, 2024-05-11T23:01:51.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041158779999999999, 2024-05-11T23:01:51.677 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.677 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.677 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:51.677 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.677 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.677 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.677 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1914 2024-05-11T23:01:51.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.677 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573229+0000", 2024-05-11T23:01:51.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573229+0000", 2024-05-11T23:01:51.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573230+0000", 2024-05-11T23:01:51.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8800000000000002e-07 2024-05-11T23:01:51.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573234+0000", 2024-05-11T23:01:51.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6070000000000001e-06 2024-05-11T23:01:51.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573244+0000", 2024-05-11T23:01:51.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9389999999999992e-06 2024-05-11T23:01:51.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573249+0000", 2024-05-11T23:01:51.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:01:51.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:51.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573297+0000", 2024-05-11T23:01:51.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7636000000000003e-05 2024-05-11T23:01:51.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:51.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573373+0000", 2024-05-11T23:01:51.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6693000000000006e-05 2024-05-11T23:01:51.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:51.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.573566+0000", 2024-05-11T23:01:51.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019307699999999999 2024-05-11T23:01:51.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:51.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.576944+0000", 2024-05-11T23:01:51.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033778900000000001 2024-05-11T23:01:51.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:51.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577244+0000", 2024-05-11T23:01:51.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00030006400000000001 2024-05-11T23:01:51.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:51.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577306+0000", 2024-05-11T23:01:51.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1625999999999996e-05 2024-05-11T23:01:51.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:51.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.577345+0000", 2024-05-11T23:01:51.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8773999999999998e-05 2024-05-11T23:01:51.682 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.682 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.682 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.682 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:1904 3.1e e25/19 3:7e2d774f:::rbd_data.111e50384849.0000000000000050:head v 25'49, mlcod=25'44)", 2024-05-11T23:01:51.682 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.574045+0000", 2024-05-11T23:01:51.682 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.081279316, 2024-05-11T23:01:51.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0058621020000000001, 2024-05-11T23:01:51.682 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.682 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.682 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:51.682 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574045+0000", 2024-05-11T23:01:51.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574045+0000", 2024-05-11T23:01:51.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574045+0000", 2024-05-11T23:01:51.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:01:51.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574049+0000", 2024-05-11T23:01:51.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5899999999999999e-06 2024-05-11T23:01:51.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574057+0000", 2024-05-11T23:01:51.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4859999999999997e-06 2024-05-11T23:01:51.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574062+0000", 2024-05-11T23:01:51.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6530000000000002e-06 2024-05-11T23:01:51.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:51.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574199+0000", 2024-05-11T23:01:51.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013656900000000001 2024-05-11T23:01:51.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:51.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.574237+0000", 2024-05-11T23:01:51.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8055000000000001e-05 2024-05-11T23:01:51.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:51.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.579872+0000", 2024-05-11T23:01:51.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005635223 2024-05-11T23:01:51.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:51.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.579907+0000", 2024-05-11T23:01:51.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5027000000000002e-05 2024-05-11T23:01:51.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.686 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.686 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:1972 3.1e e25/19 3:7e05a7a0:::rbd_data.111e50384849.0000000000000058:head v 25'50, mlcod=25'46)", 2024-05-11T23:01:51.686 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.578576+0000", 2024-05-11T23:01:51.686 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.076747919999999, 2024-05-11T23:01:51.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038930829999999999, 2024-05-11T23:01:51.686 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.686 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.687 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:51.687 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578576+0000", 2024-05-11T23:01:51.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578576+0000", 2024-05-11T23:01:51.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578577+0000", 2024-05-11T23:01:51.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8999999999999996e-07 2024-05-11T23:01:51.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578582+0000", 2024-05-11T23:01:51.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5990000000000004e-06 2024-05-11T23:01:51.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578591+0000", 2024-05-11T23:01:51.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9690000000000006e-06 2024-05-11T23:01:51.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578598+0000", 2024-05-11T23:01:51.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2179999999999996e-06 2024-05-11T23:01:51.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:51.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578657+0000", 2024-05-11T23:01:51.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9001e-05 2024-05-11T23:01:51.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:51.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.578679+0000", 2024-05-11T23:01:51.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2259000000000002e-05 2024-05-11T23:01:51.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:51.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.582439+0000", 2024-05-11T23:01:51.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037597960000000001 2024-05-11T23:01:51.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:51.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.582469+0000", 2024-05-11T23:01:51.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0550999999999999e-05 2024-05-11T23:01:51.690 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.690 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.690 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.690 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "pg_update_log_missing(3.1e epoch 25/19 rep_tid 90 entries 25'51 (0'0) error 3:7facaf60:::rbd_data.111e50384849.0000000000000061:head by client.4382.0:2005 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 25'47)", 2024-05-11T23:01:51.690 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.581892+0000", 2024-05-11T23:01:51.690 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.073432501999999, 2024-05-11T23:01:51.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042265779999999999, 2024-05-11T23:01:51.690 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.691 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.691 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:51.691 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581892+0000", 2024-05-11T23:01:51.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581892+0000", 2024-05-11T23:01:51.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581892+0000", 2024-05-11T23:01:51.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9299999999999998e-07 2024-05-11T23:01:51.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581898+0000", 2024-05-11T23:01:51.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0390000000000001e-06 2024-05-11T23:01:51.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581915+0000", 2024-05-11T23:01:51.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6418000000000001e-05 2024-05-11T23:01:51.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581929+0000", 2024-05-11T23:01:51.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4131000000000001e-05 2024-05-11T23:01:51.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:51.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.581984+0000", 2024-05-11T23:01:51.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4821999999999998e-05 2024-05-11T23:01:51.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:51.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.586118+0000", 2024-05-11T23:01:51.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004134675 2024-05-11T23:01:51.693 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.693 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.694 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.694 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3471 3.a 3:50ba23c0:::rbd_data.111e89cdde7d.00000000000000b2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:51.694 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:33.894634+0000", 2024-05-11T23:01:51.694 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.760690122, 2024-05-11T23:01:51.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053259079999999999, 2024-05-11T23:01:51.694 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.694 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.694 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:51.694 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.694 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.694 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.694 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3471 2024-05-11T23:01:51.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.695 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894634+0000", 2024-05-11T23:01:51.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894634+0000", 2024-05-11T23:01:51.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894636+0000", 2024-05-11T23:01:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.542e-06 2024-05-11T23:01:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894640+0000", 2024-05-11T23:01:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9439999999999998e-06 2024-05-11T23:01:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894651+0000", 2024-05-11T23:01:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1952e-05 2024-05-11T23:01:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894657+0000", 2024-05-11T23:01:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5280000000000003e-06 2024-05-11T23:01:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894727+0000", 2024-05-11T23:01:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0029000000000003e-05 2024-05-11T23:01:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894822+0000", 2024-05-11T23:01:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5319e-05 2024-05-11T23:01:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:51.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.894892+0000", 2024-05-11T23:01:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9542999999999995e-05 2024-05-11T23:01:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.897034+0000", 2024-05-11T23:01:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021425139999999999 2024-05-11T23:01:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.899868+0000", 2024-05-11T23:01:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028339659999999998 2024-05-11T23:01:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:51.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.899918+0000", 2024-05-11T23:01:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9548000000000001e-05 2024-05-11T23:01:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:33.899960+0000", 2024-05-11T23:01:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2023000000000001e-05 2024-05-11T23:01:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3697 3.a 3:50ba23c0:::rbd_data.111e89cdde7d.00000000000000b2:head [set-alloc-hint object_size 1048576 write_size 1048576,write 573440~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.121225+0000", 2024-05-11T23:01:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.534099374, 2024-05-11T23:01:51.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054297959999999998, 2024-05-11T23:01:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3697 2024-05-11T23:01:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121225+0000", 2024-05-11T23:01:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121225+0000", 2024-05-11T23:01:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121227+0000", 2024-05-11T23:01:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.776e-06 2024-05-11T23:01:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121236+0000", 2024-05-11T23:01:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7510000000000003e-06 2024-05-11T23:01:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121247+0000", 2024-05-11T23:01:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.115e-05 2024-05-11T23:01:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121253+0000", 2024-05-11T23:01:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.75e-06 2024-05-11T23:01:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121297+0000", 2024-05-11T23:01:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3930000000000001e-05 2024-05-11T23:01:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121350+0000", 2024-05-11T23:01:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2978e-05 2024-05-11T23:01:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.121441+0000", 2024-05-11T23:01:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0662999999999996e-05 2024-05-11T23:01:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.123760+0000", 2024-05-11T23:01:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023193549999999999 2024-05-11T23:01:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.126594+0000", 2024-05-11T23:01:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028343410000000002 2024-05-11T23:01:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.126628+0000", 2024-05-11T23:01:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3621000000000002e-05 2024-05-11T23:01:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.126655+0000", 2024-05-11T23:01:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6480999999999999e-05 2024-05-11T23:01:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.704 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:3849 3.a 3:50ba23c0:::rbd_data.111e89cdde7d.00000000000000b2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.243274+0000", 2024-05-11T23:01:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.412050405999999, 2024-05-11T23:01:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038658270000000001, 2024-05-11T23:01:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 3849 2024-05-11T23:01:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.705 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243274+0000", 2024-05-11T23:01:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243274+0000", 2024-05-11T23:01:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243275+0000", 2024-05-11T23:01:51.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3399999999999997e-07 2024-05-11T23:01:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243279+0000", 2024-05-11T23:01:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7779999999999998e-06 2024-05-11T23:01:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243294+0000", 2024-05-11T23:01:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4253000000000001e-05 2024-05-11T23:01:51.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243302+0000", 2024-05-11T23:01:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9240000000000007e-06 2024-05-11T23:01:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243364+0000", 2024-05-11T23:01:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2349999999999998e-05 2024-05-11T23:01:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243443+0000", 2024-05-11T23:01:51.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8877999999999999e-05 2024-05-11T23:01:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.243543+0000", 2024-05-11T23:01:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010016199999999999 2024-05-11T23:01:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.244177+0000", 2024-05-11T23:01:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00063432099999999997 2024-05-11T23:01:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.247082+0000", 2024-05-11T23:01:51.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029042 2024-05-11T23:01:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.247111+0000", 2024-05-11T23:01:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9839e-05 2024-05-11T23:01:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.247140+0000", 2024-05-11T23:01:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8187999999999998e-05 2024-05-11T23:01:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4161 3.a 3:50c2a80d:::rbd_data.111e89cdde7d.0000000000000087:head [set-alloc-hint object_size 1048576 write_size 1048576,write 614400~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.499080+0000", 2024-05-11T23:01:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.156244538999999, 2024-05-11T23:01:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054594780000000002, 2024-05-11T23:01:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4161 2024-05-11T23:01:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499080+0000", 2024-05-11T23:01:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499080+0000", 2024-05-11T23:01:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499081+0000", 2024-05-11T23:01:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.733e-06 2024-05-11T23:01:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499096+0000", 2024-05-11T23:01:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4721e-05 2024-05-11T23:01:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499114+0000", 2024-05-11T23:01:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8238000000000001e-05 2024-05-11T23:01:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499124+0000", 2024-05-11T23:01:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0096e-05 2024-05-11T23:01:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499186+0000", 2024-05-11T23:01:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1124000000000006e-05 2024-05-11T23:01:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499323+0000", 2024-05-11T23:01:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013730599999999999 2024-05-11T23:01:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.499458+0000", 2024-05-11T23:01:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013542599999999999 2024-05-11T23:01:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.501629+0000", 2024-05-11T23:01:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002170543 2024-05-11T23:01:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504476+0000", 2024-05-11T23:01:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028470879999999998 2024-05-11T23:01:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504508+0000", 2024-05-11T23:01:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2360000000000002e-05 2024-05-11T23:01:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:51.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.504539+0000", 2024-05-11T23:01:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0843000000000003e-05 2024-05-11T23:01:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:4252 3.a 3:50c2a80d:::rbd_data.111e89cdde7d.0000000000000087:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-05-11T23:01:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:34.600827+0000", 2024-05-11T23:01:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.054496828000001, 2024-05-11T23:01:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054938859999999999, 2024-05-11T23:01:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 4252 2024-05-11T23:01:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600827+0000", 2024-05-11T23:01:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600827+0000", 2024-05-11T23:01:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600829+0000", 2024-05-11T23:01:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2139999999999999e-06 2024-05-11T23:01:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600833+0000", 2024-05-11T23:01:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5580000000000002e-06 2024-05-11T23:01:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600844+0000", 2024-05-11T23:01:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0915e-05 2024-05-11T23:01:51.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600852+0000", 2024-05-11T23:01:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6260000000000003e-06 2024-05-11T23:01:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.600924+0000", 2024-05-11T23:01:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1884000000000005e-05 2024-05-11T23:01:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.601012+0000", 2024-05-11T23:01:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8732000000000004e-05 2024-05-11T23:01:51.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.601137+0000", 2024-05-11T23:01:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012487500000000001 2024-05-11T23:01:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.603281+0000", 2024-05-11T23:01:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002144165 2024-05-11T23:01:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606216+0000", 2024-05-11T23:01:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029350790000000002 2024-05-11T23:01:51.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606277+0000", 2024-05-11T23:01:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0752999999999999e-05 2024-05-11T23:01:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:34.606321+0000", 2024-05-11T23:01:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4085000000000002e-05 2024-05-11T23:01:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:14690 3.a 3:5038d151:::rbd_data.111e4e122501.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 770614~16808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-11T23:01:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:48.127292+0000", 2024-05-11T23:01:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.5280326190000002, 2024-05-11T23:01:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0043293070000000001, 2024-05-11T23:01:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 14690 2024-05-11T23:01:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.722 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.127292+0000", 2024-05-11T23:01:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.127292+0000", 2024-05-11T23:01:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.127293+0000", 2024-05-11T23:01:51.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8920000000000001e-06 2024-05-11T23:01:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.127335+0000", 2024-05-11T23:01:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1998e-05 2024-05-11T23:01:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.127353+0000", 2024-05-11T23:01:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.774e-05 2024-05-11T23:01:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.127361+0000", 2024-05-11T23:01:51.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7800000000000001e-06 2024-05-11T23:01:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.127428+0000", 2024-05-11T23:01:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7050000000000003e-05 2024-05-11T23:01:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.127557+0000", 2024-05-11T23:01:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00012905600000000001 2024-05-11T23:01:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.127770+0000", 2024-05-11T23:01:51.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021276499999999999 2024-05-11T23:01:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.128596+0000", 2024-05-11T23:01:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00082610699999999997 2024-05-11T23:01:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.131562+0000", 2024-05-11T23:01:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029659500000000002 2024-05-11T23:01:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:51.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.131596+0000", 2024-05-11T23:01:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4467000000000001e-05 2024-05-11T23:01:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.131621+0000", 2024-05-11T23:01:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4502e-05 2024-05-11T23:01:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:14814 3.1e e36/19 3:78813778:::rbd_object_map.111e4e122501:head v 36'223, mlcod=36'219)", 2024-05-11T23:01:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:48.213473+0000", 2024-05-11T23:01:51.727 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 3.4418512990000001, 2024-05-11T23:01:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005403471, 2024-05-11T23:01:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.213473+0000", 2024-05-11T23:01:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.213473+0000", 2024-05-11T23:01:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.213473+0000", 2024-05-11T23:01:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3799999999999998e-07 2024-05-11T23:01:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.213476+0000", 2024-05-11T23:01:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9220000000000001e-06 2024-05-11T23:01:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.213483+0000", 2024-05-11T23:01:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1029999999999996e-06 2024-05-11T23:01:51.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.213488+0000", 2024-05-11T23:01:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1970000000000004e-06 2024-05-11T23:01:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.213526+0000", 2024-05-11T23:01:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7830999999999997e-05 2024-05-11T23:01:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:51.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.213538+0000", 2024-05-11T23:01:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2102e-05 2024-05-11T23:01:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.218847+0000", 2024-05-11T23:01:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053086030000000003 2024-05-11T23:01:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.218876+0000", 2024-05-11T23:01:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9374999999999999e-05 2024-05-11T23:01:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:14995 3.10 3:0e3c085a:::rbd_data.111e4e122501.00000000000000ad:head [delete] snapc f=[f] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-11T23:01:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:48.884833+0000", 2024-05-11T23:01:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.770491013, 2024-05-11T23:01:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0055767020000000002, 2024-05-11T23:01:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 14995 2024-05-11T23:01:51.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.884833+0000", 2024-05-11T23:01:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.884833+0000", 2024-05-11T23:01:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.884834+0000", 2024-05-11T23:01:51.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1900000000000001e-07 2024-05-11T23:01:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.884839+0000", 2024-05-11T23:01:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0010000000000001e-06 2024-05-11T23:01:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.884850+0000", 2024-05-11T23:01:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1430000000000001e-05 2024-05-11T23:01:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.884858+0000", 2024-05-11T23:01:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4699999999999996e-06 2024-05-11T23:01:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.885217+0000", 2024-05-11T23:01:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035889599999999999 2024-05-11T23:01:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.885300+0000", 2024-05-11T23:01:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2991999999999994e-05 2024-05-11T23:01:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.885491+0000", 2024-05-11T23:01:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000191351 2024-05-11T23:01:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.886315+0000", 2024-05-11T23:01:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00082390399999999998 2024-05-11T23:01:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.890319+0000", 2024-05-11T23:01:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040037229999999998 2024-05-11T23:01:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.890371+0000", 2024-05-11T23:01:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2091e-05 2024-05-11T23:01:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.890410+0000", 2024-05-11T23:01:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8924999999999997e-05 2024-05-11T23:01:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:14996 3.a 3:5680116d:::rbd_data.111e4e122501.00000000000000de:head [delete] snapc f=[f] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-11T23:01:51.737 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:48.885515+0000", 2024-05-11T23:01:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.769808737, 2024-05-11T23:01:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0073724230000000003, 2024-05-11T23:01:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 14996 2024-05-11T23:01:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.885515+0000", 2024-05-11T23:01:51.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.885515+0000", 2024-05-11T23:01:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.885516+0000", 2024-05-11T23:01:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0419999999999999e-06 2024-05-11T23:01:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.885522+0000", 2024-05-11T23:01:51.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4469999999999997e-06 2024-05-11T23:01:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.885535+0000", 2024-05-11T23:01:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3254999999999999e-05 2024-05-11T23:01:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.885543+0000", 2024-05-11T23:01:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7479999999999993e-06 2024-05-11T23:01:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:51.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.885628+0000", 2024-05-11T23:01:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4593999999999997e-05 2024-05-11T23:01:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.885697+0000", 2024-05-11T23:01:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9148999999999998e-05 2024-05-11T23:01:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:01:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.885878+0000", 2024-05-11T23:01:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000181774 2024-05-11T23:01:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.890406+0000", 2024-05-11T23:01:51.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045275649999999999 2024-05-11T23:01:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.892829+0000", 2024-05-11T23:01:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024235039999999999 2024-05-11T23:01:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.892862+0000", 2024-05-11T23:01:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2159999999999997e-05 2024-05-11T23:01:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:51.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.892888+0000", 2024-05-11T23:01:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6185000000000001e-05 2024-05-11T23:01:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:14998 3.1e e37/19 3:7ad70610:::rbd_data.111e4e122501.00000000000000e0:head v 37'249, mlcod=37'246)", 2024-05-11T23:01:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:48.886508+0000", 2024-05-11T23:01:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.7688161600000001, 2024-05-11T23:01:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063572079999999996, 2024-05-11T23:01:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:51.743 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.886508+0000", 2024-05-11T23:01:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.886508+0000", 2024-05-11T23:01:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.886509+0000", 2024-05-11T23:01:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4700000000000001e-07 2024-05-11T23:01:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.886514+0000", 2024-05-11T23:01:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0669999999999999e-06 2024-05-11T23:01:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.886523+0000", 2024-05-11T23:01:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7970000000000006e-06 2024-05-11T23:01:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.886528+0000", 2024-05-11T23:01:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4750000000000001e-06 2024-05-11T23:01:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.886580+0000", 2024-05-11T23:01:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1693999999999997e-05 2024-05-11T23:01:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.886601+0000", 2024-05-11T23:01:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1129e-05 2024-05-11T23:01:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.892841+0000", 2024-05-11T23:01:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062399680000000003 2024-05-11T23:01:51.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.892865+0000", 2024-05-11T23:01:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4331000000000001e-05 2024-05-11T23:01:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:15001 3.1e e37/19 3:78813778:::rbd_object_map.111e4e122501:head v 37'252, mlcod=37'249)", 2024-05-11T23:01:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:48.897281+0000", 2024-05-11T23:01:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.7580428399999999, 2024-05-11T23:01:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004060134, 2024-05-11T23:01:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.747 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.897281+0000", 2024-05-11T23:01:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.897281+0000", 2024-05-11T23:01:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.897282+0000", 2024-05-11T23:01:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3999999999999997e-07 2024-05-11T23:01:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.897285+0000", 2024-05-11T23:01:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6119999999999999e-06 2024-05-11T23:01:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.897292+0000", 2024-05-11T23:01:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6030000000000001e-06 2024-05-11T23:01:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.897295+0000", 2024-05-11T23:01:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.456e-06 2024-05-11T23:01:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.897333+0000", 2024-05-11T23:01:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7694000000000002e-05 2024-05-11T23:01:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.897347+0000", 2024-05-11T23:01:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3692999999999999e-05 2024-05-11T23:01:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.901310+0000", 2024-05-11T23:01:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003963093 2024-05-11T23:01:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.901341+0000", 2024-05-11T23:01:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1643000000000002e-05 2024-05-11T23:01:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:15002 3.1e e37/19 3:78813778:::rbd_object_map.111e4e122501:head v 37'253, mlcod=37'249)", 2024-05-11T23:01:51.751 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:48.897854+0000", 2024-05-11T23:01:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 2.7574700280000002, 2024-05-11T23:01:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037785639999999999, 2024-05-11T23:01:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.897854+0000", 2024-05-11T23:01:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.897854+0000", 2024-05-11T23:01:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.897855+0000", 2024-05-11T23:01:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8999999999999996e-07 2024-05-11T23:01:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.897859+0000", 2024-05-11T23:01:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7319999999999998e-06 2024-05-11T23:01:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.897869+0000", 2024-05-11T23:01:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8619999999999998e-06 2024-05-11T23:01:51.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.897875+0000", 2024-05-11T23:01:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2039999999999996e-06 2024-05-11T23:01:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.897909+0000", 2024-05-11T23:01:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4754e-05 2024-05-11T23:01:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.897923+0000", 2024-05-11T23:01:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3631e-05 2024-05-11T23:01:51.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.901616+0000", 2024-05-11T23:01:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036926459999999999 2024-05-11T23:01:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:48.901633+0000", 2024-05-11T23:01:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7144999999999999e-05 2024-05-11T23:01:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.755 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.755 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:51.757 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:01:51.757 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:51.757+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:51.829 INFO:teuthology.orchestra.run.smithi005.stderr:set 3.11 pg_upmap_items mapping to [5->1,6->0] 2024-05-11T23:01:51.885 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:51.885+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:51.906 INFO:teuthology.orchestra.run.smithi005.stdout:flattening image image_client.0-clone14 2024-05-11T23:01:51.925 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:51.925 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:01:51.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.925 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23915 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.925 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913123+0000", 2024-05-11T23:01:51.925 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0033709339999999999, 2024-05-11T23:01:51.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0034014660000000001, 2024-05-11T23:01:51.926 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.926 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.926 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.926 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.926 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.926 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.926 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23915 2024-05-11T23:01:51.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.926 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913123+0000", 2024-05-11T23:01:51.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913123+0000", 2024-05-11T23:01:51.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913124+0000", 2024-05-11T23:01:51.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0499999999999999e-07 2024-05-11T23:01:51.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913126+0000", 2024-05-11T23:01:51.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0719999999999998e-06 2024-05-11T23:01:51.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913142+0000", 2024-05-11T23:01:51.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6112000000000001e-05 2024-05-11T23:01:51.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913148+0000", 2024-05-11T23:01:51.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4150000000000004e-06 2024-05-11T23:01:51.928 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.928 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.928 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.929 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24038 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.929 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913986+0000", 2024-05-11T23:01:51.929 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0025082519999999999, 2024-05-11T23:01:51.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0026034169999999998, 2024-05-11T23:01:51.929 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.929 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.929 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.929 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.929 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.929 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.929 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24038 2024-05-11T23:01:51.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.930 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913986+0000", 2024-05-11T23:01:51.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913986+0000", 2024-05-11T23:01:51.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913986+0000", 2024-05-11T23:01:51.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5999999999999999e-07 2024-05-11T23:01:51.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913989+0000", 2024-05-11T23:01:51.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099999999999999e-06 2024-05-11T23:01:51.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913995+0000", 2024-05-11T23:01:51.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6830000000000002e-06 2024-05-11T23:01:51.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914000+0000", 2024-05-11T23:01:51.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0969999999999996e-06 2024-05-11T23:01:51.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.932 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.932 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24163 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.932 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914746+0000", 2024-05-11T23:01:51.932 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0017478630000000001, 2024-05-11T23:01:51.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001897904, 2024-05-11T23:01:51.932 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.932 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.932 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.932 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.933 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24163 2024-05-11T23:01:51.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.933 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914746+0000", 2024-05-11T23:01:51.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914746+0000", 2024-05-11T23:01:51.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914747+0000", 2024-05-11T23:01:51.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1600000000000002e-07 2024-05-11T23:01:51.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914753+0000", 2024-05-11T23:01:51.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6660000000000002e-06 2024-05-11T23:01:51.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914759+0000", 2024-05-11T23:01:51.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8780000000000003e-06 2024-05-11T23:01:51.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914764+0000", 2024-05-11T23:01:51.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4610000000000001e-06 2024-05-11T23:01:51.935 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.935 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.935 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.935 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23922 3.11 3.23111871 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.935 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913157+0000", 2024-05-11T23:01:51.935 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0033375240000000001, 2024-05-11T23:01:51.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035424459999999999, 2024-05-11T23:01:51.936 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.936 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.936 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.936 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23922 2024-05-11T23:01:51.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.936 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913157+0000", 2024-05-11T23:01:51.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913157+0000", 2024-05-11T23:01:51.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913157+0000", 2024-05-11T23:01:51.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0999999999999999e-07 2024-05-11T23:01:51.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913160+0000", 2024-05-11T23:01:51.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6869999999999999e-06 2024-05-11T23:01:51.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913166+0000", 2024-05-11T23:01:51.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3119999999999999e-06 2024-05-11T23:01:51.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913171+0000", 2024-05-11T23:01:51.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7679999999999999e-06 2024-05-11T23:01:51.938 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.938 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.938 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.938 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24041 3.11 3.23111871 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.939 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914007+0000", 2024-05-11T23:01:51.939 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0024869689999999999, 2024-05-11T23:01:51.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002745497, 2024-05-11T23:01:51.939 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.939 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.939 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.939 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.939 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.939 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.939 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24041 2024-05-11T23:01:51.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.939 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914007+0000", 2024-05-11T23:01:51.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914007+0000", 2024-05-11T23:01:51.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914007+0000", 2024-05-11T23:01:51.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9799999999999999e-07 2024-05-11T23:01:51.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914010+0000", 2024-05-11T23:01:51.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3769999999999999e-06 2024-05-11T23:01:51.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914024+0000", 2024-05-11T23:01:51.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4615e-05 2024-05-11T23:01:51.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914031+0000", 2024-05-11T23:01:51.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2550000000000003e-06 2024-05-11T23:01:51.941 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.941 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.941 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.942 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24166 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.942 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914769+0000", 2024-05-11T23:01:51.942 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001724724, 2024-05-11T23:01:51.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0020380429999999998, 2024-05-11T23:01:51.942 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.942 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.942 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.942 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.942 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.942 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.942 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24166 2024-05-11T23:01:51.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.942 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914769+0000", 2024-05-11T23:01:51.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914769+0000", 2024-05-11T23:01:51.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914770+0000", 2024-05-11T23:01:51.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2899999999999999e-07 2024-05-11T23:01:51.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914771+0000", 2024-05-11T23:01:51.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.798e-06 2024-05-11T23:01:51.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914777+0000", 2024-05-11T23:01:51.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5940000000000001e-06 2024-05-11T23:01:51.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914781+0000", 2024-05-11T23:01:51.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4179999999999999e-06 2024-05-11T23:01:51.944 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.945 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.945 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.945 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23928 3.11 3.23111871 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.945 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913177+0000", 2024-05-11T23:01:51.945 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0033168260000000001, 2024-05-11T23:01:51.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003688556, 2024-05-11T23:01:51.945 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.945 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.946 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.946 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.946 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.946 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.946 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23928 2024-05-11T23:01:51.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.946 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913177+0000", 2024-05-11T23:01:51.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913177+0000", 2024-05-11T23:01:51.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913178+0000", 2024-05-11T23:01:51.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9100000000000004e-07 2024-05-11T23:01:51.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913184+0000", 2024-05-11T23:01:51.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5370000000000003e-06 2024-05-11T23:01:51.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913191+0000", 2024-05-11T23:01:51.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6699999999999997e-06 2024-05-11T23:01:51.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913196+0000", 2024-05-11T23:01:51.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1290000000000004e-06 2024-05-11T23:01:51.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.949 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.949 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24042 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.949 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914044+0000", 2024-05-11T23:01:51.949 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0024496700000000001, 2024-05-11T23:01:51.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0028763959999999998, 2024-05-11T23:01:51.949 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.949 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.949 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.950 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24042 2024-05-11T23:01:51.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.950 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914044+0000", 2024-05-11T23:01:51.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914044+0000", 2024-05-11T23:01:51.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914045+0000", 2024-05-11T23:01:51.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000001e-07 2024-05-11T23:01:51.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914047+0000", 2024-05-11T23:01:51.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5509999999999998e-06 2024-05-11T23:01:51.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914054+0000", 2024-05-11T23:01:51.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9539999999999998e-06 2024-05-11T23:01:51.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914059+0000", 2024-05-11T23:01:51.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0000000000000004e-06 2024-05-11T23:01:51.952 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.952 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.952 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.952 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24169 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.952 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914787+0000", 2024-05-11T23:01:51.952 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0017073629999999999, 2024-05-11T23:01:51.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021878610000000001, 2024-05-11T23:01:51.953 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.953 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.953 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.953 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24169 2024-05-11T23:01:51.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.953 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914787+0000", 2024-05-11T23:01:51.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914787+0000", 2024-05-11T23:01:51.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914787+0000", 2024-05-11T23:01:51.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6199999999999999e-07 2024-05-11T23:01:51.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914789+0000", 2024-05-11T23:01:51.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.886e-06 2024-05-11T23:01:51.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914803+0000", 2024-05-11T23:01:51.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4267000000000001e-05 2024-05-11T23:01:51.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914809+0000", 2024-05-11T23:01:51.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7119999999999997e-06 2024-05-11T23:01:51.956 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.956 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.956 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.956 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23934 3.11 3.23111871 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.956 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913210+0000", 2024-05-11T23:01:51.956 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0032844739999999999, 2024-05-11T23:01:51.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038218480000000001, 2024-05-11T23:01:51.956 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.956 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.956 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.956 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.957 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23934 2024-05-11T23:01:51.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.957 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913210+0000", 2024-05-11T23:01:51.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913210+0000", 2024-05-11T23:01:51.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913210+0000", 2024-05-11T23:01:51.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4300000000000003e-07 2024-05-11T23:01:51.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913213+0000", 2024-05-11T23:01:51.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.554e-06 2024-05-11T23:01:51.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913219+0000", 2024-05-11T23:01:51.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7989999999999996e-06 2024-05-11T23:01:51.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913233+0000", 2024-05-11T23:01:51.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3067999999999999e-05 2024-05-11T23:01:51.959 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.959 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.959 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.959 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24046 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.959 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914065+0000", 2024-05-11T23:01:51.959 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.002428702, 2024-05-11T23:01:51.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030183879999999999, 2024-05-11T23:01:51.959 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.960 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.960 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.960 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.960 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.960 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.960 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24046 2024-05-11T23:01:51.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.960 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914065+0000", 2024-05-11T23:01:51.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914065+0000", 2024-05-11T23:01:51.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914066+0000", 2024-05-11T23:01:51.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.58e-07 2024-05-11T23:01:51.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914068+0000", 2024-05-11T23:01:51.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8449999999999999e-06 2024-05-11T23:01:51.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914073+0000", 2024-05-11T23:01:51.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6960000000000002e-06 2024-05-11T23:01:51.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914078+0000", 2024-05-11T23:01:51.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.22e-06 2024-05-11T23:01:51.962 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.962 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.962 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.962 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24172 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.962 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914816+0000", 2024-05-11T23:01:51.962 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001677652, 2024-05-11T23:01:51.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0023234459999999998, 2024-05-11T23:01:51.963 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.963 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.963 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.963 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.963 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.963 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.963 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24172 2024-05-11T23:01:51.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.963 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914816+0000", 2024-05-11T23:01:51.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914816+0000", 2024-05-11T23:01:51.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914817+0000", 2024-05-11T23:01:51.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000002e-07 2024-05-11T23:01:51.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914819+0000", 2024-05-11T23:01:51.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3e-06 2024-05-11T23:01:51.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914825+0000", 2024-05-11T23:01:51.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2169999999999999e-06 2024-05-11T23:01:51.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914830+0000", 2024-05-11T23:01:51.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9089999999999999e-06 2024-05-11T23:01:51.965 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.965 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.965 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.966 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23940 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.966 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913240+0000", 2024-05-11T23:01:51.966 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0032538160000000001, 2024-05-11T23:01:51.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039569920000000003, 2024-05-11T23:01:51.966 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.966 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.966 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.966 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.966 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.966 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.966 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23940 2024-05-11T23:01:51.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.966 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913240+0000", 2024-05-11T23:01:51.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913240+0000", 2024-05-11T23:01:51.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913241+0000", 2024-05-11T23:01:51.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2800000000000002e-07 2024-05-11T23:01:51.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913243+0000", 2024-05-11T23:01:51.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3520000000000001e-06 2024-05-11T23:01:51.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913253+0000", 2024-05-11T23:01:51.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5759999999999995e-06 2024-05-11T23:01:51.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913258+0000", 2024-05-11T23:01:51.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3229999999999997e-06 2024-05-11T23:01:51.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.969 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.969 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.969 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24050 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.969 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914084+0000", 2024-05-11T23:01:51.969 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.002410171, 2024-05-11T23:01:51.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003171452, 2024-05-11T23:01:51.969 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.969 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.969 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.969 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24050 2024-05-11T23:01:51.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.970 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914084+0000", 2024-05-11T23:01:51.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914084+0000", 2024-05-11T23:01:51.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914084+0000", 2024-05-11T23:01:51.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:01:51.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914086+0000", 2024-05-11T23:01:51.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8199999999999999e-06 2024-05-11T23:01:51.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914092+0000", 2024-05-11T23:01:51.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7119999999999997e-06 2024-05-11T23:01:51.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914096+0000", 2024-05-11T23:01:51.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6079999999999998e-06 2024-05-11T23:01:51.972 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.972 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.972 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.972 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24175 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.972 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914837+0000", 2024-05-11T23:01:51.972 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0016574129999999999, 2024-05-11T23:01:51.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024716719999999998, 2024-05-11T23:01:51.973 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.973 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.973 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.973 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.973 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.973 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.973 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24175 2024-05-11T23:01:51.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.973 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914837+0000", 2024-05-11T23:01:51.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914837+0000", 2024-05-11T23:01:51.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914837+0000", 2024-05-11T23:01:51.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-05-11T23:01:51.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914839+0000", 2024-05-11T23:01:51.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0260000000000002e-06 2024-05-11T23:01:51.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914852+0000", 2024-05-11T23:01:51.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2981e-05 2024-05-11T23:01:51.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914857+0000", 2024-05-11T23:01:51.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1080000000000001e-06 2024-05-11T23:01:51.975 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.975 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.975 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.976 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23943 3.11 3.23111871 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.976 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913265+0000", 2024-05-11T23:01:51.976 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0032287470000000001, 2024-05-11T23:01:51.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040992900000000002, 2024-05-11T23:01:51.976 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.976 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.976 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.976 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.976 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.976 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.976 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23943 2024-05-11T23:01:51.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.976 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913265+0000", 2024-05-11T23:01:51.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913265+0000", 2024-05-11T23:01:51.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913266+0000", 2024-05-11T23:01:51.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9700000000000002e-07 2024-05-11T23:01:51.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913268+0000", 2024-05-11T23:01:51.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2900000000000001e-06 2024-05-11T23:01:51.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913274+0000", 2024-05-11T23:01:51.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4049999999999997e-06 2024-05-11T23:01:51.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913279+0000", 2024-05-11T23:01:51.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7099999999999998e-06 2024-05-11T23:01:51.979 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.979 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.979 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.979 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24055 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.979 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914115+0000", 2024-05-11T23:01:51.979 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0023790259999999998, 2024-05-11T23:01:51.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0033031169999999999, 2024-05-11T23:01:51.979 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.979 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.979 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.979 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24055 2024-05-11T23:01:51.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.980 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914115+0000", 2024-05-11T23:01:51.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914115+0000", 2024-05-11T23:01:51.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914115+0000", 2024-05-11T23:01:51.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6400000000000003e-07 2024-05-11T23:01:51.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914118+0000", 2024-05-11T23:01:51.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4219999999999999e-06 2024-05-11T23:01:51.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914132+0000", 2024-05-11T23:01:51.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3675e-05 2024-05-11T23:01:51.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914137+0000", 2024-05-11T23:01:51.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8089999999999998e-06 2024-05-11T23:01:51.981 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.981 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.982 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.982 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24178 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.982 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914871+0000", 2024-05-11T23:01:51.982 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001622603, 2024-05-11T23:01:51.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002602994, 2024-05-11T23:01:51.982 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.982 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.982 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.982 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.982 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.982 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.982 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24178 2024-05-11T23:01:51.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.983 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914871+0000", 2024-05-11T23:01:51.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914871+0000", 2024-05-11T23:01:51.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914872+0000", 2024-05-11T23:01:51.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-05-11T23:01:51.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914875+0000", 2024-05-11T23:01:51.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5569999999999998e-06 2024-05-11T23:01:51.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914881+0000", 2024-05-11T23:01:51.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7190000000000004e-06 2024-05-11T23:01:51.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914887+0000", 2024-05-11T23:01:51.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3870000000000003e-06 2024-05-11T23:01:51.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.985 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.985 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.985 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23944 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.985 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913285+0000", 2024-05-11T23:01:51.985 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0032086010000000002, 2024-05-11T23:01:51.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042496069999999999, 2024-05-11T23:01:51.985 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.985 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.985 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.985 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23944 2024-05-11T23:01:51.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.986 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913285+0000", 2024-05-11T23:01:51.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913285+0000", 2024-05-11T23:01:51.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913286+0000", 2024-05-11T23:01:51.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3999999999999997e-07 2024-05-11T23:01:51.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913288+0000", 2024-05-11T23:01:51.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1509999999999998e-06 2024-05-11T23:01:51.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913294+0000", 2024-05-11T23:01:51.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3090000000000001e-06 2024-05-11T23:01:51.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913307+0000", 2024-05-11T23:01:51.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2309e-05 2024-05-11T23:01:51.987 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.988 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.988 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24058 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.988 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914145+0000", 2024-05-11T23:01:51.988 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.002348934, 2024-05-11T23:01:51.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0034547549999999999, 2024-05-11T23:01:51.988 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.988 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.988 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.988 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24058 2024-05-11T23:01:51.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.989 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914145+0000", 2024-05-11T23:01:51.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914145+0000", 2024-05-11T23:01:51.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914146+0000", 2024-05-11T23:01:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.82e-07 2024-05-11T23:01:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914153+0000", 2024-05-11T23:01:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1969999999999999e-06 2024-05-11T23:01:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914160+0000", 2024-05-11T23:01:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.922e-06 2024-05-11T23:01:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914165+0000", 2024-05-11T23:01:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8359999999999999e-06 2024-05-11T23:01:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24180 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914893+0000", 2024-05-11T23:01:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001601005, 2024-05-11T23:01:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0027694989999999999, 2024-05-11T23:01:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.991 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.992 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.992 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24180 2024-05-11T23:01:51.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.992 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914893+0000", 2024-05-11T23:01:51.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914893+0000", 2024-05-11T23:01:51.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914893+0000", 2024-05-11T23:01:51.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3599999999999999e-07 2024-05-11T23:01:51.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914895+0000", 2024-05-11T23:01:51.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7859999999999999e-06 2024-05-11T23:01:51.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914901+0000", 2024-05-11T23:01:51.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8470000000000001e-06 2024-05-11T23:01:51.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914906+0000", 2024-05-11T23:01:51.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5260000000000004e-06 2024-05-11T23:01:51.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.994 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.994 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.994 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23947 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.994 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913314+0000", 2024-05-11T23:01:51.994 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0031798730000000002, 2024-05-11T23:01:51.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044046770000000001, 2024-05-11T23:01:51.994 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.994 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.994 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.995 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23947 2024-05-11T23:01:51.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.995 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913314+0000", 2024-05-11T23:01:51.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913314+0000", 2024-05-11T23:01:51.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913315+0000", 2024-05-11T23:01:51.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3500000000000002e-07 2024-05-11T23:01:51.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913317+0000", 2024-05-11T23:01:51.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6900000000000001e-06 2024-05-11T23:01:51.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:51.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913324+0000", 2024-05-11T23:01:51.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6000000000000003e-06 2024-05-11T23:01:51.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:51.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913332+0000", 2024-05-11T23:01:51.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5920000000000001e-06 2024-05-11T23:01:51.997 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.997 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:51.997 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:51.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.997 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24061 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:51.997 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914172+0000", 2024-05-11T23:01:51.997 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0023220910000000001, 2024-05-11T23:01:51.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0036040030000000002, 2024-05-11T23:01:51.997 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:51.997 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:51.998 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:51.998 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:51.998 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:51.998 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:51.998 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24061 2024-05-11T23:01:51.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.998 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:51.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:51.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914172+0000", 2024-05-11T23:01:51.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:51.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914172+0000", 2024-05-11T23:01:51.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:51.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:51.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914172+0000", 2024-05-11T23:01:51.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5999999999999999e-07 2024-05-11T23:01:51.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:51.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914175+0000", 2024-05-11T23:01:51.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2879999999999999e-06 2024-05-11T23:01:51.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:51.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:51.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914191+0000", 2024-05-11T23:01:52.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6365999999999999e-05 2024-05-11T23:01:52.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914199+0000", 2024-05-11T23:01:52.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4209999999999995e-06 2024-05-11T23:01:52.000 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.000 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.000 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.000 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24184 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.001 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914911+0000", 2024-05-11T23:01:52.001 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001583187, 2024-05-11T23:01:52.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0029187200000000001, 2024-05-11T23:01:52.001 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.001 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.001 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.001 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24184 2024-05-11T23:01:52.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.001 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914911+0000", 2024-05-11T23:01:52.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914911+0000", 2024-05-11T23:01:52.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914911+0000", 2024-05-11T23:01:52.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4999999999999998e-07 2024-05-11T23:01:52.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914913+0000", 2024-05-11T23:01:52.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6530000000000001e-06 2024-05-11T23:01:52.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914919+0000", 2024-05-11T23:01:52.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.699e-06 2024-05-11T23:01:52.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914927+0000", 2024-05-11T23:01:52.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2179999999999999e-06 2024-05-11T23:01:52.003 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.003 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.003 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.004 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23951 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.004 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913349+0000", 2024-05-11T23:01:52.004 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0031446920000000001, 2024-05-11T23:01:52.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045381980000000002, 2024-05-11T23:01:52.004 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.004 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.004 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.004 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.004 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.004 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.004 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23951 2024-05-11T23:01:52.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.004 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913349+0000", 2024-05-11T23:01:52.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913349+0000", 2024-05-11T23:01:52.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913350+0000", 2024-05-11T23:01:52.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6000000000000004e-07 2024-05-11T23:01:52.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913352+0000", 2024-05-11T23:01:52.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5189999999999999e-06 2024-05-11T23:01:52.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913361+0000", 2024-05-11T23:01:52.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3699999999999995e-06 2024-05-11T23:01:52.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913366+0000", 2024-05-11T23:01:52.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0200000000000002e-06 2024-05-11T23:01:52.006 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.006 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.006 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.007 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24066 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.007 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914207+0000", 2024-05-11T23:01:52.007 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.002286677, 2024-05-11T23:01:52.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0037366360000000002, 2024-05-11T23:01:52.007 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.007 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.007 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.007 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24066 2024-05-11T23:01:52.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.007 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914207+0000", 2024-05-11T23:01:52.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914207+0000", 2024-05-11T23:01:52.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914208+0000", 2024-05-11T23:01:52.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7100000000000002e-07 2024-05-11T23:01:52.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914210+0000", 2024-05-11T23:01:52.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5639999999999999e-06 2024-05-11T23:01:52.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914217+0000", 2024-05-11T23:01:52.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.596e-06 2024-05-11T23:01:52.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914226+0000", 2024-05-11T23:01:52.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0129999999999999e-06 2024-05-11T23:01:52.009 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.009 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.009 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.010 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24187 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.010 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914932+0000", 2024-05-11T23:01:52.010 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001561747, 2024-05-11T23:01:52.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0030669289999999999, 2024-05-11T23:01:52.010 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.010 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.010 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.010 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.010 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.010 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.010 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24187 2024-05-11T23:01:52.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.011 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914932+0000", 2024-05-11T23:01:52.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914932+0000", 2024-05-11T23:01:52.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914933+0000", 2024-05-11T23:01:52.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4299999999999999e-07 2024-05-11T23:01:52.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914934+0000", 2024-05-11T23:01:52.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6789999999999999e-06 2024-05-11T23:01:52.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914940+0000", 2024-05-11T23:01:52.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.677e-06 2024-05-11T23:01:52.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914945+0000", 2024-05-11T23:01:52.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5040000000000004e-06 2024-05-11T23:01:52.013 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.013 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.013 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.013 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23954 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.013 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913374+0000", 2024-05-11T23:01:52.013 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0031201670000000001, 2024-05-11T23:01:52.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0046830200000000004, 2024-05-11T23:01:52.013 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.013 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.013 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.014 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.014 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23954 2024-05-11T23:01:52.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.014 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913374+0000", 2024-05-11T23:01:52.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913374+0000", 2024-05-11T23:01:52.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913374+0000", 2024-05-11T23:01:52.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:01:52.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913385+0000", 2024-05-11T23:01:52.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0712e-05 2024-05-11T23:01:52.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913393+0000", 2024-05-11T23:01:52.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2309999999999996e-06 2024-05-11T23:01:52.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913399+0000", 2024-05-11T23:01:52.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2089999999999996e-06 2024-05-11T23:01:52.016 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.016 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.016 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.016 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24068 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.016 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914234+0000", 2024-05-11T23:01:52.016 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0022604639999999998, 2024-05-11T23:01:52.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038782909999999999, 2024-05-11T23:01:52.016 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.016 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.017 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.017 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.017 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.017 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.017 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24068 2024-05-11T23:01:52.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.017 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914234+0000", 2024-05-11T23:01:52.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914234+0000", 2024-05-11T23:01:52.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914234+0000", 2024-05-11T23:01:52.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1199999999999998e-07 2024-05-11T23:01:52.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914236+0000", 2024-05-11T23:01:52.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3070000000000001e-06 2024-05-11T23:01:52.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914243+0000", 2024-05-11T23:01:52.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3010000000000003e-06 2024-05-11T23:01:52.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914247+0000", 2024-05-11T23:01:52.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7110000000000003e-06 2024-05-11T23:01:52.019 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.019 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.019 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.020 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24190 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.020 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914950+0000", 2024-05-11T23:01:52.020 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001543951, 2024-05-11T23:01:52.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0032158859999999998, 2024-05-11T23:01:52.020 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.020 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.020 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.020 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.020 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.020 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.020 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24190 2024-05-11T23:01:52.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.020 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914950+0000", 2024-05-11T23:01:52.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914950+0000", 2024-05-11T23:01:52.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914950+0000", 2024-05-11T23:01:52.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9400000000000001e-07 2024-05-11T23:01:52.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914952+0000", 2024-05-11T23:01:52.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9609999999999999e-06 2024-05-11T23:01:52.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914959+0000", 2024-05-11T23:01:52.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.178e-06 2024-05-11T23:01:52.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914963+0000", 2024-05-11T23:01:52.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4599999999999996e-06 2024-05-11T23:01:52.022 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.023 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.023 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.023 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23958 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.023 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913409+0000", 2024-05-11T23:01:52.023 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0030853030000000002, 2024-05-11T23:01:52.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048168489999999998, 2024-05-11T23:01:52.023 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.023 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.023 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.023 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.023 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.023 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.023 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23958 2024-05-11T23:01:52.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.024 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913409+0000", 2024-05-11T23:01:52.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913409+0000", 2024-05-11T23:01:52.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913409+0000", 2024-05-11T23:01:52.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7e-07 2024-05-11T23:01:52.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913412+0000", 2024-05-11T23:01:52.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3750000000000001e-06 2024-05-11T23:01:52.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913418+0000", 2024-05-11T23:01:52.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7070000000000003e-06 2024-05-11T23:01:52.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913423+0000", 2024-05-11T23:01:52.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9819999999999999e-06 2024-05-11T23:01:52.026 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.026 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.026 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.026 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24070 3.11 3.23111871 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.026 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914254+0000", 2024-05-11T23:01:52.026 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0022404450000000002, 2024-05-11T23:01:52.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0040313340000000001, 2024-05-11T23:01:52.026 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.026 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.027 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.027 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.027 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.027 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.027 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24070 2024-05-11T23:01:52.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.027 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914254+0000", 2024-05-11T23:01:52.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914254+0000", 2024-05-11T23:01:52.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914254+0000", 2024-05-11T23:01:52.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:01:52.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914262+0000", 2024-05-11T23:01:52.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1850000000000004e-06 2024-05-11T23:01:52.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914269+0000", 2024-05-11T23:01:52.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5359999999999998e-06 2024-05-11T23:01:52.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914274+0000", 2024-05-11T23:01:52.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.028e-06 2024-05-11T23:01:52.029 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.029 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.029 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.029 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24193 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.029 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914971+0000", 2024-05-11T23:01:52.030 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001523211, 2024-05-11T23:01:52.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003369763, 2024-05-11T23:01:52.030 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.030 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.030 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.030 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24193 2024-05-11T23:01:52.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.030 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914971+0000", 2024-05-11T23:01:52.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914971+0000", 2024-05-11T23:01:52.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914971+0000", 2024-05-11T23:01:52.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1100000000000001e-07 2024-05-11T23:01:52.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914982+0000", 2024-05-11T23:01:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0706e-05 2024-05-11T23:01:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914989+0000", 2024-05-11T23:01:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5780000000000004e-06 2024-05-11T23:01:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914994+0000", 2024-05-11T23:01:52.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1359999999999996e-06 2024-05-11T23:01:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23961 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913430+0000", 2024-05-11T23:01:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.003063784, 2024-05-11T23:01:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049664490000000004, 2024-05-11T23:01:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.033 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23961 2024-05-11T23:01:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913430+0000", 2024-05-11T23:01:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913430+0000", 2024-05-11T23:01:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913431+0000", 2024-05-11T23:01:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7300000000000002e-07 2024-05-11T23:01:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913433+0000", 2024-05-11T23:01:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0990000000000001e-06 2024-05-11T23:01:52.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913439+0000", 2024-05-11T23:01:52.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8320000000000002e-06 2024-05-11T23:01:52.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913443+0000", 2024-05-11T23:01:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7859999999999996e-06 2024-05-11T23:01:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24072 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914282+0000", 2024-05-11T23:01:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.002212543, 2024-05-11T23:01:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041773390000000004, 2024-05-11T23:01:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.038 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24072 2024-05-11T23:01:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914282+0000", 2024-05-11T23:01:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914282+0000", 2024-05-11T23:01:52.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914282+0000", 2024-05-11T23:01:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9100000000000004e-07 2024-05-11T23:01:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914284+0000", 2024-05-11T23:01:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3690000000000001e-06 2024-05-11T23:01:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914291+0000", 2024-05-11T23:01:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3369999999999996e-06 2024-05-11T23:01:52.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914296+0000", 2024-05-11T23:01:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9010000000000002e-06 2024-05-11T23:01:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24195 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.915001+0000", 2024-05-11T23:01:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001493222, 2024-05-11T23:01:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035169310000000001, 2024-05-11T23:01:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.041 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24195 2024-05-11T23:01:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915001+0000", 2024-05-11T23:01:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915001+0000", 2024-05-11T23:01:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915001+0000", 2024-05-11T23:01:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:01:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915009+0000", 2024-05-11T23:01:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6399999999999997e-06 2024-05-11T23:01:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915017+0000", 2024-05-11T23:01:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.892e-06 2024-05-11T23:01:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915022+0000", 2024-05-11T23:01:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8040000000000001e-06 2024-05-11T23:01:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.044 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23964 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913450+0000", 2024-05-11T23:01:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0030442400000000001, 2024-05-11T23:01:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005126579, 2024-05-11T23:01:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23964 2024-05-11T23:01:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913450+0000", 2024-05-11T23:01:52.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913450+0000", 2024-05-11T23:01:52.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913450+0000", 2024-05-11T23:01:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6600000000000002e-07 2024-05-11T23:01:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913453+0000", 2024-05-11T23:01:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4930000000000001e-06 2024-05-11T23:01:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913465+0000", 2024-05-11T23:01:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2308e-05 2024-05-11T23:01:52.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913470+0000", 2024-05-11T23:01:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0640000000000001e-06 2024-05-11T23:01:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24076 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914312+0000", 2024-05-11T23:01:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.002182046, 2024-05-11T23:01:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0043182369999999999, 2024-05-11T23:01:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.048 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24076 2024-05-11T23:01:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914312+0000", 2024-05-11T23:01:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914312+0000", 2024-05-11T23:01:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914312+0000", 2024-05-11T23:01:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999999e-07 2024-05-11T23:01:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914318+0000", 2024-05-11T23:01:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4530000000000001e-06 2024-05-11T23:01:52.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914328+0000", 2024-05-11T23:01:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8989999999999996e-06 2024-05-11T23:01:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914333+0000", 2024-05-11T23:01:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.147e-06 2024-05-11T23:01:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24199 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.915028+0000", 2024-05-11T23:01:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0014658290000000001, 2024-05-11T23:01:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.003657815, 2024-05-11T23:01:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24199 2024-05-11T23:01:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915028+0000", 2024-05-11T23:01:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915028+0000", 2024-05-11T23:01:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915029+0000", 2024-05-11T23:01:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9799999999999999e-07 2024-05-11T23:01:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915034+0000", 2024-05-11T23:01:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1889999999999998e-06 2024-05-11T23:01:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915040+0000", 2024-05-11T23:01:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5930000000000002e-06 2024-05-11T23:01:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915046+0000", 2024-05-11T23:01:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2730000000000002e-06 2024-05-11T23:01:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.054 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23967 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913478+0000", 2024-05-11T23:01:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0030161739999999999, 2024-05-11T23:01:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052712899999999997, 2024-05-11T23:01:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23967 2024-05-11T23:01:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.055 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913478+0000", 2024-05-11T23:01:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913478+0000", 2024-05-11T23:01:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913478+0000", 2024-05-11T23:01:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4299999999999998e-07 2024-05-11T23:01:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913481+0000", 2024-05-11T23:01:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3499999999999999e-06 2024-05-11T23:01:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913487+0000", 2024-05-11T23:01:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1809999999999998e-06 2024-05-11T23:01:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913492+0000", 2024-05-11T23:01:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9389999999999996e-06 2024-05-11T23:01:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24080 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914340+0000", 2024-05-11T23:01:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0021536089999999999, 2024-05-11T23:01:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044642620000000001, 2024-05-11T23:01:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.058 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24080 2024-05-11T23:01:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914340+0000", 2024-05-11T23:01:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914340+0000", 2024-05-11T23:01:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914341+0000", 2024-05-11T23:01:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4499999999999997e-07 2024-05-11T23:01:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914343+0000", 2024-05-11T23:01:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1969999999999999e-06 2024-05-11T23:01:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914355+0000", 2024-05-11T23:01:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.222e-05 2024-05-11T23:01:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914361+0000", 2024-05-11T23:01:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2460000000000003e-06 2024-05-11T23:01:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24202 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.061 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.915051+0000", 2024-05-11T23:01:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001442682, 2024-05-11T23:01:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0038096279999999998, 2024-05-11T23:01:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24202 2024-05-11T23:01:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915051+0000", 2024-05-11T23:01:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915051+0000", 2024-05-11T23:01:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915052+0000", 2024-05-11T23:01:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7599999999999997e-07 2024-05-11T23:01:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915054+0000", 2024-05-11T23:01:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7850000000000001e-06 2024-05-11T23:01:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915059+0000", 2024-05-11T23:01:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7939999999999999e-06 2024-05-11T23:01:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915064+0000", 2024-05-11T23:01:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4599999999999996e-06 2024-05-11T23:01:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23970 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913499+0000", 2024-05-11T23:01:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0029950860000000001, 2024-05-11T23:01:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054227119999999997, 2024-05-11T23:01:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23970 2024-05-11T23:01:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913499+0000", 2024-05-11T23:01:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913499+0000", 2024-05-11T23:01:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913499+0000", 2024-05-11T23:01:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3300000000000003e-07 2024-05-11T23:01:52.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913506+0000", 2024-05-11T23:01:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7889999999999997e-06 2024-05-11T23:01:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913513+0000", 2024-05-11T23:01:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8739999999999997e-06 2024-05-11T23:01:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913518+0000", 2024-05-11T23:01:52.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8600000000000001e-06 2024-05-11T23:01:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24084 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914368+0000", 2024-05-11T23:01:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0021262540000000002, 2024-05-11T23:01:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004612494, 2024-05-11T23:01:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.068 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24084 2024-05-11T23:01:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914368+0000", 2024-05-11T23:01:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914368+0000", 2024-05-11T23:01:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914368+0000", 2024-05-11T23:01:52.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.84e-07 2024-05-11T23:01:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914370+0000", 2024-05-11T23:01:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.165e-06 2024-05-11T23:01:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914379+0000", 2024-05-11T23:01:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1769999999999998e-06 2024-05-11T23:01:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914383+0000", 2024-05-11T23:01:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6179999999999997e-06 2024-05-11T23:01:52.070 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24205 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.915069+0000", 2024-05-11T23:01:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001424862, 2024-05-11T23:01:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0039654019999999998, 2024-05-11T23:01:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.071 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24205 2024-05-11T23:01:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915069+0000", 2024-05-11T23:01:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915069+0000", 2024-05-11T23:01:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915070+0000", 2024-05-11T23:01:52.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2399999999999999e-07 2024-05-11T23:01:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915071+0000", 2024-05-11T23:01:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5579999999999999e-06 2024-05-11T23:01:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915077+0000", 2024-05-11T23:01:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5090000000000001e-06 2024-05-11T23:01:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915081+0000", 2024-05-11T23:01:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3209999999999998e-06 2024-05-11T23:01:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.073 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23973 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913546+0000", 2024-05-11T23:01:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0029483729999999998, 2024-05-11T23:01:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0055524659999999998, 2024-05-11T23:01:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23973 2024-05-11T23:01:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913546+0000", 2024-05-11T23:01:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913546+0000", 2024-05-11T23:01:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913546+0000", 2024-05-11T23:01:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8299999999999997e-07 2024-05-11T23:01:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913549+0000", 2024-05-11T23:01:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6570000000000001e-06 2024-05-11T23:01:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913556+0000", 2024-05-11T23:01:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3379999999999999e-06 2024-05-11T23:01:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913561+0000", 2024-05-11T23:01:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2700000000000004e-06 2024-05-11T23:01:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24088 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914389+0000", 2024-05-11T23:01:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0021053199999999999, 2024-05-11T23:01:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047637720000000003, 2024-05-11T23:01:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24088 2024-05-11T23:01:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914389+0000", 2024-05-11T23:01:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914389+0000", 2024-05-11T23:01:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914389+0000", 2024-05-11T23:01:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2099999999999998e-07 2024-05-11T23:01:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914391+0000", 2024-05-11T23:01:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.79e-06 2024-05-11T23:01:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914397+0000", 2024-05-11T23:01:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7509999999999997e-06 2024-05-11T23:01:52.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914401+0000", 2024-05-11T23:01:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5090000000000004e-06 2024-05-11T23:01:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24208 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.915086+0000", 2024-05-11T23:01:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00140766, 2024-05-11T23:01:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041229910000000003, 2024-05-11T23:01:52.080 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24208 2024-05-11T23:01:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915086+0000", 2024-05-11T23:01:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915086+0000", 2024-05-11T23:01:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915087+0000", 2024-05-11T23:01:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.72e-07 2024-05-11T23:01:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915089+0000", 2024-05-11T23:01:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.889e-06 2024-05-11T23:01:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915094+0000", 2024-05-11T23:01:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6980000000000003e-06 2024-05-11T23:01:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915099+0000", 2024-05-11T23:01:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8119999999999998e-06 2024-05-11T23:01:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23976 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913576+0000", 2024-05-11T23:01:52.083 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0029176240000000002, 2024-05-11T23:01:52.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0056972460000000004, 2024-05-11T23:01:52.084 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.084 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.084 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.084 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23976 2024-05-11T23:01:52.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.084 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913576+0000", 2024-05-11T23:01:52.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913576+0000", 2024-05-11T23:01:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913577+0000", 2024-05-11T23:01:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:01:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913579+0000", 2024-05-11T23:01:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5720000000000001e-06 2024-05-11T23:01:52.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913587+0000", 2024-05-11T23:01:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3560000000000004e-06 2024-05-11T23:01:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913592+0000", 2024-05-11T23:01:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5319999999999997e-06 2024-05-11T23:01:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.086 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24091 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914406+0000", 2024-05-11T23:01:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0020877589999999998, 2024-05-11T23:01:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049254800000000003, 2024-05-11T23:01:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24091 2024-05-11T23:01:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914406+0000", 2024-05-11T23:01:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914406+0000", 2024-05-11T23:01:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914407+0000", 2024-05-11T23:01:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2800000000000003e-07 2024-05-11T23:01:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914408+0000", 2024-05-11T23:01:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6899999999999999e-06 2024-05-11T23:01:52.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914414+0000", 2024-05-11T23:01:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6960000000000002e-06 2024-05-11T23:01:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914419+0000", 2024-05-11T23:01:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5079999999999999e-06 2024-05-11T23:01:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.089 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24211 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.915105+0000", 2024-05-11T23:01:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001388673, 2024-05-11T23:01:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0042805289999999999, 2024-05-11T23:01:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24211 2024-05-11T23:01:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915105+0000", 2024-05-11T23:01:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915105+0000", 2024-05-11T23:01:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915106+0000", 2024-05-11T23:01:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4299999999999998e-07 2024-05-11T23:01:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915108+0000", 2024-05-11T23:01:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0229999999999999e-06 2024-05-11T23:01:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915114+0000", 2024-05-11T23:01:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9100000000000002e-06 2024-05-11T23:01:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915119+0000", 2024-05-11T23:01:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7840000000000003e-06 2024-05-11T23:01:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.092 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23979 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913600+0000", 2024-05-11T23:01:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.002894401, 2024-05-11T23:01:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0058470479999999997, 2024-05-11T23:01:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23979 2024-05-11T23:01:52.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913600+0000", 2024-05-11T23:01:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913600+0000", 2024-05-11T23:01:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913600+0000", 2024-05-11T23:01:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:01:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913602+0000", 2024-05-11T23:01:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1129999999999999e-06 2024-05-11T23:01:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913608+0000", 2024-05-11T23:01:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.302e-06 2024-05-11T23:01:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913613+0000", 2024-05-11T23:01:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7779999999999998e-06 2024-05-11T23:01:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24095 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914424+0000", 2024-05-11T23:01:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0020698169999999998, 2024-05-11T23:01:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0050783139999999996, 2024-05-11T23:01:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.096 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24095 2024-05-11T23:01:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914424+0000", 2024-05-11T23:01:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914424+0000", 2024-05-11T23:01:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914425+0000", 2024-05-11T23:01:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2500000000000001e-07 2024-05-11T23:01:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914426+0000", 2024-05-11T23:01:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.637e-06 2024-05-11T23:01:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914432+0000", 2024-05-11T23:01:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4149999999999998e-06 2024-05-11T23:01:52.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914439+0000", 2024-05-11T23:01:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9550000000000003e-06 2024-05-11T23:01:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24214 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.915125+0000", 2024-05-11T23:01:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0013686130000000001, 2024-05-11T23:01:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044303789999999999, 2024-05-11T23:01:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.099 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24214 2024-05-11T23:01:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915125+0000", 2024-05-11T23:01:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915125+0000", 2024-05-11T23:01:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915126+0000", 2024-05-11T23:01:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:01:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915128+0000", 2024-05-11T23:01:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1689999999999999e-06 2024-05-11T23:01:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915134+0000", 2024-05-11T23:01:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4300000000000003e-06 2024-05-11T23:01:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915139+0000", 2024-05-11T23:01:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8919999999999999e-06 2024-05-11T23:01:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23982 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913621+0000", 2024-05-11T23:01:52.102 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0028730880000000002, 2024-05-11T23:01:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059949219999999998, 2024-05-11T23:01:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23982 2024-05-11T23:01:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913621+0000", 2024-05-11T23:01:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913621+0000", 2024-05-11T23:01:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913621+0000", 2024-05-11T23:01:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.27e-07 2024-05-11T23:01:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913624+0000", 2024-05-11T23:01:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.283e-06 2024-05-11T23:01:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913630+0000", 2024-05-11T23:01:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3949999999999998e-06 2024-05-11T23:01:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913635+0000", 2024-05-11T23:01:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1680000000000003e-06 2024-05-11T23:01:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24098 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.105 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914446+0000", 2024-05-11T23:01:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0020477400000000002, 2024-05-11T23:01:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052224280000000003, 2024-05-11T23:01:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24098 2024-05-11T23:01:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914446+0000", 2024-05-11T23:01:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914446+0000", 2024-05-11T23:01:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914447+0000", 2024-05-11T23:01:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7899999999999999e-07 2024-05-11T23:01:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914455+0000", 2024-05-11T23:01:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8390000000000007e-06 2024-05-11T23:01:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914461+0000", 2024-05-11T23:01:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8539999999999999e-06 2024-05-11T23:01:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914466+0000", 2024-05-11T23:01:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9699999999999998e-06 2024-05-11T23:01:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.108 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.109 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24216 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.109 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.915146+0000", 2024-05-11T23:01:52.109 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00134764, 2024-05-11T23:01:52.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0045796439999999999, 2024-05-11T23:01:52.109 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.109 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.109 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.109 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.109 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.109 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.109 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24216 2024-05-11T23:01:52.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.109 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915146+0000", 2024-05-11T23:01:52.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915146+0000", 2024-05-11T23:01:52.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915147+0000", 2024-05-11T23:01:52.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5999999999999999e-07 2024-05-11T23:01:52.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915149+0000", 2024-05-11T23:01:52.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9999999999999999e-06 2024-05-11T23:01:52.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915156+0000", 2024-05-11T23:01:52.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9280000000000004e-06 2024-05-11T23:01:52.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915175+0000", 2024-05-11T23:01:52.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9279999999999998e-05 2024-05-11T23:01:52.112 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.112 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.112 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.112 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23985 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.112 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913643+0000", 2024-05-11T23:01:52.112 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0028506880000000001, 2024-05-11T23:01:52.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061362780000000002, 2024-05-11T23:01:52.112 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.112 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.112 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.112 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.112 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.112 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.113 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23985 2024-05-11T23:01:52.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.113 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913643+0000", 2024-05-11T23:01:52.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913643+0000", 2024-05-11T23:01:52.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913644+0000", 2024-05-11T23:01:52.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.15e-07 2024-05-11T23:01:52.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913646+0000", 2024-05-11T23:01:52.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2400000000000002e-06 2024-05-11T23:01:52.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913652+0000", 2024-05-11T23:01:52.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1659999999999999e-06 2024-05-11T23:01:52.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913657+0000", 2024-05-11T23:01:52.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7380000000000002e-06 2024-05-11T23:01:52.115 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.115 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.115 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.115 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24099 3.11 3.23111871 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.115 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914473+0000", 2024-05-11T23:01:52.115 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0020207860000000001, 2024-05-11T23:01:52.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053651360000000004, 2024-05-11T23:01:52.115 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.115 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.115 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.115 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.115 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.116 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.116 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24099 2024-05-11T23:01:52.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.116 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914473+0000", 2024-05-11T23:01:52.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914473+0000", 2024-05-11T23:01:52.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914474+0000", 2024-05-11T23:01:52.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:01:52.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914476+0000", 2024-05-11T23:01:52.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1239999999999999e-06 2024-05-11T23:01:52.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914482+0000", 2024-05-11T23:01:52.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2519999999999996e-06 2024-05-11T23:01:52.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914489+0000", 2024-05-11T23:01:52.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5579999999999997e-06 2024-05-11T23:01:52.118 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.118 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.118 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.118 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24219 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.118 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.915189+0000", 2024-05-11T23:01:52.118 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001305149, 2024-05-11T23:01:52.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047035289999999997, 2024-05-11T23:01:52.118 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.118 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.118 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.119 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.119 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.119 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.119 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24219 2024-05-11T23:01:52.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.119 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915189+0000", 2024-05-11T23:01:52.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915189+0000", 2024-05-11T23:01:52.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915189+0000", 2024-05-11T23:01:52.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4799999999999998e-07 2024-05-11T23:01:52.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915192+0000", 2024-05-11T23:01:52.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7590000000000002e-06 2024-05-11T23:01:52.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915199+0000", 2024-05-11T23:01:52.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7190000000000004e-06 2024-05-11T23:01:52.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915205+0000", 2024-05-11T23:01:52.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9340000000000003e-06 2024-05-11T23:01:52.121 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.121 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.121 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.121 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23989 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.121 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913663+0000", 2024-05-11T23:01:52.121 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0028306780000000001, 2024-05-11T23:01:52.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062854859999999999, 2024-05-11T23:01:52.121 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.122 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.122 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.122 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.122 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.122 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.122 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23989 2024-05-11T23:01:52.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.122 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913663+0000", 2024-05-11T23:01:52.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913663+0000", 2024-05-11T23:01:52.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913664+0000", 2024-05-11T23:01:52.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:01:52.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913671+0000", 2024-05-11T23:01:52.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5279999999999998e-06 2024-05-11T23:01:52.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913678+0000", 2024-05-11T23:01:52.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8940000000000004e-06 2024-05-11T23:01:52.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913683+0000", 2024-05-11T23:01:52.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0379999999999999e-06 2024-05-11T23:01:52.124 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.124 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.124 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.124 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24102 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.124 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914495+0000", 2024-05-11T23:01:52.125 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001998863, 2024-05-11T23:01:52.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0055068870000000002, 2024-05-11T23:01:52.125 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.125 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.125 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.125 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24102 2024-05-11T23:01:52.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.125 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914495+0000", 2024-05-11T23:01:52.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914495+0000", 2024-05-11T23:01:52.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914496+0000", 2024-05-11T23:01:52.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-05-11T23:01:52.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914498+0000", 2024-05-11T23:01:52.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0889999999999998e-06 2024-05-11T23:01:52.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914504+0000", 2024-05-11T23:01:52.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8939999999999998e-06 2024-05-11T23:01:52.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914508+0000", 2024-05-11T23:01:52.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.797e-06 2024-05-11T23:01:52.127 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.127 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.127 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.128 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24222 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.128 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.915212+0000", 2024-05-11T23:01:52.128 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0012815979999999999, 2024-05-11T23:01:52.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0048459920000000004, 2024-05-11T23:01:52.128 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.128 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.128 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.128 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.128 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.128 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.128 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24222 2024-05-11T23:01:52.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.128 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915212+0000", 2024-05-11T23:01:52.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915212+0000", 2024-05-11T23:01:52.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915213+0000", 2024-05-11T23:01:52.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000002e-07 2024-05-11T23:01:52.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915215+0000", 2024-05-11T23:01:52.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2390000000000001e-06 2024-05-11T23:01:52.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915225+0000", 2024-05-11T23:01:52.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0134e-05 2024-05-11T23:01:52.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.915230+0000", 2024-05-11T23:01:52.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9960000000000001e-06 2024-05-11T23:01:52.130 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.130 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.130 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.131 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23883 3.11 3:89f10fe9:::rbd_data.111e4e122501.0000000000000008:f [read 348160~4096 out=4096b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.131 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.912887+0000", 2024-05-11T23:01:52.131 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.003607074, 2024-05-11T23:01:52.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.007233939, 2024-05-11T23:01:52.131 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.131 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.131 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:52.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.131 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23883 2024-05-11T23:01:52.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.131 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.912887+0000", 2024-05-11T23:01:52.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.912887+0000", 2024-05-11T23:01:52.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.912887+0000", 2024-05-11T23:01:52.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:01:52.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.912889+0000", 2024-05-11T23:01:52.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9740000000000001e-06 2024-05-11T23:01:52.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.912895+0000", 2024-05-11T23:01:52.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9229999999999999e-06 2024-05-11T23:01:52.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.912900+0000", 2024-05-11T23:01:52.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.651e-06 2024-05-11T23:01:52.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916403+0000", 2024-05-11T23:01:52.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0035033880000000001 2024-05-11T23:01:52.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916478+0000", 2024-05-11T23:01:52.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5209000000000004e-05 2024-05-11T23:01:52.134 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.134 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.134 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.135 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23991 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.135 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913690+0000", 2024-05-11T23:01:52.135 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0028042589999999999, 2024-05-11T23:01:52.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0064983810000000001, 2024-05-11T23:01:52.135 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.135 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.135 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.135 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.135 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.135 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.135 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23991 2024-05-11T23:01:52.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.135 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913690+0000", 2024-05-11T23:01:52.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913690+0000", 2024-05-11T23:01:52.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913690+0000", 2024-05-11T23:01:52.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.01e-07 2024-05-11T23:01:52.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913692+0000", 2024-05-11T23:01:52.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0999999999999998e-06 2024-05-11T23:01:52.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913708+0000", 2024-05-11T23:01:52.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5319e-05 2024-05-11T23:01:52.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913713+0000", 2024-05-11T23:01:52.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4199999999999998e-06 2024-05-11T23:01:52.137 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.137 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.137 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.138 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24106 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.138 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914515+0000", 2024-05-11T23:01:52.138 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0019794700000000001, 2024-05-11T23:01:52.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0057308239999999998, 2024-05-11T23:01:52.138 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.138 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.138 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.138 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.138 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.138 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.138 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24106 2024-05-11T23:01:52.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.138 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914515+0000", 2024-05-11T23:01:52.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914515+0000", 2024-05-11T23:01:52.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914515+0000", 2024-05-11T23:01:52.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.58e-07 2024-05-11T23:01:52.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914522+0000", 2024-05-11T23:01:52.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4590000000000001e-06 2024-05-11T23:01:52.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914529+0000", 2024-05-11T23:01:52.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3260000000000001e-06 2024-05-11T23:01:52.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914534+0000", 2024-05-11T23:01:52.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9459999999999997e-06 2024-05-11T23:01:52.140 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.141 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.141 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.141 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24225 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.141 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.916144+0000", 2024-05-11T23:01:52.141 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00034985700000000002, 2024-05-11T23:01:52.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0041577009999999998, 2024-05-11T23:01:52.141 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.141 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.141 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.141 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24225 2024-05-11T23:01:52.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.142 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916144+0000", 2024-05-11T23:01:52.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916144+0000", 2024-05-11T23:01:52.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916145+0000", 2024-05-11T23:01:52.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3300000000000002e-07 2024-05-11T23:01:52.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916149+0000", 2024-05-11T23:01:52.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6940000000000001e-06 2024-05-11T23:01:52.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916160+0000", 2024-05-11T23:01:52.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1741e-05 2024-05-11T23:01:52.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916168+0000", 2024-05-11T23:01:52.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.418e-06 2024-05-11T23:01:52.144 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.144 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.144 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.144 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23993 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.144 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913728+0000", 2024-05-11T23:01:52.144 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0027655589999999999, 2024-05-11T23:01:52.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0066295199999999999, 2024-05-11T23:01:52.144 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.144 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.144 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.144 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.144 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.144 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.145 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23993 2024-05-11T23:01:52.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.145 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913728+0000", 2024-05-11T23:01:52.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913728+0000", 2024-05-11T23:01:52.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913729+0000", 2024-05-11T23:01:52.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5199999999999997e-07 2024-05-11T23:01:52.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913732+0000", 2024-05-11T23:01:52.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7180000000000001e-06 2024-05-11T23:01:52.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913739+0000", 2024-05-11T23:01:52.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2230000000000001e-06 2024-05-11T23:01:52.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913744+0000", 2024-05-11T23:01:52.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0130000000000001e-06 2024-05-11T23:01:52.147 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.147 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.147 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.147 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24110 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.147 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914540+0000", 2024-05-11T23:01:52.147 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001954332, 2024-05-11T23:01:52.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0059416620000000003, 2024-05-11T23:01:52.147 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.147 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.147 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.147 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.148 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.148 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.148 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24110 2024-05-11T23:01:52.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.148 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914540+0000", 2024-05-11T23:01:52.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914540+0000", 2024-05-11T23:01:52.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914540+0000", 2024-05-11T23:01:52.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2899999999999999e-07 2024-05-11T23:01:52.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914542+0000", 2024-05-11T23:01:52.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8950000000000001e-06 2024-05-11T23:01:52.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914548+0000", 2024-05-11T23:01:52.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0070000000000003e-06 2024-05-11T23:01:52.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914552+0000", 2024-05-11T23:01:52.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.408e-06 2024-05-11T23:01:52.150 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.150 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.150 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.150 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24228 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.150 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.916187+0000", 2024-05-11T23:01:52.150 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00030740200000000002, 2024-05-11T23:01:52.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0043480960000000001, 2024-05-11T23:01:52.150 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.150 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.151 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.151 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.151 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.151 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.151 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24228 2024-05-11T23:01:52.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.151 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916187+0000", 2024-05-11T23:01:52.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916187+0000", 2024-05-11T23:01:52.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916187+0000", 2024-05-11T23:01:52.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9699999999999996e-07 2024-05-11T23:01:52.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916196+0000", 2024-05-11T23:01:52.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1689999999999998e-06 2024-05-11T23:01:52.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916205+0000", 2024-05-11T23:01:52.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4169999999999992e-06 2024-05-11T23:01:52.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916212+0000", 2024-05-11T23:01:52.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8719999999999996e-06 2024-05-11T23:01:52.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.153 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.153 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:23996 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.153 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913751+0000", 2024-05-11T23:01:52.153 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0027430100000000002, 2024-05-11T23:01:52.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0068415669999999998, 2024-05-11T23:01:52.153 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.154 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.154 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.154 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.154 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.154 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.154 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 23996 2024-05-11T23:01:52.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.154 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913751+0000", 2024-05-11T23:01:52.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913751+0000", 2024-05-11T23:01:52.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913752+0000", 2024-05-11T23:01:52.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8500000000000002e-07 2024-05-11T23:01:52.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913754+0000", 2024-05-11T23:01:52.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1409999999999999e-06 2024-05-11T23:01:52.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913760+0000", 2024-05-11T23:01:52.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4799999999999998e-06 2024-05-11T23:01:52.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913765+0000", 2024-05-11T23:01:52.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6739999999999996e-06 2024-05-11T23:01:52.156 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.156 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.156 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.156 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24116 3.11 3.23111871 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.156 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914560+0000", 2024-05-11T23:01:52.156 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0019335929999999999, 2024-05-11T23:01:52.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0060861190000000001, 2024-05-11T23:01:52.157 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.157 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.157 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.157 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.157 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.157 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.157 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24116 2024-05-11T23:01:52.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.157 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914560+0000", 2024-05-11T23:01:52.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914560+0000", 2024-05-11T23:01:52.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914561+0000", 2024-05-11T23:01:52.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2300000000000002e-07 2024-05-11T23:01:52.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914563+0000", 2024-05-11T23:01:52.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8589999999999999e-06 2024-05-11T23:01:52.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914568+0000", 2024-05-11T23:01:52.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7289999999999997e-06 2024-05-11T23:01:52.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914573+0000", 2024-05-11T23:01:52.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4909999999999999e-06 2024-05-11T23:01:52.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.159 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.159 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24231 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.159 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.916524+0000", 2024-05-11T23:01:52.160 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9999704, 2024-05-11T23:01:52.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.004179978, 2024-05-11T23:01:52.160 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.160 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.160 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.160 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.160 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.160 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.160 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24231 2024-05-11T23:01:52.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.160 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916524+0000", 2024-05-11T23:01:52.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916524+0000", 2024-05-11T23:01:52.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916524+0000", 2024-05-11T23:01:52.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4499999999999996e-07 2024-05-11T23:01:52.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916528+0000", 2024-05-11T23:01:52.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7170000000000002e-06 2024-05-11T23:01:52.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916539+0000", 2024-05-11T23:01:52.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.08e-05 2024-05-11T23:01:52.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916546+0000", 2024-05-11T23:01:52.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8329999999999996e-06 2024-05-11T23:01:52.162 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.162 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.162 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.163 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24000 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.163 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913771+0000", 2024-05-11T23:01:52.163 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0027231439999999998, 2024-05-11T23:01:52.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0069853789999999999, 2024-05-11T23:01:52.163 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.163 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.163 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.163 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.163 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.163 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.163 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24000 2024-05-11T23:01:52.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.163 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913771+0000", 2024-05-11T23:01:52.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913771+0000", 2024-05-11T23:01:52.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913771+0000", 2024-05-11T23:01:52.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6600000000000002e-07 2024-05-11T23:01:52.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913773+0000", 2024-05-11T23:01:52.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:01:52.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913779+0000", 2024-05-11T23:01:52.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8610000000000003e-06 2024-05-11T23:01:52.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913784+0000", 2024-05-11T23:01:52.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8890000000000001e-06 2024-05-11T23:01:52.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.165 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.166 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24122 3.11 3.23111871 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.166 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914580+0000", 2024-05-11T23:01:52.166 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001914357, 2024-05-11T23:01:52.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0062293080000000002, 2024-05-11T23:01:52.166 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.166 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.166 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.166 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24122 2024-05-11T23:01:52.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.166 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914580+0000", 2024-05-11T23:01:52.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914580+0000", 2024-05-11T23:01:52.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914580+0000", 2024-05-11T23:01:52.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5499999999999999e-07 2024-05-11T23:01:52.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914582+0000", 2024-05-11T23:01:52.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.694e-06 2024-05-11T23:01:52.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914587+0000", 2024-05-11T23:01:52.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.665e-06 2024-05-11T23:01:52.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914592+0000", 2024-05-11T23:01:52.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8199999999999996e-06 2024-05-11T23:01:52.168 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.168 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.169 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.169 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24234 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.169 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.916555+0000", 2024-05-11T23:01:52.169 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9999385, 2024-05-11T23:01:52.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0043102640000000003, 2024-05-11T23:01:52.169 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.169 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.169 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.169 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.169 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.169 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.169 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24234 2024-05-11T23:01:52.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.169 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916555+0000", 2024-05-11T23:01:52.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916555+0000", 2024-05-11T23:01:52.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916556+0000", 2024-05-11T23:01:52.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3700000000000001e-07 2024-05-11T23:01:52.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916558+0000", 2024-05-11T23:01:52.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1610000000000001e-06 2024-05-11T23:01:52.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916565+0000", 2024-05-11T23:01:52.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5780000000000004e-06 2024-05-11T23:01:52.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916570+0000", 2024-05-11T23:01:52.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0189999999999997e-06 2024-05-11T23:01:52.172 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.172 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.172 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.172 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24003 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.172 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913794+0000", 2024-05-11T23:01:52.172 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0026996749999999999, 2024-05-11T23:01:52.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0071277470000000002, 2024-05-11T23:01:52.172 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.172 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.172 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.172 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.172 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.172 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.173 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24003 2024-05-11T23:01:52.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.173 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913794+0000", 2024-05-11T23:01:52.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913794+0000", 2024-05-11T23:01:52.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913795+0000", 2024-05-11T23:01:52.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:01:52.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913797+0000", 2024-05-11T23:01:52.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9589999999999998e-06 2024-05-11T23:01:52.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913803+0000", 2024-05-11T23:01:52.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8719999999999999e-06 2024-05-11T23:01:52.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913807+0000", 2024-05-11T23:01:52.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4889999999999997e-06 2024-05-11T23:01:52.175 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.175 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.175 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.175 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24128 3.11 3.23111871 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.175 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914598+0000", 2024-05-11T23:01:52.175 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001896095, 2024-05-11T23:01:52.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063804029999999998, 2024-05-11T23:01:52.175 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.175 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.176 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.176 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.176 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.176 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.176 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24128 2024-05-11T23:01:52.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.176 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914598+0000", 2024-05-11T23:01:52.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914598+0000", 2024-05-11T23:01:52.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914598+0000", 2024-05-11T23:01:52.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:01:52.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914600+0000", 2024-05-11T23:01:52.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9589999999999998e-06 2024-05-11T23:01:52.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914607+0000", 2024-05-11T23:01:52.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2489999999999998e-06 2024-05-11T23:01:52.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914611+0000", 2024-05-11T23:01:52.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5959999999999997e-06 2024-05-11T23:01:52.178 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.178 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.178 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.178 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24237 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.178 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.916576+0000", 2024-05-11T23:01:52.178 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.999918, 2024-05-11T23:01:52.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0044560570000000002, 2024-05-11T23:01:52.179 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.179 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.179 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.179 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24237 2024-05-11T23:01:52.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.179 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916576+0000", 2024-05-11T23:01:52.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916576+0000", 2024-05-11T23:01:52.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916576+0000", 2024-05-11T23:01:52.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:01:52.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916578+0000", 2024-05-11T23:01:52.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.888e-06 2024-05-11T23:01:52.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916584+0000", 2024-05-11T23:01:52.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7930000000000003e-06 2024-05-11T23:01:52.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916589+0000", 2024-05-11T23:01:52.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8250000000000004e-06 2024-05-11T23:01:52.181 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.181 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.181 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.181 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24006 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.182 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913812+0000", 2024-05-11T23:01:52.182 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.002681559, 2024-05-11T23:01:52.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.007277403, 2024-05-11T23:01:52.182 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.182 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.182 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.182 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.182 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.182 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.182 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24006 2024-05-11T23:01:52.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.182 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913812+0000", 2024-05-11T23:01:52.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913812+0000", 2024-05-11T23:01:52.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913813+0000", 2024-05-11T23:01:52.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3200000000000001e-07 2024-05-11T23:01:52.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913820+0000", 2024-05-11T23:01:52.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.323e-06 2024-05-11T23:01:52.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913826+0000", 2024-05-11T23:01:52.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1190000000000002e-06 2024-05-11T23:01:52.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913831+0000", 2024-05-11T23:01:52.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5650000000000003e-06 2024-05-11T23:01:52.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.184 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.185 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24134 3.11 3.23111871 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.185 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914617+0000", 2024-05-11T23:01:52.185 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001876752, 2024-05-11T23:01:52.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0065248700000000003, 2024-05-11T23:01:52.185 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.185 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.185 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.185 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.185 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24134 2024-05-11T23:01:52.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.185 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914617+0000", 2024-05-11T23:01:52.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914617+0000", 2024-05-11T23:01:52.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914618+0000", 2024-05-11T23:01:52.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:01:52.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914620+0000", 2024-05-11T23:01:52.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:01:52.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914626+0000", 2024-05-11T23:01:52.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2400000000000004e-06 2024-05-11T23:01:52.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914631+0000", 2024-05-11T23:01:52.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7859999999999996e-06 2024-05-11T23:01:52.187 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.188 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.188 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.188 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24240 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.188 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.916598+0000", 2024-05-11T23:01:52.188 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9998965, 2024-05-11T23:01:52.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0046031759999999996, 2024-05-11T23:01:52.188 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.188 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.188 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.188 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.188 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.188 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.188 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24240 2024-05-11T23:01:52.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.189 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916598+0000", 2024-05-11T23:01:52.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916598+0000", 2024-05-11T23:01:52.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916598+0000", 2024-05-11T23:01:52.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:01:52.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916600+0000", 2024-05-11T23:01:52.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0209999999999998e-06 2024-05-11T23:01:52.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916606+0000", 2024-05-11T23:01:52.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8069999999999996e-06 2024-05-11T23:01:52.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916611+0000", 2024-05-11T23:01:52.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8099999999999997e-06 2024-05-11T23:01:52.191 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.191 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.191 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.191 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24011 3.11 3.23111871 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.191 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913837+0000", 2024-05-11T23:01:52.191 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0026574960000000001, 2024-05-11T23:01:52.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0074223869999999999, 2024-05-11T23:01:52.191 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.191 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.191 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.192 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.192 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24011 2024-05-11T23:01:52.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.192 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913837+0000", 2024-05-11T23:01:52.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913837+0000", 2024-05-11T23:01:52.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913837+0000", 2024-05-11T23:01:52.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.46e-07 2024-05-11T23:01:52.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913839+0000", 2024-05-11T23:01:52.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.874e-06 2024-05-11T23:01:52.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913845+0000", 2024-05-11T23:01:52.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:01:52.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913849+0000", 2024-05-11T23:01:52.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.42e-06 2024-05-11T23:01:52.194 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.194 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.194 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.194 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24140 3.11 3.23111871 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.194 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914637+0000", 2024-05-11T23:01:52.194 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0018565529999999999, 2024-05-11T23:01:52.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0066769899999999998, 2024-05-11T23:01:52.194 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.194 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.194 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.195 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.195 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.195 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.195 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24140 2024-05-11T23:01:52.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.195 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914637+0000", 2024-05-11T23:01:52.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914637+0000", 2024-05-11T23:01:52.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914638+0000", 2024-05-11T23:01:52.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.65e-07 2024-05-11T23:01:52.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914640+0000", 2024-05-11T23:01:52.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1289999999999999e-06 2024-05-11T23:01:52.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914646+0000", 2024-05-11T23:01:52.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.144e-06 2024-05-11T23:01:52.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914652+0000", 2024-05-11T23:01:52.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5899999999999998e-06 2024-05-11T23:01:52.197 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.197 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.197 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.197 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24243 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.197 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.916617+0000", 2024-05-11T23:01:52.197 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9998775, 2024-05-11T23:01:52.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0047529850000000004, 2024-05-11T23:01:52.197 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.197 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.198 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.198 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.198 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.198 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.198 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24243 2024-05-11T23:01:52.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.198 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916617+0000", 2024-05-11T23:01:52.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916617+0000", 2024-05-11T23:01:52.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916617+0000", 2024-05-11T23:01:52.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8100000000000003e-07 2024-05-11T23:01:52.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916619+0000", 2024-05-11T23:01:52.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8670000000000001e-06 2024-05-11T23:01:52.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916625+0000", 2024-05-11T23:01:52.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9479999999999996e-06 2024-05-11T23:01:52.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916635+0000", 2024-05-11T23:01:52.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0040999999999999e-05 2024-05-11T23:01:52.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.200 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.200 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24016 3.11 3.23111871 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.200 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913866+0000", 2024-05-11T23:01:52.200 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0026284250000000002, 2024-05-11T23:01:52.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0075613549999999996, 2024-05-11T23:01:52.201 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.201 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.201 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.201 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24016 2024-05-11T23:01:52.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.201 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913866+0000", 2024-05-11T23:01:52.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913866+0000", 2024-05-11T23:01:52.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913866+0000", 2024-05-11T23:01:52.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1600000000000001e-07 2024-05-11T23:01:52.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913869+0000", 2024-05-11T23:01:52.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.464e-06 2024-05-11T23:01:52.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913876+0000", 2024-05-11T23:01:52.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.199e-06 2024-05-11T23:01:52.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913881+0000", 2024-05-11T23:01:52.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0540000000000002e-06 2024-05-11T23:01:52.203 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.203 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.203 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.203 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24146 3.11 3.23111871 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.203 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914658+0000", 2024-05-11T23:01:52.203 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001836395, 2024-05-11T23:01:52.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0068661679999999998, 2024-05-11T23:01:52.204 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.204 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.204 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.204 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.204 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.204 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.204 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24146 2024-05-11T23:01:52.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.204 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914658+0000", 2024-05-11T23:01:52.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914658+0000", 2024-05-11T23:01:52.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914658+0000", 2024-05-11T23:01:52.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:01:52.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914660+0000", 2024-05-11T23:01:52.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0080000000000001e-06 2024-05-11T23:01:52.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914666+0000", 2024-05-11T23:01:52.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1090000000000003e-06 2024-05-11T23:01:52.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914671+0000", 2024-05-11T23:01:52.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.831e-06 2024-05-11T23:01:52.206 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.206 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.206 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.207 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24246 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.207 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.916641+0000", 2024-05-11T23:01:52.207 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9998531, 2024-05-11T23:01:52.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0049824240000000001, 2024-05-11T23:01:52.207 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.207 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.207 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.207 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.207 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.207 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.207 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24246 2024-05-11T23:01:52.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.207 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916641+0000", 2024-05-11T23:01:52.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916641+0000", 2024-05-11T23:01:52.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916641+0000", 2024-05-11T23:01:52.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-05-11T23:01:52.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916643+0000", 2024-05-11T23:01:52.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8479999999999999e-06 2024-05-11T23:01:52.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916649+0000", 2024-05-11T23:01:52.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1149999999999999e-06 2024-05-11T23:01:52.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916655+0000", 2024-05-11T23:01:52.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4870000000000002e-06 2024-05-11T23:01:52.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.209 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.210 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24022 3.11 3.23111871 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.210 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913888+0000", 2024-05-11T23:01:52.210 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.00260589, 2024-05-11T23:01:52.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0077921659999999997, 2024-05-11T23:01:52.210 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.210 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.210 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.210 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24022 2024-05-11T23:01:52.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.210 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913888+0000", 2024-05-11T23:01:52.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913888+0000", 2024-05-11T23:01:52.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913889+0000", 2024-05-11T23:01:52.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4000000000000002e-07 2024-05-11T23:01:52.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913891+0000", 2024-05-11T23:01:52.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.407e-06 2024-05-11T23:01:52.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913898+0000", 2024-05-11T23:01:52.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5119999999999997e-06 2024-05-11T23:01:52.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913903+0000", 2024-05-11T23:01:52.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.203e-06 2024-05-11T23:01:52.214 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.214 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.214 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.214 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24152 3.11 3.23111871 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.214 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914678+0000", 2024-05-11T23:01:52.214 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0018164430000000001, 2024-05-11T23:01:52.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0070567720000000002, 2024-05-11T23:01:52.214 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.214 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.214 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.215 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.215 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.215 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24152 2024-05-11T23:01:52.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.215 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914678+0000", 2024-05-11T23:01:52.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914678+0000", 2024-05-11T23:01:52.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914678+0000", 2024-05-11T23:01:52.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2e-07 2024-05-11T23:01:52.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914680+0000", 2024-05-11T23:01:52.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1390000000000002e-06 2024-05-11T23:01:52.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914686+0000", 2024-05-11T23:01:52.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.054e-06 2024-05-11T23:01:52.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914691+0000", 2024-05-11T23:01:52.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5170000000000001e-06 2024-05-11T23:01:52.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.217 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.217 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24249 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.217 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.916660+0000", 2024-05-11T23:01:52.217 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9998336, 2024-05-11T23:01:52.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005133388, 2024-05-11T23:01:52.217 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.218 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.218 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.218 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24249 2024-05-11T23:01:52.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.218 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916660+0000", 2024-05-11T23:01:52.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916660+0000", 2024-05-11T23:01:52.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916661+0000", 2024-05-11T23:01:52.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9200000000000002e-07 2024-05-11T23:01:52.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916663+0000", 2024-05-11T23:01:52.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9250000000000002e-06 2024-05-11T23:01:52.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916669+0000", 2024-05-11T23:01:52.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9830000000000001e-06 2024-05-11T23:01:52.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916676+0000", 2024-05-11T23:01:52.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.956e-06 2024-05-11T23:01:52.220 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.220 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.220 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.220 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24028 3.11 3.23111871 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.221 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913918+0000", 2024-05-11T23:01:52.221 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.002576297, 2024-05-11T23:01:52.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079295249999999998, 2024-05-11T23:01:52.221 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.221 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.221 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.221 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24028 2024-05-11T23:01:52.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.221 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913918+0000", 2024-05-11T23:01:52.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913918+0000", 2024-05-11T23:01:52.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913918+0000", 2024-05-11T23:01:52.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7599999999999997e-07 2024-05-11T23:01:52.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913921+0000", 2024-05-11T23:01:52.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3489999999999999e-06 2024-05-11T23:01:52.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913927+0000", 2024-05-11T23:01:52.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6529999999999997e-06 2024-05-11T23:01:52.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913939+0000", 2024-05-11T23:01:52.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2098e-05 2024-05-11T23:01:52.223 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.223 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.224 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.224 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24157 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.224 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914701+0000", 2024-05-11T23:01:52.224 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0017930859999999999, 2024-05-11T23:01:52.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0072038730000000004, 2024-05-11T23:01:52.224 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.224 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.224 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.224 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.224 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.224 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.224 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24157 2024-05-11T23:01:52.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.225 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914701+0000", 2024-05-11T23:01:52.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914701+0000", 2024-05-11T23:01:52.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914701+0000", 2024-05-11T23:01:52.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9399999999999995e-07 2024-05-11T23:01:52.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914708+0000", 2024-05-11T23:01:52.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3180000000000003e-06 2024-05-11T23:01:52.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914714+0000", 2024-05-11T23:01:52.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6599999999999998e-06 2024-05-11T23:01:52.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914719+0000", 2024-05-11T23:01:52.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.994e-06 2024-05-11T23:01:52.227 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.227 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.227 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.227 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24252 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.227 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.916681+0000", 2024-05-11T23:01:52.227 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9998126, 2024-05-11T23:01:52.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0052782899999999997, 2024-05-11T23:01:52.227 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.227 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.227 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.227 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.227 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.227 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.228 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24252 2024-05-11T23:01:52.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.228 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916681+0000", 2024-05-11T23:01:52.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916681+0000", 2024-05-11T23:01:52.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916682+0000", 2024-05-11T23:01:52.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6600000000000002e-07 2024-05-11T23:01:52.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916684+0000", 2024-05-11T23:01:52.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.86e-06 2024-05-11T23:01:52.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916690+0000", 2024-05-11T23:01:52.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9869999999999996e-06 2024-05-11T23:01:52.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916694+0000", 2024-05-11T23:01:52.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5449999999999997e-06 2024-05-11T23:01:52.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.230 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.230 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24035 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.230 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.913952+0000", 2024-05-11T23:01:52.230 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0025415870000000001, 2024-05-11T23:01:52.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0080654260000000005, 2024-05-11T23:01:52.230 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.230 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.230 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.231 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24035 2024-05-11T23:01:52.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.231 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913952+0000", 2024-05-11T23:01:52.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913952+0000", 2024-05-11T23:01:52.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913953+0000", 2024-05-11T23:01:52.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4899999999999995e-07 2024-05-11T23:01:52.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913961+0000", 2024-05-11T23:01:52.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3659999999999992e-06 2024-05-11T23:01:52.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913969+0000", 2024-05-11T23:01:52.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8120000000000008e-06 2024-05-11T23:01:52.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.913978+0000", 2024-05-11T23:01:52.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6279999999999994e-06 2024-05-11T23:01:52.233 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.233 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.233 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.233 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24160 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.233 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.914726+0000", 2024-05-11T23:01:52.233 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0017680549999999999, 2024-05-11T23:01:52.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0073466790000000001, 2024-05-11T23:01:52.233 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.234 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.234 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.234 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24160 2024-05-11T23:01:52.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.234 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914726+0000", 2024-05-11T23:01:52.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914726+0000", 2024-05-11T23:01:52.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914726+0000", 2024-05-11T23:01:52.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:01:52.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914728+0000", 2024-05-11T23:01:52.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8199999999999999e-06 2024-05-11T23:01:52.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914734+0000", 2024-05-11T23:01:52.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8590000000000001e-06 2024-05-11T23:01:52.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.914741+0000", 2024-05-11T23:01:52.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6499999999999999e-06 2024-05-11T23:01:52.236 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.236 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.236 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.236 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24255 3.11 3.97f08f91 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.236 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.916700+0000", 2024-05-11T23:01:52.236 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4294967295.9997945, 2024-05-11T23:01:52.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0054319169999999996, 2024-05-11T23:01:52.237 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.237 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.237 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:01:52.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.237 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24255 2024-05-11T23:01:52.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.237 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916700+0000", 2024-05-11T23:01:52.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916700+0000", 2024-05-11T23:01:52.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916700+0000", 2024-05-11T23:01:52.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6899999999999998e-07 2024-05-11T23:01:52.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916702+0000", 2024-05-11T23:01:52.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8789999999999999e-06 2024-05-11T23:01:52.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916708+0000", 2024-05-11T23:01:52.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9660000000000001e-06 2024-05-11T23:01:52.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.916712+0000", 2024-05-11T23:01:52.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4939999999999997e-06 2024-05-11T23:01:52.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.239 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.239 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-11T23:01:52.239 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 97 2024-05-11T23:01:52.239 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:52.240 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-05-11T23:01:52.241 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:01:52.242 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:52.241+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:52.341 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:52.341+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:52.399 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:52.400 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:52.400 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:01:52.400 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:01:52.400 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:52.408 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:01:52.440 INFO:teuthology.orchestra.run.smithi005.stdout:2125 writesame 0x7332c72 thru 0x7335965 data_size 0x1c(0x2cf4 bytes) 2024-05-11T23:01:52.442 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:52.442+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:52.453 INFO:teuthology.orchestra.run.smithi005.stdout:2127 write 0xde9df9c thru 0xdea41f5 (0x625a bytes) 2024-05-11T23:01:52.459 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:52.460 INFO:teuthology.orchestra.run.smithi005.stdout:2129 punch from 0x1d17389 to 0x1d1a9fe, (0x3675 bytes) 2024-05-11T23:01:52.460 INFO:teuthology.orchestra.run.smithi005.stdout:2130 read 0x5835e62 thru 0x5836d79 (0xf18 bytes) 2024-05-11T23:01:52.460 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:52.460 INFO:teuthology.orchestra.run.smithi005.stdout:2133 read 0x2d293bb thru 0x2d372c7 (0xdf0d bytes) 2024-05-11T23:01:52.462 INFO:teuthology.orchestra.run.smithi005.stdout:2136 trunc from 0xdea41f6 to 0xcc618a0 2024-05-11T23:01:52.471 INFO:teuthology.orchestra.run.smithi005.stdout:2137 write 0x8c410fa thru 0x8c48fd8 (0x7edf bytes) 2024-05-11T23:01:52.479 INFO:teuthology.orchestra.run.smithi005.stdout:2138 write 0x4fd392 thru 0x50a621 (0xd290 bytes) 2024-05-11T23:01:52.543 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:52.543 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 35.378465098, 2024-05-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.544 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:01:52.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:01:52.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:01:52.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:01:52.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25797 3.11 3:8eedd281:::rbd_data.111e70525772.0000000000000016:e [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.968533+0000", 2024-05-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.57318364600000005, 2024-05-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038615201000000002, 2024-05-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25797 2024-05-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968533+0000", 2024-05-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968533+0000", 2024-05-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968534+0000", 2024-05-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.46e-07 2024-05-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968536+0000", 2024-05-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9599999999999999e-06 2024-05-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968541+0000", 2024-05-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7130000000000002e-06 2024-05-11T23:01:52.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968546+0000", 2024-05-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5600000000000004e-06 2024-05-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.006997+0000", 2024-05-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038451066999999998 2024-05-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.007068+0000", 2024-05-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0514999999999998e-05 2024-05-11T23:01:52.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.007149+0000", 2024-05-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0939999999999994e-05 2024-05-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25804 3.11 3:8eedd281:::rbd_data.111e70525772.0000000000000016:e [read 266240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.968552+0000", 2024-05-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.57316533800000002, 2024-05-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038765423, 2024-05-11T23:01:52.554 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25804 2024-05-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968552+0000", 2024-05-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968552+0000", 2024-05-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968552+0000", 2024-05-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968554+0000", 2024-05-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4260000000000002e-06 2024-05-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968560+0000", 2024-05-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6559999999999997e-06 2024-05-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968565+0000", 2024-05-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1379999999999998e-06 2024-05-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.007178+0000", 2024-05-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038612339000000002 2024-05-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.007248+0000", 2024-05-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0628000000000002e-05 2024-05-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.007317+0000", 2024-05-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8899999999999994e-05 2024-05-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25811 3.11 3:8eedd281:::rbd_data.111e70525772.0000000000000016:e [read 282624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.558 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.968571+0000", 2024-05-11T23:01:52.559 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.57314611900000001, 2024-05-11T23:01:52.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038905382000000002, 2024-05-11T23:01:52.559 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.559 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.559 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:52.559 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.559 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.559 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.559 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25811 2024-05-11T23:01:52.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.559 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968571+0000", 2024-05-11T23:01:52.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968571+0000", 2024-05-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968571+0000", 2024-05-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.08e-07 2024-05-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968573+0000", 2024-05-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.198e-06 2024-05-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968579+0000", 2024-05-11T23:01:52.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.93e-06 2024-05-11T23:01:52.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968584+0000", 2024-05-11T23:01:52.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6569999999999996e-06 2024-05-11T23:01:52.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.007346+0000", 2024-05-11T23:01:52.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038762040999999997 2024-05-11T23:01:52.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.007413+0000", 2024-05-11T23:01:52.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6910999999999997e-05 2024-05-11T23:01:52.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:52.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.007476+0000", 2024-05-11T23:01:52.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3237000000000005e-05 2024-05-11T23:01:52.562 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.562 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.562 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.562 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25818 3.11 3:8eedd281:::rbd_data.111e70525772.0000000000000016:e [read 299008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.562 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.968590+0000", 2024-05-11T23:01:52.562 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.57312654799999996, 2024-05-11T23:01:52.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039039169999999998, 2024-05-11T23:01:52.563 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.563 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.563 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:52.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.563 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25818 2024-05-11T23:01:52.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.563 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968590+0000", 2024-05-11T23:01:52.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968590+0000", 2024-05-11T23:01:52.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968591+0000", 2024-05-11T23:01:52.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2599999999999998e-07 2024-05-11T23:01:52.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968593+0000", 2024-05-11T23:01:52.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9410000000000001e-06 2024-05-11T23:01:52.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968598+0000", 2024-05-11T23:01:52.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6829999999999996e-06 2024-05-11T23:01:52.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968604+0000", 2024-05-11T23:01:52.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3859999999999998e-06 2024-05-11T23:01:52.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.007503+0000", 2024-05-11T23:01:52.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038899182999999997 2024-05-11T23:01:52.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.007568+0000", 2024-05-11T23:01:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4616999999999995e-05 2024-05-11T23:01:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.007630+0000", 2024-05-11T23:01:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1933999999999994e-05 2024-05-11T23:01:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25825 3.11 3:8eedd281:::rbd_data.111e70525772.0000000000000016:e [read 315392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.968610+0000", 2024-05-11T23:01:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.57310714299999999, 2024-05-11T23:01:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039177046, 2024-05-11T23:01:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25825 2024-05-11T23:01:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968610+0000", 2024-05-11T23:01:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968610+0000", 2024-05-11T23:01:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968610+0000", 2024-05-11T23:01:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:01:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968621+0000", 2024-05-11T23:01:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.102e-05 2024-05-11T23:01:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968627+0000", 2024-05-11T23:01:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7690000000000002e-06 2024-05-11T23:01:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968632+0000", 2024-05-11T23:01:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5090000000000004e-06 2024-05-11T23:01:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.007656+0000", 2024-05-11T23:01:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039024279000000002 2024-05-11T23:01:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.007719+0000", 2024-05-11T23:01:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2916000000000005e-05 2024-05-11T23:01:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.007787+0000", 2024-05-11T23:01:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8157000000000005e-05 2024-05-11T23:01:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25832 3.11 3:8eedd281:::rbd_data.111e70525772.0000000000000016:e [read 331776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.968638+0000", 2024-05-11T23:01:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.57307931099999998, 2024-05-11T23:01:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039293742, 2024-05-11T23:01:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25832 2024-05-11T23:01:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968638+0000", 2024-05-11T23:01:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968638+0000", 2024-05-11T23:01:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968638+0000", 2024-05-11T23:01:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1300000000000001e-07 2024-05-11T23:01:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968640+0000", 2024-05-11T23:01:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8810000000000001e-06 2024-05-11T23:01:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968645+0000", 2024-05-11T23:01:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3179999999999997e-06 2024-05-11T23:01:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968650+0000", 2024-05-11T23:01:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4889999999999997e-06 2024-05-11T23:01:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.007814+0000", 2024-05-11T23:01:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039164217000000001 2024-05-11T23:01:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.007873+0000", 2024-05-11T23:01:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8875000000000001e-05 2024-05-11T23:01:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.007931+0000", 2024-05-11T23:01:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8548999999999998e-05 2024-05-11T23:01:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25839 3.11 3:8eedd281:::rbd_data.111e70525772.0000000000000016:e [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.968656+0000", 2024-05-11T23:01:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.57306140900000002, 2024-05-11T23:01:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039439357000000001, 2024-05-11T23:01:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25839 2024-05-11T23:01:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968656+0000", 2024-05-11T23:01:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.968656+0000", 2024-05-11T23:01:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970226+0000", 2024-05-11T23:01:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001570512 2024-05-11T23:01:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970231+0000", 2024-05-11T23:01:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:01:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970242+0000", 2024-05-11T23:01:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1462e-05 2024-05-11T23:01:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970250+0000", 2024-05-11T23:01:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5320000000000001e-06 2024-05-11T23:01:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.007960+0000", 2024-05-11T23:01:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037709714999999998 2024-05-11T23:01:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.008029+0000", 2024-05-11T23:01:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9663e-05 2024-05-11T23:01:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.008095+0000", 2024-05-11T23:01:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5674999999999997e-05 2024-05-11T23:01:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25867 3.11 3:8eedd281:::rbd_data.111e70525772.0000000000000016:e [read 413696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.970327+0000", 2024-05-11T23:01:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.57139005300000001, 2024-05-11T23:01:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038520085000000003, 2024-05-11T23:01:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25867 2024-05-11T23:01:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970327+0000", 2024-05-11T23:01:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970327+0000", 2024-05-11T23:01:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970327+0000", 2024-05-11T23:01:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3000000000000001e-07 2024-05-11T23:01:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970330+0000", 2024-05-11T23:01:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-06 2024-05-11T23:01:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970335+0000", 2024-05-11T23:01:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8440000000000003e-06 2024-05-11T23:01:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970340+0000", 2024-05-11T23:01:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4050000000000002e-06 2024-05-11T23:01:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.008704+0000", 2024-05-11T23:01:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038363737000000002 2024-05-11T23:01:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.008781+0000", 2024-05-11T23:01:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7956999999999999e-05 2024-05-11T23:01:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.008847+0000", 2024-05-11T23:01:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5512000000000006e-05 2024-05-11T23:01:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25874 3.11 3:8eedd281:::rbd_data.111e70525772.0000000000000016:e [read 430080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.970348+0000", 2024-05-11T23:01:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.57136923100000003, 2024-05-11T23:01:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038690537999999997, 2024-05-11T23:01:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25874 2024-05-11T23:01:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970348+0000", 2024-05-11T23:01:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970348+0000", 2024-05-11T23:01:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970348+0000", 2024-05-11T23:01:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3200000000000005e-07 2024-05-11T23:01:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970350+0000", 2024-05-11T23:01:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8509999999999999e-06 2024-05-11T23:01:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970356+0000", 2024-05-11T23:01:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6810000000000003e-06 2024-05-11T23:01:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970361+0000", 2024-05-11T23:01:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4369999999999998e-06 2024-05-11T23:01:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.008880+0000", 2024-05-11T23:01:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038519077999999998 2024-05-11T23:01:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.008966+0000", 2024-05-11T23:01:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5910999999999998e-05 2024-05-11T23:01:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.009038+0000", 2024-05-11T23:01:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2047999999999999e-05 2024-05-11T23:01:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25881 3.11 3:8eedd281:::rbd_data.111e70525772.0000000000000016:e [read 446464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.970367+0000", 2024-05-11T23:01:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.57134947899999999, 2024-05-11T23:01:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038834929999999997, 2024-05-11T23:01:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25881 2024-05-11T23:01:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970367+0000", 2024-05-11T23:01:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970367+0000", 2024-05-11T23:01:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970368+0000", 2024-05-11T23:01:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4700000000000001e-07 2024-05-11T23:01:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970370+0000", 2024-05-11T23:01:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0439999999999998e-06 2024-05-11T23:01:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970377+0000", 2024-05-11T23:01:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8979999999999999e-06 2024-05-11T23:01:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970382+0000", 2024-05-11T23:01:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5469999999999998e-06 2024-05-11T23:01:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.009067+0000", 2024-05-11T23:01:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038685889000000001 2024-05-11T23:01:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.009134+0000", 2024-05-11T23:01:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6396999999999995e-05 2024-05-11T23:01:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.009202+0000", 2024-05-11T23:01:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8608000000000004e-05 2024-05-11T23:01:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25888 3.11 3:8eedd281:::rbd_data.111e70525772.0000000000000016:e [read 462848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.970388+0000", 2024-05-11T23:01:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.57132941900000001, 2024-05-11T23:01:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038956590999999999, 2024-05-11T23:01:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25888 2024-05-11T23:01:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970388+0000", 2024-05-11T23:01:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970388+0000", 2024-05-11T23:01:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970388+0000", 2024-05-11T23:01:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9999999999999998e-07 2024-05-11T23:01:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970391+0000", 2024-05-11T23:01:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.931e-06 2024-05-11T23:01:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970397+0000", 2024-05-11T23:01:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6439999999999997e-06 2024-05-11T23:01:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970401+0000", 2024-05-11T23:01:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6129999999999997e-06 2024-05-11T23:01:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.009234+0000", 2024-05-11T23:01:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038832790999999998 2024-05-11T23:01:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.009298+0000", 2024-05-11T23:01:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4461000000000005e-05 2024-05-11T23:01:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.009344+0000", 2024-05-11T23:01:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5651e-05 2024-05-11T23:01:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25895 3.11 3:8eedd281:::rbd_data.111e70525772.0000000000000016:e [read 479232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.970407+0000", 2024-05-11T23:01:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.57130994999999996, 2024-05-11T23:01:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039113777000000002, 2024-05-11T23:01:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25895 2024-05-11T23:01:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970407+0000", 2024-05-11T23:01:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970407+0000", 2024-05-11T23:01:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970408+0000", 2024-05-11T23:01:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0900000000000002e-07 2024-05-11T23:01:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970409+0000", 2024-05-11T23:01:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9230000000000001e-06 2024-05-11T23:01:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970416+0000", 2024-05-11T23:01:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0950000000000001e-06 2024-05-11T23:01:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970420+0000", 2024-05-11T23:01:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6179999999999997e-06 2024-05-11T23:01:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.009366+0000", 2024-05-11T23:01:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038946016 2024-05-11T23:01:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.009430+0000", 2024-05-11T23:01:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3424999999999997e-05 2024-05-11T23:01:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.009521+0000", 2024-05-11T23:01:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1191000000000002e-05 2024-05-11T23:01:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25902 3.11 3:8eedd281:::rbd_data.111e70525772.0000000000000016:e [read 495616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.970427+0000", 2024-05-11T23:01:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.57129002299999998, 2024-05-11T23:01:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039246996999999999, 2024-05-11T23:01:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25902 2024-05-11T23:01:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970427+0000", 2024-05-11T23:01:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970427+0000", 2024-05-11T23:01:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970427+0000", 2024-05-11T23:01:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5200000000000002e-07 2024-05-11T23:01:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970429+0000", 2024-05-11T23:01:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0679999999999999e-06 2024-05-11T23:01:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970435+0000", 2024-05-11T23:01:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8810000000000001e-06 2024-05-11T23:01:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970440+0000", 2024-05-11T23:01:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5040000000000004e-06 2024-05-11T23:01:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.009547+0000", 2024-05-11T23:01:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039107613999999999 2024-05-11T23:01:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.009611+0000", 2024-05-11T23:01:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3527000000000003e-05 2024-05-11T23:01:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.009674+0000", 2024-05-11T23:01:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2951000000000001e-05 2024-05-11T23:01:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25909 3.11 3:8eedd281:::rbd_data.111e70525772.0000000000000016:e [read 512000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.970446+0000", 2024-05-11T23:01:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.57127128199999999, 2024-05-11T23:01:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039425799999999997, 2024-05-11T23:01:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25909 2024-05-11T23:01:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970446+0000", 2024-05-11T23:01:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970446+0000", 2024-05-11T23:01:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970446+0000", 2024-05-11T23:01:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.08e-07 2024-05-11T23:01:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970453+0000", 2024-05-11T23:01:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6499999999999999e-06 2024-05-11T23:01:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970459+0000", 2024-05-11T23:01:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:01:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970463+0000", 2024-05-11T23:01:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5020000000000003e-06 2024-05-11T23:01:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.009750+0000", 2024-05-11T23:01:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039286213 2024-05-11T23:01:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.009812+0000", 2024-05-11T23:01:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2967999999999997e-05 2024-05-11T23:01:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.009871+0000", 2024-05-11T23:01:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8986000000000001e-05 2024-05-11T23:01:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25915 3.11 3:8eedd281:::rbd_data.111e70525772.0000000000000016:e [read 151552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.970470+0000", 2024-05-11T23:01:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.57124663499999995, 2024-05-11T23:01:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039555695000000002, 2024-05-11T23:01:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25915 2024-05-11T23:01:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970470+0000", 2024-05-11T23:01:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970470+0000", 2024-05-11T23:01:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970471+0000", 2024-05-11T23:01:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6100000000000001e-07 2024-05-11T23:01:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970473+0000", 2024-05-11T23:01:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2050000000000001e-06 2024-05-11T23:01:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970479+0000", 2024-05-11T23:01:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8189999999999997e-06 2024-05-11T23:01:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970484+0000", 2024-05-11T23:01:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3839999999999996e-06 2024-05-11T23:01:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.009895+0000", 2024-05-11T23:01:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039411083 2024-05-11T23:01:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.009954+0000", 2024-05-11T23:01:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8474000000000003e-05 2024-05-11T23:01:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.010026+0000", 2024-05-11T23:01:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2268999999999995e-05 2024-05-11T23:01:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25923 3.11 3:8eedd281:::rbd_data.111e70525772.0000000000000016:e [read 167936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.970490+0000", 2024-05-11T23:01:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.57122720299999996, 2024-05-11T23:01:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039691489000000003, 2024-05-11T23:01:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25923 2024-05-11T23:01:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970490+0000", 2024-05-11T23:01:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970490+0000", 2024-05-11T23:01:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970490+0000", 2024-05-11T23:01:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:01:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970492+0000", 2024-05-11T23:01:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8899999999999999e-06 2024-05-11T23:01:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970498+0000", 2024-05-11T23:01:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8100000000000003e-06 2024-05-11T23:01:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970503+0000", 2024-05-11T23:01:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6179999999999997e-06 2024-05-11T23:01:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.010052+0000", 2024-05-11T23:01:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039549833 2024-05-11T23:01:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.010111+0000", 2024-05-11T23:01:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9011999999999997e-05 2024-05-11T23:01:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.010181+0000", 2024-05-11T23:01:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9826999999999994e-05 2024-05-11T23:01:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25929 3.11 3:8eedd281:::rbd_data.111e70525772.0000000000000016:e [read 184320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.970508+0000", 2024-05-11T23:01:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.57120858200000002, 2024-05-11T23:01:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039832357999999998, 2024-05-11T23:01:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25929 2024-05-11T23:01:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970508+0000", 2024-05-11T23:01:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970508+0000", 2024-05-11T23:01:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970509+0000", 2024-05-11T23:01:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3700000000000001e-07 2024-05-11T23:01:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970511+0000", 2024-05-11T23:01:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8679999999999999e-06 2024-05-11T23:01:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970516+0000", 2024-05-11T23:01:52.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4410000000000001e-06 2024-05-11T23:01:52.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970521+0000", 2024-05-11T23:01:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2320000000000001e-06 2024-05-11T23:01:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.010206+0000", 2024-05-11T23:01:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039684364999999999 2024-05-11T23:01:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.010271+0000", 2024-05-11T23:01:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5283999999999994e-05 2024-05-11T23:01:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.010341+0000", 2024-05-11T23:01:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9731e-05 2024-05-11T23:01:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25935 3.11 3:8eedd281:::rbd_data.111e70525772.0000000000000016:e [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.970529+0000", 2024-05-11T23:01:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.571188323, 2024-05-11T23:01:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039972804000000001, 2024-05-11T23:01:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25935 2024-05-11T23:01:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970529+0000", 2024-05-11T23:01:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970529+0000", 2024-05-11T23:01:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970529+0000", 2024-05-11T23:01:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1999999999999999e-07 2024-05-11T23:01:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970531+0000", 2024-05-11T23:01:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0999999999999998e-06 2024-05-11T23:01:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970538+0000", 2024-05-11T23:01:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2920000000000001e-06 2024-05-11T23:01:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970542+0000", 2024-05-11T23:01:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6190000000000002e-06 2024-05-11T23:01:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.010366+0000", 2024-05-11T23:01:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039823795000000002 2024-05-11T23:01:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.010431+0000", 2024-05-11T23:01:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5050999999999998e-05 2024-05-11T23:01:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.010501+0000", 2024-05-11T23:01:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0327000000000006e-05 2024-05-11T23:01:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25943 3.11 3:8eedd281:::rbd_data.111e70525772.0000000000000016:e [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.970548+0000", 2024-05-11T23:01:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.57116854500000003, 2024-05-11T23:01:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040110923, 2024-05-11T23:01:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:01:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25943 2024-05-11T23:01:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970548+0000", 2024-05-11T23:01:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970548+0000", 2024-05-11T23:01:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970549+0000", 2024-05-11T23:01:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3300000000000003e-07 2024-05-11T23:01:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970551+0000", 2024-05-11T23:01:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0379999999999998e-06 2024-05-11T23:01:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970556+0000", 2024-05-11T23:01:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5929999999999996e-06 2024-05-11T23:01:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.970561+0000", 2024-05-11T23:01:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3540000000000002e-06 2024-05-11T23:01:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.010526+0000", 2024-05-11T23:01:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039965423999999999 2024-05-11T23:01:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.010588+0000", 2024-05-11T23:01:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1760999999999994e-05 2024-05-11T23:01:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.010659+0000", 2024-05-11T23:01:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1320000000000002e-05 2024-05-11T23:01:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:52.631 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:52.632 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:52.542+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:52.633 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:01:52.634 INFO:teuthology.orchestra.run.smithi005.stdout:2139 read 0x8b3f212 thru 0x8b4887a (0x9669 bytes) 2024-05-11T23:01:52.634 INFO:teuthology.orchestra.run.smithi005.stdout:2140 punch from 0x58a1fa8 to 0x58ae049, (0xc0a1 bytes) 2024-05-11T23:01:52.634 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:52.634 INFO:teuthology.orchestra.run.smithi005.stdout:2142 read 0x736925a thru 0x7378147 (0xeeee bytes) 2024-05-11T23:01:52.634 INFO:teuthology.orchestra.run.smithi005.stdout:2143 write 0x2bd804 thru 0x2cca8a (0xf287 bytes) 2024-05-11T23:01:52.634 INFO:teuthology.orchestra.run.smithi005.stdout:2144 read 0xb95835f thru 0xb960fee (0x8c90 bytes) 2024-05-11T23:01:52.634 INFO:teuthology.orchestra.run.smithi005.stdout:2145 punch from 0x122f3b9 to 0x123b957, (0xc59e bytes) 2024-05-11T23:01:52.635 INFO:teuthology.orchestra.run.smithi005.stdout:2146 trunc from 0xcc618a0 to 0xc1c586f 2024-05-11T23:01:52.635 INFO:teuthology.orchestra.run.smithi005.stdout:2147 writesame 0x27b86a0 thru 0x27c6ebf data_size 0x30(0xe820 bytes) 2024-05-11T23:01:52.679 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:52.679+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:52.707 INFO:teuthology.orchestra.run.smithi005.stdout:2148 write 0x2a7cb17 thru 0x2a7e515 (0x19ff bytes) 2024-05-11T23:01:52.709 INFO:teuthology.orchestra.run.smithi005.stdout:2149 punch from 0x94e3033 to 0x94eea1e, (0xb9eb bytes) 2024-05-11T23:01:52.709 INFO:teuthology.orchestra.run.smithi005.stdout:2150 read 0x9362aca thru 0x936b387 (0x88be bytes) 2024-05-11T23:01:52.710 INFO:teuthology.orchestra.run.smithi005.stdout:2151 punch from 0x26964b to 0x26c517, (0x2ecc bytes) 2024-05-11T23:01:52.710 INFO:teuthology.orchestra.run.smithi005.stdout:2152 read 0x791a52d thru 0x7921930 (0x7404 bytes) 2024-05-11T23:01:52.710 INFO:teuthology.orchestra.run.smithi005.stdout:2154 write 0x6b0c578 thru 0x6b13a9b (0x7524 bytes) 2024-05-11T23:01:52.717 INFO:teuthology.orchestra.run.smithi005.stdout:2155 trunc from 0xc1c586f to 0xe17a35d 2024-05-11T23:01:52.723 INFO:teuthology.orchestra.run.smithi005.stdout:2156 write 0xbcdd38f thru 0xbce9c76 (0xc8e8 bytes) 2024-05-11T23:01:52.732 INFO:teuthology.orchestra.run.smithi005.stdout:2158 punch from 0x7be7e3b to 0x7bf79d6, (0xfb9b bytes) 2024-05-11T23:01:52.732 INFO:teuthology.orchestra.run.smithi005.stdout:2159 write 0xb5a4352 thru 0xb5afecf (0xbb7e bytes) 2024-05-11T23:01:52.736 INFO:teuthology.orchestra.run.smithi005.stdout:2160 write 0x4d15ff5 thru 0x4d200c1 (0xa0cd bytes) 2024-05-11T23:01:52.744 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:52.744+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:52.746 INFO:teuthology.orchestra.run.smithi005.stdout:2164 read 0xc447e74 thru 0xc455ab8 (0xdc45 bytes) 2024-05-11T23:01:52.746 INFO:teuthology.orchestra.run.smithi005.stdout:2165 read 0x144f3a1 thru 0x1452eae (0x3b0e bytes) 2024-05-11T23:01:52.746 INFO:teuthology.orchestra.run.smithi005.stdout:2166 write 0xc28182c thru 0xc2893de (0x7bb3 bytes) 2024-05-11T23:01:52.765 INFO:teuthology.orchestra.run.smithi005.stdout:2167 trunc from 0xe17a35d to 0x4391fdf 2024-05-11T23:01:52.779 INFO:teuthology.orchestra.run.smithi005.stdout:2168 read 0x7cc40 thru 0x8be18 (0xf1d9 bytes) 2024-05-11T23:01:52.780 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:52.780 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:52.780 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:52.788 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:52.788 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:52.788 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:01:52.788 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:52.797 INFO:teuthology.orchestra.run.smithi005.stdout:2176 writesame 0xd24477c thru 0xd245e13 data_size 0xc(0x1698 bytes) 2024-05-11T23:01:52.797 INFO:teuthology.orchestra.run.smithi005.stdout:2177 read 0x7d6b182 thru 0x7d74687 (0x9506 bytes) 2024-05-11T23:01:52.799 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:01:52.806 INFO:teuthology.orchestra.run.smithi005.stdout:2178 trunc from 0xd245e14 to 0x4d87569 2024-05-11T23:01:52.811 INFO:teuthology.orchestra.run.smithi005.stdout:2179 read 0xde1dcd thru 0xdeb096 (0x92ca bytes) 2024-05-11T23:01:52.828 INFO:teuthology.orchestra.run.smithi005.stdout:2182 write 0xdbe6a84 thru 0xdbee747 (0x7cc4 bytes) 2024-05-11T23:01:52.835 INFO:teuthology.orchestra.run.smithi005.stdout:2184 punch from 0x9b85e02 to 0x9b88671, (0x286f bytes) 2024-05-11T23:01:52.835 INFO:teuthology.orchestra.run.smithi005.stdout:2185 read 0x5f99ecd thru 0x5fa4bd6 (0xad0a bytes) 2024-05-11T23:01:52.835 INFO:teuthology.orchestra.run.smithi005.stdout:2186 write 0x52f72a4 thru 0x52f917e (0x1edb bytes) 2024-05-11T23:01:52.838 INFO:teuthology.orchestra.run.smithi005.stdout:2187 read 0x5f8b3e5 thru 0x5f96174 (0xad90 bytes) 2024-05-11T23:01:52.839 INFO:teuthology.orchestra.run.smithi005.stdout:2188 writesame 0x1ef28db thru 0x1f024d1 data_size 0x9(0xfbf7 bytes) 2024-05-11T23:01:52.843 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-11T23:01:52.844 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:52.845+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:52.945 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:52.945+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:52.979 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:52.979 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:52.979 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:01:52.979 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:01:52.979 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:52.991 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:01:53.075 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:53.075+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:53.090 INFO:teuthology.orchestra.run.smithi005.stdout:2189 writesame 0x78af716 thru 0x78b2308 data_size 0x1(0x2bf3 bytes) 2024-05-11T23:01:53.097 INFO:teuthology.orchestra.run.smithi005.stdout:2190 write 0x1b88f59 thru 0x1b8bce6 (0x2d8e bytes) 2024-05-11T23:01:53.099 INFO:teuthology.orchestra.run.smithi005.stdout:2191 punch from 0x853de9a to 0x8546cad, (0x8e13 bytes) 2024-05-11T23:01:53.099 INFO:teuthology.orchestra.run.smithi005.stdout:2193 read 0x682a224 thru 0x682dcd7 (0x3ab4 bytes) 2024-05-11T23:01:53.107 INFO:teuthology.orchestra.run.smithi005.stdout:2196 trunc from 0xdbee748 to 0x7eec8e6 2024-05-11T23:01:53.118 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:53.118 INFO:teuthology.orchestra.run.smithi005.stdout:2199 punch from 0x502be8d to 0x503735f, (0xb4d2 bytes) 2024-05-11T23:01:53.118 INFO:teuthology.orchestra.run.smithi005.stdout:2200 read 0x42bcd88 thru 0x42c520c (0x8485 bytes) 2024-05-11T23:01:53.118 INFO:teuthology.orchestra.run.smithi005.stdout:2202 punch from 0x602bf55 to 0x6033baf, (0x7c5a bytes) 2024-05-11T23:01:53.125 INFO:teuthology.orchestra.run.smithi005.stdout:2203 write 0x8cb0044 thru 0x8cb1a02 (0x19bf bytes) 2024-05-11T23:01:53.128 INFO:teuthology.orchestra.run.smithi005.stdout:2204 punch from 0x6307b07 to 0x630f7e6, (0x7cdf bytes) 2024-05-11T23:01:53.128 INFO:teuthology.orchestra.run.smithi005.stdout:2206 read 0xbd27dd thru 0xbd979f (0x6fc3 bytes) 2024-05-11T23:01:53.129 INFO:teuthology.orchestra.run.smithi005.stdout:2207 write 0x35cc2d3 thru 0x35ce63b (0x2369 bytes) 2024-05-11T23:01:53.146 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:53.147+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:53.154 INFO:teuthology.orchestra.run.smithi005.stdout:2208 writesame 0xee51447 thru 0xee59c56 data_size 0x38(0x8810 bytes) 2024-05-11T23:01:53.154 INFO:teuthology.orchestra.run.smithi005.stdout:2209 write 0x5166514 thru 0x516f8c0 (0x93ad bytes) 2024-05-11T23:01:53.159 INFO:teuthology.orchestra.run.smithi005.stdout:2210 write 0x57b9dc2 thru 0x57c6898 (0xcad7 bytes) 2024-05-11T23:01:53.165 INFO:teuthology.orchestra.run.smithi005.stdout:2211 read 0x583baf2 thru 0x5841fe2 (0x64f1 bytes) 2024-05-11T23:01:53.169 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:53.169 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:01:53.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:01:53.170 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:01:53.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.170 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25833 3.17 3:ec6e9d67:::rbd_data.111e70525772.000000000000001a:e [read 331776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:53.170 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.947574+0000", 2024-05-11T23:01:53.170 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.221277535, 2024-05-11T23:01:53.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019999618, 2024-05-11T23:01:53.170 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:53.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:53.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:53.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:53.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:53.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:53.170 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25833 2024-05-11T23:01:53.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.171 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:53.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:53.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947574+0000", 2024-05-11T23:01:53.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:53.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947574+0000", 2024-05-11T23:01:53.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:53.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947574+0000", 2024-05-11T23:01:53.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2800000000000002e-07 2024-05-11T23:01:53.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:53.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947576+0000", 2024-05-11T23:01:53.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.181e-06 2024-05-11T23:01:53.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:53.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947582+0000", 2024-05-11T23:01:53.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3720000000000001e-06 2024-05-11T23:01:53.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:53.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947587+0000", 2024-05-11T23:01:53.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7099999999999998e-06 2024-05-11T23:01:53.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:53.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967403+0000", 2024-05-11T23:01:53.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019815869 2024-05-11T23:01:53.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:53.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967476+0000", 2024-05-11T23:01:53.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2751999999999997e-05 2024-05-11T23:01:53.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:53.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967573+0000", 2024-05-11T23:01:53.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7306000000000007e-05 2024-05-11T23:01:53.174 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.174 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.174 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.174 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25837 3.17 3:ed395bba:::rbd_data.111e70525772.000000000000000e:e [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:53.174 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.947601+0000", 2024-05-11T23:01:53.174 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.2212498460000001, 2024-05-11T23:01:53.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020075396999999998, 2024-05-11T23:01:53.174 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:53.174 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:53.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:53.175 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:53.175 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:53.175 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:53.175 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25837 2024-05-11T23:01:53.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.175 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:53.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:53.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947601+0000", 2024-05-11T23:01:53.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:53.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947601+0000", 2024-05-11T23:01:53.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:53.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947602+0000", 2024-05-11T23:01:53.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.06e-07 2024-05-11T23:01:53.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:53.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947604+0000", 2024-05-11T23:01:53.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4930000000000001e-06 2024-05-11T23:01:53.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:53.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947611+0000", 2024-05-11T23:01:53.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:01:53.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:53.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947616+0000", 2024-05-11T23:01:53.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9289999999999997e-06 2024-05-11T23:01:53.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:53.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967584+0000", 2024-05-11T23:01:53.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019968769000000001 2024-05-11T23:01:53.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:53.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967632+0000", 2024-05-11T23:01:53.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7474e-05 2024-05-11T23:01:53.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:53.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967677+0000", 2024-05-11T23:01:53.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4669999999999998e-05 2024-05-11T23:01:53.178 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.178 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.179 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.179 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25840 3.17 3:ec6e9d67:::rbd_data.111e70525772.000000000000001a:e [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:53.179 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.947627+0000", 2024-05-11T23:01:53.179 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.221224265, 2024-05-11T23:01:53.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020165684999999999, 2024-05-11T23:01:53.179 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:53.179 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:53.179 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:53.179 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:53.179 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:53.179 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:53.179 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25840 2024-05-11T23:01:53.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.180 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:53.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:53.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947627+0000", 2024-05-11T23:01:53.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:53.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947627+0000", 2024-05-11T23:01:53.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:53.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947627+0000", 2024-05-11T23:01:53.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4199999999999996e-07 2024-05-11T23:01:53.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:53.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947630+0000", 2024-05-11T23:01:53.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.311e-06 2024-05-11T23:01:53.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:53.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947636+0000", 2024-05-11T23:01:53.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2790000000000004e-06 2024-05-11T23:01:53.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:53.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947641+0000", 2024-05-11T23:01:53.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9280000000000001e-06 2024-05-11T23:01:53.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:53.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967707+0000", 2024-05-11T23:01:53.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020066148999999998 2024-05-11T23:01:53.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:53.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967748+0000", 2024-05-11T23:01:53.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1247e-05 2024-05-11T23:01:53.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:53.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967792+0000", 2024-05-11T23:01:53.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4229e-05 2024-05-11T23:01:53.183 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.183 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.183 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.183 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25844 3.17 3:ed395bba:::rbd_data.111e70525772.000000000000000e:e [read 364544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:53.183 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.947768+0000", 2024-05-11T23:01:53.183 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.2210830480000001, 2024-05-11T23:01:53.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020136108999999999, 2024-05-11T23:01:53.184 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:53.184 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:53.184 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:53.184 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:53.184 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:53.184 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:53.184 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25844 2024-05-11T23:01:53.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.184 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:53.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:53.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947768+0000", 2024-05-11T23:01:53.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:53.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947768+0000", 2024-05-11T23:01:53.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:53.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947768+0000", 2024-05-11T23:01:53.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8100000000000003e-07 2024-05-11T23:01:53.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:53.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947771+0000", 2024-05-11T23:01:53.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4420000000000001e-06 2024-05-11T23:01:53.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:53.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947778+0000", 2024-05-11T23:01:53.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.888e-06 2024-05-11T23:01:53.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:53.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947782+0000", 2024-05-11T23:01:53.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9790000000000004e-06 2024-05-11T23:01:53.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:53.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967821+0000", 2024-05-11T23:01:53.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020038944999999999 2024-05-11T23:01:53.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:53.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967862+0000", 2024-05-11T23:01:53.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1242999999999998e-05 2024-05-11T23:01:53.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:53.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967904+0000", 2024-05-11T23:01:53.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2131e-05 2024-05-11T23:01:53.188 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.188 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.188 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.188 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25854 3.17 3:ec6e9d67:::rbd_data.111e70525772.000000000000001a:e [read 380928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:53.188 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948270+0000", 2024-05-11T23:01:53.188 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.220581017, 2024-05-11T23:01:53.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020068936999999998, 2024-05-11T23:01:53.189 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:53.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:53.189 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:53.189 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:53.189 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:53.189 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:53.189 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25854 2024-05-11T23:01:53.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.189 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:53.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:53.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948270+0000", 2024-05-11T23:01:53.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:53.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948270+0000", 2024-05-11T23:01:53.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:53.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948271+0000", 2024-05-11T23:01:53.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2399999999999998e-07 2024-05-11T23:01:53.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:53.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948273+0000", 2024-05-11T23:01:53.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5299999999999999e-06 2024-05-11T23:01:53.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:53.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948281+0000", 2024-05-11T23:01:53.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3280000000000006e-06 2024-05-11T23:01:53.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:53.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948287+0000", 2024-05-11T23:01:53.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6060000000000002e-06 2024-05-11T23:01:53.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:53.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968223+0000", 2024-05-11T23:01:53.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019936344000000002 2024-05-11T23:01:53.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:53.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968290+0000", 2024-05-11T23:01:53.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6576999999999996e-05 2024-05-11T23:01:53.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:53.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968339+0000", 2024-05-11T23:01:53.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9027999999999999e-05 2024-05-11T23:01:53.192 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.193 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.193 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.193 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25858 3.17 3.5dda9cb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:53.193 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948295+0000", 2024-05-11T23:01:53.193 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.2205558409999999, 2024-05-11T23:01:53.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020076373000000002, 2024-05-11T23:01:53.193 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:53.193 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:53.193 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:53.193 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:53.193 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:53.193 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:53.194 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25858 2024-05-11T23:01:53.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.194 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:53.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:53.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948295+0000", 2024-05-11T23:01:53.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:53.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948295+0000", 2024-05-11T23:01:53.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:53.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948296+0000", 2024-05-11T23:01:53.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7100000000000002e-07 2024-05-11T23:01:53.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:53.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948298+0000", 2024-05-11T23:01:53.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6089999999999999e-06 2024-05-11T23:01:53.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:53.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948305+0000", 2024-05-11T23:01:53.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0520000000000004e-06 2024-05-11T23:01:53.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:53.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948310+0000", 2024-05-11T23:01:53.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9239999999999998e-06 2024-05-11T23:01:53.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:53.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968366+0000", 2024-05-11T23:01:53.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020055988 2024-05-11T23:01:53.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:53.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968372+0000", 2024-05-11T23:01:53.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3290000000000001e-06 2024-05-11T23:01:53.196 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.196 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.196 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.197 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25861 3.17 3.e6b97637 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:53.197 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948318+0000", 2024-05-11T23:01:53.197 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.2205331850000001, 2024-05-11T23:01:53.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020083304999999999, 2024-05-11T23:01:53.197 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:53.197 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:53.197 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:53.197 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:53.197 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:53.197 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:53.197 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25861 2024-05-11T23:01:53.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.198 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:53.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:53.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948318+0000", 2024-05-11T23:01:53.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:53.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948318+0000", 2024-05-11T23:01:53.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:53.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948318+0000", 2024-05-11T23:01:53.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:01:53.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:53.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948321+0000", 2024-05-11T23:01:53.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.26e-06 2024-05-11T23:01:53.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:53.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948343+0000", 2024-05-11T23:01:53.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2563999999999999e-05 2024-05-11T23:01:53.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:53.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948351+0000", 2024-05-11T23:01:53.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6490000000000008e-06 2024-05-11T23:01:53.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:53.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968397+0000", 2024-05-11T23:01:53.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020045754999999998 2024-05-11T23:01:53.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:53.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968401+0000", 2024-05-11T23:01:53.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6519999999999997e-06 2024-05-11T23:01:53.201 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.201 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.201 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.201 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25865 3.17 3.5dda9cb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:53.201 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948361+0000", 2024-05-11T23:01:53.201 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.2204901260000001, 2024-05-11T23:01:53.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020065274000000001, 2024-05-11T23:01:53.201 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:53.201 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:53.201 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:53.201 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:53.201 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:53.201 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:53.202 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25865 2024-05-11T23:01:53.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.202 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:53.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:53.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948361+0000", 2024-05-11T23:01:53.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:53.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948361+0000", 2024-05-11T23:01:53.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:53.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948362+0000", 2024-05-11T23:01:53.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2900000000000003e-07 2024-05-11T23:01:53.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:53.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948365+0000", 2024-05-11T23:01:53.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0769999999999999e-06 2024-05-11T23:01:53.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:53.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948372+0000", 2024-05-11T23:01:53.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6299999999999998e-06 2024-05-11T23:01:53.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:53.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948377+0000", 2024-05-11T23:01:53.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0139999999999998e-06 2024-05-11T23:01:53.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:53.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968422+0000", 2024-05-11T23:01:53.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020044472000000001 2024-05-11T23:01:53.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:53.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968426+0000", 2024-05-11T23:01:53.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4519999999999999e-06 2024-05-11T23:01:53.205 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.205 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.205 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.205 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25868 3.17 3.e6b97637 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:53.205 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948385+0000", 2024-05-11T23:01:53.205 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.220466131, 2024-05-11T23:01:53.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020082010000000001, 2024-05-11T23:01:53.206 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:53.206 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:53.206 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:53.206 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:53.206 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:53.206 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:53.206 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25868 2024-05-11T23:01:53.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.206 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:53.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:53.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948385+0000", 2024-05-11T23:01:53.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:53.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948385+0000", 2024-05-11T23:01:53.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:53.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948385+0000", 2024-05-11T23:01:53.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6300000000000001e-07 2024-05-11T23:01:53.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:53.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948387+0000", 2024-05-11T23:01:53.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8169999999999999e-06 2024-05-11T23:01:53.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:53.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948391+0000", 2024-05-11T23:01:53.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8310000000000003e-06 2024-05-11T23:01:53.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:53.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948394+0000", 2024-05-11T23:01:53.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7209999999999999e-06 2024-05-11T23:01:53.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:53.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968458+0000", 2024-05-11T23:01:53.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020064354 2024-05-11T23:01:53.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:53.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968467+0000", 2024-05-11T23:01:53.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9239999999999996e-06 2024-05-11T23:01:53.209 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.209 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.209 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.209 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25871 3.17 3.5dda9cb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:53.209 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948397+0000", 2024-05-11T23:01:53.209 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.2204537520000001, 2024-05-11T23:01:53.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020115161999999999, 2024-05-11T23:01:53.209 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:53.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:53.210 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:53.210 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:53.210 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:53.210 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:53.210 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25871 2024-05-11T23:01:53.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.210 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:53.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:53.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948397+0000", 2024-05-11T23:01:53.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:53.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948397+0000", 2024-05-11T23:01:53.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:53.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948398+0000", 2024-05-11T23:01:53.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.22e-07 2024-05-11T23:01:53.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:53.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948402+0000", 2024-05-11T23:01:53.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2799999999999997e-06 2024-05-11T23:01:53.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:53.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948405+0000", 2024-05-11T23:01:53.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3830000000000001e-06 2024-05-11T23:01:53.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:53.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948410+0000", 2024-05-11T23:01:53.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3680000000000004e-06 2024-05-11T23:01:53.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:53.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968506+0000", 2024-05-11T23:01:53.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020096394 2024-05-11T23:01:53.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:53.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968512+0000", 2024-05-11T23:01:53.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5150000000000003e-06 2024-05-11T23:01:53.213 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.213 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.213 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.213 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25875 3.17 3.e6b97637 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:53.213 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948413+0000", 2024-05-11T23:01:53.213 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.220437996, 2024-05-11T23:01:53.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020142189000000001, 2024-05-11T23:01:53.213 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:53.213 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:53.214 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:53.214 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:53.214 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:53.214 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:53.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25875 2024-05-11T23:01:53.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.214 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:53.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:53.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948413+0000", 2024-05-11T23:01:53.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:53.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948413+0000", 2024-05-11T23:01:53.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:53.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948413+0000", 2024-05-11T23:01:53.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:01:53.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:53.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948415+0000", 2024-05-11T23:01:53.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2440000000000001e-06 2024-05-11T23:01:53.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:53.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948418+0000", 2024-05-11T23:01:53.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-06 2024-05-11T23:01:53.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:53.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948421+0000", 2024-05-11T23:01:53.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.706e-06 2024-05-11T23:01:53.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:53.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968546+0000", 2024-05-11T23:01:53.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020125034999999999 2024-05-11T23:01:53.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:53.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968555+0000", 2024-05-11T23:01:53.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6840000000000008e-06 2024-05-11T23:01:53.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.217 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.217 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25879 3.17 3.5dda9cb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:53.217 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948424+0000", 2024-05-11T23:01:53.217 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.220427339, 2024-05-11T23:01:53.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020150983000000001, 2024-05-11T23:01:53.217 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:53.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:53.218 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:53.218 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:53.218 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:53.218 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:53.218 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25879 2024-05-11T23:01:53.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.218 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:53.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:53.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948424+0000", 2024-05-11T23:01:53.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:53.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948424+0000", 2024-05-11T23:01:53.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:53.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948424+0000", 2024-05-11T23:01:53.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:01:53.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:53.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948425+0000", 2024-05-11T23:01:53.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.167e-06 2024-05-11T23:01:53.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:53.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948428+0000", 2024-05-11T23:01:53.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2049999999999998e-06 2024-05-11T23:01:53.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:53.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948431+0000", 2024-05-11T23:01:53.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5749999999999999e-06 2024-05-11T23:01:53.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:53.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968568+0000", 2024-05-11T23:01:53.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020137192000000002 2024-05-11T23:01:53.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:53.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968575+0000", 2024-05-11T23:01:53.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.635e-06 2024-05-11T23:01:53.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.221 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.221 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25882 3.17 3.e6b97637 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:53.221 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948434+0000", 2024-05-11T23:01:53.221 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.22041694, 2024-05-11T23:01:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020171597999999999, 2024-05-11T23:01:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25882 2024-05-11T23:01:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948434+0000", 2024-05-11T23:01:53.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:53.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948434+0000", 2024-05-11T23:01:53.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:53.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948434+0000", 2024-05-11T23:01:53.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:01:53.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:53.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948435+0000", 2024-05-11T23:01:53.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1570000000000001e-06 2024-05-11T23:01:53.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:53.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948440+0000", 2024-05-11T23:01:53.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1989999999999999e-06 2024-05-11T23:01:53.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:53.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948444+0000", 2024-05-11T23:01:53.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.121e-06 2024-05-11T23:01:53.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:53.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968599+0000", 2024-05-11T23:01:53.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020155619 2024-05-11T23:01:53.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:53.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968606+0000", 2024-05-11T23:01:53.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2709999999999998e-06 2024-05-11T23:01:53.225 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.225 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.225 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.226 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25886 3.17 3.5dda9cb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:53.226 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948487+0000", 2024-05-11T23:01:53.226 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.220363823, 2024-05-11T23:01:53.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020152224999999999, 2024-05-11T23:01:53.226 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:53.226 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:53.226 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:53.226 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:53.226 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:53.226 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:53.226 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25886 2024-05-11T23:01:53.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.226 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:53.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:53.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948487+0000", 2024-05-11T23:01:53.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:53.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948487+0000", 2024-05-11T23:01:53.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:53.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948488+0000", 2024-05-11T23:01:53.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4199999999999995e-07 2024-05-11T23:01:53.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:53.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948491+0000", 2024-05-11T23:01:53.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.304e-06 2024-05-11T23:01:53.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:53.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948499+0000", 2024-05-11T23:01:53.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8979999999999996e-06 2024-05-11T23:01:53.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:53.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948506+0000", 2024-05-11T23:01:53.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8290000000000002e-06 2024-05-11T23:01:53.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:53.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968631+0000", 2024-05-11T23:01:53.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020125546000000001 2024-05-11T23:01:53.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:53.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968639+0000", 2024-05-11T23:01:53.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0059999999999992e-06 2024-05-11T23:01:53.229 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.229 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.229 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.230 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25889 3.17 3.e6b97637 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:53.230 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948515+0000", 2024-05-11T23:01:53.230 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.220335768, 2024-05-11T23:01:53.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020155683000000001, 2024-05-11T23:01:53.230 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:53.230 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:53.230 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:53.230 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:53.230 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:53.230 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:53.230 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25889 2024-05-11T23:01:53.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.231 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:53.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:53.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948515+0000", 2024-05-11T23:01:53.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:53.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948515+0000", 2024-05-11T23:01:53.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:53.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948516+0000", 2024-05-11T23:01:53.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3599999999999999e-07 2024-05-11T23:01:53.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:53.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948518+0000", 2024-05-11T23:01:53.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4320000000000002e-06 2024-05-11T23:01:53.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:53.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948532+0000", 2024-05-11T23:01:53.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4078e-05 2024-05-11T23:01:53.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:53.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948538+0000", 2024-05-11T23:01:53.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6899999999999997e-06 2024-05-11T23:01:53.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:53.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968666+0000", 2024-05-11T23:01:53.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020128070000000001 2024-05-11T23:01:53.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:53.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968671+0000", 2024-05-11T23:01:53.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.977e-06 2024-05-11T23:01:53.236 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.236 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.236 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.236 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25893 3.17 3.5dda9cb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:53.236 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948546+0000", 2024-05-11T23:01:53.236 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.220305379, 2024-05-11T23:01:53.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020162342999999999, 2024-05-11T23:01:53.236 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:53.237 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:53.237 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:53.237 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:53.237 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:53.237 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:53.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25893 2024-05-11T23:01:53.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.237 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:53.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:53.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948546+0000", 2024-05-11T23:01:53.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:53.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948546+0000", 2024-05-11T23:01:53.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:53.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948546+0000", 2024-05-11T23:01:53.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3000000000000001e-07 2024-05-11T23:01:53.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:53.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948549+0000", 2024-05-11T23:01:53.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4040000000000002e-06 2024-05-11T23:01:53.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:53.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948555+0000", 2024-05-11T23:01:53.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7660000000000001e-06 2024-05-11T23:01:53.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:53.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948564+0000", 2024-05-11T23:01:53.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8580000000000006e-06 2024-05-11T23:01:53.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:53.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968699+0000", 2024-05-11T23:01:53.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020134927 2024-05-11T23:01:53.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:53.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968708+0000", 2024-05-11T23:01:53.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8580000000000006e-06 2024-05-11T23:01:53.240 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.240 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.240 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.240 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25896 3.17 3.e6b97637 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:53.240 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948572+0000", 2024-05-11T23:01:53.240 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.220279251, 2024-05-11T23:01:53.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020178925, 2024-05-11T23:01:53.240 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:53.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:53.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:53.241 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:53.241 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:53.241 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:53.241 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25896 2024-05-11T23:01:53.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.241 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:53.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:53.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948572+0000", 2024-05-11T23:01:53.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:53.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948572+0000", 2024-05-11T23:01:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948572+0000", 2024-05-11T23:01:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.96e-07 2024-05-11T23:01:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948575+0000", 2024-05-11T23:01:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4250000000000001e-06 2024-05-11T23:01:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:53.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948581+0000", 2024-05-11T23:01:53.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7070000000000003e-06 2024-05-11T23:01:53.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:53.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948586+0000", 2024-05-11T23:01:53.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0579999999999997e-06 2024-05-11T23:01:53.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:53.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968746+0000", 2024-05-11T23:01:53.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020159274000000001 2024-05-11T23:01:53.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:53.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968751+0000", 2024-05-11T23:01:53.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0649999999999998e-06 2024-05-11T23:01:53.244 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.244 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.244 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.244 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25900 3.17 3.5dda9cb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:53.244 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948813+0000", 2024-05-11T23:01:53.244 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.220037584, 2024-05-11T23:01:53.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019981407999999999, 2024-05-11T23:01:53.244 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:53.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:53.244 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:53.244 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:53.244 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:53.245 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:53.245 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25900 2024-05-11T23:01:53.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.245 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:53.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:53.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948813+0000", 2024-05-11T23:01:53.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:53.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948813+0000", 2024-05-11T23:01:53.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:53.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948814+0000", 2024-05-11T23:01:53.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7400000000000003e-07 2024-05-11T23:01:53.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:53.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948823+0000", 2024-05-11T23:01:53.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3430000000000002e-06 2024-05-11T23:01:53.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:53.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948831+0000", 2024-05-11T23:01:53.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1960000000000002e-06 2024-05-11T23:01:53.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:53.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948843+0000", 2024-05-11T23:01:53.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452999999999999e-05 2024-05-11T23:01:53.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:53.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968784+0000", 2024-05-11T23:01:53.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019940439000000001 2024-05-11T23:01:53.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:53.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968795+0000", 2024-05-11T23:01:53.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1303e-05 2024-05-11T23:01:53.248 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.248 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.248 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.248 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25903 3.17 3.e6b97637 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:53.248 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948852+0000", 2024-05-11T23:01:53.248 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.2199993339999999, 2024-05-11T23:01:53.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019968133999999998, 2024-05-11T23:01:53.248 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:53.248 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:53.248 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:53.248 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:53.249 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:53.249 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:53.249 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25903 2024-05-11T23:01:53.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.249 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:53.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:53.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948852+0000", 2024-05-11T23:01:53.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:53.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948852+0000", 2024-05-11T23:01:53.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:53.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948852+0000", 2024-05-11T23:01:53.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0900000000000002e-07 2024-05-11T23:01:53.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:53.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948854+0000", 2024-05-11T23:01:53.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.677e-06 2024-05-11T23:01:53.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:53.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948858+0000", 2024-05-11T23:01:53.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0779999999999997e-06 2024-05-11T23:01:53.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:53.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948861+0000", 2024-05-11T23:01:53.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3790000000000002e-06 2024-05-11T23:01:53.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:53.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968812+0000", 2024-05-11T23:01:53.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019950704999999999 2024-05-11T23:01:53.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:53.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968820+0000", 2024-05-11T23:01:53.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8860000000000004e-06 2024-05-11T23:01:53.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.252 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.252 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25907 3.17 3.5dda9cb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:53.252 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948879+0000", 2024-05-11T23:01:53.252 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.2199722959999999, 2024-05-11T23:01:53.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019955064000000002, 2024-05-11T23:01:53.252 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:53.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:53.252 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:53.252 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:53.253 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:53.253 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:53.253 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25907 2024-05-11T23:01:53.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.253 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:53.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:53.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948879+0000", 2024-05-11T23:01:53.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:53.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948879+0000", 2024-05-11T23:01:53.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:53.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:53.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948879+0000", 2024-05-11T23:01:53.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8599999999999998e-07 2024-05-11T23:01:53.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:53.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948881+0000", 2024-05-11T23:01:53.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1739999999999999e-06 2024-05-11T23:01:53.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:53.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948886+0000", 2024-05-11T23:01:53.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7600000000000002e-06 2024-05-11T23:01:53.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:53.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948889+0000", 2024-05-11T23:01:53.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.298e-06 2024-05-11T23:01:53.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:53.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968829+0000", 2024-05-11T23:01:53.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019939912000000001 2024-05-11T23:01:53.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:53.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:53.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:53.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968834+0000", 2024-05-11T23:01:53.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4340000000000002e-06 2024-05-11T23:01:53.256 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.256 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.256 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.256 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:53.256 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:53.256 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:53.257 INFO:teuthology.orchestra.run.smithi005.stdout:2212 trunc from 0xee59c57 to 0x9718f31 2024-05-11T23:01:53.257 INFO:teuthology.orchestra.run.smithi005.stdout:2213 write 0x5df1f1e thru 0x5dfee11 (0xcef4 bytes) 2024-05-11T23:01:53.259 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:01:53.260 INFO:teuthology.orchestra.run.smithi005.stdout:2214 write 0x84ac669 thru 0x84b031d (0x3cb5 bytes) 2024-05-11T23:01:53.260 INFO:teuthology.orchestra.run.smithi005.stdout:2215 read 0x824a38b thru 0x8251e89 (0x7aff bytes) 2024-05-11T23:01:53.260 INFO:teuthology.orchestra.run.smithi005.stdout:2216 write 0x36f46b9 thru 0x36ff4a8 (0xadf0 bytes) 2024-05-11T23:01:53.260 INFO:teuthology.orchestra.run.smithi005.stdout:2218 read 0x1f4bd15 thru 0x1f4e9c2 (0x2cae bytes) 2024-05-11T23:01:53.260 INFO:teuthology.orchestra.run.smithi005.stdout:2219 read 0x69eb9a2 thru 0x69f58a5 (0x9f04 bytes) 2024-05-11T23:01:53.260 INFO:teuthology.orchestra.run.smithi005.stdout:2220 read 0x8fd6839 thru 0x8fdbee5 (0x56ad bytes) 2024-05-11T23:01:53.260 INFO:teuthology.orchestra.run.smithi005.stdout:2221 clone 16 order 19 su 32768 sc 10 2024-05-11T23:01:53.260 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:53.258+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:53.359 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:53.359+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:53.410 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:53.410 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:53.410 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:01:53.410 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:53.419 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:01:53.459 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:53.460+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:53.524 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:01:53.568 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:53.568+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:53.569 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:53.569 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:53.570 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:01:53.570 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:01:53.570 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:53.579 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:01:53.652 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:01:53.652 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:01:53.661 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:53.661+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:53.670 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:01:53.670 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:01:53.687 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:01:53.687 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:01:53.705 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:01:53.705 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:01:53.722 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:01:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:01:53.731 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:53.731 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:53.740 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:01:53.741 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:01:53.741 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:01:53.761 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:01:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:01:53.762 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:53.762+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:53.779 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:01:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:01:53.829 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-05-11T23:01:53.859 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:53.859 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:53.863 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:53.863+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:53.869 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:01:53.889 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone15 from 0x9718f31 (overlap 0x344687f) to 0xa74819 2024-05-11T23:01:53.964 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:53.965+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:54.065 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:54.065+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:54.070 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:54.070 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:54.085 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:01:54.166 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:54.167+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:54.210 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-05-11T23:01:54.245 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:54.245 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:54.257 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:01:54.267 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:54.267+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:54.311 INFO:teuthology.orchestra.run.smithi005.stdout:2222 write 0xd5a949d thru 0xd5b70db (0xdc3f bytes) 2024-05-11T23:01:54.315 INFO:teuthology.orchestra.run.smithi005.stdout:2223 writesame 0xb4618d0 thru 0xb4713a3 data_size 0x6(0xfad4 bytes) 2024-05-11T23:01:54.329 INFO:teuthology.orchestra.run.smithi005.stdout:2224 writesame 0xe5ad31f thru 0xe5b8a0a data_size 0x2e(0xb6ec bytes) 2024-05-11T23:01:54.329 INFO:teuthology.orchestra.run.smithi005.stdout:2227 read 0xa14b5d2 thru 0xa14ee8b (0x38ba bytes) 2024-05-11T23:01:54.329 INFO:teuthology.orchestra.run.smithi005.stdout:2228 read 0x6b8f032 thru 0x6b90ab2 (0x1a81 bytes) 2024-05-11T23:01:54.329 INFO:teuthology.orchestra.run.smithi005.stdout:2230 punch from 0x15f6dc to 0x16d28b, (0xdbaf bytes) 2024-05-11T23:01:54.329 INFO:teuthology.orchestra.run.smithi005.stdout:2231 read 0xcdb1571 thru 0xcdbcffa (0xba8a bytes) 2024-05-11T23:01:54.338 INFO:teuthology.orchestra.run.smithi005.stdout:2232 trunc from 0xe5b8a0b to 0x55f0119 2024-05-11T23:01:54.368 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:54.369+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:54.381 INFO:teuthology.orchestra.run.smithi005.stdout:2233 read 0x510621a thru 0x510f0bf (0x8ea6 bytes) 2024-05-11T23:01:54.381 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:54.381 INFO:teuthology.orchestra.run.smithi005.stdout:2235 punch from 0x15f3464 to 0x15f49f0, (0x158c bytes) 2024-05-11T23:01:54.390 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:54.390 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:54.397 INFO:teuthology.orchestra.run.smithi005.stdout:2236 writesame 0xc94896e thru 0xc94cf14 data_size 0x1(0x45a7 bytes) 2024-05-11T23:01:54.397 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:54.397 INFO:teuthology.orchestra.run.smithi005.stdout:2238 write 0xaebb09b thru 0xaec31d8 (0x813e bytes) 2024-05-11T23:01:54.401 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:01:54.403 INFO:teuthology.orchestra.run.smithi005.stdout:2239 trunc from 0xc94cf15 to 0x9e39241 2024-05-11T23:01:54.415 INFO:teuthology.orchestra.run.smithi005.stdout:2241 punch from 0x50a8039 to 0x50ae57a, (0x6541 bytes) 2024-05-11T23:01:54.415 INFO:teuthology.orchestra.run.smithi005.stdout:2242 read 0x5265017 thru 0x5273736 (0xe720 bytes) 2024-05-11T23:01:54.416 INFO:teuthology.orchestra.run.smithi005.stdout:2244 punch from 0x3e45d4a to 0x3e47d5e, (0x2014 bytes) 2024-05-11T23:01:54.416 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:54.416 INFO:teuthology.orchestra.run.smithi005.stdout:2246 read 0x11ee274 thru 0x11fb0b8 (0xce45 bytes) 2024-05-11T23:01:54.425 INFO:teuthology.orchestra.run.smithi005.stdout:2247 write 0xbb16baa thru 0xbb181a2 (0x15f9 bytes) 2024-05-11T23:01:54.431 INFO:teuthology.orchestra.run.smithi005.stdout:2248 clone 17 order 24 su 65536 sc 2 2024-05-11T23:01:54.469 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:54.469+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:54.512 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:54.512 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:54.521 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:01:54.570 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:54.571+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:54.670 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:54.671 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:54.671 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:54.671+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:54.679 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:01:54.772 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:54.772+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:54.796 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:54.796 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:54.806 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:01:54.874 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:54.874+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:54.907 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone16 from 0xbb181a3 (overlap 0x55f0119) to 0x1130038 2024-05-11T23:01:54.933 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:54.934 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:54.948 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:01:54.998 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:54.995+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:55.075 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:55.076+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:55.136 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:55.136 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:55.149 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:01:55.176 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:55.176+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:55.277 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:55.278+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:55.296 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-05-11T23:01:55.316 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:55.316 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:55.316 INFO:teuthology.orchestra.run.smithi005.stdout:2249 writesame 0x5d75e1b thru 0x5d82957 data_size 0x2f(0xcb3d bytes) 2024-05-11T23:01:55.328 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:01:55.378 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:55.378+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:55.391 INFO:teuthology.orchestra.run.smithi005.stdout:2250 read 0x96974 thru 0x9a416 (0x3aa3 bytes) 2024-05-11T23:01:55.391 INFO:teuthology.orchestra.run.smithi005.stdout:2251 read 0x845df53 thru 0x8466edb (0x8f89 bytes) 2024-05-11T23:01:55.392 INFO:teuthology.orchestra.run.smithi005.stdout:2252 punch from 0x42cf10a to 0x42d1974, (0x286a bytes) 2024-05-11T23:01:55.392 INFO:teuthology.orchestra.run.smithi005.stdout:2253 read 0x4f09899 thru 0x4f1446f (0xabd7 bytes) 2024-05-11T23:01:55.392 INFO:teuthology.orchestra.run.smithi005.stdout:2254 punch from 0x53dee94 to 0x53e6618, (0x7784 bytes) 2024-05-11T23:01:55.395 INFO:teuthology.orchestra.run.smithi005.stdout:2255 trunc from 0xbb181a3 to 0xe6db5f5 2024-05-11T23:01:55.400 INFO:teuthology.orchestra.run.smithi005.stdout:2256 read 0x771df42 thru 0x7723951 (0x5a10 bytes) 2024-05-11T23:01:55.400 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:55.401 INFO:teuthology.orchestra.run.smithi005.stdout:2259 trunc from 0xe6db5f5 to 0xdf50d81 2024-05-11T23:01:55.406 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:55.412 INFO:teuthology.orchestra.run.smithi005.stdout:2263 trunc from 0xdf50d81 to 0x891ae1c 2024-05-11T23:01:55.422 INFO:teuthology.orchestra.run.smithi005.stdout:2264 read 0x67ad0c2 thru 0x67b9a67 (0xc9a6 bytes) 2024-05-11T23:01:55.427 INFO:teuthology.orchestra.run.smithi005.stdout:2265 trunc from 0x891ae1c to 0xca7fd68 2024-05-11T23:01:55.432 INFO:teuthology.orchestra.run.smithi005.stdout:2266 punch from 0x4eaf64b to 0x4eb89a2, (0x9357 bytes) 2024-05-11T23:01:55.432 INFO:teuthology.orchestra.run.smithi005.stdout:2267 write 0x868be11 thru 0x868bf9e (0x18e bytes) 2024-05-11T23:01:55.434 INFO:teuthology.orchestra.run.smithi005.stdout:2268 punch from 0xb76ca8b to 0xb77c4de, (0xfa53 bytes) 2024-05-11T23:01:55.444 INFO:teuthology.orchestra.run.smithi005.stdout:2269 trunc from 0xca7fd68 to 0x3c1d2e8 2024-05-11T23:01:55.479 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:55.479+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:55.507 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:55.507 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:55.522 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:01:55.580 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:55.581+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:55.645 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:55.656 INFO:teuthology.orchestra.run.smithi005.stdout:2271 write 0x7ad77be thru 0x7ade3f9 (0x6c3c bytes) 2024-05-11T23:01:55.658 INFO:teuthology.orchestra.run.smithi005.stdout:2272 read 0xfa7065 thru 0xfb08e6 (0x9882 bytes) 2024-05-11T23:01:55.660 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:55.660 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:55.666 INFO:teuthology.orchestra.run.smithi005.stdout:2274 trunc from 0x7ade3fa to 0x1777e9f 2024-05-11T23:01:55.669 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:01:55.680 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:55.680+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:55.781 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:55.782+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:55.791 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:01:55.847 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-11T23:01:55.881 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:55.881 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:55.882 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:55.883+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:55.893 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:01:55.957 INFO:teuthology.orchestra.run.smithi005.stdout:2276 trunc from 0x1777e9f to 0x108c071 2024-05-11T23:01:55.962 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:01:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:01:55.976 INFO:teuthology.orchestra.run.smithi005.stdout:2277 write 0x60f4ce3 thru 0x60f508a (0x3a8 bytes) 2024-05-11T23:01:55.979 INFO:teuthology.orchestra.run.smithi005.stdout:2279 read 0x5ee28f3 thru 0x5eef238 (0xc946 bytes) 2024-05-11T23:01:55.979 INFO:teuthology.orchestra.run.smithi005.stdout:2280 write 0x5e2ab6c thru 0x5e33131 (0x85c6 bytes) 2024-05-11T23:01:55.983 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:55.984+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:55.984 INFO:teuthology.orchestra.run.smithi005.stdout:2281 read 0x32cfa3f thru 0x32d9641 (0x9c03 bytes) 2024-05-11T23:01:55.984 INFO:teuthology.orchestra.run.smithi005.stdout:2282 read 0x4b1c83c thru 0x4b22578 (0x5d3d bytes) 2024-05-11T23:01:55.984 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:55.984 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:55.987 INFO:teuthology.orchestra.run.smithi005.stdout:2286 trunc from 0x60f508b to 0x8d7792b 2024-05-11T23:01:55.988 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:01:55.988 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:01:55.992 INFO:teuthology.orchestra.run.smithi005.stdout:2287 read 0x3fc0b59 thru 0x3fc2140 (0x15e8 bytes) 2024-05-11T23:01:56.001 INFO:teuthology.orchestra.run.smithi005.stdout:2288 write 0xbd88832 thru 0xbd8cd61 (0x4530 bytes) 2024-05-11T23:01:56.005 INFO:teuthology.orchestra.run.smithi005.stdout:2290 trunc from 0xbd8cd62 to 0xb7527a5 2024-05-11T23:01:56.008 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:01:56.008 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:01:56.011 INFO:teuthology.orchestra.run.smithi005.stdout:2291 write 0x7168fdd thru 0x71704d0 (0x74f4 bytes) 2024-05-11T23:01:56.014 INFO:teuthology.orchestra.run.smithi005.stdout:2292 punch from 0x6f9b740 to 0x6fa5944, (0xa204 bytes) 2024-05-11T23:01:56.014 INFO:teuthology.orchestra.run.smithi005.stdout:2293 writesame 0xaf0950d thru 0xaf0fd39 data_size 0x1(0x682d bytes) 2024-05-11T23:01:56.027 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:56.027 INFO:teuthology.orchestra.run.smithi005.stdout:2295 writesame 0x8878205 thru 0x8886048 data_size 0x32(0xde44 bytes) 2024-05-11T23:01:56.031 INFO:teuthology.orchestra.run.smithi005.stdout:2296 punch from 0x1570408 to 0x157d1a5, (0xcd9d bytes) 2024-05-11T23:01:56.031 INFO:teuthology.orchestra.run.smithi005.stdout:2297 read 0x3455213 thru 0x3463b96 (0xe984 bytes) 2024-05-11T23:01:56.031 INFO:teuthology.orchestra.run.smithi005.stdout:2299 read 0x1dd7278 thru 0x1ddaa39 (0x37c2 bytes) 2024-05-11T23:01:56.031 INFO:teuthology.orchestra.run.smithi005.stdout:2300 read 0x62ce84c thru 0x62d53cc (0x6b81 bytes) 2024-05-11T23:01:56.031 INFO:teuthology.orchestra.run.smithi005.stdout:2301 write 0xaea0bed thru 0xaea15ea (0x9fe bytes) 2024-05-11T23:01:56.039 INFO:teuthology.orchestra.run.smithi005.stdout:2302 writesame 0xc34c524 thru 0xc34c983 data_size 0x10(0x460 bytes) 2024-05-11T23:01:56.039 INFO:teuthology.orchestra.run.smithi005.stdout:2304 punch from 0x566d73e to 0x567587a, (0x813c bytes) 2024-05-11T23:01:56.039 INFO:teuthology.orchestra.run.smithi005.stdout:2305 read 0x62c8479 thru 0x62d547e (0xd006 bytes) 2024-05-11T23:01:56.040 INFO:teuthology.orchestra.run.smithi005.stdout:2307 read 0x9e58d85 thru 0x9e6092f (0x7bab bytes) 2024-05-11T23:01:56.040 INFO:teuthology.orchestra.run.smithi005.stdout:2308 write 0x9a4cd91 thru 0x9a56a7c (0x9cec bytes) 2024-05-11T23:01:56.042 INFO:teuthology.orchestra.run.smithi005.stdout:2309 read 0xba56dc9 thru 0xba59382 (0x25ba bytes) 2024-05-11T23:01:56.042 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:56.043 INFO:teuthology.orchestra.run.smithi005.stdout:2311 trunc from 0xc34c984 to 0xde58a54 2024-05-11T23:01:56.045 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:56.045 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:56.049 INFO:teuthology.orchestra.run.smithi005.stdout:2312 punch from 0xc3bfbe7 to 0xc3c8c42, (0x905b bytes) 2024-05-11T23:01:56.049 INFO:teuthology.orchestra.run.smithi005.stdout:2313 write 0x5d4a945 thru 0x5d508cd (0x5f89 bytes) 2024-05-11T23:01:56.050 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:56.050 INFO:teuthology.orchestra.run.smithi005.stdout:2316 write 0x2379ad9 thru 0x23813e3 (0x790b bytes) 2024-05-11T23:01:56.054 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:56.054 INFO:teuthology.orchestra.run.smithi005.stdout:2319 write 0xd703eda thru 0xd70f168 (0xb28f bytes) 2024-05-11T23:01:56.056 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:01:56.057 INFO:teuthology.orchestra.run.smithi005.stdout:2320 writesame 0xdf3606 thru 0xdfb236 data_size 0x1(0x7c31 bytes) 2024-05-11T23:01:56.071 INFO:teuthology.orchestra.run.smithi005.stdout:2322 read 0xc20f1fd thru 0xc21d8f2 (0xe6f6 bytes) 2024-05-11T23:01:56.072 INFO:teuthology.orchestra.run.smithi005.stdout:2323 punch from 0x9dd4c50 to 0x9dd8f63, (0x4313 bytes) 2024-05-11T23:01:56.072 INFO:teuthology.orchestra.run.smithi005.stdout:2324 read 0x70011ca thru 0x70029f1 (0x1828 bytes) 2024-05-11T23:01:56.072 INFO:teuthology.orchestra.run.smithi005.stdout:2325 writesame 0xd120366 thru 0xd12b5e7 data_size 0x2(0xb282 bytes) 2024-05-11T23:01:56.084 INFO:teuthology.orchestra.run.smithi005.stdout:2327 read 0x4db4193 thru 0x4dbd96f (0x97dd bytes) 2024-05-11T23:01:56.092 INFO:teuthology.orchestra.run.smithi005.stdout:2328 trunc from 0xde58a54 to 0x7420fe5 2024-05-11T23:01:56.097 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:56.098+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:56.103 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:56.103 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:56.103 INFO:teuthology.orchestra.run.smithi005.stdout:2332 punch from 0x4107859 to 0x410c70a, (0x4eb1 bytes) 2024-05-11T23:01:56.103 INFO:teuthology.orchestra.run.smithi005.stdout:2333 clone 18 order 23 su 2097152 sc 15 2024-05-11T23:01:56.185 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:56.185+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:56.201 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:56.201 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:56.210 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:01:56.226 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:01:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:01:56.244 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:01:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:01:56.262 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:01:56.262 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:01:56.280 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:01:56.281 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:01:56.286 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:56.286+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:56.298 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:01:56.298 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:01:56.357 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:56.357 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:56.357 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:01:56.357 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:56.366 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:01:56.387 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:56.387+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:56.471 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:56.471 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:01:56.471 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:01:56.471 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:01:56.471 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:56.479 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:01:56.523 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:56.524+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:56.586 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:01:56.586 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:01:56.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:01:56.586 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:01:56.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.586 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:01:56.586 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:01:56.586 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 39.422080678999997, 2024-05-11T23:01:56.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:01:56.587 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:56.587 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:56.587 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:56.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:56.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:01:56.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:01:56.587 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:01:56.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.587 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:56.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:56.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:01:56.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:56.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:01:56.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:56.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:01:56.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:01:56.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:56.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:01:56.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:01:56.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:56.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:01:56.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:01:56.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:56.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:01:56.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:01:56.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:01:56.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:01:56.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:01:56.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:01:56.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:01:56.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:01:56.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:01:56.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:01:56.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:01:56.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:01:56.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:01:56.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:01:56.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:01:56.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:56.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:01:56.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:01:56.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:01:56.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:01:56.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:01:56.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:01:56.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:01:56.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:01:56.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:56.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:01:56.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:01:56.592 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.592 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.592 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.593 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31872 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:56.593 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506353+0000", 2024-05-11T23:01:56.593 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0789794590000001, 2024-05-11T23:01:56.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040469576, 2024-05-11T23:01:56.593 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:56.593 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:56.593 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:56.593 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:56.593 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:56.593 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:56.593 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31872 2024-05-11T23:01:56.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.593 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:56.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:56.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506353+0000", 2024-05-11T23:01:56.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:56.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506353+0000", 2024-05-11T23:01:56.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:56.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506353+0000", 2024-05-11T23:01:56.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:01:56.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:56.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506356+0000", 2024-05-11T23:01:56.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0339999999999999e-06 2024-05-11T23:01:56.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:56.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506362+0000", 2024-05-11T23:01:56.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:01:56.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:56.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506367+0000", 2024-05-11T23:01:56.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3129999999999998e-06 2024-05-11T23:01:56.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546813+0000", 2024-05-11T23:01:56.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040445980999999999 2024-05-11T23:01:56.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:56.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546823+0000", 2024-05-11T23:01:56.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7669999999999998e-06 2024-05-11T23:01:56.596 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.596 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.596 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.596 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31873 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:56.597 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506374+0000", 2024-05-11T23:01:56.597 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0789589199999998, 2024-05-11T23:01:56.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040466416999999998, 2024-05-11T23:01:56.597 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:56.597 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:56.597 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:56.597 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:56.597 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:56.597 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:56.597 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31873 2024-05-11T23:01:56.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.597 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:56.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:56.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506374+0000", 2024-05-11T23:01:56.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:56.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506374+0000", 2024-05-11T23:01:56.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:56.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506374+0000", 2024-05-11T23:01:56.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1600000000000002e-07 2024-05-11T23:01:56.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:56.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506376+0000", 2024-05-11T23:01:56.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.075e-06 2024-05-11T23:01:56.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:56.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506383+0000", 2024-05-11T23:01:56.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4189999999999999e-06 2024-05-11T23:01:56.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:56.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506388+0000", 2024-05-11T23:01:56.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.147e-06 2024-05-11T23:01:56.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546836+0000", 2024-05-11T23:01:56.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040448005000000002 2024-05-11T23:01:56.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:56.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546840+0000", 2024-05-11T23:01:56.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3549999999999998e-06 2024-05-11T23:01:56.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.600 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.600 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31874 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:56.600 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506394+0000", 2024-05-11T23:01:56.600 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0789385989999998, 2024-05-11T23:01:56.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040469948999999998, 2024-05-11T23:01:56.600 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:56.600 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:56.601 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:56.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:56.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:56.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:56.601 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31874 2024-05-11T23:01:56.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.601 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:56.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:56.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506394+0000", 2024-05-11T23:01:56.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:56.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506394+0000", 2024-05-11T23:01:56.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:56.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506394+0000", 2024-05-11T23:01:56.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0900000000000002e-07 2024-05-11T23:01:56.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:56.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506396+0000", 2024-05-11T23:01:56.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0669999999999999e-06 2024-05-11T23:01:56.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:56.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506403+0000", 2024-05-11T23:01:56.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4509999999999997e-06 2024-05-11T23:01:56.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:56.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506408+0000", 2024-05-11T23:01:56.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0810000000000001e-06 2024-05-11T23:01:56.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546855+0000", 2024-05-11T23:01:56.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040446825999999998 2024-05-11T23:01:56.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:56.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546864+0000", 2024-05-11T23:01:56.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.115e-06 2024-05-11T23:01:56.603 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.603 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.604 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.604 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31878 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:56.604 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506510+0000", 2024-05-11T23:01:56.604 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.078822712, 2024-05-11T23:01:56.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040458415999999997, 2024-05-11T23:01:56.604 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:56.604 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:56.604 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:56.604 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:56.604 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:56.604 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:56.604 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31878 2024-05-11T23:01:56.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.604 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:56.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:56.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506510+0000", 2024-05-11T23:01:56.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:56.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506510+0000", 2024-05-11T23:01:56.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:56.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506510+0000", 2024-05-11T23:01:56.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-07 2024-05-11T23:01:56.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:56.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506512+0000", 2024-05-11T23:01:56.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0470000000000001e-06 2024-05-11T23:01:56.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:56.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506519+0000", 2024-05-11T23:01:56.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3130000000000004e-06 2024-05-11T23:01:56.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:56.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506523+0000", 2024-05-11T23:01:56.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6999999999999999e-06 2024-05-11T23:01:56.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546959+0000", 2024-05-11T23:01:56.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040436110999999997 2024-05-11T23:01:56.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:56.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546968+0000", 2024-05-11T23:01:56.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.884e-06 2024-05-11T23:01:56.607 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.607 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.607 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.608 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31880 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:56.608 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506550+0000", 2024-05-11T23:01:56.608 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0787828370000003, 2024-05-11T23:01:56.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040454849000000001, 2024-05-11T23:01:56.608 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:56.608 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:56.608 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:56.608 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:56.608 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:56.608 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:56.608 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31880 2024-05-11T23:01:56.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.608 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:56.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:56.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506550+0000", 2024-05-11T23:01:56.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:56.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506550+0000", 2024-05-11T23:01:56.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:56.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506550+0000", 2024-05-11T23:01:56.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7300000000000002e-07 2024-05-11T23:01:56.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506552+0000", 2024-05-11T23:01:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.012e-06 2024-05-11T23:01:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506558+0000", 2024-05-11T23:01:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:01:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506563+0000", 2024-05-11T23:01:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0000000000000004e-06 2024-05-11T23:01:56.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546997+0000", 2024-05-11T23:01:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040433416 2024-05-11T23:01:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547005+0000", 2024-05-11T23:01:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9440000000000005e-06 2024-05-11T23:01:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31881 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:56.612 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506570+0000", 2024-05-11T23:01:56.612 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0787626269999997, 2024-05-11T23:01:56.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040460073999999999, 2024-05-11T23:01:56.612 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:56.612 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:56.612 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:56.612 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:56.612 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:56.612 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:56.612 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31881 2024-05-11T23:01:56.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.612 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:56.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:56.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506570+0000", 2024-05-11T23:01:56.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:56.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506570+0000", 2024-05-11T23:01:56.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:56.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506570+0000", 2024-05-11T23:01:56.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:01:56.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:56.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506577+0000", 2024-05-11T23:01:56.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0099999999999998e-06 2024-05-11T23:01:56.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:56.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506584+0000", 2024-05-11T23:01:56.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7689999999999999e-06 2024-05-11T23:01:56.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:56.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506589+0000", 2024-05-11T23:01:56.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2340000000000002e-06 2024-05-11T23:01:56.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547022+0000", 2024-05-11T23:01:56.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040432526000000003 2024-05-11T23:01:56.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:56.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547030+0000", 2024-05-11T23:01:56.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1650000000000006e-06 2024-05-11T23:01:56.615 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.615 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.615 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.616 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31882 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:56.616 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506596+0000", 2024-05-11T23:01:56.616 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0787364070000001, 2024-05-11T23:01:56.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040473955999999998, 2024-05-11T23:01:56.616 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:56.616 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:56.616 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:56.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:56.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:56.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:56.616 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31882 2024-05-11T23:01:56.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.616 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:56.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:56.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506596+0000", 2024-05-11T23:01:56.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:56.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506596+0000", 2024-05-11T23:01:56.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:56.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506597+0000", 2024-05-11T23:01:56.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5299999999999999e-07 2024-05-11T23:01:56.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:56.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506599+0000", 2024-05-11T23:01:56.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2900000000000001e-06 2024-05-11T23:01:56.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:56.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506605+0000", 2024-05-11T23:01:56.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5259999999999999e-06 2024-05-11T23:01:56.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:56.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506610+0000", 2024-05-11T23:01:56.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1030000000000001e-06 2024-05-11T23:01:56.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547061+0000", 2024-05-11T23:01:56.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040450451999999998 2024-05-11T23:01:56.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:56.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547070+0000", 2024-05-11T23:01:56.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.132e-06 2024-05-11T23:01:56.619 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.619 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.619 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.619 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31883 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:56.619 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506618+0000", 2024-05-11T23:01:56.619 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.078714594, 2024-05-11T23:01:56.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040494963000000002, 2024-05-11T23:01:56.620 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:56.620 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:56.620 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:56.620 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:56.620 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:56.620 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:56.620 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31883 2024-05-11T23:01:56.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.620 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:56.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:56.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506618+0000", 2024-05-11T23:01:56.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:56.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506618+0000", 2024-05-11T23:01:56.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:56.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506619+0000", 2024-05-11T23:01:56.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1799999999999995e-07 2024-05-11T23:01:56.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:56.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506621+0000", 2024-05-11T23:01:56.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2879999999999999e-06 2024-05-11T23:01:56.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:56.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506628+0000", 2024-05-11T23:01:56.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9600000000000003e-06 2024-05-11T23:01:56.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:56.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506633+0000", 2024-05-11T23:01:56.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4229999999999996e-06 2024-05-11T23:01:56.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547102+0000", 2024-05-11T23:01:56.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040469172999999997 2024-05-11T23:01:56.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:56.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547113+0000", 2024-05-11T23:01:56.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0501e-05 2024-05-11T23:01:56.623 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.623 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.623 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.623 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31884 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:56.623 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506640+0000", 2024-05-11T23:01:56.623 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0786923130000003, 2024-05-11T23:01:56.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040498244000000003, 2024-05-11T23:01:56.623 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:56.623 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:56.623 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:56.623 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:56.623 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:56.623 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:56.624 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31884 2024-05-11T23:01:56.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.624 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:56.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:56.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506640+0000", 2024-05-11T23:01:56.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:56.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506640+0000", 2024-05-11T23:01:56.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:56.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506641+0000", 2024-05-11T23:01:56.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3000000000000001e-07 2024-05-11T23:01:56.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:56.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506643+0000", 2024-05-11T23:01:56.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0200000000000001e-06 2024-05-11T23:01:56.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:56.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506649+0000", 2024-05-11T23:01:56.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1689999999999997e-06 2024-05-11T23:01:56.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:56.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506654+0000", 2024-05-11T23:01:56.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3800000000000002e-06 2024-05-11T23:01:56.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547131+0000", 2024-05-11T23:01:56.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040476810000000002 2024-05-11T23:01:56.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:56.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547138+0000", 2024-05-11T23:01:56.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.435e-06 2024-05-11T23:01:56.626 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.626 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.626 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.627 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31885 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:56.627 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506660+0000", 2024-05-11T23:01:56.627 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0786721379999999, 2024-05-11T23:01:56.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040504498, 2024-05-11T23:01:56.627 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:56.627 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:56.627 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:56.627 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:56.627 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:56.627 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:56.627 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31885 2024-05-11T23:01:56.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.627 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:56.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:56.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506660+0000", 2024-05-11T23:01:56.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:56.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506660+0000", 2024-05-11T23:01:56.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:56.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506661+0000", 2024-05-11T23:01:56.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4900000000000001e-07 2024-05-11T23:01:56.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:56.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506663+0000", 2024-05-11T23:01:56.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0660000000000002e-06 2024-05-11T23:01:56.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:56.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506669+0000", 2024-05-11T23:01:56.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3829999999999997e-06 2024-05-11T23:01:56.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:56.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506675+0000", 2024-05-11T23:01:56.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3750000000000002e-06 2024-05-11T23:01:56.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547156+0000", 2024-05-11T23:01:56.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040481389 2024-05-11T23:01:56.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:56.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547165+0000", 2024-05-11T23:01:56.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-05-11T23:01:56.630 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.630 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.630 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.630 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31886 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:56.630 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506681+0000", 2024-05-11T23:01:56.630 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0786517079999998, 2024-05-11T23:01:56.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040511004000000003, 2024-05-11T23:01:56.631 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:56.631 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:56.631 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:56.631 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:56.631 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:56.631 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:56.631 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31886 2024-05-11T23:01:56.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.631 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:56.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:56.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506681+0000", 2024-05-11T23:01:56.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:56.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506681+0000", 2024-05-11T23:01:56.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:56.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506681+0000", 2024-05-11T23:01:56.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:01:56.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:56.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506683+0000", 2024-05-11T23:01:56.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-05-11T23:01:56.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:56.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506703+0000", 2024-05-11T23:01:56.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0224000000000002e-05 2024-05-11T23:01:56.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:56.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506709+0000", 2024-05-11T23:01:56.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5799999999999999e-06 2024-05-11T23:01:56.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547183+0000", 2024-05-11T23:01:56.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040474060999999999 2024-05-11T23:01:56.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:56.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547192+0000", 2024-05-11T23:01:56.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6910000000000003e-06 2024-05-11T23:01:56.634 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.634 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.634 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.634 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31887 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:56.634 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506716+0000", 2024-05-11T23:01:56.634 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0786168849999997, 2024-05-11T23:01:56.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040522425000000001, 2024-05-11T23:01:56.634 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:56.634 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:56.634 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:56.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:56.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31887 2024-05-11T23:01:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506716+0000", 2024-05-11T23:01:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506716+0000", 2024-05-11T23:01:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:56.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506716+0000", 2024-05-11T23:01:56.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8799999999999998e-07 2024-05-11T23:01:56.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:56.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506721+0000", 2024-05-11T23:01:56.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5630000000000002e-06 2024-05-11T23:01:56.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:56.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506733+0000", 2024-05-11T23:01:56.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2418999999999999e-05 2024-05-11T23:01:56.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:56.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506739+0000", 2024-05-11T23:01:56.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5149999999999997e-06 2024-05-11T23:01:56.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547229+0000", 2024-05-11T23:01:56.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040490809000000003 2024-05-11T23:01:56.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:56.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547238+0000", 2024-05-11T23:01:56.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7309999999999999e-06 2024-05-11T23:01:56.637 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.637 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.637 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.638 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31888 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:56.638 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506745+0000", 2024-05-11T23:01:56.638 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0785871460000003, 2024-05-11T23:01:56.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040530374000000001, 2024-05-11T23:01:56.638 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:56.638 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:56.638 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:56.638 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:56.638 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:56.638 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:56.638 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31888 2024-05-11T23:01:56.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.639 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:56.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:56.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506745+0000", 2024-05-11T23:01:56.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:56.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506745+0000", 2024-05-11T23:01:56.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:56.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506746+0000", 2024-05-11T23:01:56.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2599999999999998e-07 2024-05-11T23:01:56.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:56.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506748+0000", 2024-05-11T23:01:56.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2800000000000002e-06 2024-05-11T23:01:56.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:56.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506769+0000", 2024-05-11T23:01:56.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.137e-05 2024-05-11T23:01:56.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:56.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506779+0000", 2024-05-11T23:01:56.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4909999999999994e-06 2024-05-11T23:01:56.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547270+0000", 2024-05-11T23:01:56.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040491335000000003 2024-05-11T23:01:56.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:56.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547276+0000", 2024-05-11T23:01:56.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4720000000000003e-06 2024-05-11T23:01:56.641 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.641 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.641 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.642 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31889 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:56.642 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506788+0000", 2024-05-11T23:01:56.642 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0785444359999996, 2024-05-11T23:01:56.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040514673000000001, 2024-05-11T23:01:56.642 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:56.642 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:56.642 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:56.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:56.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:56.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:56.642 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31889 2024-05-11T23:01:56.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.642 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:56.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:56.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506788+0000", 2024-05-11T23:01:56.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:56.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506788+0000", 2024-05-11T23:01:56.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:56.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506789+0000", 2024-05-11T23:01:56.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1099999999999996e-07 2024-05-11T23:01:56.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:56.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506791+0000", 2024-05-11T23:01:56.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7030000000000002e-06 2024-05-11T23:01:56.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:56.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506799+0000", 2024-05-11T23:01:56.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1899999999999998e-06 2024-05-11T23:01:56.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:56.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506804+0000", 2024-05-11T23:01:56.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2549999999999997e-06 2024-05-11T23:01:56.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547294+0000", 2024-05-11T23:01:56.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040489875000000002 2024-05-11T23:01:56.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:56.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547303+0000", 2024-05-11T23:01:56.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1390000000000001e-06 2024-05-11T23:01:56.645 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.645 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.645 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.646 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31890 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:56.646 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506811+0000", 2024-05-11T23:01:56.646 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0785219169999998, 2024-05-11T23:01:56.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040521258999999997, 2024-05-11T23:01:56.646 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:56.646 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:56.646 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:56.646 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:56.646 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:56.646 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:56.646 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31890 2024-05-11T23:01:56.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.646 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:56.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:56.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506811+0000", 2024-05-11T23:01:56.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:56.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506811+0000", 2024-05-11T23:01:56.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:56.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506811+0000", 2024-05-11T23:01:56.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999998e-07 2024-05-11T23:01:56.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:56.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506816+0000", 2024-05-11T23:01:56.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9459999999999997e-06 2024-05-11T23:01:56.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:56.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506823+0000", 2024-05-11T23:01:56.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.153e-06 2024-05-11T23:01:56.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:56.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506840+0000", 2024-05-11T23:01:56.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6793e-05 2024-05-11T23:01:56.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547319+0000", 2024-05-11T23:01:56.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040478675999999998 2024-05-11T23:01:56.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:56.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547332+0000", 2024-05-11T23:01:56.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3236e-05 2024-05-11T23:01:56.649 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.649 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.649 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.649 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31891 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:56.650 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506865+0000", 2024-05-11T23:01:56.650 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0784674929999998, 2024-05-11T23:01:56.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040491271000000002, 2024-05-11T23:01:56.650 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:56.650 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:56.650 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:56.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:56.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:56.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:56.650 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31891 2024-05-11T23:01:56.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.650 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:56.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:56.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506865+0000", 2024-05-11T23:01:56.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:56.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506865+0000", 2024-05-11T23:01:56.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:56.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506866+0000", 2024-05-11T23:01:56.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0399999999999996e-07 2024-05-11T23:01:56.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:56.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506868+0000", 2024-05-11T23:01:56.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6979999999999998e-06 2024-05-11T23:01:56.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:56.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506876+0000", 2024-05-11T23:01:56.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2169999999999994e-06 2024-05-11T23:01:56.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:56.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506889+0000", 2024-05-11T23:01:56.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2497000000000001e-05 2024-05-11T23:01:56.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547350+0000", 2024-05-11T23:01:56.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040460782000000001 2024-05-11T23:01:56.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:56.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547356+0000", 2024-05-11T23:01:56.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5729999999999996e-06 2024-05-11T23:01:56.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.653 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.653 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31892 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:56.653 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506897+0000", 2024-05-11T23:01:56.653 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0784350439999999, 2024-05-11T23:01:56.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040501916999999998, 2024-05-11T23:01:56.653 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:56.653 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:56.653 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:56.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:56.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:56.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:56.654 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31892 2024-05-11T23:01:56.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.654 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:56.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:56.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506897+0000", 2024-05-11T23:01:56.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:56.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506897+0000", 2024-05-11T23:01:56.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:56.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506898+0000", 2024-05-11T23:01:56.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0400000000000002e-07 2024-05-11T23:01:56.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:56.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506900+0000", 2024-05-11T23:01:56.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.26e-06 2024-05-11T23:01:56.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:56.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506910+0000", 2024-05-11T23:01:56.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0237999999999999e-05 2024-05-11T23:01:56.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:56.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506916+0000", 2024-05-11T23:01:56.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6849999999999998e-06 2024-05-11T23:01:56.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547388+0000", 2024-05-11T23:01:56.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040472373999999998 2024-05-11T23:01:56.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:56.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547399+0000", 2024-05-11T23:01:56.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0956e-05 2024-05-11T23:01:56.656 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.657 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.657 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.657 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31893 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:56.657 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506923+0000", 2024-05-11T23:01:56.657 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0784095630000001, 2024-05-11T23:01:56.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040510939000000003, 2024-05-11T23:01:56.657 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:56.657 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:56.657 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:56.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:56.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:56.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:56.657 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31893 2024-05-11T23:01:56.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.658 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:56.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:56.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506923+0000", 2024-05-11T23:01:56.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:56.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506923+0000", 2024-05-11T23:01:56.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:56.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506923+0000", 2024-05-11T23:01:56.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2300000000000002e-07 2024-05-11T23:01:56.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:56.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506936+0000", 2024-05-11T23:01:56.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2639e-05 2024-05-11T23:01:56.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:56.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506949+0000", 2024-05-11T23:01:56.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2866e-05 2024-05-11T23:01:56.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:56.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506955+0000", 2024-05-11T23:01:56.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1530000000000002e-06 2024-05-11T23:01:56.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547425+0000", 2024-05-11T23:01:56.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040469752999999997 2024-05-11T23:01:56.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:56.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547434+0000", 2024-05-11T23:01:56.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1050000000000001e-06 2024-05-11T23:01:56.660 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.660 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.660 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.661 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31894 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:56.661 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506964+0000", 2024-05-11T23:01:56.661 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.0783689440000002, 2024-05-11T23:01:56.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040492497000000002, 2024-05-11T23:01:56.661 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:01:56.661 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:01:56.661 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:01:56.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:01:56.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:01:56.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:56.661 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31894 2024-05-11T23:01:56.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.661 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:01:56.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:01:56.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506964+0000", 2024-05-11T23:01:56.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:01:56.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506964+0000", 2024-05-11T23:01:56.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:01:56.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:01:56.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506964+0000", 2024-05-11T23:01:56.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0100000000000005e-07 2024-05-11T23:01:56.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:01:56.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506967+0000", 2024-05-11T23:01:56.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.475e-06 2024-05-11T23:01:56.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:01:56.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506979+0000", 2024-05-11T23:01:56.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1847e-05 2024-05-11T23:01:56.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:01:56.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506984+0000", 2024-05-11T23:01:56.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7599999999999999e-06 2024-05-11T23:01:56.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:01:56.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547448+0000", 2024-05-11T23:01:56.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040463784000000003 2024-05-11T23:01:56.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:01:56.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:01:56.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:01:56.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547456+0000", 2024-05-11T23:01:56.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0299999999999994e-06 2024-05-11T23:01:56.664 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.664 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.664 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.664 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:01:56.664 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:01:56.664 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:01:56.666 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:56.666+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:56.666 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:01:56.766 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:56.767+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:56.780 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:56.780 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:56.780 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:01:56.781 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:56.790 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:01:56.844 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:01:56.844 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:01:56.845 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:01:56.845 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-05-11T23:01:56.848 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-05-11T23:01:56.907 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:56.908 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:56.908 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:01:56.908 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:01:56.908 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:56.908 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:56.908+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:56.917 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:01:56.924 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone17 from 0x7420fe5 (overlap 0x108c071) to 0x0 2024-05-11T23:01:56.969 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:56.969+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:57.010 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-05-11T23:01:57.069 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:57.070+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:57.078 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:57.078 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:01:57.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:01:57.078 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:01:57.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.079 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25833 3.17 3:ec6e9d67:::rbd_data.111e70525772.000000000000001a:e [read 331776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.079 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.947574+0000", 2024-05-11T23:01:57.079 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.1290475640000004, 2024-05-11T23:01:57.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019999618, 2024-05-11T23:01:57.079 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.079 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.079 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.079 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.079 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.079 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.079 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25833 2024-05-11T23:01:57.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.079 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947574+0000", 2024-05-11T23:01:57.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947574+0000", 2024-05-11T23:01:57.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947574+0000", 2024-05-11T23:01:57.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2800000000000002e-07 2024-05-11T23:01:57.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947576+0000", 2024-05-11T23:01:57.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.181e-06 2024-05-11T23:01:57.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947582+0000", 2024-05-11T23:01:57.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3720000000000001e-06 2024-05-11T23:01:57.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947587+0000", 2024-05-11T23:01:57.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7099999999999998e-06 2024-05-11T23:01:57.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967403+0000", 2024-05-11T23:01:57.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019815869 2024-05-11T23:01:57.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967476+0000", 2024-05-11T23:01:57.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2751999999999997e-05 2024-05-11T23:01:57.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967573+0000", 2024-05-11T23:01:57.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7306000000000007e-05 2024-05-11T23:01:57.082 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.082 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.082 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.083 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25837 3.17 3:ed395bba:::rbd_data.111e70525772.000000000000000e:e [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.083 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.947601+0000", 2024-05-11T23:01:57.083 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.129019875, 2024-05-11T23:01:57.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020075396999999998, 2024-05-11T23:01:57.083 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.083 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.083 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.083 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.083 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.083 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.083 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25837 2024-05-11T23:01:57.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.084 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947601+0000", 2024-05-11T23:01:57.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947601+0000", 2024-05-11T23:01:57.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947602+0000", 2024-05-11T23:01:57.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.06e-07 2024-05-11T23:01:57.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947604+0000", 2024-05-11T23:01:57.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4930000000000001e-06 2024-05-11T23:01:57.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947611+0000", 2024-05-11T23:01:57.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:01:57.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947616+0000", 2024-05-11T23:01:57.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9289999999999997e-06 2024-05-11T23:01:57.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967584+0000", 2024-05-11T23:01:57.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019968769000000001 2024-05-11T23:01:57.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967632+0000", 2024-05-11T23:01:57.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7474e-05 2024-05-11T23:01:57.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967677+0000", 2024-05-11T23:01:57.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4669999999999998e-05 2024-05-11T23:01:57.087 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.087 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.087 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.087 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25840 3.17 3:ec6e9d67:::rbd_data.111e70525772.000000000000001a:e [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.087 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.947627+0000", 2024-05-11T23:01:57.088 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.128994294, 2024-05-11T23:01:57.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020165684999999999, 2024-05-11T23:01:57.088 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.088 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.088 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.088 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.088 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.088 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.088 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25840 2024-05-11T23:01:57.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.088 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947627+0000", 2024-05-11T23:01:57.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947627+0000", 2024-05-11T23:01:57.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947627+0000", 2024-05-11T23:01:57.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4199999999999996e-07 2024-05-11T23:01:57.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947630+0000", 2024-05-11T23:01:57.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.311e-06 2024-05-11T23:01:57.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947636+0000", 2024-05-11T23:01:57.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2790000000000004e-06 2024-05-11T23:01:57.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947641+0000", 2024-05-11T23:01:57.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9280000000000001e-06 2024-05-11T23:01:57.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967707+0000", 2024-05-11T23:01:57.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020066148999999998 2024-05-11T23:01:57.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967748+0000", 2024-05-11T23:01:57.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1247e-05 2024-05-11T23:01:57.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967792+0000", 2024-05-11T23:01:57.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4229e-05 2024-05-11T23:01:57.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.092 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.092 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25844 3.17 3:ed395bba:::rbd_data.111e70525772.000000000000000e:e [read 364544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.092 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.947768+0000", 2024-05-11T23:01:57.092 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.1288530769999996, 2024-05-11T23:01:57.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020136108999999999, 2024-05-11T23:01:57.092 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.092 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.092 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.092 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.092 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.092 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.093 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25844 2024-05-11T23:01:57.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.093 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947768+0000", 2024-05-11T23:01:57.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947768+0000", 2024-05-11T23:01:57.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947768+0000", 2024-05-11T23:01:57.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8100000000000003e-07 2024-05-11T23:01:57.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947771+0000", 2024-05-11T23:01:57.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4420000000000001e-06 2024-05-11T23:01:57.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947778+0000", 2024-05-11T23:01:57.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.888e-06 2024-05-11T23:01:57.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947782+0000", 2024-05-11T23:01:57.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9790000000000004e-06 2024-05-11T23:01:57.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967821+0000", 2024-05-11T23:01:57.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020038944999999999 2024-05-11T23:01:57.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967862+0000", 2024-05-11T23:01:57.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1242999999999998e-05 2024-05-11T23:01:57.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967904+0000", 2024-05-11T23:01:57.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2131e-05 2024-05-11T23:01:57.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.096 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.096 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25854 3.17 3:ec6e9d67:::rbd_data.111e70525772.000000000000001a:e [read 380928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.096 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948270+0000", 2024-05-11T23:01:57.096 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.1283510459999997, 2024-05-11T23:01:57.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020068936999999998, 2024-05-11T23:01:57.097 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.097 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.097 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.097 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.097 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.097 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.097 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25854 2024-05-11T23:01:57.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.097 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948270+0000", 2024-05-11T23:01:57.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948270+0000", 2024-05-11T23:01:57.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948271+0000", 2024-05-11T23:01:57.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2399999999999998e-07 2024-05-11T23:01:57.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948273+0000", 2024-05-11T23:01:57.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5299999999999999e-06 2024-05-11T23:01:57.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948281+0000", 2024-05-11T23:01:57.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3280000000000006e-06 2024-05-11T23:01:57.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948287+0000", 2024-05-11T23:01:57.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6060000000000002e-06 2024-05-11T23:01:57.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968223+0000", 2024-05-11T23:01:57.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019936344000000002 2024-05-11T23:01:57.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968290+0000", 2024-05-11T23:01:57.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6576999999999996e-05 2024-05-11T23:01:57.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968339+0000", 2024-05-11T23:01:57.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9027999999999999e-05 2024-05-11T23:01:57.100 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.100 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.100 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.101 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25858 3.17 3.5dda9cb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.101 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948295+0000", 2024-05-11T23:01:57.101 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.1283258700000003, 2024-05-11T23:01:57.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020076373000000002, 2024-05-11T23:01:57.101 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.101 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.101 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:57.101 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.101 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.101 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.101 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25858 2024-05-11T23:01:57.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.102 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948295+0000", 2024-05-11T23:01:57.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948295+0000", 2024-05-11T23:01:57.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948296+0000", 2024-05-11T23:01:57.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7100000000000002e-07 2024-05-11T23:01:57.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948298+0000", 2024-05-11T23:01:57.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6089999999999999e-06 2024-05-11T23:01:57.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948305+0000", 2024-05-11T23:01:57.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0520000000000004e-06 2024-05-11T23:01:57.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948310+0000", 2024-05-11T23:01:57.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9239999999999998e-06 2024-05-11T23:01:57.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968366+0000", 2024-05-11T23:01:57.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020055988 2024-05-11T23:01:57.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968372+0000", 2024-05-11T23:01:57.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3290000000000001e-06 2024-05-11T23:01:57.104 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.104 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.104 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.104 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25861 3.17 3.e6b97637 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.104 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948318+0000", 2024-05-11T23:01:57.105 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.1283032139999998, 2024-05-11T23:01:57.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020083304999999999, 2024-05-11T23:01:57.105 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.105 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.105 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:57.105 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.105 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.105 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.105 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25861 2024-05-11T23:01:57.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.105 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948318+0000", 2024-05-11T23:01:57.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948318+0000", 2024-05-11T23:01:57.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948318+0000", 2024-05-11T23:01:57.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:01:57.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948321+0000", 2024-05-11T23:01:57.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.26e-06 2024-05-11T23:01:57.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948343+0000", 2024-05-11T23:01:57.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2563999999999999e-05 2024-05-11T23:01:57.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948351+0000", 2024-05-11T23:01:57.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6490000000000008e-06 2024-05-11T23:01:57.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968397+0000", 2024-05-11T23:01:57.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020045754999999998 2024-05-11T23:01:57.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968401+0000", 2024-05-11T23:01:57.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6519999999999997e-06 2024-05-11T23:01:57.108 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.108 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.108 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.108 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25865 3.17 3.5dda9cb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.108 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948361+0000", 2024-05-11T23:01:57.108 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.1282601549999995, 2024-05-11T23:01:57.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020065274000000001, 2024-05-11T23:01:57.109 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.109 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:57.109 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.109 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.109 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.109 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25865 2024-05-11T23:01:57.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.109 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948361+0000", 2024-05-11T23:01:57.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948361+0000", 2024-05-11T23:01:57.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948362+0000", 2024-05-11T23:01:57.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2900000000000003e-07 2024-05-11T23:01:57.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948365+0000", 2024-05-11T23:01:57.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0769999999999999e-06 2024-05-11T23:01:57.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948372+0000", 2024-05-11T23:01:57.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6299999999999998e-06 2024-05-11T23:01:57.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948377+0000", 2024-05-11T23:01:57.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0139999999999998e-06 2024-05-11T23:01:57.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968422+0000", 2024-05-11T23:01:57.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020044472000000001 2024-05-11T23:01:57.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968426+0000", 2024-05-11T23:01:57.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4519999999999999e-06 2024-05-11T23:01:57.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.112 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.112 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25868 3.17 3.e6b97637 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.112 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948385+0000", 2024-05-11T23:01:57.112 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.1282361600000002, 2024-05-11T23:01:57.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020082010000000001, 2024-05-11T23:01:57.112 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.112 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:57.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.113 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25868 2024-05-11T23:01:57.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.113 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948385+0000", 2024-05-11T23:01:57.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948385+0000", 2024-05-11T23:01:57.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948385+0000", 2024-05-11T23:01:57.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6300000000000001e-07 2024-05-11T23:01:57.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948387+0000", 2024-05-11T23:01:57.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8169999999999999e-06 2024-05-11T23:01:57.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948391+0000", 2024-05-11T23:01:57.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8310000000000003e-06 2024-05-11T23:01:57.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948394+0000", 2024-05-11T23:01:57.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7209999999999999e-06 2024-05-11T23:01:57.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968458+0000", 2024-05-11T23:01:57.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020064354 2024-05-11T23:01:57.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968467+0000", 2024-05-11T23:01:57.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9239999999999996e-06 2024-05-11T23:01:57.115 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.115 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.116 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.116 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25871 3.17 3.5dda9cb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.116 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948397+0000", 2024-05-11T23:01:57.116 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.128223781, 2024-05-11T23:01:57.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020115161999999999, 2024-05-11T23:01:57.116 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.116 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.116 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:57.116 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.116 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.116 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.116 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25871 2024-05-11T23:01:57.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.116 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948397+0000", 2024-05-11T23:01:57.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948397+0000", 2024-05-11T23:01:57.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948398+0000", 2024-05-11T23:01:57.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.22e-07 2024-05-11T23:01:57.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948402+0000", 2024-05-11T23:01:57.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2799999999999997e-06 2024-05-11T23:01:57.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948405+0000", 2024-05-11T23:01:57.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3830000000000001e-06 2024-05-11T23:01:57.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948410+0000", 2024-05-11T23:01:57.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3680000000000004e-06 2024-05-11T23:01:57.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968506+0000", 2024-05-11T23:01:57.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020096394 2024-05-11T23:01:57.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968512+0000", 2024-05-11T23:01:57.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5150000000000003e-06 2024-05-11T23:01:57.119 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.119 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.119 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.120 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25875 3.17 3.e6b97637 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.120 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948413+0000", 2024-05-11T23:01:57.120 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.1282080250000002, 2024-05-11T23:01:57.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020142189000000001, 2024-05-11T23:01:57.120 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.120 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.120 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:57.120 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.120 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.120 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.120 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25875 2024-05-11T23:01:57.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.121 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948413+0000", 2024-05-11T23:01:57.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948413+0000", 2024-05-11T23:01:57.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948413+0000", 2024-05-11T23:01:57.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:01:57.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948415+0000", 2024-05-11T23:01:57.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2440000000000001e-06 2024-05-11T23:01:57.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948418+0000", 2024-05-11T23:01:57.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3000000000000002e-06 2024-05-11T23:01:57.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948421+0000", 2024-05-11T23:01:57.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.706e-06 2024-05-11T23:01:57.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968546+0000", 2024-05-11T23:01:57.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020125034999999999 2024-05-11T23:01:57.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968555+0000", 2024-05-11T23:01:57.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6840000000000008e-06 2024-05-11T23:01:57.126 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.126 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.126 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.127 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25879 3.17 3.5dda9cb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.127 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948424+0000", 2024-05-11T23:01:57.127 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.1281973680000004, 2024-05-11T23:01:57.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020150983000000001, 2024-05-11T23:01:57.127 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.127 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.127 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:57.127 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.127 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.127 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.127 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25879 2024-05-11T23:01:57.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.127 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948424+0000", 2024-05-11T23:01:57.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948424+0000", 2024-05-11T23:01:57.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948424+0000", 2024-05-11T23:01:57.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:01:57.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948425+0000", 2024-05-11T23:01:57.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.167e-06 2024-05-11T23:01:57.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948428+0000", 2024-05-11T23:01:57.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2049999999999998e-06 2024-05-11T23:01:57.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948431+0000", 2024-05-11T23:01:57.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5749999999999999e-06 2024-05-11T23:01:57.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968568+0000", 2024-05-11T23:01:57.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020137192000000002 2024-05-11T23:01:57.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968575+0000", 2024-05-11T23:01:57.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.635e-06 2024-05-11T23:01:57.130 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.130 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.130 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.131 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25882 3.17 3.e6b97637 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.131 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948434+0000", 2024-05-11T23:01:57.131 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.1281869689999997, 2024-05-11T23:01:57.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020171597999999999, 2024-05-11T23:01:57.131 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.131 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.131 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:57.131 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.131 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.131 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.131 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25882 2024-05-11T23:01:57.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.131 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948434+0000", 2024-05-11T23:01:57.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948434+0000", 2024-05-11T23:01:57.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948434+0000", 2024-05-11T23:01:57.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:01:57.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948435+0000", 2024-05-11T23:01:57.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1570000000000001e-06 2024-05-11T23:01:57.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948440+0000", 2024-05-11T23:01:57.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1989999999999999e-06 2024-05-11T23:01:57.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948444+0000", 2024-05-11T23:01:57.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.121e-06 2024-05-11T23:01:57.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968599+0000", 2024-05-11T23:01:57.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020155619 2024-05-11T23:01:57.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968606+0000", 2024-05-11T23:01:57.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2709999999999998e-06 2024-05-11T23:01:57.134 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.134 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.134 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.134 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25886 3.17 3.5dda9cb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948487+0000", 2024-05-11T23:01:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.1281338520000004, 2024-05-11T23:01:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020152224999999999, 2024-05-11T23:01:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25886 2024-05-11T23:01:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948487+0000", 2024-05-11T23:01:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948487+0000", 2024-05-11T23:01:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948488+0000", 2024-05-11T23:01:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4199999999999995e-07 2024-05-11T23:01:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948491+0000", 2024-05-11T23:01:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.304e-06 2024-05-11T23:01:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948499+0000", 2024-05-11T23:01:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8979999999999996e-06 2024-05-11T23:01:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948506+0000", 2024-05-11T23:01:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8290000000000002e-06 2024-05-11T23:01:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968631+0000", 2024-05-11T23:01:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020125546000000001 2024-05-11T23:01:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968639+0000", 2024-05-11T23:01:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0059999999999992e-06 2024-05-11T23:01:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.138 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25889 3.17 3.e6b97637 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948515+0000", 2024-05-11T23:01:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.1281057969999999, 2024-05-11T23:01:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020155683000000001, 2024-05-11T23:01:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25889 2024-05-11T23:01:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948515+0000", 2024-05-11T23:01:57.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948515+0000", 2024-05-11T23:01:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948516+0000", 2024-05-11T23:01:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3599999999999999e-07 2024-05-11T23:01:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948518+0000", 2024-05-11T23:01:57.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4320000000000002e-06 2024-05-11T23:01:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948532+0000", 2024-05-11T23:01:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4078e-05 2024-05-11T23:01:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948538+0000", 2024-05-11T23:01:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6899999999999997e-06 2024-05-11T23:01:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968666+0000", 2024-05-11T23:01:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020128070000000001 2024-05-11T23:01:57.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968671+0000", 2024-05-11T23:01:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.977e-06 2024-05-11T23:01:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25893 3.17 3.5dda9cb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948546+0000", 2024-05-11T23:01:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.1280754079999999, 2024-05-11T23:01:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020162342999999999, 2024-05-11T23:01:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.142 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25893 2024-05-11T23:01:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948546+0000", 2024-05-11T23:01:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948546+0000", 2024-05-11T23:01:57.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948546+0000", 2024-05-11T23:01:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3000000000000001e-07 2024-05-11T23:01:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948549+0000", 2024-05-11T23:01:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4040000000000002e-06 2024-05-11T23:01:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948555+0000", 2024-05-11T23:01:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7660000000000001e-06 2024-05-11T23:01:57.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948564+0000", 2024-05-11T23:01:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8580000000000006e-06 2024-05-11T23:01:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968699+0000", 2024-05-11T23:01:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020134927 2024-05-11T23:01:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968708+0000", 2024-05-11T23:01:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8580000000000006e-06 2024-05-11T23:01:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.145 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25896 3.17 3.e6b97637 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948572+0000", 2024-05-11T23:01:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.1280492799999999, 2024-05-11T23:01:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020178925, 2024-05-11T23:01:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25896 2024-05-11T23:01:57.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948572+0000", 2024-05-11T23:01:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948572+0000", 2024-05-11T23:01:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948572+0000", 2024-05-11T23:01:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.96e-07 2024-05-11T23:01:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948575+0000", 2024-05-11T23:01:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4250000000000001e-06 2024-05-11T23:01:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948581+0000", 2024-05-11T23:01:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7070000000000003e-06 2024-05-11T23:01:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948586+0000", 2024-05-11T23:01:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0579999999999997e-06 2024-05-11T23:01:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968746+0000", 2024-05-11T23:01:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020159274000000001 2024-05-11T23:01:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968751+0000", 2024-05-11T23:01:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0649999999999998e-06 2024-05-11T23:01:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.149 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25900 3.17 3.5dda9cb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948813+0000", 2024-05-11T23:01:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.1278076129999999, 2024-05-11T23:01:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019981407999999999, 2024-05-11T23:01:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25900 2024-05-11T23:01:57.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948813+0000", 2024-05-11T23:01:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948813+0000", 2024-05-11T23:01:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948814+0000", 2024-05-11T23:01:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7400000000000003e-07 2024-05-11T23:01:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948823+0000", 2024-05-11T23:01:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3430000000000002e-06 2024-05-11T23:01:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948831+0000", 2024-05-11T23:01:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1960000000000002e-06 2024-05-11T23:01:57.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948843+0000", 2024-05-11T23:01:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2452999999999999e-05 2024-05-11T23:01:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968784+0000", 2024-05-11T23:01:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019940439000000001 2024-05-11T23:01:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968795+0000", 2024-05-11T23:01:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1303e-05 2024-05-11T23:01:57.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25903 3.17 3.e6b97637 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948852+0000", 2024-05-11T23:01:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.1277693629999996, 2024-05-11T23:01:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019968133999999998, 2024-05-11T23:01:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.154 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25903 2024-05-11T23:01:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948852+0000", 2024-05-11T23:01:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948852+0000", 2024-05-11T23:01:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948852+0000", 2024-05-11T23:01:57.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0900000000000002e-07 2024-05-11T23:01:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948854+0000", 2024-05-11T23:01:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.677e-06 2024-05-11T23:01:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948858+0000", 2024-05-11T23:01:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0779999999999997e-06 2024-05-11T23:01:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948861+0000", 2024-05-11T23:01:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3790000000000002e-06 2024-05-11T23:01:57.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968812+0000", 2024-05-11T23:01:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019950704999999999 2024-05-11T23:01:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968820+0000", 2024-05-11T23:01:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8860000000000004e-06 2024-05-11T23:01:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25907 3.17 3.5dda9cb7 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.157 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948879+0000", 2024-05-11T23:01:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.1277423249999998, 2024-05-11T23:01:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019955064000000002, 2024-05-11T23:01:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:01:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25907 2024-05-11T23:01:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948879+0000", 2024-05-11T23:01:57.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948879+0000", 2024-05-11T23:01:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948879+0000", 2024-05-11T23:01:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8599999999999998e-07 2024-05-11T23:01:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948881+0000", 2024-05-11T23:01:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1739999999999999e-06 2024-05-11T23:01:57.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948886+0000", 2024-05-11T23:01:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7600000000000002e-06 2024-05-11T23:01:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948889+0000", 2024-05-11T23:01:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.298e-06 2024-05-11T23:01:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968829+0000", 2024-05-11T23:01:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019939912000000001 2024-05-11T23:01:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968834+0000", 2024-05-11T23:01:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4340000000000002e-06 2024-05-11T23:01:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.161 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.161 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:57.162 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:01:57.165 INFO:teuthology.orchestra.run.smithi005.stdout:2335 trunc from 0x7420fe5 to 0x6ceb807 2024-05-11T23:01:57.197 INFO:teuthology.orchestra.run.smithi005.stdout:2337 writesame 0xd321979 thru 0xd32b550 data_size 0x4(0x9bd8 bytes) 2024-05-11T23:01:57.197 INFO:teuthology.orchestra.run.smithi005.stdout:2339 write 0x24341e7 thru 0x243ffd9 (0xbdf3 bytes) 2024-05-11T23:01:57.198 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:57.197+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:57.210 INFO:teuthology.orchestra.run.smithi005.stdout:2340 write 0xd88254a thru 0xd892375 (0xfe2c bytes) 2024-05-11T23:01:57.213 INFO:teuthology.orchestra.run.smithi005.stdout:2341 read 0x17c369e thru 0x17ca5a3 (0x6f06 bytes) 2024-05-11T23:01:57.214 INFO:teuthology.orchestra.run.smithi005.stdout:2343 write 0x7fb5772 thru 0x7fbf40c (0x9c9b bytes) 2024-05-11T23:01:57.215 INFO:teuthology.orchestra.run.smithi005.stdout:2344 write 0x7442eb0 thru 0x7444d0c (0x1e5d bytes) 2024-05-11T23:01:57.217 INFO:teuthology.orchestra.run.smithi005.stdout:2346 read 0x844004 thru 0x84aefc (0x6ef9 bytes) 2024-05-11T23:01:57.217 INFO:teuthology.orchestra.run.smithi005.stdout:2349 read 0x2a37a38 thru 0x2a3d5e6 (0x5baf bytes) 2024-05-11T23:01:57.218 INFO:teuthology.orchestra.run.smithi005.stdout:2350 read 0xbc6e785 thru 0xbc77e2c (0x96a8 bytes) 2024-05-11T23:01:57.225 INFO:teuthology.orchestra.run.smithi005.stdout:2351 trunc from 0xd892376 to 0x80867b4 2024-05-11T23:01:57.232 INFO:teuthology.orchestra.run.smithi005.stdout:2352 read 0x76bbc07 thru 0x76c364d (0x7a47 bytes) 2024-05-11T23:01:57.232 INFO:teuthology.orchestra.run.smithi005.stdout:2353 writesame 0x2a21ce6 thru 0x2a29bb8 data_size 0x1(0x7ed3 bytes) 2024-05-11T23:01:57.253 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-11T23:01:57.253+0000 7fd16247e640 -1 mon.a@0(leader).osd e43 definitely_dead 0 2024-05-11T23:01:57.255 INFO:teuthology.orchestra.run.smithi005.stdout:2354 read 0x430515b thru 0x4311172 (0xc018 bytes) 2024-05-11T23:01:57.255 INFO:teuthology.orchestra.run.smithi005.stdout:2355 write 0x73f6d39 thru 0x73fc988 (0x5c50 bytes) 2024-05-11T23:01:57.257 INFO:teuthology.orchestra.run.smithi005.stdout:2356 punch from 0x6a44691 to 0x6a4f2f9, (0xac68 bytes) 2024-05-11T23:01:57.257 INFO:teuthology.orchestra.run.smithi005.stdout:2358 write 0x313c110 thru 0x313e7e7 (0x26d8 bytes) 2024-05-11T23:01:57.258 INFO:teuthology.orchestra.run.smithi005.stdout:2359 read 0x7dd1769 thru 0x7dd2d30 (0x15c8 bytes) 2024-05-11T23:01:57.266 INFO:teuthology.orchestra.run.smithi005.stdout:2361 write 0x9d1a8ba thru 0x9d29c42 (0xf389 bytes) 2024-05-11T23:01:57.268 INFO:teuthology.orchestra.run.smithi005.stdout:2362 punch from 0x218c1db to 0x219956e, (0xd393 bytes) 2024-05-11T23:01:57.268 INFO:teuthology.orchestra.run.smithi005.stdout:2366 write 0x3473fc8 thru 0x3482350 (0xe389 bytes) 2024-05-11T23:01:57.271 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:57.271+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:57.277 INFO:teuthology.orchestra.run.smithi005.stdout:2367 writesame 0xbee5b00 thru 0xbef0bbd data_size 0x3(0xb0be bytes) 2024-05-11T23:01:57.278 INFO:teuthology.orchestra.run.smithi005.stdout:2368 read 0x34987d4 thru 0x349e8ae (0x60db bytes) 2024-05-11T23:01:57.278 INFO:teuthology.orchestra.run.smithi005.stdout:2369 read 0x57608c8 thru 0x576ecab (0xe3e4 bytes) 2024-05-11T23:01:57.278 INFO:teuthology.orchestra.run.smithi005.stdout:2370 read 0x294a3a7 thru 0x294cbdf (0x2839 bytes) 2024-05-11T23:01:57.278 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:57.287 INFO:teuthology.orchestra.run.smithi005.stdout:2375 writesame 0xde5e070 thru 0xde665b8 data_size 0x1(0x8549 bytes) 2024-05-11T23:01:57.287 INFO:teuthology.orchestra.run.smithi005.stdout:2376 write 0x57fa3ae thru 0x58001d2 (0x5e25 bytes) 2024-05-11T23:01:57.290 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:57.290 INFO:teuthology.orchestra.run.smithi005.stdout:2378 punch from 0x95a99f5 to 0x95b4667, (0xac72 bytes) 2024-05-11T23:01:57.290 INFO:teuthology.orchestra.run.smithi005.stdout:2379 writesame 0x57b0dbe thru 0x57b8ab2 data_size 0x1(0x7cf5 bytes) 2024-05-11T23:01:57.302 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:57.302 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:57.302 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:01:57.302 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:57.310 INFO:teuthology.orchestra.run.smithi005.stdout:2381 write 0x83f449d thru 0x83f7c67 (0x37cb bytes) 2024-05-11T23:01:57.314 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:01:57.314 INFO:teuthology.orchestra.run.smithi005.stdout:2382 write 0x36287c2 thru 0x3629aa2 (0x12e1 bytes) 2024-05-11T23:01:57.315 INFO:teuthology.orchestra.run.smithi005.stdout:2383 punch from 0x1eda27 to 0x1f2a03, (0x4fdc bytes) 2024-05-11T23:01:57.315 INFO:teuthology.orchestra.run.smithi005.stdout:2385 punch from 0x24cd5be to 0x24db33e, (0xdd80 bytes) 2024-05-11T23:01:57.315 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:57.327 INFO:teuthology.orchestra.run.smithi005.stdout:2389 trunc from 0xde665b9 to 0x2d21848 2024-05-11T23:01:57.372 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:57.372+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:57.374 INFO:teuthology.orchestra.run.smithi005.stdout:2390 write 0x6ae7c31 thru 0x6af73b2 (0xf782 bytes) 2024-05-11T23:01:57.376 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:57.385 INFO:teuthology.orchestra.run.smithi005.stdout:2392 write 0x972632b thru 0x972c57b (0x6251 bytes) 2024-05-11T23:01:57.393 INFO:teuthology.orchestra.run.smithi005.stdout:2393 trunc from 0x972c57c to 0x4c87e64 2024-05-11T23:01:57.405 INFO:teuthology.orchestra.run.smithi005.stdout:2394 punch from 0x1840594 to 0x1848418, (0x7e84 bytes) 2024-05-11T23:01:57.405 INFO:teuthology.orchestra.run.smithi005.stdout:2395 write 0x2e3f4d8 thru 0x2e4db49 (0xe672 bytes) 2024-05-11T23:01:57.412 INFO:teuthology.orchestra.run.smithi005.stdout:2397 write 0x506141b thru 0x506effc (0xdbe2 bytes) 2024-05-11T23:01:57.414 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:01:57.414 INFO:teuthology.orchestra.run.smithi005.stdout:2400 clone 19 order 25 su 16777216 sc 15 2024-05-11T23:01:57.440 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:57.440 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:57.440 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:01:57.440 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:01:57.440 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:57.449 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:01:57.490 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:57.490+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:57.574 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:57.575+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:57.600 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:01:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:01:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:01:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28442 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 442368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081216+0000", 2024-05-11T23:01:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5181236699999996, 2024-05-11T23:01:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031581206000000001, 2024-05-11T23:01:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28442 2024-05-11T23:01:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081216+0000", 2024-05-11T23:01:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081216+0000", 2024-05-11T23:01:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081216+0000", 2024-05-11T23:01:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5699999999999999e-07 2024-05-11T23:01:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081217+0000", 2024-05-11T23:01:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.111e-06 2024-05-11T23:01:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081225+0000", 2024-05-11T23:01:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9230000000000002e-06 2024-05-11T23:01:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081228+0000", 2024-05-11T23:01:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2109999999999998e-06 2024-05-11T23:01:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112669+0000", 2024-05-11T23:01:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031440659000000003 2024-05-11T23:01:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112736+0000", 2024-05-11T23:01:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7575000000000003e-05 2024-05-11T23:01:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112797+0000", 2024-05-11T23:01:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0470000000000002e-05 2024-05-11T23:01:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28445 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:01:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5181073930000002, 2024-05-11T23:01:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031719429, 2024-05-11T23:01:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28445 2024-05-11T23:01:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:01:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:01:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:01:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-05-11T23:01:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081234+0000", 2024-05-11T23:01:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.578e-06 2024-05-11T23:01:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081237+0000", 2024-05-11T23:01:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3340000000000002e-06 2024-05-11T23:01:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081240+0000", 2024-05-11T23:01:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7429999999999998e-06 2024-05-11T23:01:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112828+0000", 2024-05-11T23:01:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031587941000000001 2024-05-11T23:01:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112878+0000", 2024-05-11T23:01:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0535999999999999e-05 2024-05-11T23:01:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112951+0000", 2024-05-11T23:01:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3072999999999996e-05 2024-05-11T23:01:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28448 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 507904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:01:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5180748919999996, 2024-05-11T23:01:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031916923, 2024-05-11T23:01:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28448 2024-05-11T23:01:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:01:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:01:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081265+0000", 2024-05-11T23:01:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2899999999999999e-07 2024-05-11T23:01:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081267+0000", 2024-05-11T23:01:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6749999999999998e-06 2024-05-11T23:01:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081273+0000", 2024-05-11T23:01:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4049999999999999e-06 2024-05-11T23:01:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081278+0000", 2024-05-11T23:01:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0139999999999998e-06 2024-05-11T23:01:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112996+0000", 2024-05-11T23:01:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031718283 2024-05-11T23:01:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113080+0000", 2024-05-11T23:01:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4334999999999998e-05 2024-05-11T23:01:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113181+0000", 2024-05-11T23:01:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000100882 2024-05-11T23:01:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28451 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 540672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:01:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5180463900000003, 2024-05-11T23:01:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032054412999999997, 2024-05-11T23:01:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28451 2024-05-11T23:01:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:01:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:01:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:01:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2499999999999995e-07 2024-05-11T23:01:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081300+0000", 2024-05-11T23:01:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0929999999999999e-06 2024-05-11T23:01:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081306+0000", 2024-05-11T23:01:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6290000000000004e-06 2024-05-11T23:01:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081311+0000", 2024-05-11T23:01:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9289999999999997e-06 2024-05-11T23:01:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113208+0000", 2024-05-11T23:01:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031897162 2024-05-11T23:01:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113277+0000", 2024-05-11T23:01:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8811000000000002e-05 2024-05-11T23:01:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113347+0000", 2024-05-11T23:01:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0264000000000004e-05 2024-05-11T23:01:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28454 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 573440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:01:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5179717689999999, 2024-05-11T23:01:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032210564999999997, 2024-05-11T23:01:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28454 2024-05-11T23:01:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:01:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:01:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:01:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.99e-07 2024-05-11T23:01:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081371+0000", 2024-05-11T23:01:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1200000000000002e-06 2024-05-11T23:01:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081378+0000", 2024-05-11T23:01:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6089999999999997e-06 2024-05-11T23:01:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081384+0000", 2024-05-11T23:01:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0859999999999998e-06 2024-05-11T23:01:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113388+0000", 2024-05-11T23:01:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032004248999999999 2024-05-11T23:01:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113470+0000", 2024-05-11T23:01:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1871000000000003e-05 2024-05-11T23:01:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113578+0000", 2024-05-11T23:01:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000108031 2024-05-11T23:01:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28457 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 606208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:01:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5179415450000002, 2024-05-11T23:01:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032410130000000002, 2024-05-11T23:01:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28457 2024-05-11T23:01:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:01:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:01:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:01:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.32e-07 2024-05-11T23:01:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081400+0000", 2024-05-11T23:01:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9120000000000001e-06 2024-05-11T23:01:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081404+0000", 2024-05-11T23:01:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7900000000000001e-06 2024-05-11T23:01:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081407+0000", 2024-05-11T23:01:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8839999999999998e-06 2024-05-11T23:01:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113621+0000", 2024-05-11T23:01:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032213841 2024-05-11T23:01:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113702+0000", 2024-05-11T23:01:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1574000000000002e-05 2024-05-11T23:01:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113808+0000", 2024-05-11T23:01:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000105697 2024-05-11T23:01:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28460 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 638976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:01:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.517928714, 2024-05-11T23:01:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032616804999999999, 2024-05-11T23:01:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28460 2024-05-11T23:01:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:01:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:01:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:01:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5900000000000003e-07 2024-05-11T23:01:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081412+0000", 2024-05-11T23:01:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1990000000000001e-06 2024-05-11T23:01:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081415+0000", 2024-05-11T23:01:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.971e-06 2024-05-11T23:01:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081418+0000", 2024-05-11T23:01:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-05-11T23:01:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113853+0000", 2024-05-11T23:01:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032435155 2024-05-11T23:01:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113936+0000", 2024-05-11T23:01:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3441000000000003e-05 2024-05-11T23:01:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114027+0000", 2024-05-11T23:01:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1108999999999998e-05 2024-05-11T23:01:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28463 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 671744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:01:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5179117919999996, 2024-05-11T23:01:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032814958999999998, 2024-05-11T23:01:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28463 2024-05-11T23:01:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:01:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:01:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:01:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4999999999999998e-07 2024-05-11T23:01:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081430+0000", 2024-05-11T23:01:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6109999999999999e-06 2024-05-11T23:01:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081433+0000", 2024-05-11T23:01:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5590000000000001e-06 2024-05-11T23:01:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081437+0000", 2024-05-11T23:01:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3950000000000001e-06 2024-05-11T23:01:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114061+0000", 2024-05-11T23:01:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032624092 2024-05-11T23:01:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114142+0000", 2024-05-11T23:01:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0893e-05 2024-05-11T23:01:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114242+0000", 2024-05-11T23:01:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010095899999999999 2024-05-11T23:01:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28466 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 704512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:01:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5178992579999999, 2024-05-11T23:01:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032933658999999997, 2024-05-11T23:01:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28466 2024-05-11T23:01:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:01:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:01:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:01:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4700000000000002e-07 2024-05-11T23:01:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081442+0000", 2024-05-11T23:01:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.296e-06 2024-05-11T23:01:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081445+0000", 2024-05-11T23:01:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9009999999999998e-06 2024-05-11T23:01:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081447+0000", 2024-05-11T23:01:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4200000000000001e-06 2024-05-11T23:01:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114280+0000", 2024-05-11T23:01:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032832667000000003 2024-05-11T23:01:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114325+0000", 2024-05-11T23:01:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5018000000000002e-05 2024-05-11T23:01:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114374+0000", 2024-05-11T23:01:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.901e-05 2024-05-11T23:01:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28468 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 737280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:01:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5178792530000003, 2024-05-11T23:01:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033118776000000003, 2024-05-11T23:01:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28468 2024-05-11T23:01:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:01:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:01:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:01:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3799999999999998e-07 2024-05-11T23:01:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081462+0000", 2024-05-11T23:01:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4899999999999999e-06 2024-05-11T23:01:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081465+0000", 2024-05-11T23:01:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4079999999999998e-06 2024-05-11T23:01:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081468+0000", 2024-05-11T23:01:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7e-06 2024-05-11T23:01:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114415+0000", 2024-05-11T23:01:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032946766000000002 2024-05-11T23:01:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114497+0000", 2024-05-11T23:01:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1854000000000006e-05 2024-05-11T23:01:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114579+0000", 2024-05-11T23:01:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2120000000000007e-05 2024-05-11T23:01:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28471 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 770048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:01:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5178645460000002, 2024-05-11T23:01:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033234970000000003, 2024-05-11T23:01:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28471 2024-05-11T23:01:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:01:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:01:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:01:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-05-11T23:01:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081476+0000", 2024-05-11T23:01:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3120000000000001e-06 2024-05-11T23:01:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081480+0000", 2024-05-11T23:01:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1470000000000001e-06 2024-05-11T23:01:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081482+0000", 2024-05-11T23:01:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7480000000000002e-06 2024-05-11T23:01:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114616+0000", 2024-05-11T23:01:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033133322 2024-05-11T23:01:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114662+0000", 2024-05-11T23:01:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6020999999999999e-05 2024-05-11T23:01:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114710+0000", 2024-05-11T23:01:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8059999999999997e-05 2024-05-11T23:01:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28474 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:01:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5178484719999998, 2024-05-11T23:01:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033336431, 2024-05-11T23:01:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28474 2024-05-11T23:01:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:01:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:01:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:01:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.15e-07 2024-05-11T23:01:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081493+0000", 2024-05-11T23:01:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4950000000000001e-06 2024-05-11T23:01:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081496+0000", 2024-05-11T23:01:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.309e-06 2024-05-11T23:01:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081499+0000", 2024-05-11T23:01:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4570000000000001e-06 2024-05-11T23:01:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114740+0000", 2024-05-11T23:01:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033240619999999999 2024-05-11T23:01:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114781+0000", 2024-05-11T23:01:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1106999999999998e-05 2024-05-11T23:01:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114827+0000", 2024-05-11T23:01:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6028e-05 2024-05-11T23:01:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28477 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 835584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:01:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5178314579999999, 2024-05-11T23:01:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033527225000000001, 2024-05-11T23:01:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28477 2024-05-11T23:01:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:01:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:01:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:01:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1300000000000001e-07 2024-05-11T23:01:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081510+0000", 2024-05-11T23:01:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4610000000000001e-06 2024-05-11T23:01:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081513+0000", 2024-05-11T23:01:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2559999999999998e-06 2024-05-11T23:01:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081516+0000", 2024-05-11T23:01:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7099999999999999e-06 2024-05-11T23:01:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114869+0000", 2024-05-11T23:01:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033353718999999997 2024-05-11T23:01:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114954+0000", 2024-05-11T23:01:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4969000000000006e-05 2024-05-11T23:01:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115035+0000", 2024-05-11T23:01:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0697000000000003e-05 2024-05-11T23:01:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28480 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 868352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:01:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.517814488, 2024-05-11T23:01:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033695939000000001, 2024-05-11T23:01:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28480 2024-05-11T23:01:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:01:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:01:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:01:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.46e-07 2024-05-11T23:01:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081526+0000", 2024-05-11T23:01:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.347e-06 2024-05-11T23:01:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081530+0000", 2024-05-11T23:01:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.348e-06 2024-05-11T23:01:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081533+0000", 2024-05-11T23:01:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4369999999999999e-06 2024-05-11T23:01:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115078+0000", 2024-05-11T23:01:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033545063 2024-05-11T23:01:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115164+0000", 2024-05-11T23:01:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5662999999999996e-05 2024-05-11T23:01:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115221+0000", 2024-05-11T23:01:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6734999999999998e-05 2024-05-11T23:01:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28483 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:01:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5177973150000001, 2024-05-11T23:01:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033803466999999997, 2024-05-11T23:01:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28483 2024-05-11T23:01:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:01:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:01:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:01:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4999999999999998e-07 2024-05-11T23:01:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081544+0000", 2024-05-11T23:01:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:01:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081547+0000", 2024-05-11T23:01:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3139999999999999e-06 2024-05-11T23:01:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081550+0000", 2024-05-11T23:01:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7779999999999999e-06 2024-05-11T23:01:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115253+0000", 2024-05-11T23:01:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033703516000000003 2024-05-11T23:01:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115302+0000", 2024-05-11T23:01:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8869999999999998e-05 2024-05-11T23:01:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115345+0000", 2024-05-11T23:01:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3340999999999997e-05 2024-05-11T23:01:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28487 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 933888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:01:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5177861139999997, 2024-05-11T23:01:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033903918999999998, 2024-05-11T23:01:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28487 2024-05-11T23:01:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:01:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:01:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:01:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9999999999999999e-07 2024-05-11T23:01:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081559+0000", 2024-05-11T23:01:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-05-11T23:01:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081563+0000", 2024-05-11T23:01:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2370000000000003e-06 2024-05-11T23:01:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081566+0000", 2024-05-11T23:01:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6199999999999999e-06 2024-05-11T23:01:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115375+0000", 2024-05-11T23:01:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033808887000000003 2024-05-11T23:01:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115413+0000", 2024-05-11T23:01:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8309000000000002e-05 2024-05-11T23:01:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115457+0000", 2024-05-11T23:01:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3809e-05 2024-05-11T23:01:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28490 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 966656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:01:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5177695900000003, 2024-05-11T23:01:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034114903000000002, 2024-05-11T23:01:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28490 2024-05-11T23:01:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:01:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:01:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:01:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4400000000000001e-07 2024-05-11T23:01:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081571+0000", 2024-05-11T23:01:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2130000000000001e-06 2024-05-11T23:01:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081574+0000", 2024-05-11T23:01:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8270000000000002e-06 2024-05-11T23:01:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081577+0000", 2024-05-11T23:01:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-05-11T23:01:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115507+0000", 2024-05-11T23:01:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033929925999999999 2024-05-11T23:01:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115599+0000", 2024-05-11T23:01:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1571999999999997e-05 2024-05-11T23:01:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115685+0000", 2024-05-11T23:01:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6071999999999999e-05 2024-05-11T23:01:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28493 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 999424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:01:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5177574869999999, 2024-05-11T23:01:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034232944000000001, 2024-05-11T23:01:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28493 2024-05-11T23:01:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:01:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:01:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:01:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1300000000000001e-07 2024-05-11T23:01:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081584+0000", 2024-05-11T23:01:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9479999999999998e-06 2024-05-11T23:01:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081590+0000", 2024-05-11T23:01:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.728e-06 2024-05-11T23:01:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081595+0000", 2024-05-11T23:01:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8579999999999999e-06 2024-05-11T23:01:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115723+0000", 2024-05-11T23:01:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034128473999999999 2024-05-11T23:01:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115768+0000", 2024-05-11T23:01:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5108999999999998e-05 2024-05-11T23:01:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115815+0000", 2024-05-11T23:01:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6514000000000002e-05 2024-05-11T23:01:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.676 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28496 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:01:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5177380280000001, 2024-05-11T23:01:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034427754999999997, 2024-05-11T23:01:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:01:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.677 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28496 2024-05-11T23:01:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:01:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:01:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.678 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081602+0000", 2024-05-11T23:01:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:01:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081604+0000", 2024-05-11T23:01:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3999999999999999e-06 2024-05-11T23:01:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081610+0000", 2024-05-11T23:01:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.224e-06 2024-05-11T23:01:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081615+0000", 2024-05-11T23:01:57.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8269999999999997e-06 2024-05-11T23:01:57.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115859+0000", 2024-05-11T23:01:57.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034243498999999997 2024-05-11T23:01:57.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115946+0000", 2024-05-11T23:01:57.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7122999999999999e-05 2024-05-11T23:01:57.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116029+0000", 2024-05-11T23:01:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3332999999999996e-05 2024-05-11T23:01:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28498 3.0 3:05d5189e:::rbd_object_map.111e5e4d8275.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:01:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:01:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5162193760000005, 2024-05-11T23:01:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.035035108000000002, 2024-05-11T23:01:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:57.681 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:01:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:01:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:01:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28498 2024-05-11T23:01:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:01:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:01:57.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083121+0000", 2024-05-11T23:01:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5700000000000001e-07 2024-05-11T23:01:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083126+0000", 2024-05-11T23:01:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8860000000000003e-06 2024-05-11T23:01:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:57.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083135+0000", 2024-05-11T23:01:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8369999999999992e-06 2024-05-11T23:01:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083142+0000", 2024-05-11T23:01:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4779999999999996e-06 2024-05-11T23:01:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116072+0000", 2024-05-11T23:01:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032929991999999998 2024-05-11T23:01:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:57.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116158+0000", 2024-05-11T23:01:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6077999999999998e-05 2024-05-11T23:01:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-05-11T23:01:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116513+0000", 2024-05-11T23:01:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00035508300000000002 2024-05-11T23:01:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:01:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118025+0000", 2024-05-11T23:01:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015118919999999999 2024-05-11T23:01:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:01:57.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118039+0000", 2024-05-11T23:01:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4188999999999999e-05 2024-05-11T23:01:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118094+0000", 2024-05-11T23:01:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4929000000000002e-05 2024-05-11T23:01:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118155+0000", 2024-05-11T23:01:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0887000000000002e-05 2024-05-11T23:01:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.686 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.687 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:57.687 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:57.687 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:57.688 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:01:57.688 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:57.688+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:57.789 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:57.789+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:57.838 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:57.839 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:57.839 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:01:57.839 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:57.847 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:01:57.865 INFO:teuthology.orchestra.run.smithi005.stderr:marked out osd.3. 2024-05-11T23:01:57.889 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:57.890+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:57.977 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:57.977 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:57.977 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:01:57.977 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:01:57.977 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:57.986 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:01:57.990 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:57.991+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:58.091 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:58.091+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:58.096 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:58.097 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:01:58.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:01:58.097 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:01:58.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.097 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:25950 3.0 e40/19 3:05d5189e:::rbd_object_map.111e5e4d8275.0000000000000011:head v 40'116, mlcod=39'112)", 2024-05-11T23:01:58.097 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.973119+0000", 2024-05-11T23:01:58.097 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.1229232309999997, 2024-05-11T23:01:58.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001399029, 2024-05-11T23:01:58.097 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:58.097 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:58.097 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:58.097 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:58.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:58.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.973119+0000", 2024-05-11T23:01:58.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:58.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.973119+0000", 2024-05-11T23:01:58.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:58.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.973120+0000", 2024-05-11T23:01:58.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8800000000000003e-07 2024-05-11T23:01:58.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:58.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.973129+0000", 2024-05-11T23:01:58.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3379999999999994e-06 2024-05-11T23:01:58.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:58.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.973138+0000", 2024-05-11T23:01:58.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7609999999999996e-06 2024-05-11T23:01:58.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:58.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.973148+0000", 2024-05-11T23:01:58.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0332e-05 2024-05-11T23:01:58.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:58.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.973277+0000", 2024-05-11T23:01:58.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012877200000000001 2024-05-11T23:01:58.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:58.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.973300+0000", 2024-05-11T23:01:58.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2359000000000001e-05 2024-05-11T23:01:58.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:58.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.974481+0000", 2024-05-11T23:01:58.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00118143 2024-05-11T23:01:58.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:58.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.974518+0000", 2024-05-11T23:01:58.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7549000000000003e-05 2024-05-11T23:01:58.101 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.101 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.101 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.101 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:25952 3.9 e40/19 3:908f208a:::rbd_data.111e5e4d8275.0000000000000002:head v 40'65, mlcod=39'63)", 2024-05-11T23:01:58.101 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.977473+0000", 2024-05-11T23:01:58.101 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.1185701589999999, 2024-05-11T23:01:58.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0037656679999999998, 2024-05-11T23:01:58.102 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:58.102 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:58.102 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:58.102 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:58.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:58.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.977473+0000", 2024-05-11T23:01:58.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:58.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.977473+0000", 2024-05-11T23:01:58.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:58.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.977473+0000", 2024-05-11T23:01:58.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7300000000000002e-07 2024-05-11T23:01:58.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:58.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979179+0000", 2024-05-11T23:01:58.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017062869999999999 2024-05-11T23:01:58.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:58.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979190+0000", 2024-05-11T23:01:58.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0341e-05 2024-05-11T23:01:58.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:58.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979194+0000", 2024-05-11T23:01:58.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7369999999999997e-06 2024-05-11T23:01:58.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:58.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979250+0000", 2024-05-11T23:01:58.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6060999999999997e-05 2024-05-11T23:01:58.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:58.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979275+0000", 2024-05-11T23:01:58.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4267999999999999e-05 2024-05-11T23:01:58.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:58.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.981206+0000", 2024-05-11T23:01:58.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001931316 2024-05-11T23:01:58.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:58.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.981238+0000", 2024-05-11T23:01:58.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2285e-05 2024-05-11T23:01:58.105 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.105 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.105 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.105 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:25953 3.9 e40/19 3:908f208a:::rbd_data.111e5e4d8275.0000000000000002:head v 40'67, mlcod=39'63)", 2024-05-11T23:01:58.105 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.979457+0000", 2024-05-11T23:01:58.106 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.1165861680000004, 2024-05-11T23:01:58.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002255132, 2024-05-11T23:01:58.106 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:58.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:58.106 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:58.106 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:58.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:58.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979457+0000", 2024-05-11T23:01:58.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:58.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979457+0000", 2024-05-11T23:01:58.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:58.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979457+0000", 2024-05-11T23:01:58.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3600000000000004e-07 2024-05-11T23:01:58.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:58.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979462+0000", 2024-05-11T23:01:58.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5109999999999997e-06 2024-05-11T23:01:58.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:58.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979469+0000", 2024-05-11T23:01:58.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6979999999999998e-06 2024-05-11T23:01:58.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:58.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979474+0000", 2024-05-11T23:01:58.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-05-11T23:01:58.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:58.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.980421+0000", 2024-05-11T23:01:58.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00094687500000000002 2024-05-11T23:01:58.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:58.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.980444+0000", 2024-05-11T23:01:58.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2878e-05 2024-05-11T23:01:58.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:58.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.981657+0000", 2024-05-11T23:01:58.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0012138749999999999 2024-05-11T23:01:58.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:58.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.981712+0000", 2024-05-11T23:01:58.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4282e-05 2024-05-11T23:01:58.109 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.109 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.109 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.109 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:26861 3.0 e40/19 3:05d5189e:::rbd_object_map.111e5e4d8275.0000000000000011:head v 40'121, mlcod=40'119)", 2024-05-11T23:01:58.110 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.008555+0000", 2024-05-11T23:01:58.110 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.0874875829999997, 2024-05-11T23:01:58.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015206460000000001, 2024-05-11T23:01:58.110 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:58.110 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:58.110 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:58.110 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:58.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:58.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008555+0000", 2024-05-11T23:01:58.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:58.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008555+0000", 2024-05-11T23:01:58.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:58.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008557+0000", 2024-05-11T23:01:58.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8840000000000001e-06 2024-05-11T23:01:58.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:58.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008566+0000", 2024-05-11T23:01:58.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2790000000000004e-06 2024-05-11T23:01:58.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:58.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008586+0000", 2024-05-11T23:01:58.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9956999999999999e-05 2024-05-11T23:01:58.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:58.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008596+0000", 2024-05-11T23:01:58.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8900000000000002e-06 2024-05-11T23:01:58.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:58.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008666+0000", 2024-05-11T23:01:58.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9814000000000006e-05 2024-05-11T23:01:58.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:58.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008688+0000", 2024-05-11T23:01:58.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2365e-05 2024-05-11T23:01:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.010029+0000", 2024-05-11T23:01:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0013403880000000001 2024-05-11T23:01:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.010076+0000", 2024-05-11T23:01:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7068999999999999e-05 2024-05-11T23:01:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:27374 3.0 e40/19 3:05d5189e:::rbd_object_map.111e5e4d8275.0000000000000011:head v 40'127, mlcod=40'124)", 2024-05-11T23:01:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.034130+0000", 2024-05-11T23:01:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.0619122760000002, 2024-05-11T23:01:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001526082, 2024-05-11T23:01:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034130+0000", 2024-05-11T23:01:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034130+0000", 2024-05-11T23:01:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034138+0000", 2024-05-11T23:01:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8960000000000003e-06 2024-05-11T23:01:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034154+0000", 2024-05-11T23:01:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6152999999999999e-05 2024-05-11T23:01:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034171+0000", 2024-05-11T23:01:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6121e-05 2024-05-11T23:01:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034177+0000", 2024-05-11T23:01:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1079999999999998e-06 2024-05-11T23:01:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034612+0000", 2024-05-11T23:01:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043566900000000002 2024-05-11T23:01:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034631+0000", 2024-05-11T23:01:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8643000000000001e-05 2024-05-11T23:01:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.035619+0000", 2024-05-11T23:01:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00098804600000000002 2024-05-11T23:01:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.035657+0000", 2024-05-11T23:01:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7446e-05 2024-05-11T23:01:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:27790 3.0 e40/19 3:05d5189e:::rbd_object_map.111e5e4d8275.0000000000000011:head v 40'130, mlcod=40'128)", 2024-05-11T23:01:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.060478+0000", 2024-05-11T23:01:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.0355649390000004, 2024-05-11T23:01:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0014537739999999999, 2024-05-11T23:01:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060478+0000", 2024-05-11T23:01:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060478+0000", 2024-05-11T23:01:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060479+0000", 2024-05-11T23:01:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8999999999999995e-07 2024-05-11T23:01:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060488+0000", 2024-05-11T23:01:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9879999999999993e-06 2024-05-11T23:01:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060499+0000", 2024-05-11T23:01:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1489e-05 2024-05-11T23:01:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060505+0000", 2024-05-11T23:01:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5330000000000002e-06 2024-05-11T23:01:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060560+0000", 2024-05-11T23:01:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5384999999999999e-05 2024-05-11T23:01:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060582+0000", 2024-05-11T23:01:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1889999999999999e-05 2024-05-11T23:01:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.061883+0000", 2024-05-11T23:01:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0013005989999999999 2024-05-11T23:01:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.061932+0000", 2024-05-11T23:01:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8999999999999998e-05 2024-05-11T23:01:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:28504 3.9 e40/19 3:92d63908:::rbd_data.111e5e4d8275.0000000000000013:head v 40'68, mlcod=40'65)", 2024-05-11T23:01:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.092954+0000", 2024-05-11T23:01:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.003088612, 2024-05-11T23:01:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0030983059999999999, 2024-05-11T23:01:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.092954+0000", 2024-05-11T23:01:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.092954+0000", 2024-05-11T23:01:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.092955+0000", 2024-05-11T23:01:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.13e-06 2024-05-11T23:01:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094110+0000", 2024-05-11T23:01:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0011551669999999999 2024-05-11T23:01:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094124+0000", 2024-05-11T23:01:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3244999999999999e-05 2024-05-11T23:01:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094131+0000", 2024-05-11T23:01:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0090000000000001e-06 2024-05-11T23:01:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094187+0000", 2024-05-11T23:01:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6218999999999998e-05 2024-05-11T23:01:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094205+0000", 2024-05-11T23:01:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8148e-05 2024-05-11T23:01:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.096006+0000", 2024-05-11T23:01:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018005619999999999 2024-05-11T23:01:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.096052+0000", 2024-05-11T23:01:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6826000000000002e-05 2024-05-11T23:01:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:28505 3.9 e40/19 3:92d63908:::rbd_data.111e5e4d8275.0000000000000013:head v 40'70, mlcod=40'65)", 2024-05-11T23:01:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.094530+0000", 2024-05-11T23:01:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.0015126780000001, 2024-05-11T23:01:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021142140000000001, 2024-05-11T23:01:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094530+0000", 2024-05-11T23:01:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094530+0000", 2024-05-11T23:01:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094531+0000", 2024-05-11T23:01:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9400000000000004e-07 2024-05-11T23:01:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094536+0000", 2024-05-11T23:01:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.575e-06 2024-05-11T23:01:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094549+0000", 2024-05-11T23:01:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2857000000000001e-05 2024-05-11T23:01:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094556+0000", 2024-05-11T23:01:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6050000000000003e-06 2024-05-11T23:01:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.095097+0000", 2024-05-11T23:01:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00054118299999999999 2024-05-11T23:01:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.095119+0000", 2024-05-11T23:01:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1673000000000001e-05 2024-05-11T23:01:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.096603+0000", 2024-05-11T23:01:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001484153 2024-05-11T23:01:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.096644+0000", 2024-05-11T23:01:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1374000000000001e-05 2024-05-11T23:01:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:28824 3.9 e40/19 3:9050c332:::rbd_data.111e5e4d8275.000000000000001d:head v 40'71, mlcod=40'68)", 2024-05-11T23:01:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.141602+0000", 2024-05-11T23:01:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.9544404350000004, 2024-05-11T23:01:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0033604630000000002, 2024-05-11T23:01:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.141602+0000", 2024-05-11T23:01:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.141602+0000", 2024-05-11T23:01:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.141604+0000", 2024-05-11T23:01:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4300000000000001e-06 2024-05-11T23:01:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.142793+0000", 2024-05-11T23:01:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0011888409999999999 2024-05-11T23:01:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.142815+0000", 2024-05-11T23:01:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2432000000000001e-05 2024-05-11T23:01:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.142826+0000", 2024-05-11T23:01:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0943e-05 2024-05-11T23:01:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.142901+0000", 2024-05-11T23:01:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5283000000000004e-05 2024-05-11T23:01:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.142935+0000", 2024-05-11T23:01:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4014000000000003e-05 2024-05-11T23:01:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144916+0000", 2024-05-11T23:01:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019805019999999999 2024-05-11T23:01:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144963+0000", 2024-05-11T23:01:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7018000000000003e-05 2024-05-11T23:01:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:28825 3.9 e40/19 3:9050c332:::rbd_data.111e5e4d8275.000000000000001d:head v 40'73, mlcod=40'68)", 2024-05-11T23:01:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.143269+0000", 2024-05-11T23:01:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.9527735489999998, 2024-05-11T23:01:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0020932170000000001, 2024-05-11T23:01:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.143269+0000", 2024-05-11T23:01:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.143269+0000", 2024-05-11T23:01:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.143270+0000", 2024-05-11T23:01:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1200000000000001e-07 2024-05-11T23:01:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.143278+0000", 2024-05-11T23:01:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1079999999999993e-06 2024-05-11T23:01:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.143289+0000", 2024-05-11T23:01:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0961000000000001e-05 2024-05-11T23:01:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.143297+0000", 2024-05-11T23:01:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3480000000000004e-06 2024-05-11T23:01:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144016+0000", 2024-05-11T23:01:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00071809800000000002 2024-05-11T23:01:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144043+0000", 2024-05-11T23:01:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7746000000000002e-05 2024-05-11T23:01:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145309+0000", 2024-05-11T23:01:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0012660060000000001 2024-05-11T23:01:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145362+0000", 2024-05-11T23:01:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3038000000000003e-05 2024-05-11T23:01:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:28828 3.9 e40/19 3:93a019b9:::rbd_data.111e5e4d8275.000000000000001f:head v 40'74, mlcod=40'68)", 2024-05-11T23:01:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.144176+0000", 2024-05-11T23:01:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.9518669539999998, 2024-05-11T23:01:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0029854930000000001, 2024-05-11T23:01:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144176+0000", 2024-05-11T23:01:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144176+0000", 2024-05-11T23:01:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144177+0000", 2024-05-11T23:01:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9899999999999999e-07 2024-05-11T23:01:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145289+0000", 2024-05-11T23:01:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0011121670000000001 2024-05-11T23:01:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145304+0000", 2024-05-11T23:01:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5590000000000002e-05 2024-05-11T23:01:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145312+0000", 2024-05-11T23:01:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8529999999999992e-06 2024-05-11T23:01:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145393+0000", 2024-05-11T23:01:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1050999999999994e-05 2024-05-11T23:01:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145412+0000", 2024-05-11T23:01:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8992999999999999e-05 2024-05-11T23:01:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.147116+0000", 2024-05-11T23:01:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017039589999999999 2024-05-11T23:01:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.147161+0000", 2024-05-11T23:01:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5080999999999997e-05 2024-05-11T23:01:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:28829 3.9 e40/19 3:93a019b9:::rbd_data.111e5e4d8275.000000000000001f:head v 40'76, mlcod=40'71)", 2024-05-11T23:01:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.145924+0000", 2024-05-11T23:01:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.950118518, 2024-05-11T23:01:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015381520000000001, 2024-05-11T23:01:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145924+0000", 2024-05-11T23:01:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145924+0000", 2024-05-11T23:01:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145925+0000", 2024-05-11T23:01:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.99e-07 2024-05-11T23:01:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145933+0000", 2024-05-11T23:01:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7090000000000002e-06 2024-05-11T23:01:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145944+0000", 2024-05-11T23:01:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1177e-05 2024-05-11T23:01:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145951+0000", 2024-05-11T23:01:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6450000000000005e-06 2024-05-11T23:01:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.146272+0000", 2024-05-11T23:01:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032017699999999999 2024-05-11T23:01:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.146290+0000", 2024-05-11T23:01:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7977000000000001e-05 2024-05-11T23:01:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.147432+0000", 2024-05-11T23:01:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0011421879999999999 2024-05-11T23:01:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.147462+0000", 2024-05-11T23:01:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0580000000000002e-05 2024-05-11T23:01:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:34962 3.9 e40/19 3:96260786:::rbd_data.111ea078695e.0000000000000028:head v 40'77, mlcod=40'74)", 2024-05-11T23:01:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.700882+0000", 2024-05-11T23:01:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.3951606720000003, 2024-05-11T23:01:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001779938, 2024-05-11T23:01:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.700882+0000", 2024-05-11T23:01:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.700882+0000", 2024-05-11T23:01:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.700884+0000", 2024-05-11T23:01:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5030000000000001e-06 2024-05-11T23:01:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.701472+0000", 2024-05-11T23:01:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00058797299999999999 2024-05-11T23:01:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.701491+0000", 2024-05-11T23:01:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9850000000000001e-05 2024-05-11T23:01:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.701505+0000", 2024-05-11T23:01:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3549e-05 2024-05-11T23:01:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.701574+0000", 2024-05-11T23:01:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8937999999999996e-05 2024-05-11T23:01:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.701598+0000", 2024-05-11T23:01:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4519e-05 2024-05-11T23:01:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.702631+0000", 2024-05-11T23:01:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001032942 2024-05-11T23:01:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.702662+0000", 2024-05-11T23:01:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0664000000000003e-05 2024-05-11T23:01:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:37451 3.0 e40/19 3:071059df:::rbd_data.111ea078695e.000000000000001f:head v 40'145, mlcod=40'143)", 2024-05-11T23:01:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:53.078457+0000", 2024-05-11T23:01:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.0175852040000004, 2024-05-11T23:01:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0020281650000000002, 2024-05-11T23:01:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.078457+0000", 2024-05-11T23:01:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.078457+0000", 2024-05-11T23:01:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.078459+0000", 2024-05-11T23:01:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4500000000000001e-06 2024-05-11T23:01:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.078988+0000", 2024-05-11T23:01:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00052866000000000002 2024-05-11T23:01:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.079007+0000", 2024-05-11T23:01:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9398e-05 2024-05-11T23:01:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.079018+0000", 2024-05-11T23:01:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0834e-05 2024-05-11T23:01:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.079086+0000", 2024-05-11T23:01:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8578000000000006e-05 2024-05-11T23:01:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.079109+0000", 2024-05-11T23:01:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2189e-05 2024-05-11T23:01:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.080429+0000", 2024-05-11T23:01:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0013205070000000001 2024-05-11T23:01:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.080486+0000", 2024-05-11T23:01:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6548999999999997e-05 2024-05-11T23:01:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:37457 3.9 e40/19 3:95e7444d:::rbd_data.111ea078695e.0000000000000022:head v 40'79, mlcod=40'77)", 2024-05-11T23:01:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:53.087914+0000", 2024-05-11T23:01:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.0081285050000002, 2024-05-11T23:01:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0023284349999999998, 2024-05-11T23:01:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.087914+0000", 2024-05-11T23:01:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.087914+0000", 2024-05-11T23:01:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.087916+0000", 2024-05-11T23:01:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7260000000000001e-06 2024-05-11T23:01:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.088797+0000", 2024-05-11T23:01:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00088118400000000003 2024-05-11T23:01:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.088814+0000", 2024-05-11T23:01:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6484999999999999e-05 2024-05-11T23:01:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.088820+0000", 2024-05-11T23:01:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8029999999999999e-06 2024-05-11T23:01:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.088872+0000", 2024-05-11T23:01:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1415999999999997e-05 2024-05-11T23:01:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.088890+0000", 2024-05-11T23:01:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8529000000000002e-05 2024-05-11T23:01:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.090179+0000", 2024-05-11T23:01:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0012889539999999999 2024-05-11T23:01:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.090243+0000", 2024-05-11T23:01:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3337999999999996e-05 2024-05-11T23:01:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60295 3.9 e42/19 3:92a0646f:::rbd_data.111e1df04b77.000000000000000a:head v 42'117, mlcod=42'115)", 2024-05-11T23:01:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.021408+0000", 2024-05-11T23:01:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.074634927, 2024-05-11T23:01:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0013993829999999999, 2024-05-11T23:01:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.021408+0000", 2024-05-11T23:01:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.021408+0000", 2024-05-11T23:01:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.021408+0000", 2024-05-11T23:01:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3900000000000004e-07 2024-05-11T23:01:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022002+0000", 2024-05-11T23:01:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000593992 2024-05-11T23:01:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022012+0000", 2024-05-11T23:01:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8390000000000002e-06 2024-05-11T23:01:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022017+0000", 2024-05-11T23:01:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6789999999999996e-06 2024-05-11T23:01:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022069+0000", 2024-05-11T23:01:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1934000000000001e-05 2024-05-11T23:01:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022089+0000", 2024-05-11T23:01:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9650999999999999e-05 2024-05-11T23:01:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022777+0000", 2024-05-11T23:01:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000688797 2024-05-11T23:01:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022807+0000", 2024-05-11T23:01:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9852000000000001e-05 2024-05-11T23:01:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60336 3.9 e42/19 3:94f86c79:::rbd_data.111e1df04b77.0000000000000001:head v 42'119, mlcod=42'117)", 2024-05-11T23:01:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.065103+0000", 2024-05-11T23:01:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 2.0309394589999998, 2024-05-11T23:01:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0016346970000000001, 2024-05-11T23:01:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065103+0000", 2024-05-11T23:01:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065103+0000", 2024-05-11T23:01:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065104+0000", 2024-05-11T23:01:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.054e-06 2024-05-11T23:01:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065864+0000", 2024-05-11T23:01:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00075959699999999998 2024-05-11T23:01:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065879+0000", 2024-05-11T23:01:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4953999999999999e-05 2024-05-11T23:01:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065887+0000", 2024-05-11T23:01:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1329999999999999e-06 2024-05-11T23:01:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065954+0000", 2024-05-11T23:01:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6675999999999997e-05 2024-05-11T23:01:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065978+0000", 2024-05-11T23:01:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4243000000000002e-05 2024-05-11T23:01:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.066707+0000", 2024-05-11T23:01:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072877 2024-05-11T23:01:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.066738+0000", 2024-05-11T23:01:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1269999999999997e-05 2024-05-11T23:01:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60471 3.4 e43/19 3:2079ea20:::rbd_data.111e1df04b77.0000000000000000:head v 43'84, mlcod=42'81)", 2024-05-11T23:01:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:01:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.16916869, 2024-05-11T23:01:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0079043050000000004, 2024-05-11T23:01:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:01:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:01:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:01:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-05-11T23:01:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926880+0000", 2024-05-11T23:01:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:01:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926893+0000", 2024-05-11T23:01:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3256e-05 2024-05-11T23:01:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926899+0000", 2024-05-11T23:01:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9850000000000003e-06 2024-05-11T23:01:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926982+0000", 2024-05-11T23:01:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2997000000000005e-05 2024-05-11T23:01:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927005+0000", 2024-05-11T23:01:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3428e-05 2024-05-11T23:01:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.934747+0000", 2024-05-11T23:01:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0077415820000000003 2024-05-11T23:01:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.934778+0000", 2024-05-11T23:01:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1479999999999997e-05 2024-05-11T23:01:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60472 3.9 e43/19 3:94f86c79:::rbd_data.111e1df04b77.0000000000000001:head v 43'122, mlcod=42'119)", 2024-05-11T23:01:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.927124+0000", 2024-05-11T23:01:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.1689189899999999, 2024-05-11T23:01:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0062295900000000001, 2024-05-11T23:01:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927124+0000", 2024-05-11T23:01:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927124+0000", 2024-05-11T23:01:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927124+0000", 2024-05-11T23:01:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:01:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927128+0000", 2024-05-11T23:01:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3390000000000003e-06 2024-05-11T23:01:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927139+0000", 2024-05-11T23:01:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1161e-05 2024-05-11T23:01:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927147+0000", 2024-05-11T23:01:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1910000000000003e-06 2024-05-11T23:01:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927199+0000", 2024-05-11T23:01:58.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.24e-05 2024-05-11T23:01:58.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:58.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927223+0000", 2024-05-11T23:01:58.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3811999999999998e-05 2024-05-11T23:01:58.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:58.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.933287+0000", 2024-05-11T23:01:58.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.006063723 2024-05-11T23:01:58.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:58.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.933353+0000", 2024-05-11T23:01:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6744999999999999e-05 2024-05-11T23:01:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60478 3.4 e43/19 3:2113d5af:::rbd_data.111e1df04b77.0000000000000007:head v 43'86, mlcod=42'82)", 2024-05-11T23:01:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.935643+0000", 2024-05-11T23:01:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 1.1603993699999999, 2024-05-11T23:01:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0016792039999999999, 2024-05-11T23:01:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:01:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:01:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:01:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:01:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:01:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.935643+0000", 2024-05-11T23:01:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:01:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.935643+0000", 2024-05-11T23:01:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:01:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:01:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.935645+0000", 2024-05-11T23:01:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4419999999999999e-06 2024-05-11T23:01:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:01:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.935653+0000", 2024-05-11T23:01:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0460000000000006e-06 2024-05-11T23:01:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:01:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.935669+0000", 2024-05-11T23:01:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5809000000000001e-05 2024-05-11T23:01:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:01:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.935676+0000", 2024-05-11T23:01:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3100000000000003e-06 2024-05-11T23:01:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:01:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.936410+0000", 2024-05-11T23:01:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00073357399999999997 2024-05-11T23:01:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:01:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.936423+0000", 2024-05-11T23:01:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3762e-05 2024-05-11T23:01:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:01:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.937274+0000", 2024-05-11T23:01:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00085112600000000001 2024-05-11T23:01:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:01:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:01:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:01:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.937323+0000", 2024-05-11T23:01:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8134999999999999e-05 2024-05-11T23:01:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:01:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:01:58.181 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:58.182 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:01:58.192 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:58.192+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:58.293 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:58.294+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:58.311 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:01:58.332 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:01:58.332 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:01:58.332 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:01:58.332 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:01:58.342 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:01:58.427 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:58.427+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:58.440 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:01:58.440 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:01:58.458 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:01:58.458 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:01:58.477 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:01:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:01:58.494 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:01:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:01:58.494 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:01:58.494+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:01:58.496 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:58.497 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:58.506 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:01:58.512 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:01:58.513 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:01:58.530 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:01:58.531 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:01:58.548 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:01:58.548 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:01:58.566 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:01:58.566 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:01:58.618 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:01:58.618+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:01:58.622 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:58.622 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:58.631 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:01:58.696 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:01:58.697+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:01:58.742 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:58.742 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:58.751 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:01:58.797 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:58.798+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:58.856 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:58.856 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:58.867 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:01:58.867 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-11T23:01:58.911 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:01:58.911+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:01:58.999 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:59.000+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:59.015 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:59.016 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:59.025 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:01:59.108 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:59.109+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:59.170 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:59.170 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:59.179 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:01:59.219 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:01:59.220+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:01:59.302 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:59.303+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:59.323 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:59.324 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:59.332 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:01:59.416 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:59.417+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:59.479 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:59.480 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:59.488 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:01:59.529 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:59.530+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:59.604 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:59.605+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:59.633 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:59.633 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:59.642 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:01:59.726 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:01:59.727+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:01:59.788 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:59.788 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:59.797 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:01:59.838 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:01:59.839+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:01:59.871 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-05-11T23:01:59.906 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:01:59.907+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:01:59.918 INFO:teuthology.orchestra.run.smithi005.stdout:leaving image image_client.0-clone18 intact 2024-05-11T23:01:59.975 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:01:59.975 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:01:59.987 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:02:00.027 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:00.028+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:00.031 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-05-11T23:02:00.033 INFO:teuthology.orchestra.run.smithi005.stdout:2401 read 0x4a57270 thru 0x4a5ab4a (0x38db bytes) 2024-05-11T23:02:00.034 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:00.034 INFO:teuthology.orchestra.run.smithi005.stdout:2403 write 0x1962391 thru 0x1969e1a (0x7a8a bytes) 2024-05-11T23:02:00.042 INFO:teuthology.orchestra.run.smithi005.stdout:2404 read 0x3c1dbbf thru 0x3c27372 (0x97b4 bytes) 2024-05-11T23:02:00.042 INFO:teuthology.orchestra.run.smithi005.stdout:2406 write 0x374e496 thru 0x375a25b (0xbdc6 bytes) 2024-05-11T23:02:00.062 INFO:teuthology.orchestra.run.smithi005.stdout:2407 write 0xc304f85 thru 0xc3108a7 (0xb923 bytes) 2024-05-11T23:02:00.065 INFO:teuthology.orchestra.run.smithi005.stdout:2409 punch from 0x31b07b to 0x3240af, (0x9034 bytes) 2024-05-11T23:02:00.072 INFO:teuthology.orchestra.run.smithi005.stdout:2410 write 0xc9b0317 thru 0xc9b05d4 (0x2be bytes) 2024-05-11T23:02:00.073 INFO:teuthology.orchestra.run.smithi005.stdout:2412 write 0x6daf675 thru 0x6db1d32 (0x26be bytes) 2024-05-11T23:02:00.077 INFO:teuthology.orchestra.run.smithi005.stdout:2414 read 0xbcfe1e9 thru 0xbd02e98 (0x4cb0 bytes) 2024-05-11T23:02:00.077 INFO:teuthology.orchestra.run.smithi005.stdout:2415 punch from 0x2eaa217 to 0x2eaca51, (0x283a bytes) 2024-05-11T23:02:00.077 INFO:teuthology.orchestra.run.smithi005.stdout:2416 punch from 0x31f3b79 to 0x31f94c2, (0x5949 bytes) 2024-05-11T23:02:00.078 INFO:teuthology.orchestra.run.smithi005.stdout:2417 write 0x1352d2e thru 0x1352e71 (0x144 bytes) 2024-05-11T23:02:00.086 INFO:teuthology.orchestra.run.smithi005.stdout:2418 writesame 0xdc7da17 thru 0xdc7f7d5 data_size 0x1(0x1dbf bytes) 2024-05-11T23:02:00.086 INFO:teuthology.orchestra.run.smithi005.stdout:2419 writesame 0xafaf4c4 thru 0xafb00f6 data_size 0x3(0xc33 bytes) 2024-05-11T23:02:00.090 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:00.090 INFO:teuthology.orchestra.run.smithi005.stdout:2425 write 0xcee0b49 thru 0xceeefc4 (0xe47c bytes) 2024-05-11T23:02:00.091 INFO:teuthology.orchestra.run.smithi005.stdout:2426 write 0x57fdb1d thru 0x58020ca (0x45ae bytes) 2024-05-11T23:02:00.096 INFO:teuthology.orchestra.run.smithi005.stdout:2429 punch from 0xb16bb26 to 0xb174660, (0x8b3a bytes) 2024-05-11T23:02:00.096 INFO:teuthology.orchestra.run.smithi005.stdout:2430 writesame 0x40dbb5f thru 0x40de3f1 data_size 0x11(0x2893 bytes) 2024-05-11T23:02:00.106 INFO:teuthology.orchestra.run.smithi005.stdout:2431 trunc from 0xdc7f7d6 to 0x884dc52 2024-05-11T23:02:00.108 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:00.109+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:00.120 INFO:teuthology.orchestra.run.smithi005.stdout:2432 read 0xae1325 thru 0xaec580 (0xb25c bytes) 2024-05-11T23:02:00.121 INFO:teuthology.orchestra.run.smithi005.stdout:2433 read 0x28d0501 thru 0x28d2358 (0x1e58 bytes) 2024-05-11T23:02:00.121 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:00.121 INFO:teuthology.orchestra.run.smithi005.stdout:2436 read 0x2155e77 thru 0x2158087 (0x2211 bytes) 2024-05-11T23:02:00.133 INFO:teuthology.orchestra.run.smithi005.stdout:2438 write 0xd3470b1 thru 0xd3496e5 (0x2635 bytes) 2024-05-11T23:02:00.136 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:00.136 INFO:teuthology.orchestra.run.smithi005.stdout:2440 read 0x3fe1a54 thru 0x3feaac3 (0x9070 bytes) 2024-05-11T23:02:00.137 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:00.137 INFO:teuthology.orchestra.run.smithi005.stdout:2442 read 0xb08eb57 thru 0xb09a604 (0xbaae bytes) 2024-05-11T23:02:00.137 INFO:teuthology.orchestra.run.smithi005.stdout:2445 write 0x71801cd thru 0x7187d09 (0x7b3d bytes) 2024-05-11T23:02:00.140 INFO:teuthology.orchestra.run.smithi005.stdout:2446 read 0x59a1014 thru 0x59aa6c8 (0x96b5 bytes) 2024-05-11T23:02:00.141 INFO:teuthology.orchestra.run.smithi005.stdout:2449 read 0x94b0a80 thru 0x94bb0bb (0xa63c bytes) 2024-05-11T23:02:00.141 INFO:teuthology.orchestra.run.smithi005.stdout:2450 punch from 0x654482b to 0x654c20e, (0x79e3 bytes) 2024-05-11T23:02:00.141 INFO:teuthology.orchestra.run.smithi005.stdout:2451 read 0xaa62ec6 thru 0xaa721dd (0xf318 bytes) 2024-05-11T23:02:00.141 INFO:teuthology.orchestra.run.smithi005.stdout:2452 write 0x3b4348e thru 0x3b52112 (0xec85 bytes) 2024-05-11T23:02:00.142 INFO:teuthology.orchestra.run.smithi005.stdout:2453 write 0x96ac87e thru 0x96b217b (0x58fe bytes) 2024-05-11T23:02:00.145 INFO:teuthology.orchestra.run.smithi005.stdout:2455 punch from 0xa154269 to 0xa15f6d0, (0xb467 bytes) 2024-05-11T23:02:00.145 INFO:teuthology.orchestra.run.smithi005.stdout:2456 writesame 0x98fe953 thru 0x9902015 data_size 0x1(0x36c3 bytes) 2024-05-11T23:02:00.152 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:00.152 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:00.152 INFO:teuthology.orchestra.run.smithi005.stdout:2462 write 0x2d0e23e thru 0x2d0ee65 (0xc28 bytes) 2024-05-11T23:02:00.156 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:00.156 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:00.157 INFO:teuthology.orchestra.run.smithi005.stdout:2464 writesame 0x9642536 thru 0x96476f3 data_size 0x1(0x51be bytes) 2024-05-11T23:02:00.166 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:02:00.167 INFO:teuthology.orchestra.run.smithi005.stdout:2466 read 0x606c1c thru 0x612c18 (0xbffd bytes) 2024-05-11T23:02:00.167 INFO:teuthology.orchestra.run.smithi005.stdout:2467 write 0xc3b3348 thru 0xc3bc796 (0x944f bytes) 2024-05-11T23:02:00.169 INFO:teuthology.orchestra.run.smithi005.stdout:2468 read 0x6f3624 thru 0x6f4f87 (0x1964 bytes) 2024-05-11T23:02:00.170 INFO:teuthology.orchestra.run.smithi005.stdout:2469 read 0x58cac96 thru 0x58d1330 (0x669b bytes) 2024-05-11T23:02:00.176 INFO:teuthology.orchestra.run.smithi005.stdout:2471 writesame 0xe070b00 thru 0xe0733c6 data_size 0xd(0x28c7 bytes) 2024-05-11T23:02:00.177 INFO:teuthology.orchestra.run.smithi005.stdout:2472 read 0x236ca9 thru 0x2409d3 (0x9d2b bytes) 2024-05-11T23:02:00.188 INFO:teuthology.orchestra.run.smithi005.stdout:2473 trunc from 0xe0733c7 to 0x45f72ff 2024-05-11T23:02:00.205 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:00.205 INFO:teuthology.orchestra.run.smithi005.stdout:2475 punch from 0x157bd3c to 0x1585737, (0x99fb bytes) 2024-05-11T23:02:00.206 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:00.206 INFO:teuthology.orchestra.run.smithi005.stdout:2477 punch from 0x2b47712 to 0x2b4f255, (0x7b43 bytes) 2024-05-11T23:02:00.210 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:00.209+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:00.215 INFO:teuthology.orchestra.run.smithi005.stdout:2478 write 0x740aa93 thru 0x741514c (0xa6ba bytes) 2024-05-11T23:02:00.218 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:00.218 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:00.221 INFO:teuthology.orchestra.run.smithi005.stdout:2483 trunc from 0x741514d to 0xa08079d 2024-05-11T23:02:00.227 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:00.227 INFO:teuthology.orchestra.run.smithi005.stdout:2485 write 0x4aae9af thru 0x4ab6028 (0x767a bytes) 2024-05-11T23:02:00.229 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:00.229 INFO:teuthology.orchestra.run.smithi005.stdout:2487 punch from 0x633fc43 to 0x6348481, (0x883e bytes) 2024-05-11T23:02:00.229 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:00.229 INFO:teuthology.orchestra.run.smithi005.stdout:2490 punch from 0x6fa0e28 to 0x6fae299, (0xd471 bytes) 2024-05-11T23:02:00.229 INFO:teuthology.orchestra.run.smithi005.stdout:2491 punch from 0x2f89796 to 0x2f987ef, (0xf059 bytes) 2024-05-11T23:02:00.229 INFO:teuthology.orchestra.run.smithi005.stdout:2492 read 0x1a4b94a thru 0x1a4d959 (0x2010 bytes) 2024-05-11T23:02:00.229 INFO:teuthology.orchestra.run.smithi005.stdout:2496 read 0x5acf03c thru 0x5adecc4 (0xfc89 bytes) 2024-05-11T23:02:00.229 INFO:teuthology.orchestra.run.smithi005.stdout:2497 trunc from 0xa08079d to 0xac1f2cd 2024-05-11T23:02:00.235 INFO:teuthology.orchestra.run.smithi005.stdout:2498 read 0x9dcab5a thru 0x9dcfcd9 (0x5180 bytes) 2024-05-11T23:02:00.235 INFO:teuthology.orchestra.run.smithi005.stdout:2499 read 0x7f7d73d thru 0x7f83f25 (0x67e9 bytes) 2024-05-11T23:02:00.235 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:00.244 INFO:teuthology.orchestra.run.smithi005.stdout:2503 write 0xd2beba9 thru 0xd2cc71f (0xdb77 bytes) 2024-05-11T23:02:00.247 INFO:teuthology.orchestra.run.smithi005.stdout:2506 write 0xbd09020 thru 0xbd17b40 (0xeb21 bytes) 2024-05-11T23:02:00.250 INFO:teuthology.orchestra.run.smithi005.stdout:2507 read 0x8292d72 thru 0x82a2032 (0xf2c1 bytes) 2024-05-11T23:02:00.250 INFO:teuthology.orchestra.run.smithi005.stdout:2508 read 0x8e6f4e3 thru 0x8e70bad (0x16cb bytes) 2024-05-11T23:02:00.259 INFO:teuthology.orchestra.run.smithi005.stdout:2509 trunc from 0xd2cc720 to 0x54a2962 2024-05-11T23:02:00.272 INFO:teuthology.orchestra.run.smithi005.stdout:2510 punch from 0x16621e3 to 0x166a928, (0x8745 bytes) 2024-05-11T23:02:00.286 INFO:teuthology.orchestra.run.smithi005.stdout:2513 write 0xbf471a9 thru 0xbf4883c (0x1694 bytes) 2024-05-11T23:02:00.289 INFO:teuthology.orchestra.run.smithi005.stdout:2514 read 0x59d3f95 thru 0x59e023d (0xc2a9 bytes) 2024-05-11T23:02:00.296 INFO:teuthology.orchestra.run.smithi005.stdout:2516 write 0xc8d78fc thru 0xc8e0943 (0x9048 bytes) 2024-05-11T23:02:00.299 INFO:teuthology.orchestra.run.smithi005.stdout:2517 clone 20 order 25 su 8388608 sc 15 2024-05-11T23:02:00.310 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:00.310+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:00.316 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:00.316 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:00.325 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:02:00.411 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:00.412+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:00.470 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:00.471 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:00.480 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:02:00.524 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:00.524+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:00.578 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:02:00.590 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:00.590 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:00.599 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:02:00.613 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:00.613+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:00.706 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:00.706 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:00.713 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:00.713+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:00.724 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:00.724 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:00.742 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:00.742 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:00.752 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:00.752 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:00.752 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:00.752 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:00.752 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:00.759 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:00.760 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:00.761 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:02:00.782 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:00.782 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:00.801 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:00.801 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:00.814 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:00.814+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:00.819 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:00.819 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:00.837 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:00.837 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:00.895 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:00.895 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:00.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:00.895 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:00.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.895 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25840 3.17 3:ec6e9d67:::rbd_data.111e70525772.000000000000001a:e [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:00.895 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.947627+0000", 2024-05-11T23:02:00.896 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.9465485630000003, 2024-05-11T23:02:00.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020165684999999999, 2024-05-11T23:02:00.896 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:00.896 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:00.896 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:00.896 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:00.896 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:00.896 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:00.896 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25840 2024-05-11T23:02:00.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.896 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:00.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:00.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947627+0000", 2024-05-11T23:02:00.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:00.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947627+0000", 2024-05-11T23:02:00.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:00.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947627+0000", 2024-05-11T23:02:00.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4199999999999996e-07 2024-05-11T23:02:00.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:00.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947630+0000", 2024-05-11T23:02:00.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.311e-06 2024-05-11T23:02:00.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:00.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947636+0000", 2024-05-11T23:02:00.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2790000000000004e-06 2024-05-11T23:02:00.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:00.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.947641+0000", 2024-05-11T23:02:00.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9280000000000001e-06 2024-05-11T23:02:00.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:00.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967707+0000", 2024-05-11T23:02:00.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020066148999999998 2024-05-11T23:02:00.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:00.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967748+0000", 2024-05-11T23:02:00.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1247e-05 2024-05-11T23:02:00.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:00.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.967792+0000", 2024-05-11T23:02:00.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4229e-05 2024-05-11T23:02:00.899 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.899 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.899 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.900 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25882 3.17 3.e6b97637 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:00.900 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948434+0000", 2024-05-11T23:02:00.900 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.9457412380000001, 2024-05-11T23:02:00.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020171597999999999, 2024-05-11T23:02:00.900 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:00.900 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:00.900 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:00.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:00.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:00.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:00.900 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25882 2024-05-11T23:02:00.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.900 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:00.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:00.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948434+0000", 2024-05-11T23:02:00.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:00.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948434+0000", 2024-05-11T23:02:00.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:00.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948434+0000", 2024-05-11T23:02:00.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:02:00.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:00.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948435+0000", 2024-05-11T23:02:00.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1570000000000001e-06 2024-05-11T23:02:00.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:00.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948440+0000", 2024-05-11T23:02:00.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1989999999999999e-06 2024-05-11T23:02:00.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:00.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948444+0000", 2024-05-11T23:02:00.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.121e-06 2024-05-11T23:02:00.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:00.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968599+0000", 2024-05-11T23:02:00.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020155619 2024-05-11T23:02:00.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:00.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968606+0000", 2024-05-11T23:02:00.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2709999999999998e-06 2024-05-11T23:02:00.903 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.903 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.903 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.903 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:25896 3.17 3.e6b97637 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:00.903 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.948572+0000", 2024-05-11T23:02:00.904 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.9456035489999994, 2024-05-11T23:02:00.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020178925, 2024-05-11T23:02:00.904 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:00.904 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:00.904 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:00.904 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:00.904 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:00.904 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:00.904 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 25896 2024-05-11T23:02:00.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.904 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:00.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:00.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948572+0000", 2024-05-11T23:02:00.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:00.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948572+0000", 2024-05-11T23:02:00.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:00.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948572+0000", 2024-05-11T23:02:00.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.96e-07 2024-05-11T23:02:00.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:00.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948575+0000", 2024-05-11T23:02:00.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4250000000000001e-06 2024-05-11T23:02:00.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:00.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948581+0000", 2024-05-11T23:02:00.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7070000000000003e-06 2024-05-11T23:02:00.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:00.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.948586+0000", 2024-05-11T23:02:00.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0579999999999997e-06 2024-05-11T23:02:00.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:00.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968746+0000", 2024-05-11T23:02:00.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020159274000000001 2024-05-11T23:02:00.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:00.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.968751+0000", 2024-05-11T23:02:00.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0649999999999998e-06 2024-05-11T23:02:00.907 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.907 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.907 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.907 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:63113 3.6 3:62b1b5a1:::rbd_data.111e5e4d8275.0000000000000009:11 [read 462848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-05-11T23:02:00.907 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:57.122105+0000", 2024-05-11T23:02:00.907 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7720700000000003, 2024-05-11T23:02:00.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020256603000000001, 2024-05-11T23:02:00.907 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:00.908 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:00.908 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:00.908 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:00.908 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:00.908 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:00.908 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 63113 2024-05-11T23:02:00.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.908 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:00.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:00.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122105+0000", 2024-05-11T23:02:00.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:00.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122105+0000", 2024-05-11T23:02:00.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:00.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122106+0000", 2024-05-11T23:02:00.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:02:00.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:00.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122108+0000", 2024-05-11T23:02:00.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1940000000000001e-06 2024-05-11T23:02:00.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:00.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122115+0000", 2024-05-11T23:02:00.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.861e-06 2024-05-11T23:02:00.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:00.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122120+0000", 2024-05-11T23:02:00.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.023e-06 2024-05-11T23:02:00.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:00.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.142169+0000", 2024-05-11T23:02:00.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020049589 2024-05-11T23:02:00.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:00.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.142256+0000", 2024-05-11T23:02:00.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6749999999999994e-05 2024-05-11T23:02:00.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:00.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.142362+0000", 2024-05-11T23:02:00.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010576100000000001 2024-05-11T23:02:00.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.911 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.912 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:63122 3.6 3:62b1b5a1:::rbd_data.111e5e4d8275.0000000000000009:11 [read 831488~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-05-11T23:02:00.912 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:57.122149+0000", 2024-05-11T23:02:00.912 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7720262770000001, 2024-05-11T23:02:00.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020356496000000002, 2024-05-11T23:02:00.912 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:00.912 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:00.912 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:00.912 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:00.912 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:00.912 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:00.912 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 63122 2024-05-11T23:02:00.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.912 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:00.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:00.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122149+0000", 2024-05-11T23:02:00.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:00.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122149+0000", 2024-05-11T23:02:00.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:00.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122149+0000", 2024-05-11T23:02:00.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3300000000000003e-07 2024-05-11T23:02:00.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:00.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122156+0000", 2024-05-11T23:02:00.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0530000000000003e-06 2024-05-11T23:02:00.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:00.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122163+0000", 2024-05-11T23:02:00.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1890000000000001e-06 2024-05-11T23:02:00.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:00.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122168+0000", 2024-05-11T23:02:00.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7010000000000001e-06 2024-05-11T23:02:00.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:00.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.142399+0000", 2024-05-11T23:02:00.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02023024 2024-05-11T23:02:00.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:00.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.142447+0000", 2024-05-11T23:02:00.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8062000000000002e-05 2024-05-11T23:02:00.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:00.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.142506+0000", 2024-05-11T23:02:00.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8817999999999998e-05 2024-05-11T23:02:00.915 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.915 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.915 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.916 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:63126 3.6 3:62b1b5a1:::rbd_data.111e5e4d8275.0000000000000009:11 [read 380928~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-05-11T23:02:00.916 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:57.122206+0000", 2024-05-11T23:02:00.916 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.771969275, 2024-05-11T23:02:00.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020532847, 2024-05-11T23:02:00.916 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:00.916 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:00.916 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:00.916 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:00.916 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:00.916 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:00.916 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 63126 2024-05-11T23:02:00.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.916 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:00.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:00.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122206+0000", 2024-05-11T23:02:00.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:00.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122206+0000", 2024-05-11T23:02:00.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:00.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122207+0000", 2024-05-11T23:02:00.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5299999999999999e-07 2024-05-11T23:02:00.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:00.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122209+0000", 2024-05-11T23:02:00.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.407e-06 2024-05-11T23:02:00.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:00.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122216+0000", 2024-05-11T23:02:00.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9319999999999999e-06 2024-05-11T23:02:00.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:00.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122221+0000", 2024-05-11T23:02:00.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9540000000000003e-06 2024-05-11T23:02:00.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:00.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.142543+0000", 2024-05-11T23:02:00.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020322389 2024-05-11T23:02:00.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:00.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.142633+0000", 2024-05-11T23:02:00.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0174999999999996e-05 2024-05-11T23:02:00.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:00.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.142739+0000", 2024-05-11T23:02:00.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000105537 2024-05-11T23:02:00.919 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.919 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.920 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.920 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:63135 3.6 3:62b1b5a1:::rbd_data.111e5e4d8275.0000000000000009:11 [read 749568~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-05-11T23:02:00.920 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:57.122275+0000", 2024-05-11T23:02:00.920 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7719004050000002, 2024-05-11T23:02:00.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020723761, 2024-05-11T23:02:00.920 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:00.920 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:00.920 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:00.920 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:00.920 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:00.920 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:00.920 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 63135 2024-05-11T23:02:00.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.921 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:00.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:00.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122275+0000", 2024-05-11T23:02:00.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:00.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122275+0000", 2024-05-11T23:02:00.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:00.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122276+0000", 2024-05-11T23:02:00.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.82e-07 2024-05-11T23:02:00.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:00.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122278+0000", 2024-05-11T23:02:00.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1560000000000002e-06 2024-05-11T23:02:00.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:00.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122285+0000", 2024-05-11T23:02:00.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8080000000000005e-06 2024-05-11T23:02:00.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:00.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122291+0000", 2024-05-11T23:02:00.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.04e-06 2024-05-11T23:02:00.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:00.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.142786+0000", 2024-05-11T23:02:00.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020495428999999999 2024-05-11T23:02:00.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:00.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.142878+0000", 2024-05-11T23:02:00.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1723000000000003e-05 2024-05-11T23:02:00.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:00.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.142999+0000", 2024-05-11T23:02:00.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000121023 2024-05-11T23:02:00.923 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.924 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.924 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.924 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:63141 3.6 3:62b1b5a1:::rbd_data.111e5e4d8275.0000000000000009:11 [read 299008~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-05-11T23:02:00.924 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:57.122320+0000", 2024-05-11T23:02:00.924 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7718550309999999, 2024-05-11T23:02:00.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020915382999999999, 2024-05-11T23:02:00.924 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:00.924 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:00.924 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:00.924 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:00.924 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:00.924 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:00.924 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 63141 2024-05-11T23:02:00.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.925 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:00.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:00.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122320+0000", 2024-05-11T23:02:00.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:00.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122320+0000", 2024-05-11T23:02:00.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:00.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122321+0000", 2024-05-11T23:02:00.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.51e-07 2024-05-11T23:02:00.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:00.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122323+0000", 2024-05-11T23:02:00.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2539999999999999e-06 2024-05-11T23:02:00.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:00.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122330+0000", 2024-05-11T23:02:00.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7449999999999998e-06 2024-05-11T23:02:00.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:00.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122335+0000", 2024-05-11T23:02:00.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9910000000000002e-06 2024-05-11T23:02:00.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:00.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.143040+0000", 2024-05-11T23:02:00.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020705616 2024-05-11T23:02:00.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:00.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.143127+0000", 2024-05-11T23:02:00.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6827000000000001e-05 2024-05-11T23:02:00.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:00.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.143236+0000", 2024-05-11T23:02:00.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000108399 2024-05-11T23:02:00.928 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.928 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.928 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.928 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:63147 3.6 3:62b1b5a1:::rbd_data.111e5e4d8275.0000000000000009:11 [read 667648~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-05-11T23:02:00.928 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:57.122342+0000", 2024-05-11T23:02:00.928 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7718329580000001, 2024-05-11T23:02:00.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021055022, 2024-05-11T23:02:00.928 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:00.928 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:00.928 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:00.928 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:00.929 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:00.929 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:00.929 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 63147 2024-05-11T23:02:00.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.929 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:00.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:00.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122342+0000", 2024-05-11T23:02:00.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:00.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122342+0000", 2024-05-11T23:02:00.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:00.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122343+0000", 2024-05-11T23:02:00.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1300000000000001e-07 2024-05-11T23:02:00.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:00.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122345+0000", 2024-05-11T23:02:00.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2110000000000001e-06 2024-05-11T23:02:00.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:00.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122352+0000", 2024-05-11T23:02:00.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.046e-06 2024-05-11T23:02:00.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:00.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122357+0000", 2024-05-11T23:02:00.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1200000000000001e-06 2024-05-11T23:02:00.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:00.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.143265+0000", 2024-05-11T23:02:00.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020907689 2024-05-11T23:02:00.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:00.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.143310+0000", 2024-05-11T23:02:00.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5583e-05 2024-05-11T23:02:00.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:00.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.143397+0000", 2024-05-11T23:02:00.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6959999999999994e-05 2024-05-11T23:02:00.932 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.932 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.932 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.932 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:63162 3.6 3:62b1b5a1:::rbd_data.111e5e4d8275.0000000000000009:11 [read 585728~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-05-11T23:02:00.932 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:57.122409+0000", 2024-05-11T23:02:00.932 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.771766301, 2024-05-11T23:02:00.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021194758000000001, 2024-05-11T23:02:00.932 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:00.932 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:00.933 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:00.933 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:00.933 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:00.933 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:00.933 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 63162 2024-05-11T23:02:00.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.933 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:00.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:00.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122409+0000", 2024-05-11T23:02:00.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:00.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122409+0000", 2024-05-11T23:02:00.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:00.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122409+0000", 2024-05-11T23:02:00.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6400000000000003e-07 2024-05-11T23:02:00.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:00.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122412+0000", 2024-05-11T23:02:00.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1019999999999999e-06 2024-05-11T23:02:00.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:00.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122418+0000", 2024-05-11T23:02:00.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.674e-06 2024-05-11T23:02:00.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:00.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122424+0000", 2024-05-11T23:02:00.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7440000000000004e-06 2024-05-11T23:02:00.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:00.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.143426+0000", 2024-05-11T23:02:00.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021002025000000001 2024-05-11T23:02:00.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:00.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.143503+0000", 2024-05-11T23:02:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7268000000000006e-05 2024-05-11T23:02:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.143604+0000", 2024-05-11T23:02:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000100481 2024-05-11T23:02:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:63174 3.6 3:62b1b5a1:::rbd_data.111e5e4d8275.0000000000000009:11 [read 503808~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-05-11T23:02:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:57.122458+0000", 2024-05-11T23:02:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7717171810000001, 2024-05-11T23:02:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021357252, 2024-05-11T23:02:00.936 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:00.937 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:00.937 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:00.937 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:00.937 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:00.937 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:00.937 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 63174 2024-05-11T23:02:00.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.937 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:00.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:00.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122458+0000", 2024-05-11T23:02:00.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:00.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122458+0000", 2024-05-11T23:02:00.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:00.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122459+0000", 2024-05-11T23:02:00.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4000000000000002e-07 2024-05-11T23:02:00.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:00.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122461+0000", 2024-05-11T23:02:00.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2230000000000001e-06 2024-05-11T23:02:00.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:00.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122467+0000", 2024-05-11T23:02:00.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5209999999999999e-06 2024-05-11T23:02:00.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:00.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122472+0000", 2024-05-11T23:02:00.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1599999999999997e-06 2024-05-11T23:02:00.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:00.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.143649+0000", 2024-05-11T23:02:00.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021176186 2024-05-11T23:02:00.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:00.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.143727+0000", 2024-05-11T23:02:00.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8533000000000002e-05 2024-05-11T23:02:00.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:00.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.143815+0000", 2024-05-11T23:02:00.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8189000000000006e-05 2024-05-11T23:02:00.940 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.940 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.940 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.940 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:63180 3.6 3:62b1b5a1:::rbd_data.111e5e4d8275.0000000000000009:11 [read 872448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-05-11T23:02:00.940 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:57.122527+0000", 2024-05-11T23:02:00.940 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7716481809999998, 2024-05-11T23:02:00.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021441317000000001, 2024-05-11T23:02:00.941 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:00.941 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:00.941 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:00.941 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:00.941 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:00.941 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:00.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 63180 2024-05-11T23:02:00.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.941 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:00.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:00.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122527+0000", 2024-05-11T23:02:00.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:00.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122527+0000", 2024-05-11T23:02:00.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:00.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122528+0000", 2024-05-11T23:02:00.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.06e-07 2024-05-11T23:02:00.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:00.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122530+0000", 2024-05-11T23:02:00.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4760000000000001e-06 2024-05-11T23:02:00.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:00.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122537+0000", 2024-05-11T23:02:00.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7889999999999997e-06 2024-05-11T23:02:00.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:00.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122542+0000", 2024-05-11T23:02:00.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1150000000000002e-06 2024-05-11T23:02:00.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:00.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.143850+0000", 2024-05-11T23:02:00.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021307494999999999 2024-05-11T23:02:00.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:00.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.143897+0000", 2024-05-11T23:02:00.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7768000000000001e-05 2024-05-11T23:02:00.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:00.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.143968+0000", 2024-05-11T23:02:00.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1167999999999994e-05 2024-05-11T23:02:00.944 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.944 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.944 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.944 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:63184 3.6 3:62b1b5a1:::rbd_data.111e5e4d8275.0000000000000009:11 [read 421888~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-05-11T23:02:00.944 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:57.122549+0000", 2024-05-11T23:02:00.944 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7716261979999999, 2024-05-11T23:02:00.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021645169999999998, 2024-05-11T23:02:00.945 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:00.945 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:00.945 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:00.945 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:00.945 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:00.945 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:00.945 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 63184 2024-05-11T23:02:00.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.945 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:00.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:00.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122549+0000", 2024-05-11T23:02:00.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:00.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122549+0000", 2024-05-11T23:02:00.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:00.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122550+0000", 2024-05-11T23:02:00.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:02:00.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:00.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122552+0000", 2024-05-11T23:02:00.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.294e-06 2024-05-11T23:02:00.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:00.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122559+0000", 2024-05-11T23:02:00.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9539999999999998e-06 2024-05-11T23:02:00.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:00.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122564+0000", 2024-05-11T23:02:00.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9529999999999998e-06 2024-05-11T23:02:00.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:00.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.144007+0000", 2024-05-11T23:02:00.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021442808000000001 2024-05-11T23:02:00.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:00.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.144085+0000", 2024-05-11T23:02:00.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8673999999999999e-05 2024-05-11T23:02:00.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:00.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.144194+0000", 2024-05-11T23:02:00.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000109058 2024-05-11T23:02:00.948 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.948 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.948 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.948 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:63192 3.6 3:62b1b5a1:::rbd_data.111e5e4d8275.0000000000000009:11 [read 790528~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-05-11T23:02:00.948 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:57.122593+0000", 2024-05-11T23:02:00.948 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.771582532, 2024-05-11T23:02:00.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021831761000000002, 2024-05-11T23:02:00.949 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:00.949 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:00.949 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:00.949 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:00.949 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:00.949 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:00.949 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 63192 2024-05-11T23:02:00.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.949 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:00.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:00.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122593+0000", 2024-05-11T23:02:00.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:00.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122593+0000", 2024-05-11T23:02:00.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:00.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122593+0000", 2024-05-11T23:02:00.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4200000000000001e-07 2024-05-11T23:02:00.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:00.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122595+0000", 2024-05-11T23:02:00.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0940000000000002e-06 2024-05-11T23:02:00.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:00.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122604+0000", 2024-05-11T23:02:00.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9749999999999996e-06 2024-05-11T23:02:00.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:00.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122610+0000", 2024-05-11T23:02:00.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1869999999999996e-06 2024-05-11T23:02:00.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:00.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.144254+0000", 2024-05-11T23:02:00.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021644861000000001 2024-05-11T23:02:00.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:00.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.144350+0000", 2024-05-11T23:02:00.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5328999999999995e-05 2024-05-11T23:02:00.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:00.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.144425+0000", 2024-05-11T23:02:00.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4873e-05 2024-05-11T23:02:00.952 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.952 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.952 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.952 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:63196 3.6 3:62b1b5a1:::rbd_data.111e5e4d8275.0000000000000009:11 [read 339968~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-05-11T23:02:00.953 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:57.122657+0000", 2024-05-11T23:02:00.953 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7715186630000002, 2024-05-11T23:02:00.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021932961000000001, 2024-05-11T23:02:00.953 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:00.953 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:00.953 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:00.953 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:00.953 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:00.953 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:00.953 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 63196 2024-05-11T23:02:00.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.953 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:00.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:00.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122657+0000", 2024-05-11T23:02:00.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:00.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122657+0000", 2024-05-11T23:02:00.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:00.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122657+0000", 2024-05-11T23:02:00.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5000000000000003e-07 2024-05-11T23:02:00.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:00.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122660+0000", 2024-05-11T23:02:00.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7259999999999998e-06 2024-05-11T23:02:00.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:00.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122667+0000", 2024-05-11T23:02:00.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0350000000000004e-06 2024-05-11T23:02:00.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:00.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122672+0000", 2024-05-11T23:02:00.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2680000000000002e-06 2024-05-11T23:02:00.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:00.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.144462+0000", 2024-05-11T23:02:00.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021790018000000001 2024-05-11T23:02:00.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:00.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.144524+0000", 2024-05-11T23:02:00.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1722000000000003e-05 2024-05-11T23:02:00.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:00.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.144590+0000", 2024-05-11T23:02:00.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5642000000000006e-05 2024-05-11T23:02:00.956 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.956 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.956 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.957 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:63204 3.6 3:62b1b5a1:::rbd_data.111e5e4d8275.0000000000000009:11 [read 708608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-05-11T23:02:00.957 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:57.122701+0000", 2024-05-11T23:02:00.957 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7714744819999999, 2024-05-11T23:02:00.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022030259999999999, 2024-05-11T23:02:00.957 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:00.957 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:00.957 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:00.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:00.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:00.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:00.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 63204 2024-05-11T23:02:00.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.957 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:00.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:00.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122701+0000", 2024-05-11T23:02:00.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:00.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122701+0000", 2024-05-11T23:02:00.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:00.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122701+0000", 2024-05-11T23:02:00.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4700000000000002e-07 2024-05-11T23:02:00.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:00.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122704+0000", 2024-05-11T23:02:00.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2330000000000001e-06 2024-05-11T23:02:00.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:00.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122710+0000", 2024-05-11T23:02:00.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.596e-06 2024-05-11T23:02:00.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:00.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122715+0000", 2024-05-11T23:02:00.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0080000000000002e-06 2024-05-11T23:02:00.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:00.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.144615+0000", 2024-05-11T23:02:00.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021899946999999999 2024-05-11T23:02:00.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:00.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.144666+0000", 2024-05-11T23:02:00.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1054999999999998e-05 2024-05-11T23:02:00.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:00.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.144731+0000", 2024-05-11T23:02:00.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4974000000000005e-05 2024-05-11T23:02:00.961 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.961 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.961 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.961 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:63216 3.6 3:62b1b5a1:::rbd_data.111e5e4d8275.0000000000000009:11 [read 626688~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-05-11T23:02:00.961 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:57.122785+0000", 2024-05-11T23:02:00.961 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.771390625, 2024-05-11T23:02:00.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022148464, 2024-05-11T23:02:00.961 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:00.961 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:00.961 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:00.961 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:00.961 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:00.961 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:00.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 63216 2024-05-11T23:02:00.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.962 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:00.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:00.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122785+0000", 2024-05-11T23:02:00.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:00.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122785+0000", 2024-05-11T23:02:00.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:00.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122785+0000", 2024-05-11T23:02:00.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8100000000000003e-07 2024-05-11T23:02:00.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:00.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122787+0000", 2024-05-11T23:02:00.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2000000000000001e-06 2024-05-11T23:02:00.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:00.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122795+0000", 2024-05-11T23:02:00.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0600000000000002e-06 2024-05-11T23:02:00.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:00.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122800+0000", 2024-05-11T23:02:00.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.13e-06 2024-05-11T23:02:00.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:00.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.144792+0000", 2024-05-11T23:02:00.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021992194999999999 2024-05-11T23:02:00.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:00.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.144860+0000", 2024-05-11T23:02:00.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7719000000000007e-05 2024-05-11T23:02:00.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:00.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.144933+0000", 2024-05-11T23:02:00.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3578999999999995e-05 2024-05-11T23:02:00.965 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.965 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.966 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.966 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:63226 3.6 3:62b1b5a1:::rbd_data.111e5e4d8275.0000000000000009:11 [read 782336~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-05-11T23:02:00.966 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:57.122830+0000", 2024-05-11T23:02:00.966 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7713448839999999, 2024-05-11T23:02:00.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022243414999999999, 2024-05-11T23:02:00.966 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:00.966 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:00.966 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:00.966 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:00.966 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:00.966 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:00.966 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 63226 2024-05-11T23:02:00.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.967 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:00.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:00.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122830+0000", 2024-05-11T23:02:00.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:00.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122830+0000", 2024-05-11T23:02:00.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:00.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122831+0000", 2024-05-11T23:02:00.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2499999999999995e-07 2024-05-11T23:02:00.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:00.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122833+0000", 2024-05-11T23:02:00.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1440000000000001e-06 2024-05-11T23:02:00.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:00.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122840+0000", 2024-05-11T23:02:00.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0090000000000001e-06 2024-05-11T23:02:00.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:00.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122845+0000", 2024-05-11T23:02:00.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8709999999999996e-06 2024-05-11T23:02:00.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:00.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.144963+0000", 2024-05-11T23:02:00.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022117955000000002 2024-05-11T23:02:00.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:00.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.145009+0000", 2024-05-11T23:02:00.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5936999999999997e-05 2024-05-11T23:02:00.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:00.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.145074+0000", 2024-05-11T23:02:00.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4974000000000005e-05 2024-05-11T23:02:00.969 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:63230 3.6 3:62b1b5a1:::rbd_data.111e5e4d8275.0000000000000009:11 [read 331776~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-05-11T23:02:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:57.122874+0000", 2024-05-11T23:02:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7713017880000002, 2024-05-11T23:02:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022348384999999998, 2024-05-11T23:02:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:00.970 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 63230 2024-05-11T23:02:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122874+0000", 2024-05-11T23:02:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122874+0000", 2024-05-11T23:02:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122874+0000", 2024-05-11T23:02:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8700000000000001e-07 2024-05-11T23:02:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122876+0000", 2024-05-11T23:02:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1749999999999999e-06 2024-05-11T23:02:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122883+0000", 2024-05-11T23:02:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6100000000000002e-06 2024-05-11T23:02:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:00.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122888+0000", 2024-05-11T23:02:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8949999999999997e-06 2024-05-11T23:02:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.145107+0000", 2024-05-11T23:02:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022219467 2024-05-11T23:02:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.145161+0000", 2024-05-11T23:02:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4107999999999998e-05 2024-05-11T23:02:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:00.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.145222+0000", 2024-05-11T23:02:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0742999999999998e-05 2024-05-11T23:02:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:63238 3.6 3:62b1b5a1:::rbd_data.111e5e4d8275.0000000000000009:11 [read 700416~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e43)", 2024-05-11T23:02:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:57.122925+0000", 2024-05-11T23:02:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.7712501920000001, 2024-05-11T23:02:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022414737000000001, 2024-05-11T23:02:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:00.974 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 63238 2024-05-11T23:02:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122925+0000", 2024-05-11T23:02:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122925+0000", 2024-05-11T23:02:00.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122926+0000", 2024-05-11T23:02:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4300000000000003e-07 2024-05-11T23:02:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122928+0000", 2024-05-11T23:02:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.492e-06 2024-05-11T23:02:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122935+0000", 2024-05-11T23:02:00.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9550000000000003e-06 2024-05-11T23:02:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.122940+0000", 2024-05-11T23:02:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1399999999999999e-06 2024-05-11T23:02:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.145238+0000", 2024-05-11T23:02:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022297465999999998 2024-05-11T23:02:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.145281+0000", 2024-05-11T23:02:00.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2994000000000002e-05 2024-05-11T23:02:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:57.145340+0000", 2024-05-11T23:02:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9147000000000001e-05 2024-05-11T23:02:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:00.978 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:00.978 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:00.980 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:02:00.980 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:00.980+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:01.081 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:01.081+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:01.089 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:01.090 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:01.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.090 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:64049 3.8 3.e4e19b88 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-05-11T23:02:01.090 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:02:01.090 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.310600614, 2024-05-11T23:02:01.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.31062126600000001, 2024-05-11T23:02:01.090 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.090 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.090 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "waiting for peered", 2024-05-11T23:02:01.090 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.090 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.090 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.090 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 64049 2024-05-11T23:02:01.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.091 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:02:01.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:02:01.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779883+0000", 2024-05-11T23:02:01.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.57e-06 2024-05-11T23:02:01.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779887+0000", 2024-05-11T23:02:01.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5709999999999999e-06 2024-05-11T23:02:01.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779898+0000", 2024-05-11T23:02:01.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.042e-05 2024-05-11T23:02:01.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779905+0000", 2024-05-11T23:02:01.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0110000000000003e-06 2024-05-11T23:02:01.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779962+0000", 2024-05-11T23:02:01.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7395999999999997e-05 2024-05-11T23:02:01.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:02:01.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779969+0000", 2024-05-11T23:02:01.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2459999999999997e-06 2024-05-11T23:02:01.093 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.093 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.093 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.093 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.093 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:02:01.093 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 1 2024-05-11T23:02:01.094 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:01.098 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:02:01.182 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:01.182+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:01.208 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:01.208 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:01.208 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:01.208 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:01.208 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:01.217 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:02:01.283 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:01.284+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:01.368 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:01.369 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:01.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:01.369 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:01.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.369 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28442 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 442368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:01.369 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081216+0000", 2024-05-11T23:02:01.369 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2866332140000001, 2024-05-11T23:02:01.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031581206000000001, 2024-05-11T23:02:01.369 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.369 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.369 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:01.369 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.369 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.370 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.370 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28442 2024-05-11T23:02:01.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.370 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081216+0000", 2024-05-11T23:02:01.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081216+0000", 2024-05-11T23:02:01.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081216+0000", 2024-05-11T23:02:01.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5699999999999999e-07 2024-05-11T23:02:01.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081217+0000", 2024-05-11T23:02:01.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.111e-06 2024-05-11T23:02:01.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081225+0000", 2024-05-11T23:02:01.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9230000000000002e-06 2024-05-11T23:02:01.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081228+0000", 2024-05-11T23:02:01.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2109999999999998e-06 2024-05-11T23:02:01.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112669+0000", 2024-05-11T23:02:01.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031440659000000003 2024-05-11T23:02:01.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112736+0000", 2024-05-11T23:02:01.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7575000000000003e-05 2024-05-11T23:02:01.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112797+0000", 2024-05-11T23:02:01.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0470000000000002e-05 2024-05-11T23:02:01.373 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.373 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.373 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.373 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28445 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:01.373 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:01.373 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2866169369999998, 2024-05-11T23:02:01.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031719429, 2024-05-11T23:02:01.373 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.373 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:01.374 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.374 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.374 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.374 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28445 2024-05-11T23:02:01.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.374 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:01.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:01.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:01.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-05-11T23:02:01.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081234+0000", 2024-05-11T23:02:01.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.578e-06 2024-05-11T23:02:01.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081237+0000", 2024-05-11T23:02:01.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3340000000000002e-06 2024-05-11T23:02:01.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081240+0000", 2024-05-11T23:02:01.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7429999999999998e-06 2024-05-11T23:02:01.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112828+0000", 2024-05-11T23:02:01.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031587941000000001 2024-05-11T23:02:01.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112878+0000", 2024-05-11T23:02:01.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0535999999999999e-05 2024-05-11T23:02:01.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112951+0000", 2024-05-11T23:02:01.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3072999999999996e-05 2024-05-11T23:02:01.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.378 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.379 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28448 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 507904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:01.379 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:02:01.379 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2865844360000001, 2024-05-11T23:02:01.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031916923, 2024-05-11T23:02:01.379 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.379 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:01.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.380 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.380 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28448 2024-05-11T23:02:01.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.380 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:02:01.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:02:01.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081265+0000", 2024-05-11T23:02:01.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2899999999999999e-07 2024-05-11T23:02:01.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081267+0000", 2024-05-11T23:02:01.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6749999999999998e-06 2024-05-11T23:02:01.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081273+0000", 2024-05-11T23:02:01.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4049999999999999e-06 2024-05-11T23:02:01.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081278+0000", 2024-05-11T23:02:01.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0139999999999998e-06 2024-05-11T23:02:01.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112996+0000", 2024-05-11T23:02:01.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031718283 2024-05-11T23:02:01.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113080+0000", 2024-05-11T23:02:01.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4334999999999998e-05 2024-05-11T23:02:01.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113181+0000", 2024-05-11T23:02:01.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000100882 2024-05-11T23:02:01.383 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.383 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.383 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.383 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28451 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 540672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:01.383 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:01.383 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2865559340000008, 2024-05-11T23:02:01.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032054412999999997, 2024-05-11T23:02:01.383 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.383 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:01.384 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.384 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.384 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.384 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28451 2024-05-11T23:02:01.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.384 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:01.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:01.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:01.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2499999999999995e-07 2024-05-11T23:02:01.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081300+0000", 2024-05-11T23:02:01.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0929999999999999e-06 2024-05-11T23:02:01.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081306+0000", 2024-05-11T23:02:01.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6290000000000004e-06 2024-05-11T23:02:01.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081311+0000", 2024-05-11T23:02:01.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9289999999999997e-06 2024-05-11T23:02:01.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113208+0000", 2024-05-11T23:02:01.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031897162 2024-05-11T23:02:01.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113277+0000", 2024-05-11T23:02:01.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8811000000000002e-05 2024-05-11T23:02:01.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113347+0000", 2024-05-11T23:02:01.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0264000000000004e-05 2024-05-11T23:02:01.387 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.387 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.387 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.387 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28454 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 573440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:01.387 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:01.387 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2864813129999995, 2024-05-11T23:02:01.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032210564999999997, 2024-05-11T23:02:01.387 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.387 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.388 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:01.388 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.388 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.388 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28454 2024-05-11T23:02:01.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.388 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:01.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:01.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:01.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.99e-07 2024-05-11T23:02:01.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081371+0000", 2024-05-11T23:02:01.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1200000000000002e-06 2024-05-11T23:02:01.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081378+0000", 2024-05-11T23:02:01.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6089999999999997e-06 2024-05-11T23:02:01.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081384+0000", 2024-05-11T23:02:01.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0859999999999998e-06 2024-05-11T23:02:01.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113388+0000", 2024-05-11T23:02:01.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032004248999999999 2024-05-11T23:02:01.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113470+0000", 2024-05-11T23:02:01.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1871000000000003e-05 2024-05-11T23:02:01.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113578+0000", 2024-05-11T23:02:01.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000108031 2024-05-11T23:02:01.391 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.391 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.391 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.391 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28457 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 606208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:01.391 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:01.391 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2864510889999998, 2024-05-11T23:02:01.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032410130000000002, 2024-05-11T23:02:01.391 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.392 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.392 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:01.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.392 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28457 2024-05-11T23:02:01.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.392 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:01.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:01.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:01.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.32e-07 2024-05-11T23:02:01.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081400+0000", 2024-05-11T23:02:01.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9120000000000001e-06 2024-05-11T23:02:01.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081404+0000", 2024-05-11T23:02:01.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7900000000000001e-06 2024-05-11T23:02:01.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081407+0000", 2024-05-11T23:02:01.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8839999999999998e-06 2024-05-11T23:02:01.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113621+0000", 2024-05-11T23:02:01.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032213841 2024-05-11T23:02:01.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113702+0000", 2024-05-11T23:02:01.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1574000000000002e-05 2024-05-11T23:02:01.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113808+0000", 2024-05-11T23:02:01.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000105697 2024-05-11T23:02:01.395 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.395 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.395 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.395 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28460 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 638976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:01.395 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:01.395 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2864382580000004, 2024-05-11T23:02:01.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032616804999999999, 2024-05-11T23:02:01.396 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.396 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.396 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:01.396 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.396 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.396 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.396 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28460 2024-05-11T23:02:01.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.396 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:01.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:01.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:01.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5900000000000003e-07 2024-05-11T23:02:01.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081412+0000", 2024-05-11T23:02:01.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1990000000000001e-06 2024-05-11T23:02:01.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081415+0000", 2024-05-11T23:02:01.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.971e-06 2024-05-11T23:02:01.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081418+0000", 2024-05-11T23:02:01.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-05-11T23:02:01.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113853+0000", 2024-05-11T23:02:01.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032435155 2024-05-11T23:02:01.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113936+0000", 2024-05-11T23:02:01.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3441000000000003e-05 2024-05-11T23:02:01.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114027+0000", 2024-05-11T23:02:01.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1108999999999998e-05 2024-05-11T23:02:01.399 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.399 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.399 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.400 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28463 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 671744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:01.400 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:01.400 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2864213360000001, 2024-05-11T23:02:01.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032814958999999998, 2024-05-11T23:02:01.400 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.400 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.400 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:01.400 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.400 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.400 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28463 2024-05-11T23:02:01.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.400 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:01.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:01.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:01.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4999999999999998e-07 2024-05-11T23:02:01.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081430+0000", 2024-05-11T23:02:01.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6109999999999999e-06 2024-05-11T23:02:01.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081433+0000", 2024-05-11T23:02:01.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5590000000000001e-06 2024-05-11T23:02:01.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081437+0000", 2024-05-11T23:02:01.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3950000000000001e-06 2024-05-11T23:02:01.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114061+0000", 2024-05-11T23:02:01.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032624092 2024-05-11T23:02:01.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114142+0000", 2024-05-11T23:02:01.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0893e-05 2024-05-11T23:02:01.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114242+0000", 2024-05-11T23:02:01.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010095899999999999 2024-05-11T23:02:01.403 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.403 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.403 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.404 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28466 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 704512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:01.404 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:01.404 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2864088020000004, 2024-05-11T23:02:01.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032933658999999997, 2024-05-11T23:02:01.404 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.404 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.404 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:01.404 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.404 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.404 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.404 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28466 2024-05-11T23:02:01.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.404 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:01.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:01.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:01.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4700000000000002e-07 2024-05-11T23:02:01.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081442+0000", 2024-05-11T23:02:01.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.296e-06 2024-05-11T23:02:01.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081445+0000", 2024-05-11T23:02:01.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9009999999999998e-06 2024-05-11T23:02:01.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081447+0000", 2024-05-11T23:02:01.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4200000000000001e-06 2024-05-11T23:02:01.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114280+0000", 2024-05-11T23:02:01.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032832667000000003 2024-05-11T23:02:01.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114325+0000", 2024-05-11T23:02:01.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5018000000000002e-05 2024-05-11T23:02:01.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114374+0000", 2024-05-11T23:02:01.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.901e-05 2024-05-11T23:02:01.407 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.408 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.408 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.408 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28468 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 737280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:01.408 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:01.408 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2863887970000007, 2024-05-11T23:02:01.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033118776000000003, 2024-05-11T23:02:01.408 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.408 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.408 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:01.408 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.408 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.408 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.408 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28468 2024-05-11T23:02:01.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.409 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:01.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:01.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:01.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3799999999999998e-07 2024-05-11T23:02:01.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081462+0000", 2024-05-11T23:02:01.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4899999999999999e-06 2024-05-11T23:02:01.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081465+0000", 2024-05-11T23:02:01.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4079999999999998e-06 2024-05-11T23:02:01.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081468+0000", 2024-05-11T23:02:01.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7e-06 2024-05-11T23:02:01.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114415+0000", 2024-05-11T23:02:01.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032946766000000002 2024-05-11T23:02:01.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114497+0000", 2024-05-11T23:02:01.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1854000000000006e-05 2024-05-11T23:02:01.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114579+0000", 2024-05-11T23:02:01.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2120000000000007e-05 2024-05-11T23:02:01.412 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.412 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.412 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.412 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28471 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 770048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:01.412 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:01.412 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2863740900000007, 2024-05-11T23:02:01.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033234970000000003, 2024-05-11T23:02:01.412 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.412 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.412 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:01.412 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.412 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.413 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.413 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28471 2024-05-11T23:02:01.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.413 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:01.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:01.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:01.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-05-11T23:02:01.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081476+0000", 2024-05-11T23:02:01.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3120000000000001e-06 2024-05-11T23:02:01.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081480+0000", 2024-05-11T23:02:01.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1470000000000001e-06 2024-05-11T23:02:01.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081482+0000", 2024-05-11T23:02:01.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7480000000000002e-06 2024-05-11T23:02:01.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114616+0000", 2024-05-11T23:02:01.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033133322 2024-05-11T23:02:01.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114662+0000", 2024-05-11T23:02:01.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6020999999999999e-05 2024-05-11T23:02:01.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114710+0000", 2024-05-11T23:02:01.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8059999999999997e-05 2024-05-11T23:02:01.416 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.416 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.416 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.416 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28474 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:01.416 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:01.416 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2863580159999994, 2024-05-11T23:02:01.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033336431, 2024-05-11T23:02:01.416 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.417 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:01.417 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.417 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.417 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.417 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28474 2024-05-11T23:02:01.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.417 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:01.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:01.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:01.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.15e-07 2024-05-11T23:02:01.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081493+0000", 2024-05-11T23:02:01.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4950000000000001e-06 2024-05-11T23:02:01.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081496+0000", 2024-05-11T23:02:01.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.309e-06 2024-05-11T23:02:01.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081499+0000", 2024-05-11T23:02:01.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4570000000000001e-06 2024-05-11T23:02:01.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114740+0000", 2024-05-11T23:02:01.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033240619999999999 2024-05-11T23:02:01.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114781+0000", 2024-05-11T23:02:01.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1106999999999998e-05 2024-05-11T23:02:01.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114827+0000", 2024-05-11T23:02:01.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6028e-05 2024-05-11T23:02:01.420 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.420 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.420 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.420 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28477 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 835584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:01.420 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:01.420 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2863410020000003, 2024-05-11T23:02:01.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033527225000000001, 2024-05-11T23:02:01.420 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.420 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.421 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:01.421 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.421 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.421 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.421 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28477 2024-05-11T23:02:01.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.421 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:01.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:01.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:01.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1300000000000001e-07 2024-05-11T23:02:01.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081510+0000", 2024-05-11T23:02:01.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4610000000000001e-06 2024-05-11T23:02:01.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081513+0000", 2024-05-11T23:02:01.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2559999999999998e-06 2024-05-11T23:02:01.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081516+0000", 2024-05-11T23:02:01.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7099999999999999e-06 2024-05-11T23:02:01.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114869+0000", 2024-05-11T23:02:01.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033353718999999997 2024-05-11T23:02:01.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114954+0000", 2024-05-11T23:02:01.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4969000000000006e-05 2024-05-11T23:02:01.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115035+0000", 2024-05-11T23:02:01.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0697000000000003e-05 2024-05-11T23:02:01.424 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.424 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.424 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.424 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28480 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 868352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:01.425 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:01.425 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2863240319999996, 2024-05-11T23:02:01.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033695939000000001, 2024-05-11T23:02:01.425 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.425 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.425 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:01.425 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.425 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.425 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.425 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28480 2024-05-11T23:02:01.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.425 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:01.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:01.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:01.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.46e-07 2024-05-11T23:02:01.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081526+0000", 2024-05-11T23:02:01.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.347e-06 2024-05-11T23:02:01.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081530+0000", 2024-05-11T23:02:01.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.348e-06 2024-05-11T23:02:01.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081533+0000", 2024-05-11T23:02:01.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4369999999999999e-06 2024-05-11T23:02:01.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115078+0000", 2024-05-11T23:02:01.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033545063 2024-05-11T23:02:01.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.431 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.431 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115164+0000", 2024-05-11T23:02:01.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5662999999999996e-05 2024-05-11T23:02:01.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.431 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.431 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115221+0000", 2024-05-11T23:02:01.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6734999999999998e-05 2024-05-11T23:02:01.431 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.431 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.431 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.432 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28483 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:01.432 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:01.432 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2863068589999997, 2024-05-11T23:02:01.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033803466999999997, 2024-05-11T23:02:01.432 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.432 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.432 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:01.432 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.432 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.432 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.432 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28483 2024-05-11T23:02:01.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.432 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:01.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:01.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:01.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4999999999999998e-07 2024-05-11T23:02:01.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081544+0000", 2024-05-11T23:02:01.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:02:01.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081547+0000", 2024-05-11T23:02:01.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3139999999999999e-06 2024-05-11T23:02:01.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081550+0000", 2024-05-11T23:02:01.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7779999999999999e-06 2024-05-11T23:02:01.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115253+0000", 2024-05-11T23:02:01.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033703516000000003 2024-05-11T23:02:01.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115302+0000", 2024-05-11T23:02:01.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8869999999999998e-05 2024-05-11T23:02:01.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115345+0000", 2024-05-11T23:02:01.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3340999999999997e-05 2024-05-11T23:02:01.435 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.435 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.435 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.436 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28487 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 933888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:01.436 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:01.436 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2862956580000002, 2024-05-11T23:02:01.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033903918999999998, 2024-05-11T23:02:01.436 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.436 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.436 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:01.436 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.436 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.436 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.436 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28487 2024-05-11T23:02:01.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.437 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:01.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:01.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:01.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9999999999999999e-07 2024-05-11T23:02:01.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081559+0000", 2024-05-11T23:02:01.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-05-11T23:02:01.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081563+0000", 2024-05-11T23:02:01.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2370000000000003e-06 2024-05-11T23:02:01.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081566+0000", 2024-05-11T23:02:01.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6199999999999999e-06 2024-05-11T23:02:01.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115375+0000", 2024-05-11T23:02:01.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033808887000000003 2024-05-11T23:02:01.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115413+0000", 2024-05-11T23:02:01.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8309000000000002e-05 2024-05-11T23:02:01.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115457+0000", 2024-05-11T23:02:01.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3809e-05 2024-05-11T23:02:01.439 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.440 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.440 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.440 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28490 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 966656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:01.440 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:01.440 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2862791340000008, 2024-05-11T23:02:01.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034114903000000002, 2024-05-11T23:02:01.440 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.440 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.440 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:01.440 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.440 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.440 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.440 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28490 2024-05-11T23:02:01.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.441 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:01.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:01.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:01.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4400000000000001e-07 2024-05-11T23:02:01.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081571+0000", 2024-05-11T23:02:01.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2130000000000001e-06 2024-05-11T23:02:01.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081574+0000", 2024-05-11T23:02:01.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8270000000000002e-06 2024-05-11T23:02:01.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.443 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.443 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081577+0000", 2024-05-11T23:02:01.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-05-11T23:02:01.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.443 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.443 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115507+0000", 2024-05-11T23:02:01.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033929925999999999 2024-05-11T23:02:01.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.443 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.443 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115599+0000", 2024-05-11T23:02:01.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1571999999999997e-05 2024-05-11T23:02:01.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115685+0000", 2024-05-11T23:02:01.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6071999999999999e-05 2024-05-11T23:02:01.444 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.444 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.444 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.444 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28493 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 999424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:01.444 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:01.444 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2862670309999995, 2024-05-11T23:02:01.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034232944000000001, 2024-05-11T23:02:01.444 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.444 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.444 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:01.445 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.445 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.445 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.445 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28493 2024-05-11T23:02:01.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.445 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:01.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:01.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:01.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1300000000000001e-07 2024-05-11T23:02:01.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081584+0000", 2024-05-11T23:02:01.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9479999999999998e-06 2024-05-11T23:02:01.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081590+0000", 2024-05-11T23:02:01.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.728e-06 2024-05-11T23:02:01.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081595+0000", 2024-05-11T23:02:01.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8579999999999999e-06 2024-05-11T23:02:01.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115723+0000", 2024-05-11T23:02:01.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034128473999999999 2024-05-11T23:02:01.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115768+0000", 2024-05-11T23:02:01.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5108999999999998e-05 2024-05-11T23:02:01.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115815+0000", 2024-05-11T23:02:01.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6514000000000002e-05 2024-05-11T23:02:01.448 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.448 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.448 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.448 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28496 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:01.448 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:02:01.448 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2862475720000006, 2024-05-11T23:02:01.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034427754999999997, 2024-05-11T23:02:01.449 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.449 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.449 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:01.449 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.449 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.449 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.449 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28496 2024-05-11T23:02:01.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.449 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.449 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:02:01.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:02:01.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081602+0000", 2024-05-11T23:02:01.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:02:01.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081604+0000", 2024-05-11T23:02:01.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3999999999999999e-06 2024-05-11T23:02:01.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081610+0000", 2024-05-11T23:02:01.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.224e-06 2024-05-11T23:02:01.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081615+0000", 2024-05-11T23:02:01.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8269999999999997e-06 2024-05-11T23:02:01.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115859+0000", 2024-05-11T23:02:01.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034243498999999997 2024-05-11T23:02:01.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.452 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.452 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115946+0000", 2024-05-11T23:02:01.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7122999999999999e-05 2024-05-11T23:02:01.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.452 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.452 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116029+0000", 2024-05-11T23:02:01.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3332999999999996e-05 2024-05-11T23:02:01.452 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.452 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.452 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.452 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28498 3.0 3:05d5189e:::rbd_object_map.111e5e4d8275.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:01.453 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:02:01.453 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2847289199999992, 2024-05-11T23:02:01.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.035035108000000002, 2024-05-11T23:02:01.453 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.453 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.453 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.453 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:01.453 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:01.453 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:01.453 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28498 2024-05-11T23:02:01.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.453 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:02:01.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.454 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.454 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:02:01.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.454 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.454 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083121+0000", 2024-05-11T23:02:01.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5700000000000001e-07 2024-05-11T23:02:01.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.454 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.454 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083126+0000", 2024-05-11T23:02:01.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8860000000000003e-06 2024-05-11T23:02:01.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.455 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083135+0000", 2024-05-11T23:02:01.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8369999999999992e-06 2024-05-11T23:02:01.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.455 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083142+0000", 2024-05-11T23:02:01.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4779999999999996e-06 2024-05-11T23:02:01.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.455 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116072+0000", 2024-05-11T23:02:01.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032929991999999998 2024-05-11T23:02:01.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116158+0000", 2024-05-11T23:02:01.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6077999999999998e-05 2024-05-11T23:02:01.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-05-11T23:02:01.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116513+0000", 2024-05-11T23:02:01.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00035508300000000002 2024-05-11T23:02:01.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:01.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118025+0000", 2024-05-11T23:02:01.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015118919999999999 2024-05-11T23:02:01.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:02:01.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118039+0000", 2024-05-11T23:02:01.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4188999999999999e-05 2024-05-11T23:02:01.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118094+0000", 2024-05-11T23:02:01.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4929000000000002e-05 2024-05-11T23:02:01.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118155+0000", 2024-05-11T23:02:01.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0887000000000002e-05 2024-05-11T23:02:01.458 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.458 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.458 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.458 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.458 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.458 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:01.459 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:02:01.460 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:01.459+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:01.560 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:01.560+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:01.610 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:01.610 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:01.611 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:02:01.611 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:01.619 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:02:01.663 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:01.663+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:01.728 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:01.729 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:01.729 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:01.729 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:01.729 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:01.737 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:02:01.779 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:01.779+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:01.857 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone19 from 0xc8e0944 (overlap 0x45f72ff) to 0x14fd5b2 2024-05-11T23:02:01.863 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:01.864+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:01.894 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-11T23:02:01.909 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:25950 3.0 e40/19 3:05d5189e:::rbd_object_map.111e5e4d8275.0000000000000011:head v 40'116, mlcod=39'112)", 2024-05-11T23:02:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.973119+0000", 2024-05-11T23:02:01.910 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.9348798180000006, 2024-05-11T23:02:01.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001399029, 2024-05-11T23:02:01.910 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.910 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.910 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.910 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.973119+0000", 2024-05-11T23:02:01.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.973119+0000", 2024-05-11T23:02:01.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.973120+0000", 2024-05-11T23:02:01.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8800000000000003e-07 2024-05-11T23:02:01.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.973129+0000", 2024-05-11T23:02:01.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3379999999999994e-06 2024-05-11T23:02:01.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.973138+0000", 2024-05-11T23:02:01.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7609999999999996e-06 2024-05-11T23:02:01.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.973148+0000", 2024-05-11T23:02:01.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0332e-05 2024-05-11T23:02:01.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.973277+0000", 2024-05-11T23:02:01.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012877200000000001 2024-05-11T23:02:01.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.973300+0000", 2024-05-11T23:02:01.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2359000000000001e-05 2024-05-11T23:02:01.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.974481+0000", 2024-05-11T23:02:01.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00118143 2024-05-11T23:02:01.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.974518+0000", 2024-05-11T23:02:01.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7549000000000003e-05 2024-05-11T23:02:01.913 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.913 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.913 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.914 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:25952 3.9 e40/19 3:908f208a:::rbd_data.111e5e4d8275.0000000000000002:head v 40'65, mlcod=39'63)", 2024-05-11T23:02:01.914 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.977473+0000", 2024-05-11T23:02:01.914 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.930526746, 2024-05-11T23:02:01.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0037656679999999998, 2024-05-11T23:02:01.914 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.914 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.914 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.914 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.977473+0000", 2024-05-11T23:02:01.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.977473+0000", 2024-05-11T23:02:01.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.977473+0000", 2024-05-11T23:02:01.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7300000000000002e-07 2024-05-11T23:02:01.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979179+0000", 2024-05-11T23:02:01.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017062869999999999 2024-05-11T23:02:01.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979190+0000", 2024-05-11T23:02:01.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0341e-05 2024-05-11T23:02:01.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979194+0000", 2024-05-11T23:02:01.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7369999999999997e-06 2024-05-11T23:02:01.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979250+0000", 2024-05-11T23:02:01.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6060999999999997e-05 2024-05-11T23:02:01.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979275+0000", 2024-05-11T23:02:01.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4267999999999999e-05 2024-05-11T23:02:01.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.981206+0000", 2024-05-11T23:02:01.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001931316 2024-05-11T23:02:01.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.981238+0000", 2024-05-11T23:02:01.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2285e-05 2024-05-11T23:02:01.917 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.917 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.917 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.918 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:25953 3.9 e40/19 3:908f208a:::rbd_data.111e5e4d8275.0000000000000002:head v 40'67, mlcod=39'63)", 2024-05-11T23:02:01.918 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.979457+0000", 2024-05-11T23:02:01.918 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.9285427550000005, 2024-05-11T23:02:01.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002255132, 2024-05-11T23:02:01.918 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.918 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.918 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.918 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979457+0000", 2024-05-11T23:02:01.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979457+0000", 2024-05-11T23:02:01.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979457+0000", 2024-05-11T23:02:01.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3600000000000004e-07 2024-05-11T23:02:01.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979462+0000", 2024-05-11T23:02:01.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5109999999999997e-06 2024-05-11T23:02:01.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979469+0000", 2024-05-11T23:02:01.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6979999999999998e-06 2024-05-11T23:02:01.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979474+0000", 2024-05-11T23:02:01.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-05-11T23:02:01.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.980421+0000", 2024-05-11T23:02:01.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00094687500000000002 2024-05-11T23:02:01.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.980444+0000", 2024-05-11T23:02:01.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2878e-05 2024-05-11T23:02:01.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.981657+0000", 2024-05-11T23:02:01.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0012138749999999999 2024-05-11T23:02:01.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.981712+0000", 2024-05-11T23:02:01.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4282e-05 2024-05-11T23:02:01.922 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.922 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.922 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.922 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:26861 3.0 e40/19 3:05d5189e:::rbd_object_map.111e5e4d8275.0000000000000011:head v 40'121, mlcod=40'119)", 2024-05-11T23:02:01.922 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.008555+0000", 2024-05-11T23:02:01.922 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.8994441700000007, 2024-05-11T23:02:01.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015206460000000001, 2024-05-11T23:02:01.922 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.922 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.922 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.922 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008555+0000", 2024-05-11T23:02:01.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008555+0000", 2024-05-11T23:02:01.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008557+0000", 2024-05-11T23:02:01.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8840000000000001e-06 2024-05-11T23:02:01.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008566+0000", 2024-05-11T23:02:01.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2790000000000004e-06 2024-05-11T23:02:01.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008586+0000", 2024-05-11T23:02:01.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9956999999999999e-05 2024-05-11T23:02:01.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008596+0000", 2024-05-11T23:02:01.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8900000000000002e-06 2024-05-11T23:02:01.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008666+0000", 2024-05-11T23:02:01.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9814000000000006e-05 2024-05-11T23:02:01.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008688+0000", 2024-05-11T23:02:01.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2365e-05 2024-05-11T23:02:01.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.010029+0000", 2024-05-11T23:02:01.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0013403880000000001 2024-05-11T23:02:01.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.010076+0000", 2024-05-11T23:02:01.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7068999999999999e-05 2024-05-11T23:02:01.926 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.926 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.926 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.926 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:27374 3.0 e40/19 3:05d5189e:::rbd_object_map.111e5e4d8275.0000000000000011:head v 40'127, mlcod=40'124)", 2024-05-11T23:02:01.926 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.034130+0000", 2024-05-11T23:02:01.926 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.8738688630000002, 2024-05-11T23:02:01.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001526082, 2024-05-11T23:02:01.927 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.927 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.927 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.927 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034130+0000", 2024-05-11T23:02:01.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034130+0000", 2024-05-11T23:02:01.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034138+0000", 2024-05-11T23:02:01.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8960000000000003e-06 2024-05-11T23:02:01.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034154+0000", 2024-05-11T23:02:01.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6152999999999999e-05 2024-05-11T23:02:01.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034171+0000", 2024-05-11T23:02:01.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6121e-05 2024-05-11T23:02:01.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034177+0000", 2024-05-11T23:02:01.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1079999999999998e-06 2024-05-11T23:02:01.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034612+0000", 2024-05-11T23:02:01.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043566900000000002 2024-05-11T23:02:01.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034631+0000", 2024-05-11T23:02:01.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8643000000000001e-05 2024-05-11T23:02:01.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.035619+0000", 2024-05-11T23:02:01.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00098804600000000002 2024-05-11T23:02:01.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.035657+0000", 2024-05-11T23:02:01.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7446e-05 2024-05-11T23:02:01.930 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.930 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.930 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.930 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:27790 3.0 e40/19 3:05d5189e:::rbd_object_map.111e5e4d8275.0000000000000011:head v 40'130, mlcod=40'128)", 2024-05-11T23:02:01.930 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.060478+0000", 2024-05-11T23:02:01.931 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.8475215259999995, 2024-05-11T23:02:01.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0014537739999999999, 2024-05-11T23:02:01.931 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.931 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.931 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.931 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060478+0000", 2024-05-11T23:02:01.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060478+0000", 2024-05-11T23:02:01.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060479+0000", 2024-05-11T23:02:01.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8999999999999995e-07 2024-05-11T23:02:01.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060488+0000", 2024-05-11T23:02:01.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9879999999999993e-06 2024-05-11T23:02:01.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060499+0000", 2024-05-11T23:02:01.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1489e-05 2024-05-11T23:02:01.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060505+0000", 2024-05-11T23:02:01.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5330000000000002e-06 2024-05-11T23:02:01.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060560+0000", 2024-05-11T23:02:01.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5384999999999999e-05 2024-05-11T23:02:01.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060582+0000", 2024-05-11T23:02:01.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1889999999999999e-05 2024-05-11T23:02:01.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.061883+0000", 2024-05-11T23:02:01.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0013005989999999999 2024-05-11T23:02:01.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.061932+0000", 2024-05-11T23:02:01.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8999999999999998e-05 2024-05-11T23:02:01.934 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.934 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.934 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:28504 3.9 e40/19 3:92d63908:::rbd_data.111e5e4d8275.0000000000000013:head v 40'68, mlcod=40'65)", 2024-05-11T23:02:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.092954+0000", 2024-05-11T23:02:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.8150451990000001, 2024-05-11T23:02:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0030983059999999999, 2024-05-11T23:02:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.092954+0000", 2024-05-11T23:02:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.092954+0000", 2024-05-11T23:02:01.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.092955+0000", 2024-05-11T23:02:01.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.13e-06 2024-05-11T23:02:01.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094110+0000", 2024-05-11T23:02:01.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0011551669999999999 2024-05-11T23:02:01.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094124+0000", 2024-05-11T23:02:01.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3244999999999999e-05 2024-05-11T23:02:01.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094131+0000", 2024-05-11T23:02:01.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0090000000000001e-06 2024-05-11T23:02:01.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094187+0000", 2024-05-11T23:02:01.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6218999999999998e-05 2024-05-11T23:02:01.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094205+0000", 2024-05-11T23:02:01.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8148e-05 2024-05-11T23:02:01.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.096006+0000", 2024-05-11T23:02:01.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018005619999999999 2024-05-11T23:02:01.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.096052+0000", 2024-05-11T23:02:01.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6826000000000002e-05 2024-05-11T23:02:01.938 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.939 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.939 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.939 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:28505 3.9 e40/19 3:92d63908:::rbd_data.111e5e4d8275.0000000000000013:head v 40'70, mlcod=40'65)", 2024-05-11T23:02:01.939 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.094530+0000", 2024-05-11T23:02:01.939 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.8134692650000002, 2024-05-11T23:02:01.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021142140000000001, 2024-05-11T23:02:01.939 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.939 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.939 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.939 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094530+0000", 2024-05-11T23:02:01.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094530+0000", 2024-05-11T23:02:01.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094531+0000", 2024-05-11T23:02:01.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9400000000000004e-07 2024-05-11T23:02:01.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094536+0000", 2024-05-11T23:02:01.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.575e-06 2024-05-11T23:02:01.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094549+0000", 2024-05-11T23:02:01.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2857000000000001e-05 2024-05-11T23:02:01.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094556+0000", 2024-05-11T23:02:01.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6050000000000003e-06 2024-05-11T23:02:01.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.095097+0000", 2024-05-11T23:02:01.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00054118299999999999 2024-05-11T23:02:01.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.095119+0000", 2024-05-11T23:02:01.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1673000000000001e-05 2024-05-11T23:02:01.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.096603+0000", 2024-05-11T23:02:01.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001484153 2024-05-11T23:02:01.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.096644+0000", 2024-05-11T23:02:01.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1374000000000001e-05 2024-05-11T23:02:01.943 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.943 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.943 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.943 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:28824 3.9 e40/19 3:9050c332:::rbd_data.111e5e4d8275.000000000000001d:head v 40'71, mlcod=40'68)", 2024-05-11T23:02:01.943 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.141602+0000", 2024-05-11T23:02:01.943 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.7663970219999996, 2024-05-11T23:02:01.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0033604630000000002, 2024-05-11T23:02:01.943 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.943 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.944 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.944 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.141602+0000", 2024-05-11T23:02:01.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.141602+0000", 2024-05-11T23:02:01.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.141604+0000", 2024-05-11T23:02:01.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4300000000000001e-06 2024-05-11T23:02:01.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.142793+0000", 2024-05-11T23:02:01.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0011888409999999999 2024-05-11T23:02:01.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.142815+0000", 2024-05-11T23:02:01.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2432000000000001e-05 2024-05-11T23:02:01.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.142826+0000", 2024-05-11T23:02:01.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0943e-05 2024-05-11T23:02:01.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.142901+0000", 2024-05-11T23:02:01.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5283000000000004e-05 2024-05-11T23:02:01.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.142935+0000", 2024-05-11T23:02:01.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4014000000000003e-05 2024-05-11T23:02:01.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144916+0000", 2024-05-11T23:02:01.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019805019999999999 2024-05-11T23:02:01.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144963+0000", 2024-05-11T23:02:01.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7018000000000003e-05 2024-05-11T23:02:01.947 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.947 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.947 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.947 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:28825 3.9 e40/19 3:9050c332:::rbd_data.111e5e4d8275.000000000000001d:head v 40'73, mlcod=40'68)", 2024-05-11T23:02:01.947 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.143269+0000", 2024-05-11T23:02:01.947 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.7647301360000007, 2024-05-11T23:02:01.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0020932170000000001, 2024-05-11T23:02:01.948 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.948 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.948 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.948 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.143269+0000", 2024-05-11T23:02:01.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.143269+0000", 2024-05-11T23:02:01.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.143270+0000", 2024-05-11T23:02:01.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1200000000000001e-07 2024-05-11T23:02:01.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.143278+0000", 2024-05-11T23:02:01.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1079999999999993e-06 2024-05-11T23:02:01.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.143289+0000", 2024-05-11T23:02:01.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0961000000000001e-05 2024-05-11T23:02:01.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.143297+0000", 2024-05-11T23:02:01.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3480000000000004e-06 2024-05-11T23:02:01.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144016+0000", 2024-05-11T23:02:01.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00071809800000000002 2024-05-11T23:02:01.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144043+0000", 2024-05-11T23:02:01.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7746000000000002e-05 2024-05-11T23:02:01.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145309+0000", 2024-05-11T23:02:01.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0012660060000000001 2024-05-11T23:02:01.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145362+0000", 2024-05-11T23:02:01.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3038000000000003e-05 2024-05-11T23:02:01.951 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.951 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.951 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.951 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:28828 3.9 e40/19 3:93a019b9:::rbd_data.111e5e4d8275.000000000000001f:head v 40'74, mlcod=40'68)", 2024-05-11T23:02:01.951 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.144176+0000", 2024-05-11T23:02:01.952 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.7638235410000007, 2024-05-11T23:02:01.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0029854930000000001, 2024-05-11T23:02:01.952 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.952 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.952 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.952 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144176+0000", 2024-05-11T23:02:01.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144176+0000", 2024-05-11T23:02:01.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144177+0000", 2024-05-11T23:02:01.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9899999999999999e-07 2024-05-11T23:02:01.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145289+0000", 2024-05-11T23:02:01.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0011121670000000001 2024-05-11T23:02:01.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145304+0000", 2024-05-11T23:02:01.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5590000000000002e-05 2024-05-11T23:02:01.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145312+0000", 2024-05-11T23:02:01.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8529999999999992e-06 2024-05-11T23:02:01.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145393+0000", 2024-05-11T23:02:01.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1050999999999994e-05 2024-05-11T23:02:01.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145412+0000", 2024-05-11T23:02:01.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8992999999999999e-05 2024-05-11T23:02:01.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.147116+0000", 2024-05-11T23:02:01.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017039589999999999 2024-05-11T23:02:01.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.147161+0000", 2024-05-11T23:02:01.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5080999999999997e-05 2024-05-11T23:02:01.955 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.955 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.955 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.956 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:28829 3.9 e40/19 3:93a019b9:::rbd_data.111e5e4d8275.000000000000001f:head v 40'76, mlcod=40'71)", 2024-05-11T23:02:01.956 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.145924+0000", 2024-05-11T23:02:01.956 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.7620751049999992, 2024-05-11T23:02:01.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015381520000000001, 2024-05-11T23:02:01.956 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.956 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.956 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145924+0000", 2024-05-11T23:02:01.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145924+0000", 2024-05-11T23:02:01.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145925+0000", 2024-05-11T23:02:01.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.99e-07 2024-05-11T23:02:01.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145933+0000", 2024-05-11T23:02:01.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7090000000000002e-06 2024-05-11T23:02:01.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145944+0000", 2024-05-11T23:02:01.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1177e-05 2024-05-11T23:02:01.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145951+0000", 2024-05-11T23:02:01.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6450000000000005e-06 2024-05-11T23:02:01.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.146272+0000", 2024-05-11T23:02:01.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032017699999999999 2024-05-11T23:02:01.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.146290+0000", 2024-05-11T23:02:01.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7977000000000001e-05 2024-05-11T23:02:01.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.147432+0000", 2024-05-11T23:02:01.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0011421879999999999 2024-05-11T23:02:01.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.147462+0000", 2024-05-11T23:02:01.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0580000000000002e-05 2024-05-11T23:02:01.959 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.959 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.960 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.960 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:34962 3.9 e40/19 3:96260786:::rbd_data.111ea078695e.0000000000000028:head v 40'77, mlcod=40'74)", 2024-05-11T23:02:01.960 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.700882+0000", 2024-05-11T23:02:01.960 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.2071172590000003, 2024-05-11T23:02:01.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001779938, 2024-05-11T23:02:01.960 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.960 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.960 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.960 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.700882+0000", 2024-05-11T23:02:01.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.700882+0000", 2024-05-11T23:02:01.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.700884+0000", 2024-05-11T23:02:01.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5030000000000001e-06 2024-05-11T23:02:01.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.701472+0000", 2024-05-11T23:02:01.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00058797299999999999 2024-05-11T23:02:01.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.701491+0000", 2024-05-11T23:02:01.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9850000000000001e-05 2024-05-11T23:02:01.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.701505+0000", 2024-05-11T23:02:01.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3549e-05 2024-05-11T23:02:01.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.701574+0000", 2024-05-11T23:02:01.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8937999999999996e-05 2024-05-11T23:02:01.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.701598+0000", 2024-05-11T23:02:01.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4519e-05 2024-05-11T23:02:01.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.702631+0000", 2024-05-11T23:02:01.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001032942 2024-05-11T23:02:01.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.702662+0000", 2024-05-11T23:02:01.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0664000000000003e-05 2024-05-11T23:02:01.964 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.964 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.964 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.964 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:37451 3.0 e40/19 3:071059df:::rbd_data.111ea078695e.000000000000001f:head v 40'145, mlcod=40'143)", 2024-05-11T23:02:01.964 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:53.078457+0000", 2024-05-11T23:02:01.964 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.8295417910000005, 2024-05-11T23:02:01.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0020281650000000002, 2024-05-11T23:02:01.964 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.964 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.964 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.964 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.078457+0000", 2024-05-11T23:02:01.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.078457+0000", 2024-05-11T23:02:01.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.078459+0000", 2024-05-11T23:02:01.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4500000000000001e-06 2024-05-11T23:02:01.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.078988+0000", 2024-05-11T23:02:01.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00052866000000000002 2024-05-11T23:02:01.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.079007+0000", 2024-05-11T23:02:01.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9398e-05 2024-05-11T23:02:01.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.079018+0000", 2024-05-11T23:02:01.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0834e-05 2024-05-11T23:02:01.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.079086+0000", 2024-05-11T23:02:01.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8578000000000006e-05 2024-05-11T23:02:01.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.079109+0000", 2024-05-11T23:02:01.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2189e-05 2024-05-11T23:02:01.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.080429+0000", 2024-05-11T23:02:01.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0013205070000000001 2024-05-11T23:02:01.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.080486+0000", 2024-05-11T23:02:01.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6548999999999997e-05 2024-05-11T23:02:01.968 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.968 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.968 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.968 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:37457 3.9 e40/19 3:95e7444d:::rbd_data.111ea078695e.0000000000000022:head v 40'79, mlcod=40'77)", 2024-05-11T23:02:01.968 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:53.087914+0000", 2024-05-11T23:02:01.968 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.8200850919999993, 2024-05-11T23:02:01.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0023284349999999998, 2024-05-11T23:02:01.968 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.968 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.969 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.969 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.087914+0000", 2024-05-11T23:02:01.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.087914+0000", 2024-05-11T23:02:01.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.087916+0000", 2024-05-11T23:02:01.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7260000000000001e-06 2024-05-11T23:02:01.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.088797+0000", 2024-05-11T23:02:01.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00088118400000000003 2024-05-11T23:02:01.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.088814+0000", 2024-05-11T23:02:01.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6484999999999999e-05 2024-05-11T23:02:01.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.088820+0000", 2024-05-11T23:02:01.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8029999999999999e-06 2024-05-11T23:02:01.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.088872+0000", 2024-05-11T23:02:01.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1415999999999997e-05 2024-05-11T23:02:01.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.088890+0000", 2024-05-11T23:02:01.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8529000000000002e-05 2024-05-11T23:02:01.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.090179+0000", 2024-05-11T23:02:01.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0012889539999999999 2024-05-11T23:02:01.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.090243+0000", 2024-05-11T23:02:01.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3337999999999996e-05 2024-05-11T23:02:01.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.972 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.972 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60295 3.9 e42/19 3:92a0646f:::rbd_data.111e1df04b77.000000000000000a:head v 42'117, mlcod=42'115)", 2024-05-11T23:02:01.972 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.021408+0000", 2024-05-11T23:02:01.972 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.886591514, 2024-05-11T23:02:01.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0013993829999999999, 2024-05-11T23:02:01.973 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.973 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.973 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.973 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.021408+0000", 2024-05-11T23:02:01.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.021408+0000", 2024-05-11T23:02:01.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.021408+0000", 2024-05-11T23:02:01.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3900000000000004e-07 2024-05-11T23:02:01.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022002+0000", 2024-05-11T23:02:01.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000593992 2024-05-11T23:02:01.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022012+0000", 2024-05-11T23:02:01.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8390000000000002e-06 2024-05-11T23:02:01.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022017+0000", 2024-05-11T23:02:01.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6789999999999996e-06 2024-05-11T23:02:01.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022069+0000", 2024-05-11T23:02:01.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1934000000000001e-05 2024-05-11T23:02:01.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022089+0000", 2024-05-11T23:02:01.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9650999999999999e-05 2024-05-11T23:02:01.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022777+0000", 2024-05-11T23:02:01.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000688797 2024-05-11T23:02:01.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022807+0000", 2024-05-11T23:02:01.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9852000000000001e-05 2024-05-11T23:02:01.976 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.976 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.976 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.977 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60336 3.9 e42/19 3:94f86c79:::rbd_data.111e1df04b77.0000000000000001:head v 42'119, mlcod=42'117)", 2024-05-11T23:02:01.977 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.065103+0000", 2024-05-11T23:02:01.977 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.8428960459999999, 2024-05-11T23:02:01.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0016346970000000001, 2024-05-11T23:02:01.977 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.977 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.977 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.977 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065103+0000", 2024-05-11T23:02:01.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065103+0000", 2024-05-11T23:02:01.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065104+0000", 2024-05-11T23:02:01.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.054e-06 2024-05-11T23:02:01.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065864+0000", 2024-05-11T23:02:01.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00075959699999999998 2024-05-11T23:02:01.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065879+0000", 2024-05-11T23:02:01.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4953999999999999e-05 2024-05-11T23:02:01.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065887+0000", 2024-05-11T23:02:01.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1329999999999999e-06 2024-05-11T23:02:01.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065954+0000", 2024-05-11T23:02:01.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6675999999999997e-05 2024-05-11T23:02:01.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065978+0000", 2024-05-11T23:02:01.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4243000000000002e-05 2024-05-11T23:02:01.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.066707+0000", 2024-05-11T23:02:01.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072877 2024-05-11T23:02:01.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.066738+0000", 2024-05-11T23:02:01.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1269999999999997e-05 2024-05-11T23:02:01.980 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.981 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.981 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.981 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60471 3.4 e43/19 3:2079ea20:::rbd_data.111e1df04b77.0000000000000000:head v 43'84, mlcod=42'81)", 2024-05-11T23:02:01.981 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:01.981 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.9811252770000003, 2024-05-11T23:02:01.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0079043050000000004, 2024-05-11T23:02:01.981 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.981 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.981 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.981 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:01.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:01.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:01.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-05-11T23:02:01.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926880+0000", 2024-05-11T23:02:01.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:02:01.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926893+0000", 2024-05-11T23:02:01.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3256e-05 2024-05-11T23:02:01.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926899+0000", 2024-05-11T23:02:01.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9850000000000003e-06 2024-05-11T23:02:01.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926982+0000", 2024-05-11T23:02:01.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2997000000000005e-05 2024-05-11T23:02:01.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927005+0000", 2024-05-11T23:02:01.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3428e-05 2024-05-11T23:02:01.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.934747+0000", 2024-05-11T23:02:01.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0077415820000000003 2024-05-11T23:02:01.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.934778+0000", 2024-05-11T23:02:01.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1479999999999997e-05 2024-05-11T23:02:01.985 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.985 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.985 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.985 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60472 3.9 e43/19 3:94f86c79:::rbd_data.111e1df04b77.0000000000000001:head v 43'122, mlcod=42'119)", 2024-05-11T23:02:01.985 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.927124+0000", 2024-05-11T23:02:01.985 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.9808755769999999, 2024-05-11T23:02:01.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0062295900000000001, 2024-05-11T23:02:01.985 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.985 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.985 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.985 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927124+0000", 2024-05-11T23:02:01.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927124+0000", 2024-05-11T23:02:01.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927124+0000", 2024-05-11T23:02:01.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:01.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927128+0000", 2024-05-11T23:02:01.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3390000000000003e-06 2024-05-11T23:02:01.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927139+0000", 2024-05-11T23:02:01.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1161e-05 2024-05-11T23:02:01.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927147+0000", 2024-05-11T23:02:01.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1910000000000003e-06 2024-05-11T23:02:01.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927199+0000", 2024-05-11T23:02:01.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.24e-05 2024-05-11T23:02:01.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927223+0000", 2024-05-11T23:02:01.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3811999999999998e-05 2024-05-11T23:02:01.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.933287+0000", 2024-05-11T23:02:01.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.006063723 2024-05-11T23:02:01.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.933353+0000", 2024-05-11T23:02:01.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6744999999999999e-05 2024-05-11T23:02:01.989 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.989 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.989 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.989 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60478 3.4 e43/19 3:2113d5af:::rbd_data.111e1df04b77.0000000000000007:head v 43'86, mlcod=42'82)", 2024-05-11T23:02:01.989 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.935643+0000", 2024-05-11T23:02:01.989 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.9723559569999995, 2024-05-11T23:02:01.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0016792039999999999, 2024-05-11T23:02:01.989 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:01.989 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:01.989 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:01.990 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:01.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:01.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.935643+0000", 2024-05-11T23:02:01.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:01.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.935643+0000", 2024-05-11T23:02:01.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:01.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:01.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.935645+0000", 2024-05-11T23:02:01.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4419999999999999e-06 2024-05-11T23:02:01.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:01.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.935653+0000", 2024-05-11T23:02:01.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0460000000000006e-06 2024-05-11T23:02:01.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:01.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.935669+0000", 2024-05-11T23:02:01.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5809000000000001e-05 2024-05-11T23:02:01.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:01.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.935676+0000", 2024-05-11T23:02:01.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3100000000000003e-06 2024-05-11T23:02:01.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:01.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.936410+0000", 2024-05-11T23:02:01.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00073357399999999997 2024-05-11T23:02:01.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:01.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.936423+0000", 2024-05-11T23:02:01.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3762e-05 2024-05-11T23:02:01.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:01.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.937274+0000", 2024-05-11T23:02:01.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00085112600000000001 2024-05-11T23:02:01.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:01.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:01.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:01.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.937323+0000", 2024-05-11T23:02:01.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8134999999999999e-05 2024-05-11T23:02:01.993 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.993 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.993 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.993 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:01.994 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:01.994 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:01.995 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:02:01.996 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-05-11T23:02:01.996 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:01.995+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:02.082 INFO:teuthology.orchestra.run.smithi005.stdout:2518 write 0x85672e3 thru 0x8573b88 (0xc8a6 bytes) 2024-05-11T23:02:02.095 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:02.095+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:02.119 INFO:teuthology.orchestra.run.smithi005.stdout:2519 write 0xb8a282d thru 0xb8a6eb7 (0x468b bytes) 2024-05-11T23:02:02.128 INFO:teuthology.orchestra.run.smithi005.stdout:2521 trunc from 0xc8e0944 to 0x823c73d 2024-05-11T23:02:02.147 INFO:teuthology.orchestra.run.smithi005.stdout:2522 write 0x1eacd7b thru 0x1eb096f (0x3bf5 bytes) 2024-05-11T23:02:02.149 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:02.149 INFO:teuthology.orchestra.run.smithi005.stdout:2524 write 0x207611d thru 0x207c8a7 (0x678b bytes) 2024-05-11T23:02:02.150 INFO:teuthology.orchestra.run.smithi005.stdout:2525 writesame 0x5548e9c thru 0x554b17e data_size 0xd(0x22e3 bytes) 2024-05-11T23:02:02.158 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:02.158 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:02:02.158 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:02.163 INFO:teuthology.orchestra.run.smithi005.stdout:2526 write 0xc514575 thru 0xc51f032 (0xaabe bytes) 2024-05-11T23:02:02.165 INFO:teuthology.orchestra.run.smithi005.stdout:2528 writesame 0x4a485bf thru 0x4a4b970 data_size 0xd(0x33b2 bytes) 2024-05-11T23:02:02.168 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:02:02.173 INFO:teuthology.orchestra.run.smithi005.stdout:2529 write 0xce667a5 thru 0xce69770 (0x2fcc bytes) 2024-05-11T23:02:02.175 INFO:teuthology.orchestra.run.smithi005.stdout:2531 writesame 0x6a0f9c4 thru 0x6a13ff6 data_size 0x1(0x4633 bytes) 2024-05-11T23:02:02.185 INFO:teuthology.orchestra.run.smithi005.stdout:2533 read 0x1741094 thru 0x174b153 (0xa0c0 bytes) 2024-05-11T23:02:02.185 INFO:teuthology.orchestra.run.smithi005.stdout:2534 read 0x11781e5 thru 0x117c39b (0x41b7 bytes) 2024-05-11T23:02:02.186 INFO:teuthology.orchestra.run.smithi005.stdout:2535 write 0x55ab9b0 thru 0x55b056a (0x4bbb bytes) 2024-05-11T23:02:02.187 INFO:teuthology.orchestra.run.smithi005.stdout:2536 read 0x8e472d1 thru 0x8e553a2 (0xe0d2 bytes) 2024-05-11T23:02:02.196 INFO:teuthology.orchestra.run.smithi005.stdout:2537 trunc from 0xce69771 to 0x5a6927c 2024-05-11T23:02:02.197 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:02.198+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:02.240 INFO:teuthology.orchestra.run.smithi005.stdout:2538 writesame 0xb12c6fb thru 0xb1384f2 data_size 0x4(0xbdf8 bytes) 2024-05-11T23:02:02.241 INFO:teuthology.orchestra.run.smithi005.stdout:2539 read 0xa72f34e thru 0xa731af7 (0x27aa bytes) 2024-05-11T23:02:02.241 INFO:teuthology.orchestra.run.smithi005.stdout:2540 writesame 0xab594f6 thru 0xab68ed6 data_size 0x1(0xf9e1 bytes) 2024-05-11T23:02:02.270 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:02.270 INFO:teuthology.orchestra.run.smithi005.stdout:2543 punch from 0xa4abea6 to 0xa4afd50, (0x3eaa bytes) 2024-05-11T23:02:02.270 INFO:teuthology.orchestra.run.smithi005.stdout:2544 read 0xf9d09 thru 0x1070f5 (0xd3ed bytes) 2024-05-11T23:02:02.271 INFO:teuthology.orchestra.run.smithi005.stdout:2545 writesame 0x1f16d42 thru 0x1f1efc1 data_size 0xc(0x8280 bytes) 2024-05-11T23:02:02.273 INFO:teuthology.orchestra.run.smithi005.stdout:2546 write 0xfe3df8 thru 0xfeb34d (0x7556 bytes) 2024-05-11T23:02:02.283 INFO:teuthology.orchestra.run.smithi005.stdout:2547 write 0xe6b4f2f thru 0xe6b76c8 (0x279a bytes) 2024-05-11T23:02:02.286 INFO:teuthology.orchestra.run.smithi005.stdout:2549 write 0xaa5fc51 thru 0xaa627f9 (0x2ba9 bytes) 2024-05-11T23:02:02.288 INFO:teuthology.orchestra.run.smithi005.stdout:2551 write 0xa17a407 thru 0xa17ed80 (0x497a bytes) 2024-05-11T23:02:02.289 INFO:teuthology.orchestra.run.smithi005.stdout:2552 read 0x1088fa9 thru 0x108b8d6 (0x292e bytes) 2024-05-11T23:02:02.294 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:02.294 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:02.294 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:02.295 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:02.295 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:02.297 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:02.298+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:02.301 INFO:teuthology.orchestra.run.smithi005.stdout:2553 trunc from 0xe6b76c9 to 0x3e7ddd2 2024-05-11T23:02:02.308 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:02:02.320 INFO:teuthology.orchestra.run.smithi005.stdout:2555 trunc from 0x3e7ddd2 to 0x120a300 2024-05-11T23:02:02.343 INFO:teuthology.orchestra.run.smithi005.stdout:2556 writesame 0x364f34e thru 0x36553f0 data_size 0xd(0x60a3 bytes) 2024-05-11T23:02:02.344 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:02.344 INFO:teuthology.orchestra.run.smithi005.stdout:2560 punch from 0x35b119c to 0x35bef6f, (0xddd3 bytes) 2024-05-11T23:02:02.344 INFO:teuthology.orchestra.run.smithi005.stdout:2561 write 0xca56e1 thru 0xcae63a (0x8f5a bytes) 2024-05-11T23:02:02.351 INFO:teuthology.orchestra.run.smithi005.stdout:2562 write 0x3748fe6 thru 0x3756169 (0xd184 bytes) 2024-05-11T23:02:02.354 INFO:teuthology.orchestra.run.smithi005.stdout:2563 punch from 0x1c838a4 to 0x1c87f1f, (0x467b bytes) 2024-05-11T23:02:02.354 INFO:teuthology.orchestra.run.smithi005.stdout:2564 read 0x2613535 thru 0x2618914 (0x53e0 bytes) 2024-05-11T23:02:02.354 INFO:teuthology.orchestra.run.smithi005.stdout:2565 read 0xd4de9c thru 0xd53297 (0x53fc bytes) 2024-05-11T23:02:02.354 INFO:teuthology.orchestra.run.smithi005.stdout:2566 punch from 0x1e0ccaf to 0x1e15b44, (0x8e95 bytes) 2024-05-11T23:02:02.362 INFO:teuthology.orchestra.run.smithi005.stdout:2567 write 0x5aa8738 thru 0x5ab52c4 (0xcb8d bytes) 2024-05-11T23:02:02.373 INFO:teuthology.orchestra.run.smithi005.stdout:2568 trunc from 0x5ab52c5 to 0xc993213 2024-05-11T23:02:02.384 INFO:teuthology.orchestra.run.smithi005.stdout:2570 trunc from 0xc993213 to 0x6c80a5f 2024-05-11T23:02:02.398 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:02.398+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:02.401 INFO:teuthology.orchestra.run.smithi005.stdout:2571 writesame 0xb503307 thru 0xb506937 data_size 0x1(0x3631 bytes) 2024-05-11T23:02:02.401 INFO:teuthology.orchestra.run.smithi005.stdout:2573 read 0x1e21127 thru 0x1e2d8ac (0xc786 bytes) 2024-05-11T23:02:02.401 INFO:teuthology.orchestra.run.smithi005.stdout:2574 write 0xa89e773 thru 0xa8a0d61 (0x25ef bytes) 2024-05-11T23:02:02.402 INFO:teuthology.orchestra.run.smithi005.stdout:2575 write 0x392df12 thru 0x393a7f5 (0xc8e4 bytes) 2024-05-11T23:02:02.405 INFO:teuthology.orchestra.run.smithi005.stdout:2576 write 0x9db8005 thru 0x9dbe3ec (0x63e8 bytes) 2024-05-11T23:02:02.411 INFO:teuthology.orchestra.run.smithi005.stdout:2577 write 0x6754805 thru 0x6762dc4 (0xe5c0 bytes) 2024-05-11T23:02:02.413 INFO:teuthology.orchestra.run.smithi005.stdout:2578 write 0x273254c thru 0x2741c38 (0xf6ed bytes) 2024-05-11T23:02:02.414 INFO:teuthology.orchestra.run.smithi005.stdout:2579 punch from 0x1de9215 to 0x1decb15, (0x3900 bytes) 2024-05-11T23:02:02.414 INFO:teuthology.orchestra.run.smithi005.stdout:2580 read 0x8fadb0d thru 0x8fb780c (0x9d00 bytes) 2024-05-11T23:02:02.415 INFO:teuthology.orchestra.run.smithi005.stdout:2582 trunc from 0xb506938 to 0xc2ece28 2024-05-11T23:02:02.420 INFO:teuthology.orchestra.run.smithi005.stdout:2583 write 0x4e49acb thru 0x4e51e1c (0x8352 bytes) 2024-05-11T23:02:02.423 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:02.423 INFO:teuthology.orchestra.run.smithi005.stdout:2585 write 0xb2cc25b thru 0xb2cfab5 (0x385b bytes) 2024-05-11T23:02:02.425 INFO:teuthology.orchestra.run.smithi005.stdout:2586 write 0x1730882 thru 0x173ef24 (0xe6a3 bytes) 2024-05-11T23:02:02.426 INFO:teuthology.orchestra.run.smithi005.stdout:2587 write 0xa8347ed thru 0xa8406e4 (0xbef8 bytes) 2024-05-11T23:02:02.436 INFO:teuthology.orchestra.run.smithi005.stdout:2588 writesame 0xec6ccd1 thru 0xec75eb0 data_size 0x30(0x91e0 bytes) 2024-05-11T23:02:02.436 INFO:teuthology.orchestra.run.smithi005.stdout:2589 punch from 0x947d0d4 to 0x948bcd4, (0xec00 bytes) 2024-05-11T23:02:02.436 INFO:teuthology.orchestra.run.smithi005.stdout:2591 punch from 0xb8bf498 to 0xb8cea34, (0xf59c bytes) 2024-05-11T23:02:02.436 INFO:teuthology.orchestra.run.smithi005.stdout:2593 writesame 0x47ca231 thru 0x47d93a7 data_size 0xc3(0xf177 bytes) 2024-05-11T23:02:02.438 INFO:teuthology.orchestra.run.smithi005.stdout:2596 punch from 0x56e7eec to 0x56ec890, (0x49a4 bytes) 2024-05-11T23:02:02.440 INFO:teuthology.orchestra.run.smithi005.stdout:2597 trunc from 0xec75eb1 to 0xd5f0c28 2024-05-11T23:02:02.446 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:02.446 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:02.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:02.446 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:02.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.447 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36828 3.b 3:d385bc8e:::rbd_data.111e5e4d8275.0000000000000020:11 [read 512000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.447 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:02.447 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5126618119999993, 2024-05-11T23:02:02.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051481218000000002, 2024-05-11T23:02:02.447 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:02.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:02.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:02.447 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:02.447 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:02.447 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:02.447 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36828 2024-05-11T23:02:02.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.449 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:02.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:02.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:02.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:02.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:02.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:02.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934915+0000", 2024-05-11T23:02:02.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00181814 2024-05-11T23:02:02.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:02.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934919+0000", 2024-05-11T23:02:02.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6739999999999996e-06 2024-05-11T23:02:02.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:02.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934934+0000", 2024-05-11T23:02:02.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4415e-05 2024-05-11T23:02:02.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:02.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934942+0000", 2024-05-11T23:02:02.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8619999999999992e-06 2024-05-11T23:02:02.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.452 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:02.452 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984433+0000", 2024-05-11T23:02:02.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049490658999999999 2024-05-11T23:02:02.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.452 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:02.452 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984508+0000", 2024-05-11T23:02:02.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4394999999999995e-05 2024-05-11T23:02:02.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:02.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984578+0000", 2024-05-11T23:02:02.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0073000000000005e-05 2024-05-11T23:02:02.453 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.453 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.453 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.454 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36950 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 520192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.454 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:02.454 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5083560459999994, 2024-05-11T23:02:02.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051450856000000003, 2024-05-11T23:02:02.454 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:02.454 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:02.454 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:02.455 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:02.455 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:02.455 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:02.455 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36950 2024-05-11T23:02:02.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.455 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:02.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:02.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:02.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:02.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:02.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:02.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938914+0000", 2024-05-11T23:02:02.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015121480000000001 2024-05-11T23:02:02.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:02.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938918+0000", 2024-05-11T23:02:02.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4149999999999999e-06 2024-05-11T23:02:02.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:02.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938925+0000", 2024-05-11T23:02:02.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0779999999999998e-06 2024-05-11T23:02:02.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:02.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938929+0000", 2024-05-11T23:02:02.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2930000000000002e-06 2024-05-11T23:02:02.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:02.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988675+0000", 2024-05-11T23:02:02.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049746394999999999 2024-05-11T23:02:02.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:02.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988756+0000", 2024-05-11T23:02:02.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0499000000000003e-05 2024-05-11T23:02:02.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:02.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988853+0000", 2024-05-11T23:02:02.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7027999999999993e-05 2024-05-11T23:02:02.462 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.463 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.463 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.463 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36957 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 778240~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.463 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:02.463 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5067342569999997, 2024-05-11T23:02:02.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051328273000000001, 2024-05-11T23:02:02.464 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:02.464 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:02.464 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:02.464 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:02.464 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:02.464 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:02.465 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36957 2024-05-11T23:02:02.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.465 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:02.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:02.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:02.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:02.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:02.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:02.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:02.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2099999999999998e-07 2024-05-11T23:02:02.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:02.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939026+0000", 2024-05-11T23:02:02.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5069999999999999e-06 2024-05-11T23:02:02.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:02.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939030+0000", 2024-05-11T23:02:02.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7919999999999999e-06 2024-05-11T23:02:02.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:02.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939032+0000", 2024-05-11T23:02:02.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9129999999999999e-06 2024-05-11T23:02:02.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:02.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990178+0000", 2024-05-11T23:02:02.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051145373000000001 2024-05-11T23:02:02.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:02.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990255+0000", 2024-05-11T23:02:02.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7195999999999997e-05 2024-05-11T23:02:02.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:02.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990352+0000", 2024-05-11T23:02:02.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7170999999999995e-05 2024-05-11T23:02:02.472 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.473 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.473 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.473 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36958 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 815104~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.473 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:02.474 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5067219380000001, 2024-05-11T23:02:02.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051531625999999997, 2024-05-11T23:02:02.474 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:02.474 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:02.474 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:02.474 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:02.475 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:02.475 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:02.475 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36958 2024-05-11T23:02:02.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.475 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:02.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:02.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:02.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:02.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:02.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:02.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939037+0000", 2024-05-11T23:02:02.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0400000000000002e-07 2024-05-11T23:02:02.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:02.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939038+0000", 2024-05-11T23:02:02.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.164e-06 2024-05-11T23:02:02.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:02.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939041+0000", 2024-05-11T23:02:02.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1109999999999999e-06 2024-05-11T23:02:02.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:02.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939043+0000", 2024-05-11T23:02:02.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.554e-06 2024-05-11T23:02:02.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:02.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990382+0000", 2024-05-11T23:02:02.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051338953999999999 2024-05-11T23:02:02.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:02.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990465+0000", 2024-05-11T23:02:02.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2461999999999998e-05 2024-05-11T23:02:02.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:02.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990568+0000", 2024-05-11T23:02:02.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000103077 2024-05-11T23:02:02.483 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.483 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.483 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.484 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36959 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 851968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.484 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:02.484 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5067100419999999, 2024-05-11T23:02:02.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051758936999999998, 2024-05-11T23:02:02.485 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:02.485 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:02.485 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:02.485 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:02.485 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:02.485 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:02.485 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36959 2024-05-11T23:02:02.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.485 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:02.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:02.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:02.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:02.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:02.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:02.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:02.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.16e-07 2024-05-11T23:02:02.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:02.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939050+0000", 2024-05-11T23:02:02.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.176e-06 2024-05-11T23:02:02.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:02.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939053+0000", 2024-05-11T23:02:02.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1140000000000002e-06 2024-05-11T23:02:02.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:02.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939055+0000", 2024-05-11T23:02:02.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.621e-06 2024-05-11T23:02:02.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:02.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990610+0000", 2024-05-11T23:02:02.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051554244999999999 2024-05-11T23:02:02.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:02.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990700+0000", 2024-05-11T23:02:02.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0284999999999994e-05 2024-05-11T23:02:02.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:02.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990807+0000", 2024-05-11T23:02:02.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010728 2024-05-11T23:02:02.490 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.491 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.491 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.491 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36960 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 888832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.491 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:02.491 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5066998070000004, 2024-05-11T23:02:02.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051886831000000001, 2024-05-11T23:02:02.492 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:02.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:02.492 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:02.492 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:02.492 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:02.492 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:02.492 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36960 2024-05-11T23:02:02.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.492 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:02.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:02.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:02.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:02.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:02.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:02.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940913+0000", 2024-05-11T23:02:02.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001855104 2024-05-11T23:02:02.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:02.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940916+0000", 2024-05-11T23:02:02.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0000000000000001e-06 2024-05-11T23:02:02.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:02.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940924+0000", 2024-05-11T23:02:02.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0999999999999998e-06 2024-05-11T23:02:02.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:02.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940927+0000", 2024-05-11T23:02:02.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7160000000000001e-06 2024-05-11T23:02:02.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:02.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990826+0000", 2024-05-11T23:02:02.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049898832999999997 2024-05-11T23:02:02.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:02.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990872+0000", 2024-05-11T23:02:02.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5717000000000003e-05 2024-05-11T23:02:02.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:02.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990945+0000", 2024-05-11T23:02:02.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3361000000000005e-05 2024-05-11T23:02:02.495 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.495 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.496 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.496 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36986 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 528384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.496 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:02.496 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5046935890000004, 2024-05-11T23:02:02.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051341053999999997, 2024-05-11T23:02:02.496 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:02.496 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:02.496 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:02.496 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:02.496 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:02.496 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:02.496 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36986 2024-05-11T23:02:02.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.497 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:02.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:02.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:02.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:02.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:02.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:02.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:02.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:02:02.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:02.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941066+0000", 2024-05-11T23:02:02.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1629999999999999e-06 2024-05-11T23:02:02.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:02.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941069+0000", 2024-05-11T23:02:02.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.118e-06 2024-05-11T23:02:02.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:02.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941072+0000", 2024-05-11T23:02:02.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5720000000000001e-06 2024-05-11T23:02:02.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:02.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992292+0000", 2024-05-11T23:02:02.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051220620000000001 2024-05-11T23:02:02.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:02.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992344+0000", 2024-05-11T23:02:02.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1737000000000003e-05 2024-05-11T23:02:02.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:02.500 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992406+0000", 2024-05-11T23:02:02.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1623000000000003e-05 2024-05-11T23:02:02.500 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.500 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.500 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.501 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36987 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 565248~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.501 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:02.501 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5046834679999996, 2024-05-11T23:02:02.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051568725000000003, 2024-05-11T23:02:02.501 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:02.502 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:02.502 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:02.502 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:02.502 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:02.502 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:02.502 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36987 2024-05-11T23:02:02.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.503 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:02.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:02.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:02.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:02.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:02.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:02.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:02.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.91e-07 2024-05-11T23:02:02.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:02.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941076+0000", 2024-05-11T23:02:02.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1379999999999999e-06 2024-05-11T23:02:02.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:02.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941079+0000", 2024-05-11T23:02:02.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9739999999999998e-06 2024-05-11T23:02:02.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:02.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941082+0000", 2024-05-11T23:02:02.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-05-11T23:02:02.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:02.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992446+0000", 2024-05-11T23:02:02.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051364058999999997 2024-05-11T23:02:02.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:02.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992536+0000", 2024-05-11T23:02:02.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0085000000000002e-05 2024-05-11T23:02:02.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:02.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992643+0000", 2024-05-11T23:02:02.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107377 2024-05-11T23:02:02.510 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.510 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.510 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.511 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36988 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 602112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.511 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:02.511 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5046730989999997, 2024-05-11T23:02:02.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051700007999999999, 2024-05-11T23:02:02.511 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:02.511 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:02.512 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:02.512 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:02.512 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:02.512 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:02.512 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36988 2024-05-11T23:02:02.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.513 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:02.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:02.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:02.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:02.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:02.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:02.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:02.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.05e-07 2024-05-11T23:02:02.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:02.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941086+0000", 2024-05-11T23:02:02.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.125e-06 2024-05-11T23:02:02.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:02.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941089+0000", 2024-05-11T23:02:02.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0599999999999999e-06 2024-05-11T23:02:02.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:02.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941092+0000", 2024-05-11T23:02:02.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6529999999999998e-06 2024-05-11T23:02:02.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:02.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992667+0000", 2024-05-11T23:02:02.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051574587999999998 2024-05-11T23:02:02.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:02.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992723+0000", 2024-05-11T23:02:02.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6682999999999999e-05 2024-05-11T23:02:02.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:02.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992785+0000", 2024-05-11T23:02:02.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1693999999999996e-05 2024-05-11T23:02:02.520 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.520 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.520 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.521 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36989 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 638976~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.521 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:02.521 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5046629100000004, 2024-05-11T23:02:02.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05192811, 2024-05-11T23:02:02.521 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:02.522 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:02.522 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:02.522 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:02.522 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:02.522 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:02.522 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36989 2024-05-11T23:02:02.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.523 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:02.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:02.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:02.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:02.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:02.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:02.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942916+0000", 2024-05-11T23:02:02.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001820835 2024-05-11T23:02:02.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:02.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942920+0000", 2024-05-11T23:02:02.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5310000000000002e-06 2024-05-11T23:02:02.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:02.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942931+0000", 2024-05-11T23:02:02.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1839e-05 2024-05-11T23:02:02.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:02.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942940+0000", 2024-05-11T23:02:02.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2309999999999996e-06 2024-05-11T23:02:02.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:02.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992820+0000", 2024-05-11T23:02:02.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049880751000000001 2024-05-11T23:02:02.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:02.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992918+0000", 2024-05-11T23:02:02.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7051000000000003e-05 2024-05-11T23:02:02.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:02.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.993023+0000", 2024-05-11T23:02:02.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010587200000000001 2024-05-11T23:02:02.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.528 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.528 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36997 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 933888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.528 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:02.529 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5026290749999998, 2024-05-11T23:02:02.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051390602000000001, 2024-05-11T23:02:02.529 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:02.529 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:02.529 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:02.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:02.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:02.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:02.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36997 2024-05-11T23:02:02.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.529 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:02.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:02.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:02.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:02.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:02.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:02.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943130+0000", 2024-05-11T23:02:02.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0200000000000003e-07 2024-05-11T23:02:02.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:02.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943132+0000", 2024-05-11T23:02:02.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.305e-06 2024-05-11T23:02:02.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:02.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943138+0000", 2024-05-11T23:02:02.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4219999999999997e-06 2024-05-11T23:02:02.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:02.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943143+0000", 2024-05-11T23:02:02.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0610000000000003e-06 2024-05-11T23:02:02.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:02.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994323+0000", 2024-05-11T23:02:02.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051179592000000003 2024-05-11T23:02:02.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:02.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994412+0000", 2024-05-11T23:02:02.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8709999999999996e-05 2024-05-11T23:02:02.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:02.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994520+0000", 2024-05-11T23:02:02.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010811000000000001 2024-05-11T23:02:02.532 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.532 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.532 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.533 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36998 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 970752~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.533 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:02.533 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5026081960000006, 2024-05-11T23:02:02.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051517669000000002, 2024-05-11T23:02:02.533 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:02.533 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:02.533 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:02.533 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:02.533 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:02.533 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:02.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36998 2024-05-11T23:02:02.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.533 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:02.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:02.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:02.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:02.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:02.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:02.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:02.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:02:02.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:02.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943153+0000", 2024-05-11T23:02:02.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.306e-06 2024-05-11T23:02:02.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:02.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943160+0000", 2024-05-11T23:02:02.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.452e-06 2024-05-11T23:02:02.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:02.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943165+0000", 2024-05-11T23:02:02.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.152e-06 2024-05-11T23:02:02.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:02.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994561+0000", 2024-05-11T23:02:02.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051395765000000003 2024-05-11T23:02:02.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:02.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994607+0000", 2024-05-11T23:02:02.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6415000000000002e-05 2024-05-11T23:02:02.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:02.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994668+0000", 2024-05-11T23:02:02.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0179000000000001e-05 2024-05-11T23:02:02.536 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.536 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.536 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.537 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36999 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 1007616~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.537 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:02.537 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5025633149999997, 2024-05-11T23:02:02.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051696919000000001, 2024-05-11T23:02:02.537 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:02.537 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:02.537 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:02.537 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:02.537 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:02.537 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:02.537 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36999 2024-05-11T23:02:02.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.537 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:02.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:02.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:02.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:02.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:02.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:02.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:02.539 INFO:teuthology.orchestra.run.smithi005.stdout:2599 writesame 0x1c17140 thru 0x1c18109 data_size 0x2(0xfca bytes) 2024-05-11T23:02:02.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:02:02.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:02.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943197+0000", 2024-05-11T23:02:02.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1280000000000002e-06 2024-05-11T23:02:02.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:02.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943202+0000", 2024-05-11T23:02:02.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8910000000000003e-06 2024-05-11T23:02:02.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:02.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943206+0000", 2024-05-11T23:02:02.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3550000000000001e-06 2024-05-11T23:02:02.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:02.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994703+0000", 2024-05-11T23:02:02.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051497045999999998 2024-05-11T23:02:02.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:02.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994789+0000", 2024-05-11T23:02:02.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6458000000000005e-05 2024-05-11T23:02:02.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:02.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994892+0000", 2024-05-11T23:02:02.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102705 2024-05-11T23:02:02.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.541 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.542 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37000 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 1044480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.542 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:02.542 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5025473700000003, 2024-05-11T23:02:02.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051842371999999998, 2024-05-11T23:02:02.542 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:02.542 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:02.542 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:02.542 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:02.542 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:02.542 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:02.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37000 2024-05-11T23:02:02.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.543 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:02.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:02.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:02.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:02.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:02.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:02.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944916+0000", 2024-05-11T23:02:02.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001705097 2024-05-11T23:02:02.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:02.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944921+0000", 2024-05-11T23:02:02.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5809999999999998e-06 2024-05-11T23:02:02.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:02.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944930+0000", 2024-05-11T23:02:02.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5529999999999998e-06 2024-05-11T23:02:02.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:02.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944938+0000", 2024-05-11T23:02:02.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2519999999999999e-06 2024-05-11T23:02:02.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:02.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994911+0000", 2024-05-11T23:02:02.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049973113999999999 2024-05-11T23:02:02.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:02.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994982+0000", 2024-05-11T23:02:02.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0544999999999996e-05 2024-05-11T23:02:02.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:02.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.995053+0000", 2024-05-11T23:02:02.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1229999999999994e-05 2024-05-11T23:02:02.545 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.545 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.545 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.546 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37041 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 618496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.546 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:02.546 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5007033910000001, 2024-05-11T23:02:02.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051352768999999999, 2024-05-11T23:02:02.546 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:02.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:02.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:02.546 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:02.546 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:02.546 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:02.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37041 2024-05-11T23:02:02.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.546 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:02.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:02.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:02.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:02.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:02.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:02.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:02.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:02:02.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:02.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945056+0000", 2024-05-11T23:02:02.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:02:02.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:02.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945059+0000", 2024-05-11T23:02:02.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.965e-06 2024-05-11T23:02:02.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:02.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945062+0000", 2024-05-11T23:02:02.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:02:02.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:02.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996213+0000", 2024-05-11T23:02:02.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051151532999999999 2024-05-11T23:02:02.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:02.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996301+0000", 2024-05-11T23:02:02.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8143e-05 2024-05-11T23:02:02.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:02.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996408+0000", 2024-05-11T23:02:02.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010623700000000001 2024-05-11T23:02:02.549 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.549 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.550 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.550 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37042 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 655360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.550 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:02.550 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5006934059999999, 2024-05-11T23:02:02.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051462075000000003, 2024-05-11T23:02:02.550 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:02.550 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:02.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:02.550 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:02.550 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:02.550 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:02.550 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37042 2024-05-11T23:02:02.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.551 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:02.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:02.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:02.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:02.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:02.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:02.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:02.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0599999999999999e-07 2024-05-11T23:02:02.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:02.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945066+0000", 2024-05-11T23:02:02.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-05-11T23:02:02.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:02.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945069+0000", 2024-05-11T23:02:02.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0079999999999998e-06 2024-05-11T23:02:02.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:02.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945072+0000", 2024-05-11T23:02:02.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6740000000000001e-06 2024-05-11T23:02:02.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:02.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996422+0000", 2024-05-11T23:02:02.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051349999 2024-05-11T23:02:02.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:02.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996467+0000", 2024-05-11T23:02:02.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.528e-05 2024-05-11T23:02:02.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:02.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996527+0000", 2024-05-11T23:02:02.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9759999999999997e-05 2024-05-11T23:02:02.554 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.554 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.554 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.554 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37043 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 692224~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.554 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:02.554 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5006813329999993, 2024-05-11T23:02:02.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051679535999999998, 2024-05-11T23:02:02.554 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:02.554 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:02.554 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:02.555 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:02.555 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:02.555 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:02.555 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37043 2024-05-11T23:02:02.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.555 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:02.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:02.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:02.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:02.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:02.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:02.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:02.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8999999999999997e-07 2024-05-11T23:02:02.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:02.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945079+0000", 2024-05-11T23:02:02.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.001e-06 2024-05-11T23:02:02.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:02.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945087+0000", 2024-05-11T23:02:02.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1749999999999999e-06 2024-05-11T23:02:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945093+0000", 2024-05-11T23:02:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9639999999999997e-06 2024-05-11T23:02:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996563+0000", 2024-05-11T23:02:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051469050000000002 2024-05-11T23:02:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996651+0000", 2024-05-11T23:02:02.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8356000000000006e-05 2024-05-11T23:02:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996756+0000", 2024-05-11T23:02:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0001055 2024-05-11T23:02:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37044 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 729088~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.500647421, 2024-05-11T23:02:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051763445999999998, 2024-05-11T23:02:02.558 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37044 2024-05-11T23:02:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946915+0000", 2024-05-11T23:02:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018042760000000001 2024-05-11T23:02:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946918+0000", 2024-05-11T23:02:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.36e-06 2024-05-11T23:02:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946926+0000", 2024-05-11T23:02:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3130000000000001e-06 2024-05-11T23:02:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946930+0000", 2024-05-11T23:02:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5249999999999999e-06 2024-05-11T23:02:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996772+0000", 2024-05-11T23:02:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049841868999999997 2024-05-11T23:02:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996817+0000", 2024-05-11T23:02:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5351e-05 2024-05-11T23:02:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996874+0000", 2024-05-11T23:02:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6752000000000001e-05 2024-05-11T23:02:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37052 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 1024000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.4987405700000007, 2024-05-11T23:02:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051326140999999999, 2024-05-11T23:02:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37052 2024-05-11T23:02:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947019+0000", 2024-05-11T23:02:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.218e-06 2024-05-11T23:02:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947022+0000", 2024-05-11T23:02:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1889999999999999e-06 2024-05-11T23:02:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947025+0000", 2024-05-11T23:02:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.464e-06 2024-05-11T23:02:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998237+0000", 2024-05-11T23:02:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051212494999999997 2024-05-11T23:02:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998284+0000", 2024-05-11T23:02:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7246000000000001e-05 2024-05-11T23:02:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998344+0000", 2024-05-11T23:02:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9320000000000001e-05 2024-05-11T23:02:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.566 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37071 3.b 3:d385bc8e:::rbd_data.111e5e4d8275.0000000000000020:11 [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.4987303609999998, 2024-05-11T23:02:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051528796000000002, 2024-05-11T23:02:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37071 2024-05-11T23:02:02.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948913+0000", 2024-05-11T23:02:02.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018854970000000001 2024-05-11T23:02:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948917+0000", 2024-05-11T23:02:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8089999999999999e-06 2024-05-11T23:02:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948926+0000", 2024-05-11T23:02:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8389999999999996e-06 2024-05-11T23:02:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948930+0000", 2024-05-11T23:02:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4519999999999999e-06 2024-05-11T23:02:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998379+0000", 2024-05-11T23:02:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049449011000000001 2024-05-11T23:02:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998462+0000", 2024-05-11T23:02:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2739999999999997e-05 2024-05-11T23:02:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998557+0000", 2024-05-11T23:02:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4448000000000001e-05 2024-05-11T23:02:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.570 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.571 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.571 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:02.571 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:02.571 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:02.572 INFO:teuthology.orchestra.run.smithi005.stdout:2600 read 0x5b6a8ab thru 0x5b7694d (0xc0a3 bytes) 2024-05-11T23:02:02.572 INFO:teuthology.orchestra.run.smithi005.stdout:2601 read 0x5dfec27 thru 0x5e07710 (0x8aea bytes) 2024-05-11T23:02:02.572 INFO:teuthology.orchestra.run.smithi005.stdout:2602 read 0x9704a7c thru 0x9714855 (0xfdda bytes) 2024-05-11T23:02:02.572 INFO:teuthology.orchestra.run.smithi005.stdout:2603 punch from 0x50d94e4 to 0x50da2f8, (0xe14 bytes) 2024-05-11T23:02:02.572 INFO:teuthology.orchestra.run.smithi005.stdout:2604 write 0x1d0b5b5 thru 0x1d1698d (0xb3d9 bytes) 2024-05-11T23:02:02.573 INFO:teuthology.orchestra.run.smithi005.stdout:2606 read 0x53604e4 thru 0x536c48f (0xbfac bytes) 2024-05-11T23:02:02.573 INFO:teuthology.orchestra.run.smithi005.stdout:2607 writesame 0x88be7a4 thru 0x88c34c8 data_size 0x1d(0x4d25 bytes) 2024-05-11T23:02:02.573 INFO:teuthology.orchestra.run.smithi005.stdout:2608 write 0xb04e381 thru 0xb0500bc (0x1d3c bytes) 2024-05-11T23:02:02.573 INFO:teuthology.orchestra.run.smithi005.stdout:2609 punch from 0x506b85b to 0x507a8a5, (0xf04a bytes) 2024-05-11T23:02:02.573 INFO:teuthology.orchestra.run.smithi005.stdout:2610 read 0x61cf0db thru 0x61d5ab4 (0x69da bytes) 2024-05-11T23:02:02.573 INFO:teuthology.orchestra.run.smithi005.stdout:2611 punch from 0x22bf806 to 0x22c7d4e, (0x8548 bytes) 2024-05-11T23:02:02.573 INFO:teuthology.orchestra.run.smithi005.stdout:2612 punch from 0x5a5c587 to 0x5a5d76e, (0x11e7 bytes) 2024-05-11T23:02:02.573 INFO:teuthology.orchestra.run.smithi005.stdout:2613 write 0xb6a2014 thru 0xb6a25e4 (0x5d1 bytes) 2024-05-11T23:02:02.574 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:02:02.577 INFO:teuthology.orchestra.run.smithi005.stdout:2614 write 0xac29466 thru 0xac2dc04 (0x479f bytes) 2024-05-11T23:02:02.577 INFO:teuthology.orchestra.run.smithi005.stdout:2615 writesame 0xbf507b8 thru 0xbf55818 data_size 0x13(0x5061 bytes) 2024-05-11T23:02:02.577 INFO:teuthology.orchestra.run.smithi005.stdout:2617 write 0x9eb885a thru 0x9ec3c14 (0xb3bb bytes) 2024-05-11T23:02:02.577 INFO:teuthology.orchestra.run.smithi005.stdout:2619 read 0xa3f0af1 thru 0xa3f475b (0x3c6b bytes) 2024-05-11T23:02:02.577 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:02.577 INFO:teuthology.orchestra.run.smithi005.stdout:2622 punch from 0xa14a414 to 0xa156a5b, (0xc647 bytes) 2024-05-11T23:02:02.577 INFO:teuthology.orchestra.run.smithi005.stdout:2623 trunc from 0xd5f0c28 to 0x75287dd 2024-05-11T23:02:02.577 INFO:teuthology.orchestra.run.smithi005.stdout:2624 punch from 0x2705ce6 to 0x270c3e2, (0x66fc bytes) 2024-05-11T23:02:02.578 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:02.578 INFO:teuthology.orchestra.run.smithi005.stdout:2628 read 0x25cf26 thru 0x2678b3 (0xa98e bytes) 2024-05-11T23:02:02.578 INFO:teuthology.orchestra.run.smithi005.stdout:2630 punch from 0x4b98cda to 0x4ba472e, (0xba54 bytes) 2024-05-11T23:02:02.578 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:02.578 INFO:teuthology.orchestra.run.smithi005.stdout:2633 trunc from 0x75287dd to 0xb72ebf0 2024-05-11T23:02:02.578 INFO:teuthology.orchestra.run.smithi005.stdout:2634 write 0x95adb88 thru 0x95bb506 (0xd97f bytes) 2024-05-11T23:02:02.578 INFO:teuthology.orchestra.run.smithi005.stdout:2635 punch from 0x3b15d15 to 0x3b1e85d, (0x8b48 bytes) 2024-05-11T23:02:02.578 INFO:teuthology.orchestra.run.smithi005.stdout:2636 writesame 0x612ae4f thru 0x61339ce data_size 0x90(0x8b80 bytes) 2024-05-11T23:02:02.578 INFO:teuthology.orchestra.run.smithi005.stdout:2638 write 0x28f6c74 thru 0x2901a33 (0xadc0 bytes) 2024-05-11T23:02:02.578 INFO:teuthology.orchestra.run.smithi005.stdout:2640 write 0x4704793 thru 0x470eeed (0xa75b bytes) 2024-05-11T23:02:02.578 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:02.578 INFO:teuthology.orchestra.run.smithi005.stdout:2642 punch from 0x2448b98 to 0x244cbf2, (0x405a bytes) 2024-05-11T23:02:02.578 INFO:teuthology.orchestra.run.smithi005.stdout:2643 read 0x6a4171c thru 0x6a4f108 (0xd9ed bytes) 2024-05-11T23:02:02.578 INFO:teuthology.orchestra.run.smithi005.stdout:2644 punch from 0x82ef207 to 0x82f4f51, (0x5d4a bytes) 2024-05-11T23:02:02.579 INFO:teuthology.orchestra.run.smithi005.stdout:2646 read 0x32c7c77 thru 0x32c9e9d (0x2227 bytes) 2024-05-11T23:02:02.579 INFO:teuthology.orchestra.run.smithi005.stdout:2647 write 0x1f4300b thru 0x1f477b3 (0x47a9 bytes) 2024-05-11T23:02:02.579 INFO:teuthology.orchestra.run.smithi005.stdout:2648 write 0x342dd42 thru 0x3436a5e (0x8d1d bytes) 2024-05-11T23:02:02.579 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:02.579 INFO:teuthology.orchestra.run.smithi005.stdout:2651 write 0xd97ae4 thru 0xd9de9b (0x63b8 bytes) 2024-05-11T23:02:02.579 INFO:teuthology.orchestra.run.smithi005.stdout:2652 read 0x715ea7c thru 0x7164caf (0x6234 bytes) 2024-05-11T23:02:02.579 INFO:teuthology.orchestra.run.smithi005.stdout:2653 punch from 0xd0b0c0 to 0xd199ad, (0xe8ed bytes) 2024-05-11T23:02:02.579 INFO:teuthology.orchestra.run.smithi005.stdout:2654 read 0x585e428 thru 0x586b5c9 (0xd1a2 bytes) 2024-05-11T23:02:02.579 INFO:teuthology.orchestra.run.smithi005.stdout:2655 clone 21 order 18 su 131072 sc 11 2024-05-11T23:02:02.580 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:02.575+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:02.676 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:02.676+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:02.683 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:02.683 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:02.684 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:02.684 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:02.692 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:02:02.778 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:02.779+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:02.837 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:02.837 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:02.846 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:02:02.849 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:02:02.871 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-05-11T23:02:02.881 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:02:02.881 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:02:02.881 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [0, 1, 2, 4, 5, 6, 7] 2024-05-11T23:02:02.881 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 6 2024-05-11T23:02:02.887 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:02.888+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:02.972 INFO:teuthology.orchestra.run.smithi005.stdout:flattening image image_client.0-clone20 2024-05-11T23:02:02.978 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:02.978+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:02.999 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:02.999 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:02.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:02.999 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:02.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:02.999 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24548 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:02.999 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.927949+0000", 2024-05-11T23:02:02.999 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.070181896999999, 2024-05-11T23:02:02.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037300396999999999, 2024-05-11T23:02:03.000 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:03.000 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:03.000 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:03.000 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:03.000 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:03.000 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:03.000 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24548 2024-05-11T23:02:03.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.000 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:03.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:03.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927949+0000", 2024-05-11T23:02:03.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:03.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927949+0000", 2024-05-11T23:02:03.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:03.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927950+0000", 2024-05-11T23:02:03.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9400000000000005e-07 2024-05-11T23:02:03.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:03.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927952+0000", 2024-05-11T23:02:03.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2330000000000001e-06 2024-05-11T23:02:03.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:03.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927959+0000", 2024-05-11T23:02:03.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8820000000000003e-06 2024-05-11T23:02:03.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:03.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927964+0000", 2024-05-11T23:02:03.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.899e-06 2024-05-11T23:02:03.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:03.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.959280+0000", 2024-05-11T23:02:03.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031315904999999998 2024-05-11T23:02:03.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:03.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.959348+0000", 2024-05-11T23:02:03.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8214999999999997e-05 2024-05-11T23:02:03.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:03.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965250+0000", 2024-05-11T23:02:03.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005901669 2024-05-11T23:02:03.003 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.003 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.003 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.004 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24555 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:03.004 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.927974+0000", 2024-05-11T23:02:03.004 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.07015663, 2024-05-11T23:02:03.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.03739746, 2024-05-11T23:02:03.004 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:03.004 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:03.004 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:03.004 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:03.004 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:03.004 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:03.004 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24555 2024-05-11T23:02:03.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.004 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:03.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:03.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927974+0000", 2024-05-11T23:02:03.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:03.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927974+0000", 2024-05-11T23:02:03.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:03.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927975+0000", 2024-05-11T23:02:03.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6899999999999998e-07 2024-05-11T23:02:03.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:03.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927977+0000", 2024-05-11T23:02:03.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.074e-06 2024-05-11T23:02:03.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:03.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927984+0000", 2024-05-11T23:02:03.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7309999999999996e-06 2024-05-11T23:02:03.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:03.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927989+0000", 2024-05-11T23:02:03.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.079e-06 2024-05-11T23:02:03.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:03.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965279+0000", 2024-05-11T23:02:03.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037290271999999999 2024-05-11T23:02:03.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:03.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965324+0000", 2024-05-11T23:02:03.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4638999999999997e-05 2024-05-11T23:02:03.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:03.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965372+0000", 2024-05-11T23:02:03.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8195999999999997e-05 2024-05-11T23:02:03.007 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.007 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.008 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.008 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24562 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:03.008 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.927995+0000", 2024-05-11T23:02:03.008 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.070135694999999, 2024-05-11T23:02:03.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037703408000000001, 2024-05-11T23:02:03.008 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:03.008 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:03.008 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:03.008 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:03.008 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:03.008 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:03.008 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24562 2024-05-11T23:02:03.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.009 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:03.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:03.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927995+0000", 2024-05-11T23:02:03.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:03.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927995+0000", 2024-05-11T23:02:03.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:03.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927996+0000", 2024-05-11T23:02:03.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9499999999999998e-07 2024-05-11T23:02:03.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:03.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927998+0000", 2024-05-11T23:02:03.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0889999999999998e-06 2024-05-11T23:02:03.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:03.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928004+0000", 2024-05-11T23:02:03.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4740000000000002e-06 2024-05-11T23:02:03.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:03.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928009+0000", 2024-05-11T23:02:03.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:02:03.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:03.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965549+0000", 2024-05-11T23:02:03.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037539826999999998 2024-05-11T23:02:03.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:03.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965616+0000", 2024-05-11T23:02:03.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7387999999999999e-05 2024-05-11T23:02:03.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:03.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965699+0000", 2024-05-11T23:02:03.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2347000000000003e-05 2024-05-11T23:02:03.012 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.012 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.012 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.012 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24568 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:03.012 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928016+0000", 2024-05-11T23:02:03.012 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.070115509000001, 2024-05-11T23:02:03.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037850338999999997, 2024-05-11T23:02:03.012 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:03.013 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:03.013 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:03.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:03.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:03.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:03.013 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24568 2024-05-11T23:02:03.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.013 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:03.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:03.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928016+0000", 2024-05-11T23:02:03.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:03.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928016+0000", 2024-05-11T23:02:03.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:03.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928016+0000", 2024-05-11T23:02:03.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-05-11T23:02:03.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:03.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928025+0000", 2024-05-11T23:02:03.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.363e-06 2024-05-11T23:02:03.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:03.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928032+0000", 2024-05-11T23:02:03.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.114e-06 2024-05-11T23:02:03.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:03.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928038+0000", 2024-05-11T23:02:03.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1250000000000001e-06 2024-05-11T23:02:03.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:03.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965720+0000", 2024-05-11T23:02:03.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037682488 2024-05-11T23:02:03.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:03.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965785+0000", 2024-05-11T23:02:03.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5180999999999998e-05 2024-05-11T23:02:03.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:03.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965866+0000", 2024-05-11T23:02:03.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0687999999999997e-05 2024-05-11T23:02:03.016 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.017 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.017 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.017 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24574 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 266240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:03.017 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928045+0000", 2024-05-11T23:02:03.017 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.070086531999999, 2024-05-11T23:02:03.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037990532, 2024-05-11T23:02:03.017 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:03.017 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:03.018 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:03.018 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:03.018 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:03.018 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:03.018 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24574 2024-05-11T23:02:03.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.018 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:03.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:03.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928045+0000", 2024-05-11T23:02:03.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:03.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928045+0000", 2024-05-11T23:02:03.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:03.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928045+0000", 2024-05-11T23:02:03.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:02:03.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:03.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928047+0000", 2024-05-11T23:02:03.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9649999999999998e-06 2024-05-11T23:02:03.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:03.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928053+0000", 2024-05-11T23:02:03.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1550000000000004e-06 2024-05-11T23:02:03.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:03.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928058+0000", 2024-05-11T23:02:03.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9060000000000001e-06 2024-05-11T23:02:03.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:03.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965890+0000", 2024-05-11T23:02:03.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037831754000000002 2024-05-11T23:02:03.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:03.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965957+0000", 2024-05-11T23:02:03.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6798999999999995e-05 2024-05-11T23:02:03.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:03.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966035+0000", 2024-05-11T23:02:03.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8532e-05 2024-05-11T23:02:03.023 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.023 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.023 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.023 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24581 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:03.023 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928064+0000", 2024-05-11T23:02:03.024 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.070066715999999, 2024-05-11T23:02:03.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038134135, 2024-05-11T23:02:03.024 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:03.024 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:03.024 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:03.024 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:03.024 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:03.024 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:03.025 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24581 2024-05-11T23:02:03.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.025 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:03.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:03.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928064+0000", 2024-05-11T23:02:03.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:03.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928064+0000", 2024-05-11T23:02:03.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:03.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928065+0000", 2024-05-11T23:02:03.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3099999999999998e-07 2024-05-11T23:02:03.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:03.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928067+0000", 2024-05-11T23:02:03.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9549999999999999e-06 2024-05-11T23:02:03.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:03.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928073+0000", 2024-05-11T23:02:03.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1929999999999998e-06 2024-05-11T23:02:03.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:03.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928078+0000", 2024-05-11T23:02:03.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9289999999999997e-06 2024-05-11T23:02:03.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:03.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966058+0000", 2024-05-11T23:02:03.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037980452999999997 2024-05-11T23:02:03.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:03.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966123+0000", 2024-05-11T23:02:03.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4832000000000005e-05 2024-05-11T23:02:03.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:03.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966198+0000", 2024-05-11T23:02:03.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5341999999999998e-05 2024-05-11T23:02:03.031 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.031 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.031 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.031 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24585 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:03.032 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928095+0000", 2024-05-11T23:02:03.032 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.070036196, 2024-05-11T23:02:03.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038282688000000002, 2024-05-11T23:02:03.032 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:03.032 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:03.032 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:03.032 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:03.032 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:03.033 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:03.033 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24585 2024-05-11T23:02:03.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.033 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:03.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:03.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928095+0000", 2024-05-11T23:02:03.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:03.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928095+0000", 2024-05-11T23:02:03.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:03.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928095+0000", 2024-05-11T23:02:03.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1900000000000003e-07 2024-05-11T23:02:03.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:03.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928098+0000", 2024-05-11T23:02:03.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1880000000000001e-06 2024-05-11T23:02:03.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:03.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928105+0000", 2024-05-11T23:02:03.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3690000000000001e-06 2024-05-11T23:02:03.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:03.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928120+0000", 2024-05-11T23:02:03.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4762e-05 2024-05-11T23:02:03.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:03.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966233+0000", 2024-05-11T23:02:03.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038113526000000002 2024-05-11T23:02:03.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:03.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966300+0000", 2024-05-11T23:02:03.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6637999999999995e-05 2024-05-11T23:02:03.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:03.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966378+0000", 2024-05-11T23:02:03.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7686000000000001e-05 2024-05-11T23:02:03.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.038 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.039 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24592 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 315392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:03.039 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928128+0000", 2024-05-11T23:02:03.039 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.070002891, 2024-05-11T23:02:03.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038417382, 2024-05-11T23:02:03.039 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:03.039 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:03.039 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:03.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:03.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:03.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:03.039 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24592 2024-05-11T23:02:03.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.039 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:03.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:03.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928128+0000", 2024-05-11T23:02:03.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:03.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928128+0000", 2024-05-11T23:02:03.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:03.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928129+0000", 2024-05-11T23:02:03.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.44e-07 2024-05-11T23:02:03.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:03.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928131+0000", 2024-05-11T23:02:03.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2910000000000002e-06 2024-05-11T23:02:03.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:03.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928138+0000", 2024-05-11T23:02:03.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9430000000000003e-06 2024-05-11T23:02:03.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:03.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928143+0000", 2024-05-11T23:02:03.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4739999999999996e-06 2024-05-11T23:02:03.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:03.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966400+0000", 2024-05-11T23:02:03.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038257001999999998 2024-05-11T23:02:03.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:03.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966465+0000", 2024-05-11T23:02:03.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4468999999999995e-05 2024-05-11T23:02:03.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:03.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966546+0000", 2024-05-11T23:02:03.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0659000000000001e-05 2024-05-11T23:02:03.043 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.043 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.043 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.043 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24598 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 331776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:03.043 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928150+0000", 2024-05-11T23:02:03.043 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.069980581999999, 2024-05-11T23:02:03.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038544469999999997, 2024-05-11T23:02:03.043 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:03.043 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:03.043 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:03.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:03.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:03.044 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:03.044 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24598 2024-05-11T23:02:03.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.044 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:03.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:03.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928150+0000", 2024-05-11T23:02:03.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:03.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928150+0000", 2024-05-11T23:02:03.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:03.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928151+0000", 2024-05-11T23:02:03.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999999e-07 2024-05-11T23:02:03.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:03.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928153+0000", 2024-05-11T23:02:03.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.21e-06 2024-05-11T23:02:03.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:03.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928160+0000", 2024-05-11T23:02:03.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5200000000000003e-06 2024-05-11T23:02:03.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:03.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928165+0000", 2024-05-11T23:02:03.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9269999999999996e-06 2024-05-11T23:02:03.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:03.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966569+0000", 2024-05-11T23:02:03.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038404595999999999 2024-05-11T23:02:03.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:03.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966634+0000", 2024-05-11T23:02:03.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5221000000000005e-05 2024-05-11T23:02:03.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:03.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966695+0000", 2024-05-11T23:02:03.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0547999999999997e-05 2024-05-11T23:02:03.047 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.047 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.047 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.047 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24604 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:03.047 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928171+0000", 2024-05-11T23:02:03.048 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.06996022, 2024-05-11T23:02:03.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038676227000000001, 2024-05-11T23:02:03.048 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:03.048 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:03.048 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:03.048 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:03.048 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:03.048 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:03.048 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24604 2024-05-11T23:02:03.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.048 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:03.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:03.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928171+0000", 2024-05-11T23:02:03.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:03.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928171+0000", 2024-05-11T23:02:03.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:03.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928171+0000", 2024-05-11T23:02:03.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-05-11T23:02:03.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:03.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928173+0000", 2024-05-11T23:02:03.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0619999999999999e-06 2024-05-11T23:02:03.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:03.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928180+0000", 2024-05-11T23:02:03.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-05-11T23:02:03.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:03.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928185+0000", 2024-05-11T23:02:03.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.04e-06 2024-05-11T23:02:03.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:03.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966718+0000", 2024-05-11T23:02:03.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038532759 2024-05-11T23:02:03.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:03.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966784+0000", 2024-05-11T23:02:03.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6865000000000004e-05 2024-05-11T23:02:03.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:03.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966847+0000", 2024-05-11T23:02:03.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2694000000000007e-05 2024-05-11T23:02:03.052 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.052 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.052 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.052 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24609 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 364544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:03.052 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928191+0000", 2024-05-11T23:02:03.052 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.069940151999999, 2024-05-11T23:02:03.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038806908000000001, 2024-05-11T23:02:03.052 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:03.052 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:03.053 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:03.053 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:03.053 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:03.053 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:03.053 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24609 2024-05-11T23:02:03.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.053 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:03.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:03.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928191+0000", 2024-05-11T23:02:03.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:03.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928191+0000", 2024-05-11T23:02:03.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:03.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928191+0000", 2024-05-11T23:02:03.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-05-11T23:02:03.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:03.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928198+0000", 2024-05-11T23:02:03.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2180000000000002e-06 2024-05-11T23:02:03.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:03.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928205+0000", 2024-05-11T23:02:03.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5899999999999996e-06 2024-05-11T23:02:03.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:03.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928210+0000", 2024-05-11T23:02:03.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.118e-06 2024-05-11T23:02:03.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:03.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966869+0000", 2024-05-11T23:02:03.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038659106999999998 2024-05-11T23:02:03.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:03.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966933+0000", 2024-05-11T23:02:03.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4158999999999993e-05 2024-05-11T23:02:03.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:03.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966998+0000", 2024-05-11T23:02:03.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4352999999999999e-05 2024-05-11T23:02:03.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.056 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.057 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24616 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 380928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:03.057 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928217+0000", 2024-05-11T23:02:03.057 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.069914009, 2024-05-11T23:02:03.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038928536999999999, 2024-05-11T23:02:03.057 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:03.057 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:03.057 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:03.057 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:03.057 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:03.057 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:03.057 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24616 2024-05-11T23:02:03.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.057 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:03.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:03.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928217+0000", 2024-05-11T23:02:03.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:03.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928217+0000", 2024-05-11T23:02:03.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:03.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928217+0000", 2024-05-11T23:02:03.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-05-11T23:02:03.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:03.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928219+0000", 2024-05-11T23:02:03.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0609999999999998e-06 2024-05-11T23:02:03.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:03.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928233+0000", 2024-05-11T23:02:03.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3715e-05 2024-05-11T23:02:03.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:03.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928238+0000", 2024-05-11T23:02:03.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1889999999999998e-06 2024-05-11T23:02:03.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:03.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967021+0000", 2024-05-11T23:02:03.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038782284 2024-05-11T23:02:03.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:03.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967085+0000", 2024-05-11T23:02:03.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4319000000000005e-05 2024-05-11T23:02:03.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:03.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967146+0000", 2024-05-11T23:02:03.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0579999999999999e-05 2024-05-11T23:02:03.064 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.064 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.064 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.064 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24622 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 397312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:03.064 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928262+0000", 2024-05-11T23:02:03.064 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.069868703999999, 2024-05-11T23:02:03.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039038392999999998, 2024-05-11T23:02:03.064 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:03.064 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:03.064 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:03.064 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:03.064 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:03.064 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:03.065 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24622 2024-05-11T23:02:03.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.065 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:03.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:03.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928262+0000", 2024-05-11T23:02:03.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:03.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928262+0000", 2024-05-11T23:02:03.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:03.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928263+0000", 2024-05-11T23:02:03.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0299999999999999e-07 2024-05-11T23:02:03.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:03.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928266+0000", 2024-05-11T23:02:03.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6690000000000002e-06 2024-05-11T23:02:03.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:03.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928275+0000", 2024-05-11T23:02:03.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1630000000000002e-06 2024-05-11T23:02:03.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:03.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928280+0000", 2024-05-11T23:02:03.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6579999999999999e-06 2024-05-11T23:02:03.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:03.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967169+0000", 2024-05-11T23:02:03.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038888569999999997 2024-05-11T23:02:03.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:03.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967240+0000", 2024-05-11T23:02:03.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0947999999999999e-05 2024-05-11T23:02:03.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:03.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967301+0000", 2024-05-11T23:02:03.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0782000000000002e-05 2024-05-11T23:02:03.068 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.068 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.068 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.068 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24628 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 413696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:03.068 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928288+0000", 2024-05-11T23:02:03.068 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.069842656000001, 2024-05-11T23:02:03.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039162792000000002, 2024-05-11T23:02:03.069 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:03.069 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:03.069 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:03.069 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:03.069 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:03.069 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:03.069 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24628 2024-05-11T23:02:03.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.069 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:03.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:03.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928288+0000", 2024-05-11T23:02:03.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:03.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928288+0000", 2024-05-11T23:02:03.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:03.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928289+0000", 2024-05-11T23:02:03.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7399999999999998e-07 2024-05-11T23:02:03.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:03.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928291+0000", 2024-05-11T23:02:03.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.221e-06 2024-05-11T23:02:03.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:03.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928298+0000", 2024-05-11T23:02:03.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0520000000000004e-06 2024-05-11T23:02:03.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:03.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928304+0000", 2024-05-11T23:02:03.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4149999999999998e-06 2024-05-11T23:02:03.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:03.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967324+0000", 2024-05-11T23:02:03.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039020436999999998 2024-05-11T23:02:03.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:03.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967390+0000", 2024-05-11T23:02:03.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5821000000000006e-05 2024-05-11T23:02:03.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:03.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967451+0000", 2024-05-11T23:02:03.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1371999999999994e-05 2024-05-11T23:02:03.072 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.072 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.072 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.073 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24636 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 430080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:03.073 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928310+0000", 2024-05-11T23:02:03.073 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.069820791, 2024-05-11T23:02:03.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039286303000000002, 2024-05-11T23:02:03.073 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:03.073 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:03.073 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:03.073 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:03.073 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:03.073 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:03.073 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24636 2024-05-11T23:02:03.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.073 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:03.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:03.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928310+0000", 2024-05-11T23:02:03.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:03.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928310+0000", 2024-05-11T23:02:03.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:03.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928311+0000", 2024-05-11T23:02:03.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:02:03.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:03.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928313+0000", 2024-05-11T23:02:03.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0499999999999999e-06 2024-05-11T23:02:03.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:03.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928319+0000", 2024-05-11T23:02:03.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4049999999999997e-06 2024-05-11T23:02:03.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:03.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928324+0000", 2024-05-11T23:02:03.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:02:03.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:03.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967473+0000", 2024-05-11T23:02:03.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039149138999999999 2024-05-11T23:02:03.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:03.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967537+0000", 2024-05-11T23:02:03.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4109000000000006e-05 2024-05-11T23:02:03.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967597+0000", 2024-05-11T23:02:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9407000000000002e-05 2024-05-11T23:02:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25144 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 28672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936035+0000", 2024-05-11T23:02:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.062095886, 2024-05-11T23:02:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036140510000000001, 2024-05-11T23:02:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25144 2024-05-11T23:02:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936035+0000", 2024-05-11T23:02:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936035+0000", 2024-05-11T23:02:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936036+0000", 2024-05-11T23:02:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-05-11T23:02:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936038+0000", 2024-05-11T23:02:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6910000000000002e-06 2024-05-11T23:02:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936045+0000", 2024-05-11T23:02:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1670000000000004e-06 2024-05-11T23:02:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936049+0000", 2024-05-11T23:02:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9609999999999996e-06 2024-05-11T23:02:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972024+0000", 2024-05-11T23:02:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.035974221000000001 2024-05-11T23:02:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972076+0000", 2024-05-11T23:02:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1918e-05 2024-05-11T23:02:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972176+0000", 2024-05-11T23:02:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010006200000000001 2024-05-11T23:02:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25150 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 45056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936063+0000", 2024-05-11T23:02:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.062068353000001, 2024-05-11T23:02:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036298174000000002, 2024-05-11T23:02:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25150 2024-05-11T23:02:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936063+0000", 2024-05-11T23:02:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936063+0000", 2024-05-11T23:02:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936063+0000", 2024-05-11T23:02:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.39e-07 2024-05-11T23:02:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936069+0000", 2024-05-11T23:02:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2770000000000002e-06 2024-05-11T23:02:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936088+0000", 2024-05-11T23:02:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8841000000000001e-05 2024-05-11T23:02:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936093+0000", 2024-05-11T23:02:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2190000000000003e-06 2024-05-11T23:02:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972213+0000", 2024-05-11T23:02:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036119039999999998 2024-05-11T23:02:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972293+0000", 2024-05-11T23:02:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0437000000000003e-05 2024-05-11T23:02:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972361+0000", 2024-05-11T23:02:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7921000000000002e-05 2024-05-11T23:02:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25156 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936107+0000", 2024-05-11T23:02:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.062023828999999, 2024-05-11T23:02:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036383785000000002, 2024-05-11T23:02:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25156 2024-05-11T23:02:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936107+0000", 2024-05-11T23:02:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936107+0000", 2024-05-11T23:02:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936108+0000", 2024-05-11T23:02:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3900000000000005e-07 2024-05-11T23:02:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936110+0000", 2024-05-11T23:02:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.435e-06 2024-05-11T23:02:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936118+0000", 2024-05-11T23:02:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2889999999999998e-06 2024-05-11T23:02:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936124+0000", 2024-05-11T23:02:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2850000000000002e-06 2024-05-11T23:02:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972378+0000", 2024-05-11T23:02:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036254020999999997 2024-05-11T23:02:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972420+0000", 2024-05-11T23:02:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2577999999999998e-05 2024-05-11T23:02:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972491+0000", 2024-05-11T23:02:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0637999999999997e-05 2024-05-11T23:02:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25164 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936132+0000", 2024-05-11T23:02:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.061998586, 2024-05-11T23:02:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036532028000000001, 2024-05-11T23:02:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25164 2024-05-11T23:02:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936132+0000", 2024-05-11T23:02:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936132+0000", 2024-05-11T23:02:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936133+0000", 2024-05-11T23:02:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-05-11T23:02:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936135+0000", 2024-05-11T23:02:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0660000000000002e-06 2024-05-11T23:02:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936146+0000", 2024-05-11T23:02:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.098e-05 2024-05-11T23:02:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936153+0000", 2024-05-11T23:02:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8619999999999997e-06 2024-05-11T23:02:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972532+0000", 2024-05-11T23:02:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036379702999999999 2024-05-11T23:02:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972577+0000", 2024-05-11T23:02:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4697999999999998e-05 2024-05-11T23:02:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972664+0000", 2024-05-11T23:02:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7338999999999998e-05 2024-05-11T23:02:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25168 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 94208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936160+0000", 2024-05-11T23:02:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 11.061971308, 2024-05-11T23:02:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036655488999999999, 2024-05-11T23:02:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25168 2024-05-11T23:02:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936160+0000", 2024-05-11T23:02:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:03.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936160+0000", 2024-05-11T23:02:03.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:03.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:03.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936160+0000", 2024-05-11T23:02:03.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3000000000000001e-07 2024-05-11T23:02:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936162+0000", 2024-05-11T23:02:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.108e-06 2024-05-11T23:02:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.937848+0000", 2024-05-11T23:02:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001685284 2024-05-11T23:02:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.937858+0000", 2024-05-11T23:02:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0222e-05 2024-05-11T23:02:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972704+0000", 2024-05-11T23:02:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034845736000000002 2024-05-11T23:02:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972746+0000", 2024-05-11T23:02:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2077999999999999e-05 2024-05-11T23:02:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972815+0000", 2024-05-11T23:02:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9630999999999998e-05 2024-05-11T23:02:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:03.101 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:03.102 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:03.102 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:03.103 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:03.103 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:03.104 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:02:03.104 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:03.103+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:03.104 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:03.105 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:03.105 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:03.105 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:03.110 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:03.110 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:03.127 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:03.128 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:03.139 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-05-11T23:02:03.165 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:03.165 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:03.174 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:03.174 INFO:teuthology.orchestra.run.smithi005.stdout:2657 writesame 0x371031c thru 0x371b167 data_size 0x73(0xae4c bytes) 2024-05-11T23:02:03.181 INFO:teuthology.orchestra.run.smithi005.stdout:2658 punch from 0x9bffd5e to 0x9c020a3, (0x2345 bytes) 2024-05-11T23:02:03.182 INFO:teuthology.orchestra.run.smithi005.stdout:2659 write 0xb1f3930 thru 0xb1f7364 (0x3a35 bytes) 2024-05-11T23:02:03.184 INFO:teuthology.orchestra.run.smithi005.stdout:2660 read 0x2db9156 thru 0x2dc3b63 (0xaa0e bytes) 2024-05-11T23:02:03.185 INFO:teuthology.orchestra.run.smithi005.stdout:2663 writesame 0x4a0dedd thru 0x4a0fcad data_size 0x1(0x1dd1 bytes) 2024-05-11T23:02:03.190 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:03.190 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:03.191 INFO:teuthology.orchestra.run.smithi005.stdout:2666 read 0x3f36c3f thru 0x3f3b64f (0x4a11 bytes) 2024-05-11T23:02:03.192 INFO:teuthology.orchestra.run.smithi005.stdout:2668 punch from 0x9215da6 to 0x9218af0, (0x2d4a bytes) 2024-05-11T23:02:03.194 INFO:teuthology.orchestra.run.smithi005.stdout:2669 trunc from 0xb72ebf0 to 0xd4dd836 2024-05-11T23:02:03.200 INFO:teuthology.orchestra.run.smithi005.stdout:2670 read 0x7faea8d thru 0x7fb4eb3 (0x6427 bytes) 2024-05-11T23:02:03.201 INFO:teuthology.orchestra.run.smithi005.stdout:2671 write 0x1f7d200 thru 0x1f838e2 (0x66e3 bytes) 2024-05-11T23:02:03.203 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:03.203+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:03.205 INFO:teuthology.orchestra.run.smithi005.stdout:2674 read 0x1fb7692 thru 0x1fc6385 (0xecf4 bytes) 2024-05-11T23:02:03.220 INFO:teuthology.orchestra.run.smithi005.stdout:2675 trunc from 0xd4dd836 to 0x1a9dd36 2024-05-11T23:02:03.240 INFO:teuthology.orchestra.run.smithi005.stdout:2677 punch from 0x2e912 to 0x3d42d, (0xeb1b bytes) 2024-05-11T23:02:03.241 INFO:teuthology.orchestra.run.smithi005.stdout:2678 punch from 0x427014 to 0x434a69, (0xda55 bytes) 2024-05-11T23:02:03.256 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:03.256 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:03.261 INFO:teuthology.orchestra.run.smithi005.stdout:2679 writesame 0xec6a2bb thru 0xec6b2ba data_size 0x20(0x1000 bytes) 2024-05-11T23:02:03.261 INFO:teuthology.orchestra.run.smithi005.stdout:2682 read 0xd209991 thru 0xd216e4f (0xd4bf bytes) 2024-05-11T23:02:03.261 INFO:teuthology.orchestra.run.smithi005.stdout:2683 write 0x1a2de48 thru 0x1a359fb (0x7bb4 bytes) 2024-05-11T23:02:03.266 INFO:teuthology.orchestra.run.smithi005.stdout:2684 writesame 0x7f5ea8c thru 0x7f6cd35 data_size 0x39(0xe2aa bytes) 2024-05-11T23:02:03.267 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:02:03.270 INFO:teuthology.orchestra.run.smithi005.stdout:2685 write 0x98e1fe0 thru 0x98e54c8 (0x34e9 bytes) 2024-05-11T23:02:03.273 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:03.273 INFO:teuthology.orchestra.run.smithi005.stdout:2687 read 0x81d4edb thru 0x81df7c9 (0xa8ef bytes) 2024-05-11T23:02:03.273 INFO:teuthology.orchestra.run.smithi005.stdout:2689 read 0xc2920b1 thru 0xc296cac (0x4bfc bytes) 2024-05-11T23:02:03.273 INFO:teuthology.orchestra.run.smithi005.stdout:2690 read 0xb9bdf25 thru 0xb9ccedd (0xefb9 bytes) 2024-05-11T23:02:03.273 INFO:teuthology.orchestra.run.smithi005.stdout:2691 write 0x779617a thru 0x779f30c (0x9193 bytes) 2024-05-11T23:02:03.276 INFO:teuthology.orchestra.run.smithi005.stdout:2692 trunc from 0xec6b2bb to 0xe0040c1 2024-05-11T23:02:03.282 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:03.282 INFO:teuthology.orchestra.run.smithi005.stdout:2694 punch from 0xb2d9c9f to 0xb2e168e, (0x79ef bytes) 2024-05-11T23:02:03.282 INFO:teuthology.orchestra.run.smithi005.stdout:2695 read 0x1448ec1 thru 0x1458052 (0xf192 bytes) 2024-05-11T23:02:03.283 INFO:teuthology.orchestra.run.smithi005.stdout:2698 read 0x925c675 thru 0x92636e9 (0x7075 bytes) 2024-05-11T23:02:03.283 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:03.283 INFO:teuthology.orchestra.run.smithi005.stdout:2703 write 0x362ac9e thru 0x362d6fb (0x2a5e bytes) 2024-05-11T23:02:03.286 INFO:teuthology.orchestra.run.smithi005.stdout:2704 write 0xc0b9879 thru 0xc0c14f3 (0x7c7b bytes) 2024-05-11T23:02:03.290 INFO:teuthology.orchestra.run.smithi005.stdout:2705 writesame 0xd7d85f9 thru 0xd7e5a71 data_size 0x1(0xd479 bytes) 2024-05-11T23:02:03.292 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-11T23:02:03.292+0000 7fd16247e640 -1 mon.a@0(leader).osd e49 definitely_dead 0 2024-05-11T23:02:03.305 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:03.305+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:03.307 INFO:teuthology.orchestra.run.smithi005.stdout:2708 writesame 0xa2aa4c1 thru 0xa2b627a data_size 0xf(0xbdba bytes) 2024-05-11T23:02:03.311 INFO:teuthology.orchestra.run.smithi005.stdout:2709 write 0x9ee2f6b thru 0x9ef1724 (0xe7ba bytes) 2024-05-11T23:02:03.315 INFO:teuthology.orchestra.run.smithi005.stdout:2710 write 0x34489fe thru 0x34529da (0x9fdd bytes) 2024-05-11T23:02:03.318 INFO:teuthology.orchestra.run.smithi005.stdout:2711 read 0xdbc4399 thru 0xdbd4381 (0xffe9 bytes) 2024-05-11T23:02:03.319 INFO:teuthology.orchestra.run.smithi005.stdout:2712 writesame 0xd077f8e thru 0xd085393 data_size 0x7(0xd406 bytes) 2024-05-11T23:02:03.323 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:03.334 INFO:teuthology.orchestra.run.smithi005.stdout:2717 trunc from 0xe0040c1 to 0x3a52779 2024-05-11T23:02:03.360 INFO:teuthology.orchestra.run.smithi005.stdout:2718 write 0x92a42f8 thru 0x92afca8 (0xb9b1 bytes) 2024-05-11T23:02:03.372 INFO:teuthology.orchestra.run.smithi005.stdout:2719 write 0xac00a9d thru 0xac03c21 (0x3185 bytes) 2024-05-11T23:02:03.375 INFO:teuthology.orchestra.run.smithi005.stdout:2720 punch from 0x329fa3a to 0x32a6bc4, (0x718a bytes) 2024-05-11T23:02:03.375 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:03.375 INFO:teuthology.orchestra.run.smithi005.stdout:2722 write 0x74917fd thru 0x749ebf4 (0xd3f8 bytes) 2024-05-11T23:02:03.378 INFO:teuthology.orchestra.run.smithi005.stdout:2724 write 0x55de273 thru 0x55e6015 (0x7da3 bytes) 2024-05-11T23:02:03.380 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:03.381 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:03.382 INFO:teuthology.orchestra.run.smithi005.stdout:2725 write 0x20be8d1 thru 0x20cda73 (0xf1a3 bytes) 2024-05-11T23:02:03.391 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:02:03.392 INFO:teuthology.orchestra.run.smithi005.stdout:2726 trunc from 0xac03c22 to 0x60568aa 2024-05-11T23:02:03.402 INFO:teuthology.orchestra.run.smithi005.stdout:2727 write 0x5e6f6b6 thru 0x5e77255 (0x7ba0 bytes) 2024-05-11T23:02:03.417 INFO:teuthology.orchestra.run.smithi005.stdout:2728 write 0xad05822 thru 0xad08888 (0x3067 bytes) 2024-05-11T23:02:03.420 INFO:teuthology.orchestra.run.smithi005.stdout:2729 writesame 0x6c77b4a thru 0x6c7998b data_size 0x3(0x1e42 bytes) 2024-05-11T23:02:03.432 INFO:teuthology.orchestra.run.smithi005.stdout:2730 write 0xe70c183 thru 0xe712c07 (0x6a85 bytes) 2024-05-11T23:02:03.435 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:03.436+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:03.435 INFO:teuthology.orchestra.run.smithi005.stdout:2731 write 0xab9bdec thru 0xaba140d (0x5622 bytes) 2024-05-11T23:02:03.452 INFO:teuthology.orchestra.run.smithi005.stdout:2732 trunc from 0xe712c08 to 0x22c5f8b 2024-05-11T23:02:03.476 INFO:teuthology.orchestra.run.smithi005.stdout:2734 writesame 0x7b400a1 thru 0x7b4ab47 data_size 0x7(0xaaa7 bytes) 2024-05-11T23:02:03.477 INFO:teuthology.orchestra.run.smithi005.stdout:2735 read 0x42697d2 thru 0x426f60d (0x5e3c bytes) 2024-05-11T23:02:03.477 INFO:teuthology.orchestra.run.smithi005.stdout:2736 punch from 0x1686623 to 0x169477d, (0xe15a bytes) 2024-05-11T23:02:03.477 INFO:teuthology.orchestra.run.smithi005.stdout:2739 read 0x5b0db1e thru 0x5b1bcd1 (0xe1b4 bytes) 2024-05-11T23:02:03.477 INFO:teuthology.orchestra.run.smithi005.stdout:2740 write 0x4667808 thru 0x466b37d (0x3b76 bytes) 2024-05-11T23:02:03.493 INFO:teuthology.orchestra.run.smithi005.stdout:2743 write 0xe49688c thru 0xe4a22b3 (0xba28 bytes) 2024-05-11T23:02:03.497 INFO:teuthology.orchestra.run.smithi005.stdout:2744 write 0x6f1a6cb thru 0x6f281be (0xdaf4 bytes) 2024-05-11T23:02:03.501 INFO:teuthology.orchestra.run.smithi005.stdout:2745 punch from 0x37a751b to 0x37b6f8b, (0xfa70 bytes) 2024-05-11T23:02:03.501 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:03.501 INFO:teuthology.orchestra.run.smithi005.stdout:2747 write 0xa3e4608 thru 0xa3f2911 (0xe30a bytes) 2024-05-11T23:02:03.503 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:03.503 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:03.504 INFO:teuthology.orchestra.run.smithi005.stdout:2748 writesame 0x86e97b thru 0x872f82 data_size 0x24(0x4608 bytes) 2024-05-11T23:02:03.506 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:03.506+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:03.508 INFO:teuthology.orchestra.run.smithi005.stdout:2750 punch from 0xdab1208 to 0xdab9d1b, (0x8b13 bytes) 2024-05-11T23:02:03.508 INFO:teuthology.orchestra.run.smithi005.stdout:2751 read 0x84150ab thru 0x8420caf (0xbc05 bytes) 2024-05-11T23:02:03.509 INFO:teuthology.orchestra.run.smithi005.stdout:2752 read 0x6c5c275 thru 0x6c6ad66 (0xeaf2 bytes) 2024-05-11T23:02:03.509 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:03.509 INFO:teuthology.orchestra.run.smithi005.stdout:2754 writesame 0x76f90ed thru 0x7707f83 data_size 0x1(0xee97 bytes) 2024-05-11T23:02:03.513 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:02:03.538 INFO:teuthology.orchestra.run.smithi005.stdout:2755 write 0x32bf048 thru 0x32c2ff0 (0x3fa9 bytes) 2024-05-11T23:02:03.541 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:03.541 INFO:teuthology.orchestra.run.smithi005.stdout:2761 read 0x26cc5fa thru 0x26d0722 (0x4129 bytes) 2024-05-11T23:02:03.541 INFO:teuthology.orchestra.run.smithi005.stdout:2762 write 0x113c095 thru 0x11445e1 (0x854d bytes) 2024-05-11T23:02:03.545 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:03.545 INFO:teuthology.orchestra.run.smithi005.stdout:2764 punch from 0x11892ee to 0x1199060, (0xfd72 bytes) 2024-05-11T23:02:03.546 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:03.546 INFO:teuthology.orchestra.run.smithi005.stdout:2766 read 0x2b09701 thru 0x2b17dd1 (0xe6d1 bytes) 2024-05-11T23:02:03.546 INFO:teuthology.orchestra.run.smithi005.stdout:2767 write 0xb3a55e4 thru 0xb3abb33 (0x6550 bytes) 2024-05-11T23:02:03.548 INFO:teuthology.orchestra.run.smithi005.stdout:2770 read 0x2d0eee3 thru 0x2d1a07d (0xb19b bytes) 2024-05-11T23:02:03.548 INFO:teuthology.orchestra.run.smithi005.stdout:2771 write 0xd66b44a thru 0xd67a704 (0xf2bb bytes) 2024-05-11T23:02:03.551 INFO:teuthology.orchestra.run.smithi005.stdout:2773 punch from 0xd03fa09 to 0xd0489af, (0x8fa6 bytes) 2024-05-11T23:02:03.551 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:03.556 INFO:teuthology.orchestra.run.smithi005.stdout:2775 trunc from 0xe4a22b4 to 0xa152066 2024-05-11T23:02:03.566 INFO:teuthology.orchestra.run.smithi005.stdout:2777 punch from 0x43229d to 0x4400e4, (0xde47 bytes) 2024-05-11T23:02:03.573 INFO:teuthology.orchestra.run.smithi005.stdout:2778 trunc from 0xa152066 to 0x3d610ae 2024-05-11T23:02:03.584 INFO:teuthology.orchestra.run.smithi005.stdout:2779 read 0xc958f thru 0xccc7d (0x36ef bytes) 2024-05-11T23:02:03.585 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:03.594 INFO:teuthology.orchestra.run.smithi005.stdout:2781 write 0x69c1eea thru 0x69ceab4 (0xcbcb bytes) 2024-05-11T23:02:03.597 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:03.607 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:03.607+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:03.611 INFO:teuthology.orchestra.run.smithi005.stdout:2783 write 0xd3bf70c thru 0xd3cb304 (0xbbf9 bytes) 2024-05-11T23:02:03.615 INFO:teuthology.orchestra.run.smithi005.stdout:2784 writesame 0x3f884ae thru 0x3f8acf5 data_size 0x4(0x2848 bytes) 2024-05-11T23:02:03.619 INFO:teuthology.orchestra.run.smithi005.stdout:2785 write 0x630c542 thru 0x630df5b (0x1a1a bytes) 2024-05-11T23:02:03.622 INFO:teuthology.orchestra.run.smithi005.stdout:2786 writesame 0xa7aead7 thru 0xa7b1411 data_size 0x1(0x293b bytes) 2024-05-11T23:02:03.628 INFO:teuthology.orchestra.run.smithi005.stdout:2789 write 0xccd0c98 thru 0xcce0876 (0xfbdf bytes) 2024-05-11T23:02:03.631 INFO:teuthology.orchestra.run.smithi005.stdout:2790 write 0x1523971 thru 0x152890d (0x4f9d bytes) 2024-05-11T23:02:03.635 INFO:teuthology.orchestra.run.smithi005.stdout:2791 punch from 0x305a78a to 0x3061222, (0x6a98 bytes) 2024-05-11T23:02:03.635 INFO:teuthology.orchestra.run.smithi005.stdout:2792 read 0x25d4afe thru 0x25dfd6c (0xb26f bytes) 2024-05-11T23:02:03.635 INFO:teuthology.orchestra.run.smithi005.stdout:2793 punch from 0x62cf7de to 0x62df52d, (0xfd4f bytes) 2024-05-11T23:02:03.636 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:03.636 INFO:teuthology.orchestra.run.smithi005.stdout:2795 read 0x5622d55 thru 0x562d138 (0xa3e4 bytes) 2024-05-11T23:02:03.636 INFO:teuthology.orchestra.run.smithi005.stdout:2796 read 0xc4aeabd thru 0xc4b6f59 (0x849d bytes) 2024-05-11T23:02:03.636 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:03.636 INFO:teuthology.orchestra.run.smithi005.stdout:2799 punch from 0x13adaf9 to 0x13bd090, (0xf597 bytes) 2024-05-11T23:02:03.636 INFO:teuthology.orchestra.run.smithi005.stdout:2800 read 0xd12a2b1 thru 0xd135660 (0xb3b0 bytes) 2024-05-11T23:02:03.636 INFO:teuthology.orchestra.run.smithi005.stdout:2801 read 0xd01c288 thru 0xd021718 (0x5491 bytes) 2024-05-11T23:02:03.636 INFO:teuthology.orchestra.run.smithi005.stdout:2802 write 0x97e38f8 thru 0x97e9cd5 (0x63de bytes) 2024-05-11T23:02:03.639 INFO:teuthology.orchestra.run.smithi005.stdout:2803 writesame 0xb425201 thru 0xb42aaea data_size 0x13(0x58ea bytes) 2024-05-11T23:02:03.642 INFO:teuthology.orchestra.run.smithi005.stdout:2804 write 0x1c35293 thru 0x1c38a38 (0x37a6 bytes) 2024-05-11T23:02:03.650 INFO:teuthology.orchestra.run.smithi005.stdout:2805 write 0xd531b88 thru 0xd5382fe (0x6777 bytes) 2024-05-11T23:02:03.652 INFO:teuthology.orchestra.run.smithi005.stdout:2806 trunc from 0xd5382ff to 0xd9c9882 2024-05-11T23:02:03.657 INFO:teuthology.orchestra.run.smithi005.stdout:2808 read 0x7a9f02f thru 0x7aa9ae2 (0xaab4 bytes) 2024-05-11T23:02:03.657 INFO:teuthology.orchestra.run.smithi005.stdout:2810 write 0x89493d thru 0x899a82 (0x5146 bytes) 2024-05-11T23:02:03.662 INFO:teuthology.orchestra.run.smithi005.stdout:2811 read 0x919e7db thru 0x919ecd5 (0x4fb bytes) 2024-05-11T23:02:03.662 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:03.663 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:03.663 INFO:teuthology.orchestra.run.smithi005.stdout:2818 read 0x3d4ba95 thru 0x3d524f6 (0x6a62 bytes) 2024-05-11T23:02:03.663 INFO:teuthology.orchestra.run.smithi005.stdout:2819 read 0x9413223 thru 0x94222df (0xf0bd bytes) 2024-05-11T23:02:03.663 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:03.663 INFO:teuthology.orchestra.run.smithi005.stdout:2821 clone 22 order 19 su 4096 sc 6 2024-05-11T23:02:03.667 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:03.667 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:03.677 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:02:03.721 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:03.722+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:03.781 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:03.782 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:03.790 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:02:03.808 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:03.808+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:03.872 INFO:teuthology.orchestra.run.smithi005.stderr:marked out osd.6. 2024-05-11T23:02:03.909 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:03.910+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:03.943 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:03.944 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:03.952 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:02:04.010 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:04.011+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:04.097 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:04.097 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:04.106 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:02:04.111 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:04.111+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:04.211 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:04.212 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:04.212 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:04.212+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:04.220 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:02:04.312 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:04.313+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:04.366 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:04.366 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:04.374 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:02:04.419 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:04.419+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:04.497 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:04.497 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:04.506 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:02:04.514 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:04.514+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:04.615 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:04.616+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:04.657 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:04.657 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:04.665 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:02:04.716 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:04.717+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:04.776 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:04.776 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:04.785 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:02:04.817 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:04.818+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:04.890 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-11T23:02:04.897 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:04.898 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:04.907 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:02:04.948 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:04.948+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:05.018 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:05.018+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:05.059 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:05.059 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:05.068 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:02:05.119 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:05.119+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:05.177 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:05.177 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:05.186 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:02:05.205 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:02:05.221 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:05.220+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:05.299 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:05.299 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:05.310 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:02:05.355 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:05.355+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:05.358 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:05.359 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:05.377 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:05.377 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:05.396 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:05.396 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:05.415 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:05.415 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:05.422 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:05.422+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:05.425 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:05.425 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:05.426 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:05.426 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:05.426 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:05.432 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:05.433 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:05.435 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:02:05.451 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:05.451 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:05.469 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:05.469 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:05.487 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:05.488 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:05.524 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:05.523+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:05.563 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:05.563 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:05.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:05.563 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:05.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.564 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:25952 3.9 e40/19 3:908f208a:::rbd_data.111e5e4d8275.0000000000000002:head v 40'65, mlcod=39'63)", 2024-05-11T23:02:05.564 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.977473+0000", 2024-05-11T23:02:05.564 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.585147159, 2024-05-11T23:02:05.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0037656679999999998, 2024-05-11T23:02:05.564 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:05.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:05.564 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:05.564 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:05.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:05.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.977473+0000", 2024-05-11T23:02:05.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:05.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.977473+0000", 2024-05-11T23:02:05.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:05.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.977473+0000", 2024-05-11T23:02:05.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7300000000000002e-07 2024-05-11T23:02:05.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:05.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979179+0000", 2024-05-11T23:02:05.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017062869999999999 2024-05-11T23:02:05.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:05.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979190+0000", 2024-05-11T23:02:05.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0341e-05 2024-05-11T23:02:05.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:05.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979194+0000", 2024-05-11T23:02:05.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7369999999999997e-06 2024-05-11T23:02:05.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:05.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979250+0000", 2024-05-11T23:02:05.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6060999999999997e-05 2024-05-11T23:02:05.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:05.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979275+0000", 2024-05-11T23:02:05.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4267999999999999e-05 2024-05-11T23:02:05.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:05.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.981206+0000", 2024-05-11T23:02:05.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001931316 2024-05-11T23:02:05.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:05.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.981238+0000", 2024-05-11T23:02:05.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2285e-05 2024-05-11T23:02:05.567 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.568 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.568 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.568 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:25953 3.9 e40/19 3:908f208a:::rbd_data.111e5e4d8275.0000000000000002:head v 40'67, mlcod=39'63)", 2024-05-11T23:02:05.568 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:51.979457+0000", 2024-05-11T23:02:05.568 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.583163168, 2024-05-11T23:02:05.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002255132, 2024-05-11T23:02:05.568 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:05.568 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:05.568 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:05.568 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:05.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:05.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979457+0000", 2024-05-11T23:02:05.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:05.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979457+0000", 2024-05-11T23:02:05.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:05.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979457+0000", 2024-05-11T23:02:05.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3600000000000004e-07 2024-05-11T23:02:05.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:05.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979462+0000", 2024-05-11T23:02:05.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5109999999999997e-06 2024-05-11T23:02:05.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:05.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979469+0000", 2024-05-11T23:02:05.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6979999999999998e-06 2024-05-11T23:02:05.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:05.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.979474+0000", 2024-05-11T23:02:05.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4769999999999997e-06 2024-05-11T23:02:05.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:05.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.980421+0000", 2024-05-11T23:02:05.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00094687500000000002 2024-05-11T23:02:05.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:05.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.980444+0000", 2024-05-11T23:02:05.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2878e-05 2024-05-11T23:02:05.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:05.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.981657+0000", 2024-05-11T23:02:05.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0012138749999999999 2024-05-11T23:02:05.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:05.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:51.981712+0000", 2024-05-11T23:02:05.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4282e-05 2024-05-11T23:02:05.572 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.573 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.573 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.573 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:26861 3.0 e40/19 3:05d5189e:::rbd_object_map.111e5e4d8275.0000000000000011:head v 40'121, mlcod=40'119)", 2024-05-11T23:02:05.573 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.008555+0000", 2024-05-11T23:02:05.573 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.554064583000001, 2024-05-11T23:02:05.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015206460000000001, 2024-05-11T23:02:05.573 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:05.573 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:05.573 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:05.573 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:05.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:05.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008555+0000", 2024-05-11T23:02:05.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:05.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008555+0000", 2024-05-11T23:02:05.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:05.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008557+0000", 2024-05-11T23:02:05.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8840000000000001e-06 2024-05-11T23:02:05.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:05.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008566+0000", 2024-05-11T23:02:05.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2790000000000004e-06 2024-05-11T23:02:05.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:05.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008586+0000", 2024-05-11T23:02:05.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9956999999999999e-05 2024-05-11T23:02:05.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:05.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008596+0000", 2024-05-11T23:02:05.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8900000000000002e-06 2024-05-11T23:02:05.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:05.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008666+0000", 2024-05-11T23:02:05.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9814000000000006e-05 2024-05-11T23:02:05.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:05.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.008688+0000", 2024-05-11T23:02:05.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2365e-05 2024-05-11T23:02:05.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:05.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.010029+0000", 2024-05-11T23:02:05.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0013403880000000001 2024-05-11T23:02:05.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:05.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.010076+0000", 2024-05-11T23:02:05.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7068999999999999e-05 2024-05-11T23:02:05.577 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.577 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.577 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.577 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:27374 3.0 e40/19 3:05d5189e:::rbd_object_map.111e5e4d8275.0000000000000011:head v 40'127, mlcod=40'124)", 2024-05-11T23:02:05.577 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.034130+0000", 2024-05-11T23:02:05.577 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.528489276, 2024-05-11T23:02:05.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001526082, 2024-05-11T23:02:05.577 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:05.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:05.577 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:05.577 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:05.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:05.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034130+0000", 2024-05-11T23:02:05.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:05.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034130+0000", 2024-05-11T23:02:05.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:05.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034138+0000", 2024-05-11T23:02:05.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8960000000000003e-06 2024-05-11T23:02:05.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:05.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034154+0000", 2024-05-11T23:02:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6152999999999999e-05 2024-05-11T23:02:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034171+0000", 2024-05-11T23:02:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6121e-05 2024-05-11T23:02:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034177+0000", 2024-05-11T23:02:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1079999999999998e-06 2024-05-11T23:02:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034612+0000", 2024-05-11T23:02:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00043566900000000002 2024-05-11T23:02:05.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:05.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.034631+0000", 2024-05-11T23:02:05.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8643000000000001e-05 2024-05-11T23:02:05.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:05.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.035619+0000", 2024-05-11T23:02:05.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00098804600000000002 2024-05-11T23:02:05.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:05.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.035657+0000", 2024-05-11T23:02:05.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7446e-05 2024-05-11T23:02:05.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.580 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.581 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:27790 3.0 e40/19 3:05d5189e:::rbd_object_map.111e5e4d8275.0000000000000011:head v 40'130, mlcod=40'128)", 2024-05-11T23:02:05.581 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.060478+0000", 2024-05-11T23:02:05.581 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.502141938999999, 2024-05-11T23:02:05.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0014537739999999999, 2024-05-11T23:02:05.581 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:05.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:05.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:05.581 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:05.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:05.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060478+0000", 2024-05-11T23:02:05.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:05.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060478+0000", 2024-05-11T23:02:05.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:05.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060479+0000", 2024-05-11T23:02:05.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8999999999999995e-07 2024-05-11T23:02:05.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:05.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060488+0000", 2024-05-11T23:02:05.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9879999999999993e-06 2024-05-11T23:02:05.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:05.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060499+0000", 2024-05-11T23:02:05.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1489e-05 2024-05-11T23:02:05.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:05.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060505+0000", 2024-05-11T23:02:05.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5330000000000002e-06 2024-05-11T23:02:05.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:05.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060560+0000", 2024-05-11T23:02:05.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5384999999999999e-05 2024-05-11T23:02:05.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:05.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.060582+0000", 2024-05-11T23:02:05.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1889999999999999e-05 2024-05-11T23:02:05.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:05.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.061883+0000", 2024-05-11T23:02:05.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0013005989999999999 2024-05-11T23:02:05.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:05.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.061932+0000", 2024-05-11T23:02:05.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8999999999999998e-05 2024-05-11T23:02:05.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.584 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:28504 3.9 e40/19 3:92d63908:::rbd_data.111e5e4d8275.0000000000000013:head v 40'68, mlcod=40'65)", 2024-05-11T23:02:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.092954+0000", 2024-05-11T23:02:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.469665612, 2024-05-11T23:02:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0030983059999999999, 2024-05-11T23:02:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.092954+0000", 2024-05-11T23:02:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.092954+0000", 2024-05-11T23:02:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:05.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.092955+0000", 2024-05-11T23:02:05.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.13e-06 2024-05-11T23:02:05.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:05.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094110+0000", 2024-05-11T23:02:05.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0011551669999999999 2024-05-11T23:02:05.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:05.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094124+0000", 2024-05-11T23:02:05.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3244999999999999e-05 2024-05-11T23:02:05.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:05.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094131+0000", 2024-05-11T23:02:05.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0090000000000001e-06 2024-05-11T23:02:05.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:05.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094187+0000", 2024-05-11T23:02:05.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6218999999999998e-05 2024-05-11T23:02:05.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:05.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094205+0000", 2024-05-11T23:02:05.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8148e-05 2024-05-11T23:02:05.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:05.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.096006+0000", 2024-05-11T23:02:05.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018005619999999999 2024-05-11T23:02:05.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:05.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.096052+0000", 2024-05-11T23:02:05.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6826000000000002e-05 2024-05-11T23:02:05.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.588 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.589 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:28505 3.9 e40/19 3:92d63908:::rbd_data.111e5e4d8275.0000000000000013:head v 40'70, mlcod=40'65)", 2024-05-11T23:02:05.589 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.094530+0000", 2024-05-11T23:02:05.589 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.468089678, 2024-05-11T23:02:05.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0021142140000000001, 2024-05-11T23:02:05.589 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:05.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:05.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:05.589 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:05.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:05.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094530+0000", 2024-05-11T23:02:05.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:05.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094530+0000", 2024-05-11T23:02:05.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:05.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094531+0000", 2024-05-11T23:02:05.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9400000000000004e-07 2024-05-11T23:02:05.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:05.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094536+0000", 2024-05-11T23:02:05.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.575e-06 2024-05-11T23:02:05.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:05.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094549+0000", 2024-05-11T23:02:05.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2857000000000001e-05 2024-05-11T23:02:05.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:05.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.094556+0000", 2024-05-11T23:02:05.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6050000000000003e-06 2024-05-11T23:02:05.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:05.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.095097+0000", 2024-05-11T23:02:05.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00054118299999999999 2024-05-11T23:02:05.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:05.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.095119+0000", 2024-05-11T23:02:05.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1673000000000001e-05 2024-05-11T23:02:05.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:05.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.096603+0000", 2024-05-11T23:02:05.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001484153 2024-05-11T23:02:05.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:05.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.096644+0000", 2024-05-11T23:02:05.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1374000000000001e-05 2024-05-11T23:02:05.592 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.592 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.592 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.592 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:28824 3.9 e40/19 3:9050c332:::rbd_data.111e5e4d8275.000000000000001d:head v 40'71, mlcod=40'68)", 2024-05-11T23:02:05.593 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.141602+0000", 2024-05-11T23:02:05.593 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.421017435, 2024-05-11T23:02:05.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0033604630000000002, 2024-05-11T23:02:05.593 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:05.593 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:05.593 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:05.593 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:05.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:05.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.141602+0000", 2024-05-11T23:02:05.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:05.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.141602+0000", 2024-05-11T23:02:05.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:05.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.141604+0000", 2024-05-11T23:02:05.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4300000000000001e-06 2024-05-11T23:02:05.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:05.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.142793+0000", 2024-05-11T23:02:05.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0011888409999999999 2024-05-11T23:02:05.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:05.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.142815+0000", 2024-05-11T23:02:05.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2432000000000001e-05 2024-05-11T23:02:05.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:05.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.142826+0000", 2024-05-11T23:02:05.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0943e-05 2024-05-11T23:02:05.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:05.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.142901+0000", 2024-05-11T23:02:05.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5283000000000004e-05 2024-05-11T23:02:05.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:05.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.142935+0000", 2024-05-11T23:02:05.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4014000000000003e-05 2024-05-11T23:02:05.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:05.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144916+0000", 2024-05-11T23:02:05.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0019805019999999999 2024-05-11T23:02:05.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:05.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144963+0000", 2024-05-11T23:02:05.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7018000000000003e-05 2024-05-11T23:02:05.596 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.596 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.596 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.596 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:28825 3.9 e40/19 3:9050c332:::rbd_data.111e5e4d8275.000000000000001d:head v 40'73, mlcod=40'68)", 2024-05-11T23:02:05.597 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.143269+0000", 2024-05-11T23:02:05.597 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.419350549000001, 2024-05-11T23:02:05.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0020932170000000001, 2024-05-11T23:02:05.597 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:05.597 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:05.597 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:05.597 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:05.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:05.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.143269+0000", 2024-05-11T23:02:05.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:05.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.143269+0000", 2024-05-11T23:02:05.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:05.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.143270+0000", 2024-05-11T23:02:05.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1200000000000001e-07 2024-05-11T23:02:05.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:05.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.143278+0000", 2024-05-11T23:02:05.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1079999999999993e-06 2024-05-11T23:02:05.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:05.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.143289+0000", 2024-05-11T23:02:05.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0961000000000001e-05 2024-05-11T23:02:05.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:05.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.143297+0000", 2024-05-11T23:02:05.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3480000000000004e-06 2024-05-11T23:02:05.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:05.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144016+0000", 2024-05-11T23:02:05.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00071809800000000002 2024-05-11T23:02:05.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:05.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144043+0000", 2024-05-11T23:02:05.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7746000000000002e-05 2024-05-11T23:02:05.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:05.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145309+0000", 2024-05-11T23:02:05.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0012660060000000001 2024-05-11T23:02:05.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:05.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145362+0000", 2024-05-11T23:02:05.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3038000000000003e-05 2024-05-11T23:02:05.600 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.600 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.600 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.600 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:28828 3.9 e40/19 3:93a019b9:::rbd_data.111e5e4d8275.000000000000001f:head v 40'74, mlcod=40'68)", 2024-05-11T23:02:05.601 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.144176+0000", 2024-05-11T23:02:05.601 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.418443954000001, 2024-05-11T23:02:05.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0029854930000000001, 2024-05-11T23:02:05.601 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:05.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:05.601 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:05.601 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:05.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:05.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144176+0000", 2024-05-11T23:02:05.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:05.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144176+0000", 2024-05-11T23:02:05.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:05.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.144177+0000", 2024-05-11T23:02:05.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9899999999999999e-07 2024-05-11T23:02:05.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:05.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145289+0000", 2024-05-11T23:02:05.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0011121670000000001 2024-05-11T23:02:05.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:05.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145304+0000", 2024-05-11T23:02:05.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5590000000000002e-05 2024-05-11T23:02:05.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:05.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145312+0000", 2024-05-11T23:02:05.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8529999999999992e-06 2024-05-11T23:02:05.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:05.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145393+0000", 2024-05-11T23:02:05.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1050999999999994e-05 2024-05-11T23:02:05.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:05.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145412+0000", 2024-05-11T23:02:05.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8992999999999999e-05 2024-05-11T23:02:05.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:05.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.147116+0000", 2024-05-11T23:02:05.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017039589999999999 2024-05-11T23:02:05.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:05.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.147161+0000", 2024-05-11T23:02:05.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5080999999999997e-05 2024-05-11T23:02:05.604 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.604 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.604 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.604 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:28829 3.9 e40/19 3:93a019b9:::rbd_data.111e5e4d8275.000000000000001f:head v 40'76, mlcod=40'71)", 2024-05-11T23:02:05.604 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.145924+0000", 2024-05-11T23:02:05.605 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.416695517999999, 2024-05-11T23:02:05.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015381520000000001, 2024-05-11T23:02:05.605 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:05.605 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:05.605 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:05.605 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:05.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:05.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145924+0000", 2024-05-11T23:02:05.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:05.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145924+0000", 2024-05-11T23:02:05.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:05.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145925+0000", 2024-05-11T23:02:05.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.99e-07 2024-05-11T23:02:05.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:05.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145933+0000", 2024-05-11T23:02:05.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7090000000000002e-06 2024-05-11T23:02:05.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:05.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145944+0000", 2024-05-11T23:02:05.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1177e-05 2024-05-11T23:02:05.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:05.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.145951+0000", 2024-05-11T23:02:05.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6450000000000005e-06 2024-05-11T23:02:05.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:05.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.146272+0000", 2024-05-11T23:02:05.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032017699999999999 2024-05-11T23:02:05.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:05.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.146290+0000", 2024-05-11T23:02:05.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7977000000000001e-05 2024-05-11T23:02:05.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:05.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.147432+0000", 2024-05-11T23:02:05.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0011421879999999999 2024-05-11T23:02:05.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:05.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.147462+0000", 2024-05-11T23:02:05.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0580000000000002e-05 2024-05-11T23:02:05.608 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.608 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.608 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.608 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:34962 3.9 e40/19 3:96260786:::rbd_data.111ea078695e.0000000000000028:head v 40'77, mlcod=40'74)", 2024-05-11T23:02:05.608 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.700882+0000", 2024-05-11T23:02:05.608 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.861737672, 2024-05-11T23:02:05.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001779938, 2024-05-11T23:02:05.609 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:05.609 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:05.609 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:05.609 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:05.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:05.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.700882+0000", 2024-05-11T23:02:05.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:05.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.700882+0000", 2024-05-11T23:02:05.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:05.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.700884+0000", 2024-05-11T23:02:05.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5030000000000001e-06 2024-05-11T23:02:05.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:05.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.701472+0000", 2024-05-11T23:02:05.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00058797299999999999 2024-05-11T23:02:05.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:05.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.701491+0000", 2024-05-11T23:02:05.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9850000000000001e-05 2024-05-11T23:02:05.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:05.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.701505+0000", 2024-05-11T23:02:05.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3549e-05 2024-05-11T23:02:05.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:05.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.701574+0000", 2024-05-11T23:02:05.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8937999999999996e-05 2024-05-11T23:02:05.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:05.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.701598+0000", 2024-05-11T23:02:05.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4519e-05 2024-05-11T23:02:05.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:05.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.702631+0000", 2024-05-11T23:02:05.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001032942 2024-05-11T23:02:05.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:05.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.702662+0000", 2024-05-11T23:02:05.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0664000000000003e-05 2024-05-11T23:02:05.612 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.612 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.612 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.612 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:37451 3.0 e40/19 3:071059df:::rbd_data.111ea078695e.000000000000001f:head v 40'145, mlcod=40'143)", 2024-05-11T23:02:05.612 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:53.078457+0000", 2024-05-11T23:02:05.612 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.484162204, 2024-05-11T23:02:05.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0020281650000000002, 2024-05-11T23:02:05.613 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:05.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:05.613 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:05.613 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:05.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:05.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.078457+0000", 2024-05-11T23:02:05.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:05.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.078457+0000", 2024-05-11T23:02:05.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:05.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.078459+0000", 2024-05-11T23:02:05.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4500000000000001e-06 2024-05-11T23:02:05.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:05.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.078988+0000", 2024-05-11T23:02:05.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00052866000000000002 2024-05-11T23:02:05.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:05.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.079007+0000", 2024-05-11T23:02:05.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9398e-05 2024-05-11T23:02:05.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:05.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.079018+0000", 2024-05-11T23:02:05.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0834e-05 2024-05-11T23:02:05.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:05.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.079086+0000", 2024-05-11T23:02:05.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8578000000000006e-05 2024-05-11T23:02:05.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:05.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.079109+0000", 2024-05-11T23:02:05.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2189e-05 2024-05-11T23:02:05.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:05.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.080429+0000", 2024-05-11T23:02:05.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0013205070000000001 2024-05-11T23:02:05.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:05.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.080486+0000", 2024-05-11T23:02:05.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6548999999999997e-05 2024-05-11T23:02:05.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.616 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.616 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:37457 3.9 e40/19 3:95e7444d:::rbd_data.111ea078695e.0000000000000022:head v 40'79, mlcod=40'77)", 2024-05-11T23:02:05.616 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:53.087914+0000", 2024-05-11T23:02:05.617 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 12.474705504999999, 2024-05-11T23:02:05.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0023284349999999998, 2024-05-11T23:02:05.617 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:05.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:05.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:05.617 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:05.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:05.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.087914+0000", 2024-05-11T23:02:05.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:05.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.087914+0000", 2024-05-11T23:02:05.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:05.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.087916+0000", 2024-05-11T23:02:05.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7260000000000001e-06 2024-05-11T23:02:05.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:05.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.088797+0000", 2024-05-11T23:02:05.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00088118400000000003 2024-05-11T23:02:05.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:05.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.088814+0000", 2024-05-11T23:02:05.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6484999999999999e-05 2024-05-11T23:02:05.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:05.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.088820+0000", 2024-05-11T23:02:05.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8029999999999999e-06 2024-05-11T23:02:05.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:05.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.088872+0000", 2024-05-11T23:02:05.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1415999999999997e-05 2024-05-11T23:02:05.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:05.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.088890+0000", 2024-05-11T23:02:05.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8529000000000002e-05 2024-05-11T23:02:05.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:05.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.090179+0000", 2024-05-11T23:02:05.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0012889539999999999 2024-05-11T23:02:05.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:05.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:53.090243+0000", 2024-05-11T23:02:05.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3337999999999996e-05 2024-05-11T23:02:05.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.620 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.620 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60295 3.9 e42/19 3:92a0646f:::rbd_data.111e1df04b77.000000000000000a:head v 42'117, mlcod=42'115)", 2024-05-11T23:02:05.620 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.021408+0000", 2024-05-11T23:02:05.621 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5412119269999991, 2024-05-11T23:02:05.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0013993829999999999, 2024-05-11T23:02:05.621 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:05.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:05.621 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:05.621 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:05.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:05.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.021408+0000", 2024-05-11T23:02:05.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:05.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.021408+0000", 2024-05-11T23:02:05.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:05.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.021408+0000", 2024-05-11T23:02:05.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3900000000000004e-07 2024-05-11T23:02:05.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:05.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022002+0000", 2024-05-11T23:02:05.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000593992 2024-05-11T23:02:05.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:05.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022012+0000", 2024-05-11T23:02:05.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8390000000000002e-06 2024-05-11T23:02:05.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:05.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022017+0000", 2024-05-11T23:02:05.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6789999999999996e-06 2024-05-11T23:02:05.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:05.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022069+0000", 2024-05-11T23:02:05.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1934000000000001e-05 2024-05-11T23:02:05.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022089+0000", 2024-05-11T23:02:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9650999999999999e-05 2024-05-11T23:02:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022777+0000", 2024-05-11T23:02:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000688797 2024-05-11T23:02:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.022807+0000", 2024-05-11T23:02:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9852000000000001e-05 2024-05-11T23:02:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.625 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60336 3.9 e42/19 3:94f86c79:::rbd_data.111e1df04b77.0000000000000001:head v 42'119, mlcod=42'117)", 2024-05-11T23:02:05.625 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.065103+0000", 2024-05-11T23:02:05.625 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.4975164589999999, 2024-05-11T23:02:05.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0016346970000000001, 2024-05-11T23:02:05.625 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:05.625 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:05.625 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:05.625 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:05.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:05.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065103+0000", 2024-05-11T23:02:05.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:05.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065103+0000", 2024-05-11T23:02:05.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:05.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065104+0000", 2024-05-11T23:02:05.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.054e-06 2024-05-11T23:02:05.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:05.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065864+0000", 2024-05-11T23:02:05.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00075959699999999998 2024-05-11T23:02:05.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:05.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065879+0000", 2024-05-11T23:02:05.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4953999999999999e-05 2024-05-11T23:02:05.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:05.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065887+0000", 2024-05-11T23:02:05.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1329999999999999e-06 2024-05-11T23:02:05.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:05.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065954+0000", 2024-05-11T23:02:05.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6675999999999997e-05 2024-05-11T23:02:05.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:05.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.065978+0000", 2024-05-11T23:02:05.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4243000000000002e-05 2024-05-11T23:02:05.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:05.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.066707+0000", 2024-05-11T23:02:05.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00072877 2024-05-11T23:02:05.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:05.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.066738+0000", 2024-05-11T23:02:05.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1269999999999997e-05 2024-05-11T23:02:05.628 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.628 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.628 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.629 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60471 3.4 e43/19 3:2079ea20:::rbd_data.111e1df04b77.0000000000000000:head v 43'84, mlcod=42'81)", 2024-05-11T23:02:05.629 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:05.629 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.6357456900000003, 2024-05-11T23:02:05.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0079043050000000004, 2024-05-11T23:02:05.629 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:05.629 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:05.629 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:05.629 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:05.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:05.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:05.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:05.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:05.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:05.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:05.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-05-11T23:02:05.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:05.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926880+0000", 2024-05-11T23:02:05.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:02:05.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:05.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926893+0000", 2024-05-11T23:02:05.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3256e-05 2024-05-11T23:02:05.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:05.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926899+0000", 2024-05-11T23:02:05.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9850000000000003e-06 2024-05-11T23:02:05.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:05.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926982+0000", 2024-05-11T23:02:05.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2997000000000005e-05 2024-05-11T23:02:05.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:05.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927005+0000", 2024-05-11T23:02:05.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3428e-05 2024-05-11T23:02:05.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:05.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.934747+0000", 2024-05-11T23:02:05.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0077415820000000003 2024-05-11T23:02:05.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:05.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.934778+0000", 2024-05-11T23:02:05.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1479999999999997e-05 2024-05-11T23:02:05.632 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.632 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.632 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.632 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60472 3.9 e43/19 3:94f86c79:::rbd_data.111e1df04b77.0000000000000001:head v 43'122, mlcod=42'119)", 2024-05-11T23:02:05.632 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.927124+0000", 2024-05-11T23:02:05.633 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.6354959900000008, 2024-05-11T23:02:05.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0062295900000000001, 2024-05-11T23:02:05.633 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:05.633 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:05.633 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:05.633 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:05.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:05.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927124+0000", 2024-05-11T23:02:05.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:05.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927124+0000", 2024-05-11T23:02:05.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:05.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927124+0000", 2024-05-11T23:02:05.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:05.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:05.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927128+0000", 2024-05-11T23:02:05.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3390000000000003e-06 2024-05-11T23:02:05.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:05.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927139+0000", 2024-05-11T23:02:05.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1161e-05 2024-05-11T23:02:05.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:05.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927147+0000", 2024-05-11T23:02:05.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1910000000000003e-06 2024-05-11T23:02:05.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:05.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927199+0000", 2024-05-11T23:02:05.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.24e-05 2024-05-11T23:02:05.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:05.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927223+0000", 2024-05-11T23:02:05.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3811999999999998e-05 2024-05-11T23:02:05.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:05.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.933287+0000", 2024-05-11T23:02:05.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.006063723 2024-05-11T23:02:05.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:05.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.933353+0000", 2024-05-11T23:02:05.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6744999999999999e-05 2024-05-11T23:02:05.636 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.636 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.636 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.636 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60478 3.4 e43/19 3:2113d5af:::rbd_data.111e1df04b77.0000000000000007:head v 43'86, mlcod=42'82)", 2024-05-11T23:02:05.636 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.935643+0000", 2024-05-11T23:02:05.636 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.6269763699999995, 2024-05-11T23:02:05.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0016792039999999999, 2024-05-11T23:02:05.636 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:05.636 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:05.637 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:05.637 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:05.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:05.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.935643+0000", 2024-05-11T23:02:05.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:05.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.935643+0000", 2024-05-11T23:02:05.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:05.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.935645+0000", 2024-05-11T23:02:05.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4419999999999999e-06 2024-05-11T23:02:05.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:05.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.935653+0000", 2024-05-11T23:02:05.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0460000000000006e-06 2024-05-11T23:02:05.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:05.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.935669+0000", 2024-05-11T23:02:05.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5809000000000001e-05 2024-05-11T23:02:05.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:05.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.935676+0000", 2024-05-11T23:02:05.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3100000000000003e-06 2024-05-11T23:02:05.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:05.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.936410+0000", 2024-05-11T23:02:05.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00073357399999999997 2024-05-11T23:02:05.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:05.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.936423+0000", 2024-05-11T23:02:05.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3762e-05 2024-05-11T23:02:05.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:05.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.937274+0000", 2024-05-11T23:02:05.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00085112600000000001 2024-05-11T23:02:05.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:05.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.937323+0000", 2024-05-11T23:02:05.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8134999999999999e-05 2024-05-11T23:02:05.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.640 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.640 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:64399 3.1e e48/47 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head v 48'297, mlcod=0'0)", 2024-05-11T23:02:05.640 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:02.104619+0000", 2024-05-11T23:02:05.640 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.458000577, 2024-05-11T23:02:05.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014840949000000001, 2024-05-11T23:02:05.640 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:05.640 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:05.640 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:05.640 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:05.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:05.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.104619+0000", 2024-05-11T23:02:05.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:05.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.104619+0000", 2024-05-11T23:02:05.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:05.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:05.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.104620+0000", 2024-05-11T23:02:05.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.299e-06 2024-05-11T23:02:05.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:05.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.112866+0000", 2024-05-11T23:02:05.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082450949999999992 2024-05-11T23:02:05.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:05.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.112893+0000", 2024-05-11T23:02:05.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7131000000000002e-05 2024-05-11T23:02:05.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:05.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.112902+0000", 2024-05-11T23:02:05.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4089999999999992e-06 2024-05-11T23:02:05.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:05.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.112980+0000", 2024-05-11T23:02:05.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8375999999999996e-05 2024-05-11T23:02:05.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:05.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.113009+0000", 2024-05-11T23:02:05.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8713000000000001e-05 2024-05-11T23:02:05.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:05.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.119403+0000", 2024-05-11T23:02:05.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0063940580000000002 2024-05-11T23:02:05.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:05.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:05.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:05.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.119460+0000", 2024-05-11T23:02:05.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6867999999999998e-05 2024-05-11T23:02:05.644 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.644 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.644 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.644 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:05.644 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:05.644 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:05.645 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:02:05.646 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:05.646+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:05.746 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:05.745+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:05.796 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:05.796 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:05.796 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:02:05.796 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:05.805 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:02:05.848 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:05.848+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-05-11T23:02:05.920 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone21 from 0xd9c9882 (overlap 0x1a9dd36) to 0x0 2024-05-11T23:02:05.930 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:05.930 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:05.930 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:05.930 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:05.930 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:05.945 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:02:05.948 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:05.947+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:06.048 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:06.048+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:06.073 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-05-11T23:02:06.143 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:06.143 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:06.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:06.143 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:06.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.143 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36828 3.b 3:d385bc8e:::rbd_data.111e5e4d8275.0000000000000020:11 [read 512000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.143 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:06.143 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.208799637, 2024-05-11T23:02:06.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051481218000000002, 2024-05-11T23:02:06.144 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:06.144 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:06.144 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:06.144 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:06.144 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:06.144 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.144 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36828 2024-05-11T23:02:06.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.144 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:06.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:06.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:06.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:06.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:06.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:06.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934915+0000", 2024-05-11T23:02:06.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00181814 2024-05-11T23:02:06.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:06.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934919+0000", 2024-05-11T23:02:06.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6739999999999996e-06 2024-05-11T23:02:06.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:06.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934934+0000", 2024-05-11T23:02:06.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4415e-05 2024-05-11T23:02:06.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934942+0000", 2024-05-11T23:02:06.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8619999999999992e-06 2024-05-11T23:02:06.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:06.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984433+0000", 2024-05-11T23:02:06.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049490658999999999 2024-05-11T23:02:06.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:06.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984508+0000", 2024-05-11T23:02:06.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4394999999999995e-05 2024-05-11T23:02:06.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:06.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984578+0000", 2024-05-11T23:02:06.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0073000000000005e-05 2024-05-11T23:02:06.151 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.151 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.151 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.152 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36950 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 520192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.152 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:06.152 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.204493871, 2024-05-11T23:02:06.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051450856000000003, 2024-05-11T23:02:06.152 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:06.152 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:06.152 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:06.152 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:06.152 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:06.152 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.152 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36950 2024-05-11T23:02:06.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.152 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:06.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:06.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:06.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:06.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:06.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:06.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938914+0000", 2024-05-11T23:02:06.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015121480000000001 2024-05-11T23:02:06.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:06.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938918+0000", 2024-05-11T23:02:06.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4149999999999999e-06 2024-05-11T23:02:06.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:06.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938925+0000", 2024-05-11T23:02:06.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0779999999999998e-06 2024-05-11T23:02:06.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938929+0000", 2024-05-11T23:02:06.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2930000000000002e-06 2024-05-11T23:02:06.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:06.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988675+0000", 2024-05-11T23:02:06.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049746394999999999 2024-05-11T23:02:06.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:06.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988756+0000", 2024-05-11T23:02:06.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0499000000000003e-05 2024-05-11T23:02:06.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:06.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988853+0000", 2024-05-11T23:02:06.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7027999999999993e-05 2024-05-11T23:02:06.155 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.155 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.155 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.156 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36957 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 778240~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.156 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:06.156 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.202872082000001, 2024-05-11T23:02:06.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051328273000000001, 2024-05-11T23:02:06.156 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:06.156 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:06.156 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:06.156 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:06.156 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:06.156 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36957 2024-05-11T23:02:06.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.156 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:06.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:06.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:06.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:06.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:06.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:06.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:06.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2099999999999998e-07 2024-05-11T23:02:06.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:06.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939026+0000", 2024-05-11T23:02:06.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5069999999999999e-06 2024-05-11T23:02:06.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:06.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939030+0000", 2024-05-11T23:02:06.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7919999999999999e-06 2024-05-11T23:02:06.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939032+0000", 2024-05-11T23:02:06.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9129999999999999e-06 2024-05-11T23:02:06.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:06.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990178+0000", 2024-05-11T23:02:06.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051145373000000001 2024-05-11T23:02:06.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:06.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990255+0000", 2024-05-11T23:02:06.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7195999999999997e-05 2024-05-11T23:02:06.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:06.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990352+0000", 2024-05-11T23:02:06.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7170999999999995e-05 2024-05-11T23:02:06.159 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.159 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.159 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.160 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36958 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 815104~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.160 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:06.160 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.202859762999999, 2024-05-11T23:02:06.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051531625999999997, 2024-05-11T23:02:06.160 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:06.160 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:06.160 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:06.160 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:06.160 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:06.160 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.160 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36958 2024-05-11T23:02:06.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.160 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:06.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:06.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:06.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:06.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:06.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:06.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939037+0000", 2024-05-11T23:02:06.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0400000000000002e-07 2024-05-11T23:02:06.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:06.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939038+0000", 2024-05-11T23:02:06.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.164e-06 2024-05-11T23:02:06.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:06.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939041+0000", 2024-05-11T23:02:06.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1109999999999999e-06 2024-05-11T23:02:06.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939043+0000", 2024-05-11T23:02:06.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.554e-06 2024-05-11T23:02:06.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:06.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990382+0000", 2024-05-11T23:02:06.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051338953999999999 2024-05-11T23:02:06.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:06.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990465+0000", 2024-05-11T23:02:06.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2461999999999998e-05 2024-05-11T23:02:06.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:06.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990568+0000", 2024-05-11T23:02:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000103077 2024-05-11T23:02:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36959 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 851968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.202847866999999, 2024-05-11T23:02:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051758936999999998, 2024-05-11T23:02:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.165 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36959 2024-05-11T23:02:06.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.165 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:06.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:06.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:06.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:06.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:06.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:06.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:06.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.16e-07 2024-05-11T23:02:06.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:06.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939050+0000", 2024-05-11T23:02:06.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.176e-06 2024-05-11T23:02:06.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:06.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939053+0000", 2024-05-11T23:02:06.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1140000000000002e-06 2024-05-11T23:02:06.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939055+0000", 2024-05-11T23:02:06.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.621e-06 2024-05-11T23:02:06.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:06.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990610+0000", 2024-05-11T23:02:06.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051554244999999999 2024-05-11T23:02:06.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:06.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990700+0000", 2024-05-11T23:02:06.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0284999999999994e-05 2024-05-11T23:02:06.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:06.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990807+0000", 2024-05-11T23:02:06.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010728 2024-05-11T23:02:06.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.168 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.168 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36960 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 888832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.168 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:06.168 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.202837632, 2024-05-11T23:02:06.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051886831000000001, 2024-05-11T23:02:06.168 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:06.168 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:06.168 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:06.168 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:06.168 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:06.169 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.169 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36960 2024-05-11T23:02:06.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.169 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:06.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:06.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:06.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:06.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:06.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:06.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940913+0000", 2024-05-11T23:02:06.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001855104 2024-05-11T23:02:06.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:06.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940916+0000", 2024-05-11T23:02:06.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0000000000000001e-06 2024-05-11T23:02:06.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:06.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940924+0000", 2024-05-11T23:02:06.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0999999999999998e-06 2024-05-11T23:02:06.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940927+0000", 2024-05-11T23:02:06.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7160000000000001e-06 2024-05-11T23:02:06.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:06.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990826+0000", 2024-05-11T23:02:06.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049898832999999997 2024-05-11T23:02:06.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:06.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990872+0000", 2024-05-11T23:02:06.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5717000000000003e-05 2024-05-11T23:02:06.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:06.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990945+0000", 2024-05-11T23:02:06.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3361000000000005e-05 2024-05-11T23:02:06.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.172 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.172 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36986 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 528384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.172 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:06.172 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.200831414, 2024-05-11T23:02:06.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051341053999999997, 2024-05-11T23:02:06.172 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:06.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:06.172 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:06.172 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:06.172 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:06.173 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.173 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36986 2024-05-11T23:02:06.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.173 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:06.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:06.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:06.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:06.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:06.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:06.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:06.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:02:06.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:06.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941066+0000", 2024-05-11T23:02:06.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1629999999999999e-06 2024-05-11T23:02:06.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:06.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941069+0000", 2024-05-11T23:02:06.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.118e-06 2024-05-11T23:02:06.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941072+0000", 2024-05-11T23:02:06.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5720000000000001e-06 2024-05-11T23:02:06.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:06.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992292+0000", 2024-05-11T23:02:06.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051220620000000001 2024-05-11T23:02:06.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:06.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992344+0000", 2024-05-11T23:02:06.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1737000000000003e-05 2024-05-11T23:02:06.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:06.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992406+0000", 2024-05-11T23:02:06.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1623000000000003e-05 2024-05-11T23:02:06.176 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.176 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.176 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.176 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36987 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 565248~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.176 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:06.176 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.200821293000001, 2024-05-11T23:02:06.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051568725000000003, 2024-05-11T23:02:06.176 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:06.176 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:06.176 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:06.176 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:06.177 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:06.177 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.177 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36987 2024-05-11T23:02:06.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.177 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:06.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:06.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:06.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:06.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:06.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:06.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:06.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.91e-07 2024-05-11T23:02:06.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:06.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941076+0000", 2024-05-11T23:02:06.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1379999999999999e-06 2024-05-11T23:02:06.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:06.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941079+0000", 2024-05-11T23:02:06.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9739999999999998e-06 2024-05-11T23:02:06.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941082+0000", 2024-05-11T23:02:06.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-05-11T23:02:06.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:06.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992446+0000", 2024-05-11T23:02:06.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051364058999999997 2024-05-11T23:02:06.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:06.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992536+0000", 2024-05-11T23:02:06.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0085000000000002e-05 2024-05-11T23:02:06.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:06.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992643+0000", 2024-05-11T23:02:06.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107377 2024-05-11T23:02:06.180 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.180 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.180 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.180 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36988 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 602112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.181 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:06.181 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.200810924000001, 2024-05-11T23:02:06.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051700007999999999, 2024-05-11T23:02:06.181 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:06.181 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:06.181 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:06.181 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:06.181 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:06.181 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.181 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36988 2024-05-11T23:02:06.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.181 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:06.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:06.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:06.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:06.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:06.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:06.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:06.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.05e-07 2024-05-11T23:02:06.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:06.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941086+0000", 2024-05-11T23:02:06.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.125e-06 2024-05-11T23:02:06.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:06.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941089+0000", 2024-05-11T23:02:06.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0599999999999999e-06 2024-05-11T23:02:06.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941092+0000", 2024-05-11T23:02:06.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6529999999999998e-06 2024-05-11T23:02:06.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:06.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992667+0000", 2024-05-11T23:02:06.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051574587999999998 2024-05-11T23:02:06.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:06.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992723+0000", 2024-05-11T23:02:06.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6682999999999999e-05 2024-05-11T23:02:06.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:06.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992785+0000", 2024-05-11T23:02:06.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1693999999999996e-05 2024-05-11T23:02:06.184 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.184 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.184 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.184 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36989 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 638976~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.185 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:06.185 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.200800735, 2024-05-11T23:02:06.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05192811, 2024-05-11T23:02:06.185 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:06.185 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:06.185 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:06.185 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:06.185 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:06.185 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.185 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36989 2024-05-11T23:02:06.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.185 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:06.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:06.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:06.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:06.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:06.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:06.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942916+0000", 2024-05-11T23:02:06.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001820835 2024-05-11T23:02:06.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:06.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942920+0000", 2024-05-11T23:02:06.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5310000000000002e-06 2024-05-11T23:02:06.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:06.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942931+0000", 2024-05-11T23:02:06.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1839e-05 2024-05-11T23:02:06.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942940+0000", 2024-05-11T23:02:06.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2309999999999996e-06 2024-05-11T23:02:06.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:06.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992820+0000", 2024-05-11T23:02:06.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049880751000000001 2024-05-11T23:02:06.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:06.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992918+0000", 2024-05-11T23:02:06.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7051000000000003e-05 2024-05-11T23:02:06.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:06.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.993023+0000", 2024-05-11T23:02:06.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010587200000000001 2024-05-11T23:02:06.188 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.188 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.188 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.189 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36997 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 933888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.189 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:06.189 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.198766900000001, 2024-05-11T23:02:06.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051390602000000001, 2024-05-11T23:02:06.189 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:06.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:06.189 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:06.189 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:06.189 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:06.189 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.189 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36997 2024-05-11T23:02:06.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.189 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:06.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:06.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:06.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:06.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:06.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:06.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943130+0000", 2024-05-11T23:02:06.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0200000000000003e-07 2024-05-11T23:02:06.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:06.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943132+0000", 2024-05-11T23:02:06.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.305e-06 2024-05-11T23:02:06.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:06.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943138+0000", 2024-05-11T23:02:06.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4219999999999997e-06 2024-05-11T23:02:06.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943143+0000", 2024-05-11T23:02:06.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0610000000000003e-06 2024-05-11T23:02:06.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:06.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994323+0000", 2024-05-11T23:02:06.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051179592000000003 2024-05-11T23:02:06.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:06.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994412+0000", 2024-05-11T23:02:06.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8709999999999996e-05 2024-05-11T23:02:06.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:06.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994520+0000", 2024-05-11T23:02:06.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010811000000000001 2024-05-11T23:02:06.192 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.192 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.192 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.193 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36998 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 970752~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.193 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:06.193 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.198746021, 2024-05-11T23:02:06.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051517669000000002, 2024-05-11T23:02:06.193 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:06.193 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:06.193 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:06.193 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:06.193 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:06.193 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.193 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36998 2024-05-11T23:02:06.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.193 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:06.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:06.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:06.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:06.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:06.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:06.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:06.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:02:06.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:06.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943153+0000", 2024-05-11T23:02:06.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.306e-06 2024-05-11T23:02:06.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:06.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943160+0000", 2024-05-11T23:02:06.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.452e-06 2024-05-11T23:02:06.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943165+0000", 2024-05-11T23:02:06.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.152e-06 2024-05-11T23:02:06.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:06.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994561+0000", 2024-05-11T23:02:06.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051395765000000003 2024-05-11T23:02:06.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:06.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994607+0000", 2024-05-11T23:02:06.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6415000000000002e-05 2024-05-11T23:02:06.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:06.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994668+0000", 2024-05-11T23:02:06.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0179000000000001e-05 2024-05-11T23:02:06.196 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.196 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.196 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.196 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36999 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 1007616~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.197 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:06.197 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.198701140000001, 2024-05-11T23:02:06.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051696919000000001, 2024-05-11T23:02:06.197 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:06.197 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:06.197 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:06.197 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:06.197 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:06.197 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.197 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36999 2024-05-11T23:02:06.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.197 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:06.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:06.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:06.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:06.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:06.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:06.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:06.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:02:06.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:06.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943197+0000", 2024-05-11T23:02:06.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1280000000000002e-06 2024-05-11T23:02:06.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:06.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943202+0000", 2024-05-11T23:02:06.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8910000000000003e-06 2024-05-11T23:02:06.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943206+0000", 2024-05-11T23:02:06.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3550000000000001e-06 2024-05-11T23:02:06.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:06.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994703+0000", 2024-05-11T23:02:06.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051497045999999998 2024-05-11T23:02:06.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:06.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994789+0000", 2024-05-11T23:02:06.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6458000000000005e-05 2024-05-11T23:02:06.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:06.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994892+0000", 2024-05-11T23:02:06.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102705 2024-05-11T23:02:06.200 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.200 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.200 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.201 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37000 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 1044480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.201 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:06.201 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.198685194999999, 2024-05-11T23:02:06.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051842371999999998, 2024-05-11T23:02:06.201 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:06.201 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:06.201 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:06.201 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:06.201 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:06.201 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.201 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37000 2024-05-11T23:02:06.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.201 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:06.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:06.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:06.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:06.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:06.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:06.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944916+0000", 2024-05-11T23:02:06.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001705097 2024-05-11T23:02:06.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:06.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944921+0000", 2024-05-11T23:02:06.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5809999999999998e-06 2024-05-11T23:02:06.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:06.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944930+0000", 2024-05-11T23:02:06.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5529999999999998e-06 2024-05-11T23:02:06.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944938+0000", 2024-05-11T23:02:06.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2519999999999999e-06 2024-05-11T23:02:06.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:06.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994911+0000", 2024-05-11T23:02:06.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049973113999999999 2024-05-11T23:02:06.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:06.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994982+0000", 2024-05-11T23:02:06.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0544999999999996e-05 2024-05-11T23:02:06.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:06.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.995053+0000", 2024-05-11T23:02:06.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1229999999999994e-05 2024-05-11T23:02:06.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.204 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.205 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37041 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 618496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.205 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:06.205 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.196841215999999, 2024-05-11T23:02:06.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051352768999999999, 2024-05-11T23:02:06.205 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:06.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:06.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:06.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:06.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:06.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37041 2024-05-11T23:02:06.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.205 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:06.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:06.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:06.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:06.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:06.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:06.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:06.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:02:06.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:06.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945056+0000", 2024-05-11T23:02:06.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:02:06.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:06.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945059+0000", 2024-05-11T23:02:06.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.965e-06 2024-05-11T23:02:06.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945062+0000", 2024-05-11T23:02:06.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:02:06.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:06.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996213+0000", 2024-05-11T23:02:06.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051151532999999999 2024-05-11T23:02:06.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:06.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996301+0000", 2024-05-11T23:02:06.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8143e-05 2024-05-11T23:02:06.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:06.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996408+0000", 2024-05-11T23:02:06.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010623700000000001 2024-05-11T23:02:06.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.208 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.209 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.209 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37042 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 655360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.209 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:06.209 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.196831231000001, 2024-05-11T23:02:06.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051462075000000003, 2024-05-11T23:02:06.209 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:06.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:06.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:06.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:06.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:06.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37042 2024-05-11T23:02:06.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.210 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:06.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:06.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:06.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:06.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:06.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:06.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:06.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0599999999999999e-07 2024-05-11T23:02:06.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:06.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945066+0000", 2024-05-11T23:02:06.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-05-11T23:02:06.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:06.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945069+0000", 2024-05-11T23:02:06.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0079999999999998e-06 2024-05-11T23:02:06.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945072+0000", 2024-05-11T23:02:06.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6740000000000001e-06 2024-05-11T23:02:06.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:06.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996422+0000", 2024-05-11T23:02:06.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051349999 2024-05-11T23:02:06.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:06.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996467+0000", 2024-05-11T23:02:06.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.528e-05 2024-05-11T23:02:06.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:06.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996527+0000", 2024-05-11T23:02:06.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9759999999999997e-05 2024-05-11T23:02:06.213 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.213 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.213 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.213 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37043 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 692224~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.213 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:06.213 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.196819158, 2024-05-11T23:02:06.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051679535999999998, 2024-05-11T23:02:06.213 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:06.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:06.214 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:06.214 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:06.214 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:06.214 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37043 2024-05-11T23:02:06.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.214 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:06.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:06.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:06.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:06.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:06.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:06.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:06.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8999999999999997e-07 2024-05-11T23:02:06.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:06.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945079+0000", 2024-05-11T23:02:06.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.001e-06 2024-05-11T23:02:06.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:06.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945087+0000", 2024-05-11T23:02:06.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1749999999999999e-06 2024-05-11T23:02:06.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945093+0000", 2024-05-11T23:02:06.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9639999999999997e-06 2024-05-11T23:02:06.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:06.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996563+0000", 2024-05-11T23:02:06.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051469050000000002 2024-05-11T23:02:06.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:06.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996651+0000", 2024-05-11T23:02:06.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8356000000000006e-05 2024-05-11T23:02:06.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:06.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996756+0000", 2024-05-11T23:02:06.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0001055 2024-05-11T23:02:06.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.217 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.217 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37044 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 729088~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.217 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:06.217 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.196785245999999, 2024-05-11T23:02:06.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051763445999999998, 2024-05-11T23:02:06.218 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:06.218 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:06.218 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:06.218 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:06.218 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:06.218 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.218 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37044 2024-05-11T23:02:06.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.218 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:06.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:06.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:06.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:06.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:06.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:06.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946915+0000", 2024-05-11T23:02:06.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018042760000000001 2024-05-11T23:02:06.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:06.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946918+0000", 2024-05-11T23:02:06.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.36e-06 2024-05-11T23:02:06.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:06.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946926+0000", 2024-05-11T23:02:06.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3130000000000001e-06 2024-05-11T23:02:06.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946930+0000", 2024-05-11T23:02:06.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5249999999999999e-06 2024-05-11T23:02:06.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:06.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996772+0000", 2024-05-11T23:02:06.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049841868999999997 2024-05-11T23:02:06.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:06.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996817+0000", 2024-05-11T23:02:06.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5351e-05 2024-05-11T23:02:06.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:06.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996874+0000", 2024-05-11T23:02:06.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6752000000000001e-05 2024-05-11T23:02:06.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.221 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.221 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37052 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 1024000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.221 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:06.221 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.194878395, 2024-05-11T23:02:06.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051326140999999999, 2024-05-11T23:02:06.221 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:06.221 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:06.221 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:06.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:06.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:06.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37052 2024-05-11T23:02:06.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.222 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:06.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:06.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:06.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:06.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:06.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:06.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:06.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:06.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:06.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947019+0000", 2024-05-11T23:02:06.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.218e-06 2024-05-11T23:02:06.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:06.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947022+0000", 2024-05-11T23:02:06.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1889999999999999e-06 2024-05-11T23:02:06.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947025+0000", 2024-05-11T23:02:06.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.464e-06 2024-05-11T23:02:06.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:06.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998237+0000", 2024-05-11T23:02:06.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051212494999999997 2024-05-11T23:02:06.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:06.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998284+0000", 2024-05-11T23:02:06.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7246000000000001e-05 2024-05-11T23:02:06.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:06.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998344+0000", 2024-05-11T23:02:06.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9320000000000001e-05 2024-05-11T23:02:06.225 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.225 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.225 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.225 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37071 3.b 3:d385bc8e:::rbd_data.111e5e4d8275.0000000000000020:11 [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.225 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:06.225 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.194868186000001, 2024-05-11T23:02:06.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051528796000000002, 2024-05-11T23:02:06.225 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:06.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:06.225 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:06.225 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:06.226 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:06.226 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.226 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37071 2024-05-11T23:02:06.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.226 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:06.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:06.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:06.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:06.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:06.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:06.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:06.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948913+0000", 2024-05-11T23:02:06.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018854970000000001 2024-05-11T23:02:06.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:06.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948917+0000", 2024-05-11T23:02:06.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8089999999999999e-06 2024-05-11T23:02:06.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:06.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948926+0000", 2024-05-11T23:02:06.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8389999999999996e-06 2024-05-11T23:02:06.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948930+0000", 2024-05-11T23:02:06.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4519999999999999e-06 2024-05-11T23:02:06.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:06.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998379+0000", 2024-05-11T23:02:06.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049449011000000001 2024-05-11T23:02:06.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:06.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998462+0000", 2024-05-11T23:02:06.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2739999999999997e-05 2024-05-11T23:02:06.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:06.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:06.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:06.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998557+0000", 2024-05-11T23:02:06.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4448000000000001e-05 2024-05-11T23:02:06.229 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.229 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.229 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.229 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:06.229 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:06.229 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:06.231 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:02:06.231 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:06.230+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:06.291 INFO:teuthology.orchestra.run.smithi005.stdout:2822 write 0x25d8ee4 thru 0x25e4056 (0xb173 bytes) 2024-05-11T23:02:06.307 INFO:teuthology.orchestra.run.smithi005.stdout:2823 read 0xa462623 thru 0xa471e24 (0xf802 bytes) 2024-05-11T23:02:06.307 INFO:teuthology.orchestra.run.smithi005.stdout:2824 punch from 0xbe4ffb3 to 0xbe5ec10, (0xec5d bytes) 2024-05-11T23:02:06.307 INFO:teuthology.orchestra.run.smithi005.stdout:2825 read 0xc96b7e9 thru 0xc974263 (0x8a7b bytes) 2024-05-11T23:02:06.307 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:06.307 INFO:teuthology.orchestra.run.smithi005.stdout:2828 write 0xd87dfb7 thru 0xd88a6a7 (0xc6f1 bytes) 2024-05-11T23:02:06.318 INFO:teuthology.orchestra.run.smithi005.stdout:2829 trunc from 0xd9c9882 to 0xa58bc0d 2024-05-11T23:02:06.334 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:06.331+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:06.346 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:06.346 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:06.346 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:06.346 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:06.348 INFO:teuthology.orchestra.run.smithi005.stdout:2830 punch from 0x3d7d5e5 to 0x3d84f05, (0x7920 bytes) 2024-05-11T23:02:06.348 INFO:teuthology.orchestra.run.smithi005.stdout:2831 write 0x9231923 thru 0x92337b0 (0x1e8e bytes) 2024-05-11T23:02:06.353 INFO:teuthology.orchestra.run.smithi005.stdout:2834 punch from 0x70271ec to 0x7035ec0, (0xecd4 bytes) 2024-05-11T23:02:06.353 INFO:teuthology.orchestra.run.smithi005.stdout:2838 read 0x4903351 thru 0x490d7ab (0xa45b bytes) 2024-05-11T23:02:06.353 INFO:teuthology.orchestra.run.smithi005.stdout:2841 punch from 0x667ecb2 to 0x66807c9, (0x1b17 bytes) 2024-05-11T23:02:06.353 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:06.353 INFO:teuthology.orchestra.run.smithi005.stdout:2845 read 0x7eaa0d3 thru 0x7eaff43 (0x5e71 bytes) 2024-05-11T23:02:06.353 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:06.353 INFO:teuthology.orchestra.run.smithi005.stdout:2848 write 0x380ee41 thru 0x3813f27 (0x50e7 bytes) 2024-05-11T23:02:06.356 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:02:06.362 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:06.362 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:06.362 INFO:teuthology.orchestra.run.smithi005.stdout:2851 read 0x2da6025 thru 0x2da8beb (0x2bc7 bytes) 2024-05-11T23:02:06.362 INFO:teuthology.orchestra.run.smithi005.stdout:2853 read 0x89793b2 thru 0x897acb7 (0x1906 bytes) 2024-05-11T23:02:06.362 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:06.362 INFO:teuthology.orchestra.run.smithi005.stdout:2855 write 0x7b98f30 thru 0x7b9ae4a (0x1f1b bytes) 2024-05-11T23:02:06.370 INFO:teuthology.orchestra.run.smithi005.stdout:2857 trunc from 0xa58bc0d to 0xe1bbf88 2024-05-11T23:02:06.377 INFO:teuthology.orchestra.run.smithi005.stdout:2859 read 0x3b048d6 thru 0x3b08939 (0x4064 bytes) 2024-05-11T23:02:06.377 INFO:teuthology.orchestra.run.smithi005.stdout:2862 read 0x346fda7 thru 0x3478424 (0x867e bytes) 2024-05-11T23:02:06.377 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:06.377 INFO:teuthology.orchestra.run.smithi005.stdout:2864 write 0x1666ccb thru 0x166d2d1 (0x6607 bytes) 2024-05-11T23:02:06.440 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:06.440+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:06.486 INFO:teuthology.orchestra.run.smithi005.stdout:2866 writesame 0xafcadf8 thru 0xafd6c1d data_size 0x85(0xbe26 bytes) 2024-05-11T23:02:06.494 INFO:teuthology.orchestra.run.smithi005.stdout:2868 writesame 0x391c29b thru 0x391c61f data_size 0x1(0x385 bytes) 2024-05-11T23:02:06.497 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:06.497 INFO:teuthology.orchestra.run.smithi005.stdout:2873 write 0x1f27c41 thru 0x1f33728 (0xbae8 bytes) 2024-05-11T23:02:06.520 INFO:teuthology.orchestra.run.smithi005.stdout:2875 read 0x824ed25 thru 0x825e596 (0xf872 bytes) 2024-05-11T23:02:06.520 INFO:teuthology.orchestra.run.smithi005.stdout:2877 writesame 0x966f734 thru 0x9676a0b data_size 0x3c(0x72d8 bytes) 2024-05-11T23:02:06.523 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:06.523 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:06.523 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:06.523 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:06.523 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:06.533 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:02:06.534 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:06.533+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:06.542 INFO:teuthology.orchestra.run.smithi005.stdout:2878 trunc from 0xe1bbf88 to 0x5029314 2024-05-11T23:02:06.564 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:06.565 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:06.565 INFO:teuthology.orchestra.run.smithi005.stdout:2881 read 0x4663283 thru 0x466cb00 (0x987e bytes) 2024-05-11T23:02:06.565 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:06.565 INFO:teuthology.orchestra.run.smithi005.stdout:2885 read 0xfd5232 thru 0xfd9b45 (0x4914 bytes) 2024-05-11T23:02:06.574 INFO:teuthology.orchestra.run.smithi005.stdout:2888 writesame 0x815646d thru 0x815e809 data_size 0xb(0x839d bytes) 2024-05-11T23:02:06.575 INFO:teuthology.orchestra.run.smithi005.stdout:2889 punch from 0x38c04d0 to 0x38c528d, (0x4dbd bytes) 2024-05-11T23:02:06.575 INFO:teuthology.orchestra.run.smithi005.stdout:2890 read 0x47b5234 thru 0x47c518c (0xff59 bytes) 2024-05-11T23:02:06.577 INFO:teuthology.orchestra.run.smithi005.stdout:2891 trunc from 0x815e80a to 0xa6263af 2024-05-11T23:02:06.594 INFO:teuthology.orchestra.run.smithi005.stdout:2892 trunc from 0xa6263af to 0x45ccaa 2024-05-11T23:02:06.635 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:06.635+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:06.714 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:06.714 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:06.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24548 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.927949+0000", 2024-05-11T23:02:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.785470870999999, 2024-05-11T23:02:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037300396999999999, 2024-05-11T23:02:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:06.715 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24548 2024-05-11T23:02:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927949+0000", 2024-05-11T23:02:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927949+0000", 2024-05-11T23:02:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927950+0000", 2024-05-11T23:02:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9400000000000005e-07 2024-05-11T23:02:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927952+0000", 2024-05-11T23:02:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2330000000000001e-06 2024-05-11T23:02:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927959+0000", 2024-05-11T23:02:06.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8820000000000003e-06 2024-05-11T23:02:06.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927964+0000", 2024-05-11T23:02:06.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.899e-06 2024-05-11T23:02:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.959280+0000", 2024-05-11T23:02:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031315904999999998 2024-05-11T23:02:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.959348+0000", 2024-05-11T23:02:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8214999999999997e-05 2024-05-11T23:02:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965250+0000", 2024-05-11T23:02:06.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005901669 2024-05-11T23:02:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24555 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.927974+0000", 2024-05-11T23:02:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.785445604, 2024-05-11T23:02:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.03739746, 2024-05-11T23:02:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:06.721 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:06.722 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:06.722 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:06.722 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.722 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24555 2024-05-11T23:02:06.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.722 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927974+0000", 2024-05-11T23:02:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927974+0000", 2024-05-11T23:02:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927975+0000", 2024-05-11T23:02:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6899999999999998e-07 2024-05-11T23:02:06.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927977+0000", 2024-05-11T23:02:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.074e-06 2024-05-11T23:02:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927984+0000", 2024-05-11T23:02:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7309999999999996e-06 2024-05-11T23:02:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927989+0000", 2024-05-11T23:02:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.079e-06 2024-05-11T23:02:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965279+0000", 2024-05-11T23:02:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037290271999999999 2024-05-11T23:02:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965324+0000", 2024-05-11T23:02:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4638999999999997e-05 2024-05-11T23:02:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:06.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965372+0000", 2024-05-11T23:02:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8195999999999997e-05 2024-05-11T23:02:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24562 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.927995+0000", 2024-05-11T23:02:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.785424668999999, 2024-05-11T23:02:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037703408000000001, 2024-05-11T23:02:06.726 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24562 2024-05-11T23:02:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927995+0000", 2024-05-11T23:02:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:06.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927995+0000", 2024-05-11T23:02:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927996+0000", 2024-05-11T23:02:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9499999999999998e-07 2024-05-11T23:02:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927998+0000", 2024-05-11T23:02:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0889999999999998e-06 2024-05-11T23:02:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928004+0000", 2024-05-11T23:02:06.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4740000000000002e-06 2024-05-11T23:02:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928009+0000", 2024-05-11T23:02:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:02:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965549+0000", 2024-05-11T23:02:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037539826999999998 2024-05-11T23:02:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965616+0000", 2024-05-11T23:02:06.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7387999999999999e-05 2024-05-11T23:02:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965699+0000", 2024-05-11T23:02:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2347000000000003e-05 2024-05-11T23:02:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24568 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928016+0000", 2024-05-11T23:02:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.785404483000001, 2024-05-11T23:02:06.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037850338999999997, 2024-05-11T23:02:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24568 2024-05-11T23:02:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928016+0000", 2024-05-11T23:02:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928016+0000", 2024-05-11T23:02:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928016+0000", 2024-05-11T23:02:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-05-11T23:02:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928025+0000", 2024-05-11T23:02:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.363e-06 2024-05-11T23:02:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928032+0000", 2024-05-11T23:02:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.114e-06 2024-05-11T23:02:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928038+0000", 2024-05-11T23:02:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1250000000000001e-06 2024-05-11T23:02:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965720+0000", 2024-05-11T23:02:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037682488 2024-05-11T23:02:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965785+0000", 2024-05-11T23:02:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5180999999999998e-05 2024-05-11T23:02:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965866+0000", 2024-05-11T23:02:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0687999999999997e-05 2024-05-11T23:02:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.734 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24574 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 266240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928045+0000", 2024-05-11T23:02:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.785375505999999, 2024-05-11T23:02:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037990532, 2024-05-11T23:02:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:06.735 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24574 2024-05-11T23:02:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928045+0000", 2024-05-11T23:02:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928045+0000", 2024-05-11T23:02:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928045+0000", 2024-05-11T23:02:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:02:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928047+0000", 2024-05-11T23:02:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9649999999999998e-06 2024-05-11T23:02:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928053+0000", 2024-05-11T23:02:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1550000000000004e-06 2024-05-11T23:02:06.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928058+0000", 2024-05-11T23:02:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9060000000000001e-06 2024-05-11T23:02:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965890+0000", 2024-05-11T23:02:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037831754000000002 2024-05-11T23:02:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:06.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965957+0000", 2024-05-11T23:02:06.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6798999999999995e-05 2024-05-11T23:02:06.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:06.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966035+0000", 2024-05-11T23:02:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8532e-05 2024-05-11T23:02:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24581 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928064+0000", 2024-05-11T23:02:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.785355689999999, 2024-05-11T23:02:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038134135, 2024-05-11T23:02:06.743 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24581 2024-05-11T23:02:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928064+0000", 2024-05-11T23:02:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928064+0000", 2024-05-11T23:02:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928065+0000", 2024-05-11T23:02:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3099999999999998e-07 2024-05-11T23:02:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928067+0000", 2024-05-11T23:02:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9549999999999999e-06 2024-05-11T23:02:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928073+0000", 2024-05-11T23:02:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1929999999999998e-06 2024-05-11T23:02:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928078+0000", 2024-05-11T23:02:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9289999999999997e-06 2024-05-11T23:02:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966058+0000", 2024-05-11T23:02:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037980452999999997 2024-05-11T23:02:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966123+0000", 2024-05-11T23:02:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4832000000000005e-05 2024-05-11T23:02:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966198+0000", 2024-05-11T23:02:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5341999999999998e-05 2024-05-11T23:02:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.747 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24585 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928095+0000", 2024-05-11T23:02:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.78532517, 2024-05-11T23:02:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038282688000000002, 2024-05-11T23:02:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24585 2024-05-11T23:02:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.748 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928095+0000", 2024-05-11T23:02:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928095+0000", 2024-05-11T23:02:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928095+0000", 2024-05-11T23:02:06.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1900000000000003e-07 2024-05-11T23:02:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928098+0000", 2024-05-11T23:02:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1880000000000001e-06 2024-05-11T23:02:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928105+0000", 2024-05-11T23:02:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3690000000000001e-06 2024-05-11T23:02:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928120+0000", 2024-05-11T23:02:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4762e-05 2024-05-11T23:02:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966233+0000", 2024-05-11T23:02:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038113526000000002 2024-05-11T23:02:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966300+0000", 2024-05-11T23:02:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6637999999999995e-05 2024-05-11T23:02:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966378+0000", 2024-05-11T23:02:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7686000000000001e-05 2024-05-11T23:02:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24592 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 315392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928128+0000", 2024-05-11T23:02:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.785291865, 2024-05-11T23:02:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038417382, 2024-05-11T23:02:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:06.752 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24592 2024-05-11T23:02:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928128+0000", 2024-05-11T23:02:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928128+0000", 2024-05-11T23:02:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928129+0000", 2024-05-11T23:02:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.44e-07 2024-05-11T23:02:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928131+0000", 2024-05-11T23:02:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2910000000000002e-06 2024-05-11T23:02:06.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928138+0000", 2024-05-11T23:02:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9430000000000003e-06 2024-05-11T23:02:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928143+0000", 2024-05-11T23:02:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4739999999999996e-06 2024-05-11T23:02:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966400+0000", 2024-05-11T23:02:06.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038257001999999998 2024-05-11T23:02:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966465+0000", 2024-05-11T23:02:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4468999999999995e-05 2024-05-11T23:02:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966546+0000", 2024-05-11T23:02:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0659000000000001e-05 2024-05-11T23:02:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24598 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 331776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928150+0000", 2024-05-11T23:02:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.785269555999999, 2024-05-11T23:02:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038544469999999997, 2024-05-11T23:02:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24598 2024-05-11T23:02:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.757 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928150+0000", 2024-05-11T23:02:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928150+0000", 2024-05-11T23:02:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928151+0000", 2024-05-11T23:02:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999999e-07 2024-05-11T23:02:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928153+0000", 2024-05-11T23:02:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.21e-06 2024-05-11T23:02:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928160+0000", 2024-05-11T23:02:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5200000000000003e-06 2024-05-11T23:02:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928165+0000", 2024-05-11T23:02:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9269999999999996e-06 2024-05-11T23:02:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966569+0000", 2024-05-11T23:02:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038404595999999999 2024-05-11T23:02:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966634+0000", 2024-05-11T23:02:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5221000000000005e-05 2024-05-11T23:02:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966695+0000", 2024-05-11T23:02:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0547999999999997e-05 2024-05-11T23:02:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24604 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928171+0000", 2024-05-11T23:02:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.785249194, 2024-05-11T23:02:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038676227000000001, 2024-05-11T23:02:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:06.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24604 2024-05-11T23:02:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928171+0000", 2024-05-11T23:02:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928171+0000", 2024-05-11T23:02:06.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928171+0000", 2024-05-11T23:02:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-05-11T23:02:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928173+0000", 2024-05-11T23:02:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0619999999999999e-06 2024-05-11T23:02:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928180+0000", 2024-05-11T23:02:06.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-05-11T23:02:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928185+0000", 2024-05-11T23:02:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.04e-06 2024-05-11T23:02:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966718+0000", 2024-05-11T23:02:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038532759 2024-05-11T23:02:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966784+0000", 2024-05-11T23:02:06.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6865000000000004e-05 2024-05-11T23:02:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966847+0000", 2024-05-11T23:02:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2694000000000007e-05 2024-05-11T23:02:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24609 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 364544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928191+0000", 2024-05-11T23:02:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.785229126000001, 2024-05-11T23:02:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038806908000000001, 2024-05-11T23:02:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:06.765 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24609 2024-05-11T23:02:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928191+0000", 2024-05-11T23:02:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928191+0000", 2024-05-11T23:02:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928191+0000", 2024-05-11T23:02:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-05-11T23:02:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928198+0000", 2024-05-11T23:02:06.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2180000000000002e-06 2024-05-11T23:02:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928205+0000", 2024-05-11T23:02:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5899999999999996e-06 2024-05-11T23:02:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928210+0000", 2024-05-11T23:02:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.118e-06 2024-05-11T23:02:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966869+0000", 2024-05-11T23:02:06.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038659106999999998 2024-05-11T23:02:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966933+0000", 2024-05-11T23:02:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4158999999999993e-05 2024-05-11T23:02:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966998+0000", 2024-05-11T23:02:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4352999999999999e-05 2024-05-11T23:02:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24616 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 380928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928217+0000", 2024-05-11T23:02:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.785202983, 2024-05-11T23:02:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038928536999999999, 2024-05-11T23:02:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24616 2024-05-11T23:02:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:06.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928217+0000", 2024-05-11T23:02:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928217+0000", 2024-05-11T23:02:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928217+0000", 2024-05-11T23:02:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-05-11T23:02:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928219+0000", 2024-05-11T23:02:06.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0609999999999998e-06 2024-05-11T23:02:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928233+0000", 2024-05-11T23:02:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3715e-05 2024-05-11T23:02:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928238+0000", 2024-05-11T23:02:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1889999999999998e-06 2024-05-11T23:02:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967021+0000", 2024-05-11T23:02:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038782284 2024-05-11T23:02:06.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967085+0000", 2024-05-11T23:02:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4319000000000005e-05 2024-05-11T23:02:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967146+0000", 2024-05-11T23:02:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0579999999999999e-05 2024-05-11T23:02:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24622 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 397312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928262+0000", 2024-05-11T23:02:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.785157678000001, 2024-05-11T23:02:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039038392999999998, 2024-05-11T23:02:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24622 2024-05-11T23:02:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.774 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928262+0000", 2024-05-11T23:02:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928262+0000", 2024-05-11T23:02:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928263+0000", 2024-05-11T23:02:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0299999999999999e-07 2024-05-11T23:02:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928266+0000", 2024-05-11T23:02:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6690000000000002e-06 2024-05-11T23:02:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928275+0000", 2024-05-11T23:02:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1630000000000002e-06 2024-05-11T23:02:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928280+0000", 2024-05-11T23:02:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6579999999999999e-06 2024-05-11T23:02:06.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967169+0000", 2024-05-11T23:02:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038888569999999997 2024-05-11T23:02:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967240+0000", 2024-05-11T23:02:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0947999999999999e-05 2024-05-11T23:02:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967301+0000", 2024-05-11T23:02:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0782000000000002e-05 2024-05-11T23:02:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24628 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 413696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928288+0000", 2024-05-11T23:02:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.78513163, 2024-05-11T23:02:06.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039162792000000002, 2024-05-11T23:02:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24628 2024-05-11T23:02:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928288+0000", 2024-05-11T23:02:06.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928288+0000", 2024-05-11T23:02:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928289+0000", 2024-05-11T23:02:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7399999999999998e-07 2024-05-11T23:02:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928291+0000", 2024-05-11T23:02:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.221e-06 2024-05-11T23:02:06.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928298+0000", 2024-05-11T23:02:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0520000000000004e-06 2024-05-11T23:02:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928304+0000", 2024-05-11T23:02:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4149999999999998e-06 2024-05-11T23:02:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967324+0000", 2024-05-11T23:02:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039020436999999998 2024-05-11T23:02:06.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967390+0000", 2024-05-11T23:02:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5821000000000006e-05 2024-05-11T23:02:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967451+0000", 2024-05-11T23:02:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1371999999999994e-05 2024-05-11T23:02:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24636 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 430080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928310+0000", 2024-05-11T23:02:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.785109765, 2024-05-11T23:02:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039286303000000002, 2024-05-11T23:02:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24636 2024-05-11T23:02:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928310+0000", 2024-05-11T23:02:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928310+0000", 2024-05-11T23:02:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928311+0000", 2024-05-11T23:02:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:02:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928313+0000", 2024-05-11T23:02:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0499999999999999e-06 2024-05-11T23:02:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928319+0000", 2024-05-11T23:02:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4049999999999997e-06 2024-05-11T23:02:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928324+0000", 2024-05-11T23:02:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:02:06.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967473+0000", 2024-05-11T23:02:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039149138999999999 2024-05-11T23:02:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967537+0000", 2024-05-11T23:02:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4109000000000006e-05 2024-05-11T23:02:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967597+0000", 2024-05-11T23:02:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9407000000000002e-05 2024-05-11T23:02:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25144 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 28672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936035+0000", 2024-05-11T23:02:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.77738486, 2024-05-11T23:02:06.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036140510000000001, 2024-05-11T23:02:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25144 2024-05-11T23:02:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936035+0000", 2024-05-11T23:02:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936035+0000", 2024-05-11T23:02:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936036+0000", 2024-05-11T23:02:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-05-11T23:02:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936038+0000", 2024-05-11T23:02:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6910000000000002e-06 2024-05-11T23:02:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936045+0000", 2024-05-11T23:02:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1670000000000004e-06 2024-05-11T23:02:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936049+0000", 2024-05-11T23:02:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9609999999999996e-06 2024-05-11T23:02:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972024+0000", 2024-05-11T23:02:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.035974221000000001 2024-05-11T23:02:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972076+0000", 2024-05-11T23:02:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1918e-05 2024-05-11T23:02:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972176+0000", 2024-05-11T23:02:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010006200000000001 2024-05-11T23:02:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.791 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25150 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 45056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936063+0000", 2024-05-11T23:02:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.777357327000001, 2024-05-11T23:02:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036298174000000002, 2024-05-11T23:02:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25150 2024-05-11T23:02:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.792 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936063+0000", 2024-05-11T23:02:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936063+0000", 2024-05-11T23:02:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936063+0000", 2024-05-11T23:02:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.39e-07 2024-05-11T23:02:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936069+0000", 2024-05-11T23:02:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2770000000000002e-06 2024-05-11T23:02:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936088+0000", 2024-05-11T23:02:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8841000000000001e-05 2024-05-11T23:02:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936093+0000", 2024-05-11T23:02:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2190000000000003e-06 2024-05-11T23:02:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972213+0000", 2024-05-11T23:02:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036119039999999998 2024-05-11T23:02:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972293+0000", 2024-05-11T23:02:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0437000000000003e-05 2024-05-11T23:02:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972361+0000", 2024-05-11T23:02:06.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7921000000000002e-05 2024-05-11T23:02:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25156 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936107+0000", 2024-05-11T23:02:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.777312803000001, 2024-05-11T23:02:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036383785000000002, 2024-05-11T23:02:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:06.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25156 2024-05-11T23:02:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936107+0000", 2024-05-11T23:02:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936107+0000", 2024-05-11T23:02:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:06.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936108+0000", 2024-05-11T23:02:06.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3900000000000005e-07 2024-05-11T23:02:06.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:06.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936110+0000", 2024-05-11T23:02:06.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.435e-06 2024-05-11T23:02:06.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936118+0000", 2024-05-11T23:02:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2889999999999998e-06 2024-05-11T23:02:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936124+0000", 2024-05-11T23:02:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2850000000000002e-06 2024-05-11T23:02:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972378+0000", 2024-05-11T23:02:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036254020999999997 2024-05-11T23:02:06.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:06.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972420+0000", 2024-05-11T23:02:06.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2577999999999998e-05 2024-05-11T23:02:06.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:06.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972491+0000", 2024-05-11T23:02:06.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0637999999999997e-05 2024-05-11T23:02:06.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.800 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.800 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25164 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.801 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936132+0000", 2024-05-11T23:02:06.801 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.77728756, 2024-05-11T23:02:06.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036532028000000001, 2024-05-11T23:02:06.801 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:06.801 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:06.801 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:06.801 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:06.801 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:06.801 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.801 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25164 2024-05-11T23:02:06.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.801 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:06.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:06.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936132+0000", 2024-05-11T23:02:06.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:06.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936132+0000", 2024-05-11T23:02:06.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:06.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936133+0000", 2024-05-11T23:02:06.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-05-11T23:02:06.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:06.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936135+0000", 2024-05-11T23:02:06.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0660000000000002e-06 2024-05-11T23:02:06.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:06.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936146+0000", 2024-05-11T23:02:06.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.098e-05 2024-05-11T23:02:06.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936153+0000", 2024-05-11T23:02:06.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8619999999999997e-06 2024-05-11T23:02:06.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:06.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972532+0000", 2024-05-11T23:02:06.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036379702999999999 2024-05-11T23:02:06.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:06.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972577+0000", 2024-05-11T23:02:06.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4697999999999998e-05 2024-05-11T23:02:06.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:06.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972664+0000", 2024-05-11T23:02:06.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7338999999999998e-05 2024-05-11T23:02:06.804 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.804 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.804 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.805 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25168 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 94208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:06.805 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936160+0000", 2024-05-11T23:02:06.805 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.777260282, 2024-05-11T23:02:06.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036655488999999999, 2024-05-11T23:02:06.805 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:06.805 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:06.805 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:06.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:06.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:06.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:06.805 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25168 2024-05-11T23:02:06.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.805 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:06.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:06.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936160+0000", 2024-05-11T23:02:06.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:06.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936160+0000", 2024-05-11T23:02:06.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:06.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:06.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936160+0000", 2024-05-11T23:02:06.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3000000000000001e-07 2024-05-11T23:02:06.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:06.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936162+0000", 2024-05-11T23:02:06.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.108e-06 2024-05-11T23:02:06.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:06.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.937848+0000", 2024-05-11T23:02:06.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001685284 2024-05-11T23:02:06.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:06.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.937858+0000", 2024-05-11T23:02:06.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0222e-05 2024-05-11T23:02:06.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:06.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972704+0000", 2024-05-11T23:02:06.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034845736000000002 2024-05-11T23:02:06.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:06.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972746+0000", 2024-05-11T23:02:06.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2077999999999999e-05 2024-05-11T23:02:06.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:06.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:06.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:06.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972815+0000", 2024-05-11T23:02:06.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9630999999999998e-05 2024-05-11T23:02:06.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.809 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.809 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:06.809 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:06.809 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:06.811 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:02:06.812 INFO:teuthology.orchestra.run.smithi005.stdout:2893 read 0x200fd thru 0x25404 (0x5308 bytes) 2024-05-11T23:02:06.812 INFO:teuthology.orchestra.run.smithi005.stdout:2894 punch from 0x25ad7f to 0x26179d, (0x6a1e bytes) 2024-05-11T23:02:06.812 INFO:teuthology.orchestra.run.smithi005.stdout:2896 read 0x141092 thru 0x147968 (0x68d7 bytes) 2024-05-11T23:02:06.812 INFO:teuthology.orchestra.run.smithi005.stdout:2897 write 0x5fef401 thru 0x5ff975f (0xa35f bytes) 2024-05-11T23:02:06.813 INFO:teuthology.orchestra.run.smithi005.stdout:2899 punch from 0x4a1e262 to 0x4a2de7d, (0xfc1b bytes) 2024-05-11T23:02:06.813 INFO:teuthology.orchestra.run.smithi005.stdout:2901 writesame 0x5ae796 thru 0x5bc580 data_size 0x1d(0xddeb bytes) 2024-05-11T23:02:06.813 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:06.813 INFO:teuthology.orchestra.run.smithi005.stdout:2904 punch from 0x3ae31d3 to 0x3ae9511, (0x633e bytes) 2024-05-11T23:02:06.813 INFO:teuthology.orchestra.run.smithi005.stdout:2905 read 0x2df05f2 thru 0x2df9318 (0x8d27 bytes) 2024-05-11T23:02:06.813 INFO:teuthology.orchestra.run.smithi005.stdout:2906 write 0x8022f1b thru 0x8025cf9 (0x2ddf bytes) 2024-05-11T23:02:06.813 INFO:teuthology.orchestra.run.smithi005.stdout:2907 write 0xc1deb5 thru 0xc2c25e (0xe3aa bytes) 2024-05-11T23:02:06.813 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:06.813 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:06.813 INFO:teuthology.orchestra.run.smithi005.stdout:2911 read 0x323a06e thru 0x3249051 (0xefe4 bytes) 2024-05-11T23:02:06.813 INFO:teuthology.orchestra.run.smithi005.stdout:2912 trunc from 0x8025cfa to 0xe023785 2024-05-11T23:02:06.813 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:06.813 INFO:teuthology.orchestra.run.smithi005.stdout:2914 trunc from 0xe023785 to 0x5c8b726 2024-05-11T23:02:06.814 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:06.810+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:06.814 INFO:teuthology.orchestra.run.smithi005.stdout:2915 writesame 0xd27458c thru 0xd284458 data_size 0x11(0xfecd bytes) 2024-05-11T23:02:06.814 INFO:teuthology.orchestra.run.smithi005.stdout:2916 punch from 0x96fefb5 to 0x9703431, (0x447c bytes) 2024-05-11T23:02:06.814 INFO:teuthology.orchestra.run.smithi005.stdout:2917 read 0x27d058 thru 0x28ae48 (0xddf1 bytes) 2024-05-11T23:02:06.817 INFO:teuthology.orchestra.run.smithi005.stdout:2918 read 0x4c964c3 thru 0x4ca426f (0xddad bytes) 2024-05-11T23:02:06.817 INFO:teuthology.orchestra.run.smithi005.stdout:2921 read 0x9ab05e7 thru 0x9abf54d (0xef67 bytes) 2024-05-11T23:02:06.818 INFO:teuthology.orchestra.run.smithi005.stdout:2922 read 0x9ee6c66 thru 0x9eea288 (0x3623 bytes) 2024-05-11T23:02:06.818 INFO:teuthology.orchestra.run.smithi005.stdout:2923 punch from 0xbcac0c6 to 0xbcb68b4, (0xa7ee bytes) 2024-05-11T23:02:06.823 INFO:teuthology.orchestra.run.smithi005.stdout:2924 trunc from 0xd284459 to 0x8189d06 2024-05-11T23:02:06.829 INFO:teuthology.orchestra.run.smithi005.stdout:2925 punch from 0x80d667 to 0x817b34, (0xa4cd bytes) 2024-05-11T23:02:06.829 INFO:teuthology.orchestra.run.smithi005.stdout:2927 punch from 0x7a2760a to 0x7a276e8, (0xde bytes) 2024-05-11T23:02:06.829 INFO:teuthology.orchestra.run.smithi005.stdout:2928 read 0x3e0093 thru 0x3e0b34 (0xaa2 bytes) 2024-05-11T23:02:06.829 INFO:teuthology.orchestra.run.smithi005.stdout:2929 read 0x27db415 thru 0x27e9dbf (0xe9ab bytes) 2024-05-11T23:02:06.843 INFO:teuthology.orchestra.run.smithi005.stdout:2930 write 0xeb48a5f thru 0xeb4d0dc (0x467e bytes) 2024-05-11T23:02:06.850 INFO:teuthology.orchestra.run.smithi005.stdout:2933 write 0x2c233bc thru 0x2c2384f (0x494 bytes) 2024-05-11T23:02:06.853 INFO:teuthology.orchestra.run.smithi005.stdout:2934 write 0x829e1ca thru 0x82aa377 (0xc1ae bytes) 2024-05-11T23:02:06.860 INFO:teuthology.orchestra.run.smithi005.stdout:2935 read 0x5eaf2bb thru 0x5eb7b1e (0x8864 bytes) 2024-05-11T23:02:06.860 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:06.872 INFO:teuthology.orchestra.run.smithi005.stdout:2938 trunc from 0xeb4d0dd to 0x3d5fd92 2024-05-11T23:02:06.892 INFO:teuthology.orchestra.run.smithi005.stdout:2941 write 0x95e1fc thru 0x9614dd (0x32e2 bytes) 2024-05-11T23:02:06.911 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:06.911+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:06.929 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:06.929 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:06.929 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:06.929 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:06.938 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:02:07.012 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:07.012+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:07.044 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:07.044 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:07.044 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:07.044 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:07.044 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:07.053 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:02:07.113 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:07.114+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:07.159 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:07.159 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:07.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:07.159 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:07.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.160 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67817 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:07.160 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:07.160 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.53864915199999996, 2024-05-11T23:02:07.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044148845999999999, 2024-05-11T23:02:07.160 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.160 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.160 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.160 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.161 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.161 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.161 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67817 2024-05-11T23:02:07.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.161 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:07.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:07.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620227+0000", 2024-05-11T23:02:07.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00046792099999999999 2024-05-11T23:02:07.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620232+0000", 2024-05-11T23:02:07.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3050000000000003e-06 2024-05-11T23:02:07.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620243+0000", 2024-05-11T23:02:07.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1289e-05 2024-05-11T23:02:07.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620250+0000", 2024-05-11T23:02:07.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2130000000000002e-06 2024-05-11T23:02:07.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663745+0000", 2024-05-11T23:02:07.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043494865000000001 2024-05-11T23:02:07.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663815+0000", 2024-05-11T23:02:07.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0284999999999995e-05 2024-05-11T23:02:07.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663908+0000", 2024-05-11T23:02:07.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2967999999999995e-05 2024-05-11T23:02:07.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.165 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.166 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67820 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4902912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:07.166 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:07.166 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.53809964499999996, 2024-05-11T23:02:07.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044096781000000002, 2024-05-11T23:02:07.166 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.166 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.166 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.166 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67820 2024-05-11T23:02:07.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.167 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:07.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:07.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620310+0000", 2024-05-11T23:02:07.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9800000000000004e-07 2024-05-11T23:02:07.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620312+0000", 2024-05-11T23:02:07.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.272e-06 2024-05-11T23:02:07.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620319+0000", 2024-05-11T23:02:07.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9310000000000002e-06 2024-05-11T23:02:07.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620323+0000", 2024-05-11T23:02:07.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:02:07.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664284+0000", 2024-05-11T23:02:07.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043960908999999999 2024-05-11T23:02:07.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664327+0000", 2024-05-11T23:02:07.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2811000000000001e-05 2024-05-11T23:02:07.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664406+0000", 2024-05-11T23:02:07.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8588e-05 2024-05-11T23:02:07.169 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.169 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.170 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.170 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67821 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4927488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:07.170 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:07.170 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.53807923499999999, 2024-05-11T23:02:07.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044203340000000001, 2024-05-11T23:02:07.170 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.170 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.170 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.170 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.170 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.170 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.170 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67821 2024-05-11T23:02:07.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.171 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:07.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:07.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620330+0000", 2024-05-11T23:02:07.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:02:07.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620332+0000", 2024-05-11T23:02:07.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:02:07.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620338+0000", 2024-05-11T23:02:07.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6250000000000004e-06 2024-05-11T23:02:07.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620342+0000", 2024-05-11T23:02:07.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7609999999999998e-06 2024-05-11T23:02:07.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664433+0000", 2024-05-11T23:02:07.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044090455000000001 2024-05-11T23:02:07.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664476+0000", 2024-05-11T23:02:07.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2861999999999997e-05 2024-05-11T23:02:07.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664533+0000", 2024-05-11T23:02:07.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7156e-05 2024-05-11T23:02:07.174 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.174 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.174 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.174 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67822 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4976640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:07.174 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:07.174 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.53806019699999996, 2024-05-11T23:02:07.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044409416, 2024-05-11T23:02:07.174 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.174 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.174 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.174 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67822 2024-05-11T23:02:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620349+0000", 2024-05-11T23:02:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.51e-07 2024-05-11T23:02:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620351+0000", 2024-05-11T23:02:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.181e-06 2024-05-11T23:02:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620358+0000", 2024-05-11T23:02:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3060000000000003e-06 2024-05-11T23:02:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620362+0000", 2024-05-11T23:02:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7949999999999998e-06 2024-05-11T23:02:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664572+0000", 2024-05-11T23:02:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044209318999999997 2024-05-11T23:02:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664654+0000", 2024-05-11T23:02:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2367999999999995e-05 2024-05-11T23:02:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664758+0000", 2024-05-11T23:02:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103896 2024-05-11T23:02:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67823 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4952064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:07.178 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.53804014700000002, 2024-05-11T23:02:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044623293000000001, 2024-05-11T23:02:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67823 2024-05-11T23:02:07.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622228+0000", 2024-05-11T23:02:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018592280000000001 2024-05-11T23:02:07.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622232+0000", 2024-05-11T23:02:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.403e-06 2024-05-11T23:02:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622248+0000", 2024-05-11T23:02:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5449000000000001e-05 2024-05-11T23:02:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622254+0000", 2024-05-11T23:02:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5819999999999998e-06 2024-05-11T23:02:07.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664800+0000", 2024-05-11T23:02:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042546048000000003 2024-05-11T23:02:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664886+0000", 2024-05-11T23:02:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5525000000000005e-05 2024-05-11T23:02:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664992+0000", 2024-05-11T23:02:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010605800000000001 2024-05-11T23:02:07.182 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67833 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5099520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.53593367400000003, 2024-05-11T23:02:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044076606999999997, 2024-05-11T23:02:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67833 2024-05-11T23:02:07.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8099999999999998e-07 2024-05-11T23:02:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622477+0000", 2024-05-11T23:02:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0700000000000001e-06 2024-05-11T23:02:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622483+0000", 2024-05-11T23:02:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0780000000000003e-06 2024-05-11T23:02:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622487+0000", 2024-05-11T23:02:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.352e-06 2024-05-11T23:02:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666460+0000", 2024-05-11T23:02:07.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043973085000000002 2024-05-11T23:02:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666498+0000", 2024-05-11T23:02:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8441e-05 2024-05-11T23:02:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666552+0000", 2024-05-11T23:02:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3199999999999999e-05 2024-05-11T23:02:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67834 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3944448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.53589539600000002, 2024-05-11T23:02:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044166551999999998, 2024-05-11T23:02:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67834 2024-05-11T23:02:07.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622514+0000", 2024-05-11T23:02:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1100000000000002e-07 2024-05-11T23:02:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622515+0000", 2024-05-11T23:02:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:02:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622519+0000", 2024-05-11T23:02:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8260000000000003e-06 2024-05-11T23:02:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622522+0000", 2024-05-11T23:02:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8899999999999999e-06 2024-05-11T23:02:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666580+0000", 2024-05-11T23:02:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044058235000000001 2024-05-11T23:02:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666624+0000", 2024-05-11T23:02:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3501999999999998e-05 2024-05-11T23:02:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666680+0000", 2024-05-11T23:02:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6042999999999999e-05 2024-05-11T23:02:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67835 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.53588285999999996, 2024-05-11T23:02:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044272019000000003, 2024-05-11T23:02:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.192 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67835 2024-05-11T23:02:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1699999999999999e-07 2024-05-11T23:02:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622527+0000", 2024-05-11T23:02:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2109999999999999e-06 2024-05-11T23:02:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622531+0000", 2024-05-11T23:02:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4910000000000001e-06 2024-05-11T23:02:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622534+0000", 2024-05-11T23:02:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7080000000000002e-06 2024-05-11T23:02:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666703+0000", 2024-05-11T23:02:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044169377000000003 2024-05-11T23:02:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666745+0000", 2024-05-11T23:02:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1616000000000003e-05 2024-05-11T23:02:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666798+0000", 2024-05-11T23:02:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3298999999999999e-05 2024-05-11T23:02:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67836 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.53587220000000002, 2024-05-11T23:02:07.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044373692999999999, 2024-05-11T23:02:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67836 2024-05-11T23:02:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622537+0000", 2024-05-11T23:02:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622538+0000", 2024-05-11T23:02:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1510000000000001e-06 2024-05-11T23:02:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622541+0000", 2024-05-11T23:02:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:02:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622543+0000", 2024-05-11T23:02:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4870000000000001e-06 2024-05-11T23:02:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666821+0000", 2024-05-11T23:02:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044278038999999998 2024-05-11T23:02:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666859+0000", 2024-05-11T23:02:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7224000000000001e-05 2024-05-11T23:02:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666910+0000", 2024-05-11T23:02:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1629999999999999e-05 2024-05-11T23:02:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67837 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.53586247300000001, 2024-05-11T23:02:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044477883000000003, 2024-05-11T23:02:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67837 2024-05-11T23:02:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622548+0000", 2024-05-11T23:02:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:02:07.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622550+0000", 2024-05-11T23:02:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-05-11T23:02:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622553+0000", 2024-05-11T23:02:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.43e-06 2024-05-11T23:02:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666932+0000", 2024-05-11T23:02:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044379584 2024-05-11T23:02:07.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666971+0000", 2024-05-11T23:02:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8427000000000003e-05 2024-05-11T23:02:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667024+0000", 2024-05-11T23:02:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3226000000000002e-05 2024-05-11T23:02:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67838 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:07.204 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.53585295600000005, 2024-05-11T23:02:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044580374999999998, 2024-05-11T23:02:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67838 2024-05-11T23:02:07.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622557+0000", 2024-05-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.102e-06 2024-05-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622560+0000", 2024-05-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622562+0000", 2024-05-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667045+0000", 2024-05-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044482936000000001 2024-05-11T23:02:07.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667083+0000", 2024-05-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7443e-05 2024-05-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667136+0000", 2024-05-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3257000000000002e-05 2024-05-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.209 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67839 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5394432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.53584321400000001, 2024-05-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04470263, 2024-05-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67839 2024-05-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:07.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622566+0000", 2024-05-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-05-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622572+0000", 2024-05-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5610000000000004e-06 2024-05-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622576+0000", 2024-05-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3210000000000001e-06 2024-05-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622578+0000", 2024-05-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4710000000000001e-06 2024-05-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667160+0000", 2024-05-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044582220999999998 2024-05-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667205+0000", 2024-05-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5049000000000002e-05 2024-05-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667268+0000", 2024-05-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2788999999999998e-05 2024-05-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67840 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5419008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.53582758399999997, 2024-05-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044796006999999999, 2024-05-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67840 2024-05-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622583+0000", 2024-05-11T23:02:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4389999999999999e-06 2024-05-11T23:02:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622586+0000", 2024-05-11T23:02:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:02:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622588+0000", 2024-05-11T23:02:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4540000000000001e-06 2024-05-11T23:02:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667287+0000", 2024-05-11T23:02:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044698882000000002 2024-05-11T23:02:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667324+0000", 2024-05-11T23:02:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7196000000000001e-05 2024-05-11T23:02:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667377+0000", 2024-05-11T23:02:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2927999999999999e-05 2024-05-11T23:02:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.217 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67841 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5443584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.53581781100000003, 2024-05-11T23:02:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044907337999999998, 2024-05-11T23:02:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67841 2024-05-11T23:02:07.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:07.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622592+0000", 2024-05-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-05-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622595+0000", 2024-05-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622598+0000", 2024-05-11T23:02:07.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5909999999999998e-06 2024-05-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667401+0000", 2024-05-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044802980999999999 2024-05-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667439+0000", 2024-05-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8840000000000001e-05 2024-05-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667498+0000", 2024-05-11T23:02:07.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8709000000000003e-05 2024-05-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67842 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5468160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.53580820399999995, 2024-05-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045005876, 2024-05-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.222 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67842 2024-05-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622601+0000", 2024-05-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622602+0000", 2024-05-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0780000000000001e-06 2024-05-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622605+0000", 2024-05-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.914e-06 2024-05-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622607+0000", 2024-05-11T23:02:07.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667516+0000", 2024-05-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044909156999999998 2024-05-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667554+0000", 2024-05-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7920000000000003e-05 2024-05-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667606+0000", 2024-05-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2111000000000003e-05 2024-05-11T23:02:07.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67843 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4067328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.53579867800000003, 2024-05-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045119014999999998, 2024-05-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67843 2024-05-11T23:02:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624228+0000", 2024-05-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001617867 2024-05-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624232+0000", 2024-05-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-05-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624240+0000", 2024-05-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4589999999999998e-06 2024-05-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624247+0000", 2024-05-11T23:02:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-05-11T23:02:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667629+0000", 2024-05-11T23:02:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043382012999999997 2024-05-11T23:02:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667674+0000", 2024-05-11T23:02:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5172000000000001e-05 2024-05-11T23:02:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667729+0000", 2024-05-11T23:02:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5086000000000001e-05 2024-05-11T23:02:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67849 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4190208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.53404055699999997, 2024-05-11T23:02:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044061433999999997, 2024-05-11T23:02:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.230 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67849 2024-05-11T23:02:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624369+0000", 2024-05-11T23:02:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5900000000000002e-07 2024-05-11T23:02:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624371+0000", 2024-05-11T23:02:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0310000000000001e-06 2024-05-11T23:02:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624376+0000", 2024-05-11T23:02:07.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4029999999999998e-06 2024-05-11T23:02:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624381+0000", 2024-05-11T23:02:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4349999999999996e-06 2024-05-11T23:02:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668331+0000", 2024-05-11T23:02:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043949191999999998 2024-05-11T23:02:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668370+0000", 2024-05-11T23:02:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9227999999999998e-05 2024-05-11T23:02:07.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668430+0000", 2024-05-11T23:02:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9685999999999997e-05 2024-05-11T23:02:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67850 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.53401932500000004, 2024-05-11T23:02:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044151953000000001, 2024-05-11T23:02:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.234 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67850 2024-05-11T23:02:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:07.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624390+0000", 2024-05-11T23:02:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:02:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624392+0000", 2024-05-11T23:02:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1880000000000001e-06 2024-05-11T23:02:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624399+0000", 2024-05-11T23:02:07.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7390000000000002e-06 2024-05-11T23:02:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624405+0000", 2024-05-11T23:02:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1059999999999996e-06 2024-05-11T23:02:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668450+0000", 2024-05-11T23:02:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044045147999999999 2024-05-11T23:02:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668487+0000", 2024-05-11T23:02:07.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7450000000000002e-05 2024-05-11T23:02:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668541+0000", 2024-05-11T23:02:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3952000000000001e-05 2024-05-11T23:02:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67851 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5541888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.53399274299999999, 2024-05-11T23:02:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044264413000000002, 2024-05-11T23:02:07.238 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67851 2024-05-11T23:02:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-05-11T23:02:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624419+0000", 2024-05-11T23:02:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.21e-06 2024-05-11T23:02:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624424+0000", 2024-05-11T23:02:07.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9050000000000002e-06 2024-05-11T23:02:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624430+0000", 2024-05-11T23:02:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-05-11T23:02:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668566+0000", 2024-05-11T23:02:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044136208000000003 2024-05-11T23:02:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668614+0000", 2024-05-11T23:02:07.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8022000000000002e-05 2024-05-11T23:02:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668680+0000", 2024-05-11T23:02:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6181999999999998e-05 2024-05-11T23:02:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67852 3.16 3.1a4a4af6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.53397227599999997, 2024-05-11T23:02:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044278287999999999, 2024-05-11T23:02:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.242 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67852 2024-05-11T23:02:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:07.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626226+0000", 2024-05-11T23:02:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017893 2024-05-11T23:02:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626229+0000", 2024-05-11T23:02:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0709999999999999e-06 2024-05-11T23:02:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626239+0000", 2024-05-11T23:02:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0507e-05 2024-05-11T23:02:07.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626244+0000", 2024-05-11T23:02:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4880000000000001e-06 2024-05-11T23:02:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668706+0000", 2024-05-11T23:02:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042462054999999999 2024-05-11T23:02:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668715+0000", 2024-05-11T23:02:07.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.867e-06 2024-05-11T23:02:07.246 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.246 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.246 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.246 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.246 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.246 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:07.247 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:02:07.248 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:07.247+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:07.348 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:07.349+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:07.392 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:07.393 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:07.393 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:07.393 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:07.401 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:02:07.449 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:07.449+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:07.500 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:02:07.507 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:07.507 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:07.507 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:07.507 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:07.507 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:07.516 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:02:07.557 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:07.557+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:07.634 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:07.634 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:07.651 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:07.651+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:07.652 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:07.652 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:07.663 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.009084886, 2024-05-11T23:02:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:02:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:02:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:02:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:02:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:02:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:02:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:02:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:02:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:02:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:02:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:02:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:02:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:02:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:02:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:02:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:07.668 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.005703697, 2024-05-11T23:02:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:02:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:02:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:07.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:02:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:02:07.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:02:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:02:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:02:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:02:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:02:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:02:07.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:02:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:02:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:02:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:02:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:02:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.005684990000001, 2024-05-11T23:02:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:02:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:02:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:02:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:02:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:02:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:02:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:02:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:02:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:02:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:02:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:02:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:02:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:02:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:02:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:02:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:02:07.677 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.005666847000001, 2024-05-11T23:02:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:02:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.678 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:02:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:02:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:02:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:02:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:02:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:02:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:02:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:02:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:02:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:02:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:02:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:02:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:02:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:02:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.00563936, 2024-05-11T23:02:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:02:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:02:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:02:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:02:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:02:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:02:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:02:07.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:02:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:02:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:02:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:02:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:02:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:02:07.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:02:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:02:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:02:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.005618832, 2024-05-11T23:02:07.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:02:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:02:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:02:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:02:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:02:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:02:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:02:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:02:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:02:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:02:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:02:07.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:02:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:02:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:02:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:02:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.690 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.005599057, 2024-05-11T23:02:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:02:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:02:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:02:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:02:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:02:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:02:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:02:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:02:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:02:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:02:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:02:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:02:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:02:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:02:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:02:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:02:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.694 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.005580251, 2024-05-11T23:02:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:02:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:02:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.695 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:02:07.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:02:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:02:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:02:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:02:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:02:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:02:07.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:02:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:02:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:02:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:02:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:02:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:02:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:02:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.005561328000001, 2024-05-11T23:02:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:02:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.699 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:02:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:02:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:02:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:02:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:02:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:02:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:02:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:02:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:02:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:02:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:02:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:02:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:02:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:02:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.005542666, 2024-05-11T23:02:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:02:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.703 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:02:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:07.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:02:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:02:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:02:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:02:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:02:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:02:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:02:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:02:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:02:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:02:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:02:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:02:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:02:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:07.707 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.005197504, 2024-05-11T23:02:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:02:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:02:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:02:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:02:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:02:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:02:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:02:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:02:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:02:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:02:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:02:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:02:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:02:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:02:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:02:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:02:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.005171954, 2024-05-11T23:02:07.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:02:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:02:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:07.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:02:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:02:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:02:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:02:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:02:07.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:02:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:02:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:02:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:02:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:02:07.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:02:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:02:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:02:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.005089696000001, 2024-05-11T23:02:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:02:07.715 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:02:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:02:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:02:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:02:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:02:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:02:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:02:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:02:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:02:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:02:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:02:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:02:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:02:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:02:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.005067249, 2024-05-11T23:02:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:02:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:02:07.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:02:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:02:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:02:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:02:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:02:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:02:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:02:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:02:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:02:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:02:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:02:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:02:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:02:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:02:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.005047791000001, 2024-05-11T23:02:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:02:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.724 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:02:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:02:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:02:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:02:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:02:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:02:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:02:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:02:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:02:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:02:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:02:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:02:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:02:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:02:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.005027731, 2024-05-11T23:02:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:02:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.728 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:02:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:02:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:02:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:02:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:02:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:02:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:02:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:02:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:02:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:02:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:02:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:02:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:02:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:02:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.005008460999999, 2024-05-11T23:02:07.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:02:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:02:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:02:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:02:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:02:07.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:02:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:02:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:02:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:02:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:02:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:02:07.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:02:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:02:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:02:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:02:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.736 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.004462394999999, 2024-05-11T23:02:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:02:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:02:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:02:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:02:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:02:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:02:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:02:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:02:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:02:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:02:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:02:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:02:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:02:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:02:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:02:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:02:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.740 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.004430611, 2024-05-11T23:02:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:02:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:02:07.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:07.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:02:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:02:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:02:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:02:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:02:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:02:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:02:07.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:02:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:02:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:02:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:02:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:02:07.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:02:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:02:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 5.567185147, 2024-05-11T23:02:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:02:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:07.745 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:02:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:02:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:02:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:02:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:02:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:02:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:02:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:02:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:02:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:02:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:02:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:02:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:02:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:02:07.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:02:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:02:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:02:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:02:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:02:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:02:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:07.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:02:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:02:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:02:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:02:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:07.750 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:07.750 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:07.751 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:07.752 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:07.752 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:07.752 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:02:07.753 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:07.753 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:07.754 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:07.754 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:07.754 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:07.754 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:07.755 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:07.755+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:07.757 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:07.758 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:07.856 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:07.857+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:07.890 INFO:teuthology.orchestra.run.smithi005.stdout:2942 punch from 0x1ef7929 to 0x1f04a13, (0xd0ea bytes) 2024-05-11T23:02:07.890 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:07.890 INFO:teuthology.orchestra.run.smithi005.stdout:2945 writesame 0x349413f thru 0x349d1e1 data_size 0x1(0x90a3 bytes) 2024-05-11T23:02:07.895 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-11T23:02:07.902 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:07.902 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:07.912 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:02:07.914 INFO:teuthology.orchestra.run.smithi005.stdout:2946 write 0x769c41f thru 0x76a1a2a (0x560c bytes) 2024-05-11T23:02:07.924 INFO:teuthology.orchestra.run.smithi005.stdout:2947 punch from 0x5172e1f to 0x5181a47, (0xec28 bytes) 2024-05-11T23:02:07.924 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:07.924 INFO:teuthology.orchestra.run.smithi005.stdout:2949 punch from 0x3296ddc to 0x3297a48, (0xc6c bytes) 2024-05-11T23:02:07.937 INFO:teuthology.orchestra.run.smithi005.stdout:2950 writesame 0xd175900 thru 0xd17d8ae data_size 0x1(0x7faf bytes) 2024-05-11T23:02:07.937 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:07.937 INFO:teuthology.orchestra.run.smithi005.stdout:2953 read 0x3846d13 thru 0x384c384 (0x5672 bytes) 2024-05-11T23:02:07.942 INFO:teuthology.orchestra.run.smithi005.stdout:2954 writesame 0xd908e8f thru 0xd9107d8 data_size 0x8a(0x794a bytes) 2024-05-11T23:02:07.953 INFO:teuthology.orchestra.run.smithi005.stdout:2956 trunc from 0xd9107d9 to 0x405330a 2024-05-11T23:02:07.957 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:07.957+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:07.967 INFO:teuthology.orchestra.run.smithi005.stdout:2957 read 0x13d0a0d thru 0x13dbfb7 (0xb5ab bytes) 2024-05-11T23:02:07.967 INFO:teuthology.orchestra.run.smithi005.stdout:2958 read 0x37e059a thru 0x37eb981 (0xb3e8 bytes) 2024-05-11T23:02:07.978 INFO:teuthology.orchestra.run.smithi005.stdout:2959 write 0xc925a78 thru 0xc92a740 (0x4cc9 bytes) 2024-05-11T23:02:07.985 INFO:teuthology.orchestra.run.smithi005.stdout:2960 write 0x5e6c3eb thru 0x5e6f721 (0x3337 bytes) 2024-05-11T23:02:07.991 INFO:teuthology.orchestra.run.smithi005.stdout:2961 read 0x3c23d55 thru 0x3c274e1 (0x378d bytes) 2024-05-11T23:02:07.993 INFO:teuthology.orchestra.run.smithi005.stdout:2962 trunc from 0xc92a741 to 0xae7036d 2024-05-11T23:02:08.003 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:08.003 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:08.003 INFO:teuthology.orchestra.run.smithi005.stdout:2965 write 0x135ee00 thru 0x1367bb2 (0x8db3 bytes) 2024-05-11T23:02:08.011 INFO:teuthology.orchestra.run.smithi005.stdout:2966 read 0x7f25ba0 thru 0x7f28785 (0x2be6 bytes) 2024-05-11T23:02:08.012 INFO:teuthology.orchestra.run.smithi005.stdout:2968 trunc from 0xae7036d to 0x9c0db14 2024-05-11T23:02:08.017 INFO:teuthology.orchestra.run.smithi005.stdout:2969 write 0x84151cd thru 0x841ce2f (0x7c63 bytes) 2024-05-11T23:02:08.028 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:08.028 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:08.037 INFO:teuthology.orchestra.run.smithi005.stdout:2970 write 0xe19b017 thru 0xe1a7f14 (0xcefe bytes) 2024-05-11T23:02:08.038 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:02:08.058 INFO:teuthology.orchestra.run.smithi005.stdout:2971 trunc from 0xe1a7f15 to 0x26187e1 2024-05-11T23:02:08.071 INFO:teuthology.orchestra.run.smithi005.stdout:2973 read 0x6abac1 thru 0x6b4189 (0x86c9 bytes) 2024-05-11T23:02:08.071 INFO:teuthology.orchestra.run.smithi005.stdout:2976 read 0x1f319e9 thru 0x1f3f5fe (0xdc16 bytes) 2024-05-11T23:02:08.079 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:08.079+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:08.081 INFO:teuthology.orchestra.run.smithi005.stdout:2977 writesame 0x693d685 thru 0x694393d data_size 0x1(0x62b9 bytes) 2024-05-11T23:02:08.082 INFO:teuthology.orchestra.run.smithi005.stdout:2978 writesame 0x5d47dff thru 0x5d568a3 data_size 0x1(0xeaa5 bytes) 2024-05-11T23:02:08.095 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:08.096 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:08.103 INFO:teuthology.orchestra.run.smithi005.stdout:2981 write 0x7c5103e thru 0x7c5d546 (0xc509 bytes) 2024-05-11T23:02:08.111 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:08.111 INFO:teuthology.orchestra.run.smithi005.stdout:2984 punch from 0x212e813 to 0x212f5ee, (0xddb bytes) 2024-05-11T23:02:08.111 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:08.111 INFO:teuthology.orchestra.run.smithi005.stdout:2986 read 0x622699c thru 0x6228ba7 (0x220c bytes) 2024-05-11T23:02:08.119 INFO:teuthology.orchestra.run.smithi005.stdout:2989 trunc from 0x7c5d547 to 0x12dfd89 2024-05-11T23:02:08.150 INFO:teuthology.orchestra.run.smithi005.stdout:2990 write 0x8ba71e9 thru 0x8ba9cf7 (0x2b0f bytes) 2024-05-11T23:02:08.155 INFO:teuthology.orchestra.run.smithi005.stdout:2993 write 0x10e8664 thru 0x10ec67a (0x4017 bytes) 2024-05-11T23:02:08.161 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:08.158+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:08.163 INFO:teuthology.orchestra.run.smithi005.stdout:2994 writesame 0x5921f38 thru 0x5927665 data_size 0x1(0x572e bytes) 2024-05-11T23:02:08.175 INFO:teuthology.orchestra.run.smithi005.stdout:2995 read 0x53ee16d thru 0x53f0cce (0x2b62 bytes) 2024-05-11T23:02:08.175 INFO:teuthology.orchestra.run.smithi005.stdout:2996 punch from 0x461d62e to 0x4629a1e, (0xc3f0 bytes) 2024-05-11T23:02:08.186 INFO:teuthology.orchestra.run.smithi005.stdout:2998 write 0xc06e12e thru 0xc074736 (0x6609 bytes) 2024-05-11T23:02:08.199 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:08.199 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:08.199 INFO:teuthology.orchestra.run.smithi005.stdout:2999 trunc from 0xc074737 to 0xcd11fd3 2024-05-11T23:02:08.206 INFO:teuthology.orchestra.run.smithi005.stdout:3001 clone 23 order 25 su 2097152 sc 14 2024-05-11T23:02:08.208 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:02:08.260 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:08.260+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:08.319 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:08.319 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:08.328 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:02:08.375 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:08.374+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:08.437 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:08.437 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:08.446 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:02:08.462 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:08.461+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:08.563 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:08.563+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:08.596 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:08.596 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:08.605 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:02:08.664 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:08.663+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:08.715 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:08.715 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:08.723 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:02:08.765 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:08.764+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:08.832 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:08.833 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:08.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:02:08.886 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:08.886+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:08.890 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7] out_osds: [3, 6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:02:08.890 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:02:08.890 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-05-11T23:02:08.890 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-11T23:02:08.890 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-11T23:02:08.891 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-05-11T23:02:08.893 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-05-11T23:02:08.961 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone22 from 0xcd11fd3 (overlap 0x45ccaa) to 0x0 2024-05-11T23:02:08.962 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:08.962 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:08.966 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:08.966+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:08.976 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:02:09.012 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:09.012 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-11T23:02:09.013 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:09.069 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:09.068+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:09.129 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-05-11T23:02:09.130 INFO:teuthology.orchestra.run.smithi005.stdout:3002 punch from 0xbcbe6cb to 0xbcc841f, (0x9d54 bytes) 2024-05-11T23:02:09.133 INFO:teuthology.orchestra.run.smithi005.stdout:3003 punch from 0x8407604 to 0x840b5a2, (0x3f9e bytes) 2024-05-11T23:02:09.136 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:09.136 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:09.138 INFO:teuthology.orchestra.run.smithi005.stdout:3004 writesame 0xce0b92f thru 0xce0f0ea data_size 0x29(0x37bc bytes) 2024-05-11T23:02:09.143 INFO:teuthology.orchestra.run.smithi005.stdout:3005 write 0x83957b6 thru 0x8397e7a (0x26c5 bytes) 2024-05-11T23:02:09.146 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:02:09.148 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.155 INFO:teuthology.orchestra.run.smithi005.stdout:3007 writesame 0xdfc6aa0 thru 0xdfd558c data_size 0x1(0xeaed bytes) 2024-05-11T23:02:09.159 INFO:teuthology.orchestra.run.smithi005.stdout:3008 read 0x52a5643 thru 0x52ae27a (0x8c38 bytes) 2024-05-11T23:02:09.159 INFO:teuthology.orchestra.run.smithi005.stdout:3010 writesame 0x4a30782 thru 0x4a30790 data_size 0x1(0xf bytes) 2024-05-11T23:02:09.164 INFO:teuthology.orchestra.run.smithi005.stdout:3011 trunc from 0xdfd558d to 0xb352ace 2024-05-11T23:02:09.170 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:09.169+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:09.196 INFO:teuthology.orchestra.run.smithi005.stdout:3013 write 0xda9b568 thru 0xdaa2bcf (0x7668 bytes) 2024-05-11T23:02:09.204 INFO:teuthology.orchestra.run.smithi005.stdout:3016 write 0xe97e5ab thru 0xe98a005 (0xba5b bytes) 2024-05-11T23:02:09.205 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.205 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.215 INFO:teuthology.orchestra.run.smithi005.stdout:3020 trunc from 0xe98a006 to 0x65c664f 2024-05-11T23:02:09.223 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.223 INFO:teuthology.orchestra.run.smithi005.stdout:3023 read 0x38fdc3c thru 0x38fe4a3 (0x868 bytes) 2024-05-11T23:02:09.235 INFO:teuthology.orchestra.run.smithi005.stdout:3024 write 0xb9a51e0 thru 0xb9a53af (0x1d0 bytes) 2024-05-11T23:02:09.236 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.244 INFO:teuthology.orchestra.run.smithi005.stdout:3027 write 0xcdcd245 thru 0xcdcfa85 (0x2841 bytes) 2024-05-11T23:02:09.252 INFO:teuthology.orchestra.run.smithi005.stdout:3028 writesame 0xe119650 thru 0xe12796f data_size 0x9e(0xe320 bytes) 2024-05-11T23:02:09.253 INFO:teuthology.orchestra.run.smithi005.stdout:3030 write 0xd58f0ce thru 0xd59cd0c (0xdc3f bytes) 2024-05-11T23:02:09.254 INFO:teuthology.orchestra.run.smithi005.stdout:3031 write 0xa2d132c thru 0xa2d7450 (0x6125 bytes) 2024-05-11T23:02:09.256 INFO:teuthology.orchestra.run.smithi005.stdout:3032 write 0xa117c4e thru 0xa125734 (0xdae7 bytes) 2024-05-11T23:02:09.258 INFO:teuthology.orchestra.run.smithi005.stdout:3035 read 0x848189f thru 0x8484084 (0x27e6 bytes) 2024-05-11T23:02:09.258 INFO:teuthology.orchestra.run.smithi005.stdout:3036 write 0x94b9269 thru 0x94c3536 (0xa2ce bytes) 2024-05-11T23:02:09.260 INFO:teuthology.orchestra.run.smithi005.stdout:3037 read 0xa6618a1 thru 0xa667d16 (0x6476 bytes) 2024-05-11T23:02:09.260 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.260 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.260 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.261 INFO:teuthology.orchestra.run.smithi005.stdout:3043 punch from 0x282e375 to 0x283dafd, (0xf788 bytes) 2024-05-11T23:02:09.261 INFO:teuthology.orchestra.run.smithi005.stdout:3044 read 0x8c79adf thru 0x8c880ff (0xe621 bytes) 2024-05-11T23:02:09.261 INFO:teuthology.orchestra.run.smithi005.stdout:3045 punch from 0xd251f45 to 0xd253836, (0x18f1 bytes) 2024-05-11T23:02:09.261 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.261 INFO:teuthology.orchestra.run.smithi005.stdout:3047 write 0x20ae516 thru 0x20b1866 (0x3351 bytes) 2024-05-11T23:02:09.262 INFO:teuthology.orchestra.run.smithi005.stdout:3048 punch from 0x19ab590 to 0x19b9b6a, (0xe5da bytes) 2024-05-11T23:02:09.262 INFO:teuthology.orchestra.run.smithi005.stdout:3049 write 0x4945a42 thru 0x495060b (0xabca bytes) 2024-05-11T23:02:09.268 INFO:teuthology.orchestra.run.smithi005.stdout:3050 trunc from 0xe127970 to 0xa2b7fb1 2024-05-11T23:02:09.271 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:09.271+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:09.272 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:09.272 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:09.275 INFO:teuthology.orchestra.run.smithi005.stdout:3051 punch from 0x42621e2 to 0x4271981, (0xf79f bytes) 2024-05-11T23:02:09.275 INFO:teuthology.orchestra.run.smithi005.stdout:3052 read 0x1d308b0 thru 0x1d3c333 (0xba84 bytes) 2024-05-11T23:02:09.284 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:02:09.285 INFO:teuthology.orchestra.run.smithi005.stdout:3053 trunc from 0xa2b7fb1 to 0x1fdd28a 2024-05-11T23:02:09.305 INFO:teuthology.orchestra.run.smithi005.stdout:3054 write 0x8f482c0 thru 0x8f554f2 (0xd233 bytes) 2024-05-11T23:02:09.307 INFO:teuthology.orchestra.run.smithi005.stdout:3055 punch from 0x932913 to 0x938494, (0x5b81 bytes) 2024-05-11T23:02:09.307 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.307 INFO:teuthology.orchestra.run.smithi005.stdout:3058 punch from 0x7a37111 to 0x7a460f5, (0xefe4 bytes) 2024-05-11T23:02:09.315 INFO:teuthology.orchestra.run.smithi005.stdout:3059 writesame 0xb8b7317 thru 0xb8b8a18 data_size 0x26(0x1702 bytes) 2024-05-11T23:02:09.317 INFO:teuthology.orchestra.run.smithi005.stdout:3060 punch from 0x65edd6a to 0x65fb645, (0xd8db bytes) 2024-05-11T23:02:09.317 INFO:teuthology.orchestra.run.smithi005.stdout:3061 write 0x5e6e13e thru 0x5e7a8e6 (0xc7a9 bytes) 2024-05-11T23:02:09.318 INFO:teuthology.orchestra.run.smithi005.stdout:3065 write 0x544c156 thru 0x545634f (0xa1fa bytes) 2024-05-11T23:02:09.320 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.320 INFO:teuthology.orchestra.run.smithi005.stdout:3067 read 0x8c008ce thru 0x8c02d1b (0x244e bytes) 2024-05-11T23:02:09.321 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.321 INFO:teuthology.orchestra.run.smithi005.stdout:3069 punch from 0x7123717 to 0x712bcee, (0x85d7 bytes) 2024-05-11T23:02:09.321 INFO:teuthology.orchestra.run.smithi005.stdout:3070 read 0xa733445 thru 0xa73feb5 (0xca71 bytes) 2024-05-11T23:02:09.321 INFO:teuthology.orchestra.run.smithi005.stdout:3071 writesame 0x7a5e0ca thru 0x7a6d87f data_size 0x27(0xf7b6 bytes) 2024-05-11T23:02:09.322 INFO:teuthology.orchestra.run.smithi005.stdout:3072 read 0x18c70b2 thru 0x18ccbc3 (0x5b12 bytes) 2024-05-11T23:02:09.323 INFO:teuthology.orchestra.run.smithi005.stdout:3073 write 0x56ddd82 thru 0x56e5fc9 (0x8248 bytes) 2024-05-11T23:02:09.332 INFO:teuthology.orchestra.run.smithi005.stdout:3074 writesame 0xe56e2c3 thru 0xe57788d data_size 0x1(0x95cb bytes) 2024-05-11T23:02:09.349 INFO:teuthology.orchestra.run.smithi005.stdout:3078 trunc from 0xe57788e to 0x89b0b1 2024-05-11T23:02:09.372 INFO:teuthology.orchestra.run.smithi005.stdout:3079 read 0x712c5d thru 0x71b0bc (0x8460 bytes) 2024-05-11T23:02:09.372 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:09.372+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:09.373 INFO:teuthology.orchestra.run.smithi005.stdout:3080 read 0x1d40d6 thru 0x1da5d0 (0x64fb bytes) 2024-05-11T23:02:09.387 INFO:teuthology.orchestra.run.smithi005.stdout:3081 writesame 0x6f1a3a9 thru 0x6f1b7c9 data_size 0x1(0x1421 bytes) 2024-05-11T23:02:09.387 INFO:teuthology.orchestra.run.smithi005.stdout:3082 read 0x3bf9164 thru 0x3c072d2 (0xe16f bytes) 2024-05-11T23:02:09.387 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.387 INFO:teuthology.orchestra.run.smithi005.stdout:3086 read 0x16c5b85 thru 0x16c81d4 (0x2650 bytes) 2024-05-11T23:02:09.387 INFO:teuthology.orchestra.run.smithi005.stdout:3087 punch from 0x53923f2 to 0x539a523, (0x8131 bytes) 2024-05-11T23:02:09.387 INFO:teuthology.orchestra.run.smithi005.stdout:3088 read 0x581ebd9 thru 0x582d73f (0xeb67 bytes) 2024-05-11T23:02:09.387 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.388 INFO:teuthology.orchestra.run.smithi005.stdout:3090 read 0x49e5491 thru 0x49ed2da (0x7e4a bytes) 2024-05-11T23:02:09.388 INFO:teuthology.orchestra.run.smithi005.stdout:3093 trunc from 0x6f1b7ca to 0x65e97e7 2024-05-11T23:02:09.393 INFO:teuthology.orchestra.run.smithi005.stdout:3096 write 0xe2caaf thru 0xe3625c (0x97ae bytes) 2024-05-11T23:02:09.396 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.397 INFO:teuthology.orchestra.run.smithi005.stdout:3101 read 0x5a67139 thru 0x5a712e5 (0xa1ad bytes) 2024-05-11T23:02:09.397 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.402 INFO:teuthology.orchestra.run.smithi005.stdout:3105 trunc from 0x65e97e7 to 0x34dacb7 2024-05-11T23:02:09.409 INFO:teuthology.orchestra.run.smithi005.stdout:3106 trunc from 0x34dacb7 to 0xedd6bb 2024-05-11T23:02:09.410 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:09.410 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:09.415 INFO:teuthology.orchestra.run.smithi005.stdout:3107 read 0x36221 thru 0x3df23 (0x7d03 bytes) 2024-05-11T23:02:09.416 INFO:teuthology.orchestra.run.smithi005.stdout:3108 read 0x1efcd6 thru 0x1f707e (0x73a9 bytes) 2024-05-11T23:02:09.417 INFO:teuthology.orchestra.run.smithi005.stdout:3109 punch from 0xac96d6 to 0xad3b0a, (0xa434 bytes) 2024-05-11T23:02:09.417 INFO:teuthology.orchestra.run.smithi005.stdout:3110 read 0x8c3bbe thru 0x8ce921 (0xad64 bytes) 2024-05-11T23:02:09.417 INFO:teuthology.orchestra.run.smithi005.stdout:3111 punch from 0x9d317a to 0x9d4dff, (0x1c85 bytes) 2024-05-11T23:02:09.417 INFO:teuthology.orchestra.run.smithi005.stdout:3112 read 0xe9a1bc thru 0xea13f0 (0x7235 bytes) 2024-05-11T23:02:09.421 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:02:09.436 INFO:teuthology.orchestra.run.smithi005.stdout:3113 writesame 0xca90a62 thru 0xca9525b data_size 0x53(0x47fa bytes) 2024-05-11T23:02:09.438 INFO:teuthology.orchestra.run.smithi005.stdout:3114 read 0x8c5c0ec thru 0x8c5c955 (0x86a bytes) 2024-05-11T23:02:09.438 INFO:teuthology.orchestra.run.smithi005.stdout:3117 punch from 0xa005c33 to 0xa00c39b, (0x6768 bytes) 2024-05-11T23:02:09.438 INFO:teuthology.orchestra.run.smithi005.stdout:3118 read 0x4ccd2aa thru 0x4cd7ceb (0xaa42 bytes) 2024-05-11T23:02:09.438 INFO:teuthology.orchestra.run.smithi005.stdout:3119 read 0x780df12 thru 0x780f5fc (0x16eb bytes) 2024-05-11T23:02:09.445 INFO:teuthology.orchestra.run.smithi005.stdout:3120 writesame 0xe19d9f6 thru 0xe1a74c2 data_size 0x1(0x9acd bytes) 2024-05-11T23:02:09.446 INFO:teuthology.orchestra.run.smithi005.stdout:3121 read 0x18eddd8 thru 0x18f1931 (0x3b5a bytes) 2024-05-11T23:02:09.446 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.447 INFO:teuthology.orchestra.run.smithi005.stdout:3123 trunc from 0xe1a74c3 to 0xd071a21 2024-05-11T23:02:09.454 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.454 INFO:teuthology.orchestra.run.smithi005.stdout:3125 read 0x13de210 thru 0x13ed6b6 (0xf4a7 bytes) 2024-05-11T23:02:09.454 INFO:teuthology.orchestra.run.smithi005.stdout:3128 read 0x9ff45a thru 0xa09735 (0xa2dc bytes) 2024-05-11T23:02:09.454 INFO:teuthology.orchestra.run.smithi005.stdout:3130 write 0x4e3b488 thru 0x4e3cc58 (0x17d1 bytes) 2024-05-11T23:02:09.457 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.457 INFO:teuthology.orchestra.run.smithi005.stdout:3132 punch from 0xa570287 to 0xa572197, (0x1f10 bytes) 2024-05-11T23:02:09.471 INFO:teuthology.orchestra.run.smithi005.stdout:3133 trunc from 0xd071a21 to 0xdf45b0 2024-05-11T23:02:09.473 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:09.473+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:09.480 INFO:teuthology.orchestra.run.smithi005.stdout:3134 punch from 0xa9715a to 0xa9ebc6, (0x7a6c bytes) 2024-05-11T23:02:09.498 INFO:teuthology.orchestra.run.smithi005.stdout:3136 write 0xba491d3 thru 0xba4ad9b (0x1bc9 bytes) 2024-05-11T23:02:09.501 INFO:teuthology.orchestra.run.smithi005.stdout:3137 write 0x57f739 thru 0x581655 (0x1f1d bytes) 2024-05-11T23:02:09.502 INFO:teuthology.orchestra.run.smithi005.stdout:3138 writesame 0x2431669 thru 0x2436353 data_size 0x7(0x4ceb bytes) 2024-05-11T23:02:09.511 INFO:teuthology.orchestra.run.smithi005.stdout:3139 write 0xda782fa thru 0xda7d05c (0x4d63 bytes) 2024-05-11T23:02:09.520 INFO:teuthology.orchestra.run.smithi005.stdout:3140 writesame 0xe1b3a45 thru 0xe1b46ca data_size 0x7(0xc86 bytes) 2024-05-11T23:02:09.521 INFO:teuthology.orchestra.run.smithi005.stdout:3141 read 0xd480a00 thru 0xd4845d3 (0x3bd4 bytes) 2024-05-11T23:02:09.521 INFO:teuthology.orchestra.run.smithi005.stdout:3142 read 0x38c2428 thru 0x38ca1f8 (0x7dd1 bytes) 2024-05-11T23:02:09.521 INFO:teuthology.orchestra.run.smithi005.stdout:3143 writesame 0x43d112 thru 0x43ff44 data_size 0x1(0x2e33 bytes) 2024-05-11T23:02:09.527 INFO:teuthology.orchestra.run.smithi005.stdout:3144 write 0x4dc41b3 thru 0x4dca944 (0x6792 bytes) 2024-05-11T23:02:09.531 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.531 INFO:teuthology.orchestra.run.smithi005.stdout:3146 read 0xd3acafd thru 0xd3afacb (0x2fcf bytes) 2024-05-11T23:02:09.531 INFO:teuthology.orchestra.run.smithi005.stdout:3148 writesame 0xc610c8 thru 0xc68e1c data_size 0xf(0x7d55 bytes) 2024-05-11T23:02:09.533 INFO:teuthology.orchestra.run.smithi005.stdout:3149 write 0x9a4e424 thru 0x9a5395d (0x553a bytes) 2024-05-11T23:02:09.536 INFO:teuthology.orchestra.run.smithi005.stdout:3151 write 0x8390cca thru 0x839f4f4 (0xe82b bytes) 2024-05-11T23:02:09.538 INFO:teuthology.orchestra.run.smithi005.stdout:3152 punch from 0xa94c5f9 to 0xa95913e, (0xcb45 bytes) 2024-05-11T23:02:09.538 INFO:teuthology.orchestra.run.smithi005.stdout:3153 read 0xc2401ec thru 0xc246140 (0x5f55 bytes) 2024-05-11T23:02:09.538 INFO:teuthology.orchestra.run.smithi005.stdout:3156 punch from 0x5ddb052 to 0x5de9fdc, (0xef8a bytes) 2024-05-11T23:02:09.538 INFO:teuthology.orchestra.run.smithi005.stdout:3157 writesame 0x83afdb2 thru 0x83be1c8 data_size 0x1(0xe417 bytes) 2024-05-11T23:02:09.558 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:09.558 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:09.565 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:09.565 INFO:teuthology.orchestra.run.smithi005.stdout:3159 writesame 0x4675eaf thru 0x4681c8e data_size 0x3e(0xbde0 bytes) 2024-05-11T23:02:09.567 INFO:teuthology.orchestra.run.smithi005.stdout:3160 read 0x4c0405e thru 0x4c07201 (0x31a4 bytes) 2024-05-11T23:02:09.568 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:02:09.568 INFO:teuthology.orchestra.run.smithi005.stdout:3163 clone 24 order 20 su 524288 sc 13 2024-05-11T23:02:09.574 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:09.573+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:09.675 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:09.675+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:09.719 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:09.719 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:09.728 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:02:09.769 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:02:09.814 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:09.814+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:09.876 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:09.876+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:09.882 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:09.882 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:09.891 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:02:09.897 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:09.897 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:09.918 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:09.919 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:09.936 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:09.936 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:09.955 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:09.955 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:09.973 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:09.974 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:09.977 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:09.976+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:09.977 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone23 from 0xe1b46cb (overlap 0x89b0b1) to 0xdc4de 2024-05-11T23:02:09.993 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:09.993 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:10.011 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:10.012 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:10.012 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:10.012 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:10.012 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:10.012 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:10.021 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:02:10.033 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:10.034 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:10.054 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-05-11T23:02:10.056 INFO:teuthology.orchestra.run.smithi005.stdout:3164 read 0x8c26af0 thru 0x8c368d8 (0xfde9 bytes) 2024-05-11T23:02:10.057 INFO:teuthology.orchestra.run.smithi005.stdout:3165 write 0x592d541 thru 0x593c40e (0xeece bytes) 2024-05-11T23:02:10.064 INFO:teuthology.orchestra.run.smithi005.stdout:3166 punch from 0x88f074e to 0x88fcc2f, (0xc4e1 bytes) 2024-05-11T23:02:10.064 INFO:teuthology.orchestra.run.smithi005.stdout:3168 write 0x32ee94e thru 0x32fb6ab (0xcd5e bytes) 2024-05-11T23:02:10.067 INFO:teuthology.orchestra.run.smithi005.stdout:3169 read 0x95d904 thru 0x9667e5 (0x8ee2 bytes) 2024-05-11T23:02:10.068 INFO:teuthology.orchestra.run.smithi005.stdout:3172 trunc from 0xe1b46cb to 0xe80e60c 2024-05-11T23:02:10.073 INFO:teuthology.orchestra.run.smithi005.stdout:3175 write 0x902a350 thru 0x9035903 (0xb5b4 bytes) 2024-05-11T23:02:10.077 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:10.077 INFO:teuthology.orchestra.run.smithi005.stdout:3177 write 0x96c5bcc thru 0x96d2754 (0xcb89 bytes) 2024-05-11T23:02:10.077 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:10.077+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:10.078 INFO:teuthology.orchestra.run.smithi005.stdout:3178 punch from 0xa64966f to 0xa64f72e, (0x60bf bytes) 2024-05-11T23:02:10.084 INFO:teuthology.orchestra.run.smithi005.stdout:3179 trunc from 0xe80e60c to 0x9669a47 2024-05-11T23:02:10.109 INFO:teuthology.orchestra.run.smithi005.stdout:3180 trunc from 0x9669a47 to 0x32b7af3 2024-05-11T23:02:10.128 INFO:teuthology.orchestra.run.smithi005.stdout:3181 write 0x64f1b5 thru 0x65867a (0x94c6 bytes) 2024-05-11T23:02:10.133 INFO:teuthology.orchestra.run.smithi005.stdout:3183 read 0x149824e thru 0x1498fb7 (0xd6a bytes) 2024-05-11T23:02:10.133 INFO:teuthology.orchestra.run.smithi005.stdout:3184 writesame 0x267ed2 thru 0x274669 data_size 0xc(0xc798 bytes) 2024-05-11T23:02:10.140 INFO:teuthology.orchestra.run.smithi005.stdout:3185 trunc from 0x32b7af3 to 0x3ea49fd 2024-05-11T23:02:10.151 INFO:teuthology.orchestra.run.smithi005.stdout:3186 write 0x41d4b2f thru 0x41d6a8a (0x1f5c bytes) 2024-05-11T23:02:10.153 INFO:teuthology.orchestra.run.smithi005.stdout:3188 write 0x19c0a15 thru 0x19c3efa (0x34e6 bytes) 2024-05-11T23:02:10.156 INFO:teuthology.orchestra.run.smithi005.stdout:3189 read 0x28645a thru 0x293902 (0xd4a9 bytes) 2024-05-11T23:02:10.157 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:10.157 INFO:teuthology.orchestra.run.smithi005.stdout:3191 write 0x3a3921c thru 0x3a3ae42 (0x1c27 bytes) 2024-05-11T23:02:10.173 INFO:teuthology.orchestra.run.smithi005.stdout:3192 write 0xa42c3d2 thru 0xa436b15 (0xa744 bytes) 2024-05-11T23:02:10.176 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:10.177 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:10.177 INFO:teuthology.orchestra.run.smithi005.stdout:3197 read 0x1c8571c thru 0x1c86b25 (0x140a bytes) 2024-05-11T23:02:10.177 INFO:teuthology.orchestra.run.smithi005.stdout:3198 punch from 0x6bc7b4c to 0x6bcaf2a, (0x33de bytes) 2024-05-11T23:02:10.178 INFO:teuthology.orchestra.run.smithi005.stdout:3199 read 0x5d61045 thru 0x5d6b7a0 (0xa75c bytes) 2024-05-11T23:02:10.178 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:10.177+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:10.179 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:10.179 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:10.179 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:10.179 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:10.179 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:10.185 INFO:teuthology.orchestra.run.smithi005.stdout:3201 writesame 0xc33fb0e thru 0xc3498a3 data_size 0x2(0x9d96 bytes) 2024-05-11T23:02:10.185 INFO:teuthology.orchestra.run.smithi005.stdout:3202 punch from 0x5100e8b to 0x5105598, (0x470d bytes) 2024-05-11T23:02:10.186 INFO:teuthology.orchestra.run.smithi005.stdout:3204 trunc from 0xc3498a4 to 0xd997d8e 2024-05-11T23:02:10.189 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:02:10.192 INFO:teuthology.orchestra.run.smithi005.stdout:3205 punch from 0x27bea92 to 0x27bedc3, (0x331 bytes) 2024-05-11T23:02:10.192 INFO:teuthology.orchestra.run.smithi005.stdout:3206 write 0xc721e88 thru 0xc726f9e (0x5117 bytes) 2024-05-11T23:02:10.195 INFO:teuthology.orchestra.run.smithi005.stdout:3207 read 0x7630c8b thru 0x7632e32 (0x21a8 bytes) 2024-05-11T23:02:10.195 INFO:teuthology.orchestra.run.smithi005.stdout:3208 read 0x6dc53f8 thru 0x6dccea7 (0x7ab0 bytes) 2024-05-11T23:02:10.195 INFO:teuthology.orchestra.run.smithi005.stdout:3209 read 0x52eafb0 thru 0x52efe54 (0x4ea5 bytes) 2024-05-11T23:02:10.195 INFO:teuthology.orchestra.run.smithi005.stdout:3211 read 0x5079159 thru 0x5088a7a (0xf922 bytes) 2024-05-11T23:02:10.195 INFO:teuthology.orchestra.run.smithi005.stdout:3212 read 0xb26b711 thru 0xb26d266 (0x1b56 bytes) 2024-05-11T23:02:10.195 INFO:teuthology.orchestra.run.smithi005.stdout:3213 write 0x6cadee0 thru 0x6cb2c8d (0x4dae bytes) 2024-05-11T23:02:10.198 INFO:teuthology.orchestra.run.smithi005.stdout:3214 read 0x8305649 thru 0x830a875 (0x522d bytes) 2024-05-11T23:02:10.198 INFO:teuthology.orchestra.run.smithi005.stdout:3216 read 0xac453eb thru 0xac4941d (0x4033 bytes) 2024-05-11T23:02:10.206 INFO:teuthology.orchestra.run.smithi005.stdout:3218 trunc from 0xd997d8e to 0x66ea0d2 2024-05-11T23:02:10.225 INFO:teuthology.orchestra.run.smithi005.stdout:3219 trunc from 0x66ea0d2 to 0xe328600 2024-05-11T23:02:10.368 INFO:teuthology.orchestra.run.smithi005.stdout:3222 writesame 0x867af4d thru 0x868190c data_size 0x40(0x69c0 bytes) 2024-05-11T23:02:10.368 INFO:teuthology.orchestra.run.smithi005.stdout:3223 trunc from 0xe328600 to 0xe4daf6b 2024-05-11T23:02:10.368 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:10.368 INFO:teuthology.orchestra.run.smithi005.stdout:3225 write 0x95c1633 thru 0x95d1469 (0xfe37 bytes) 2024-05-11T23:02:10.368 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:10.368 INFO:teuthology.orchestra.run.smithi005.stdout:3227 writesame 0xeb06cd6 thru 0xeb079d0 data_size 0x1(0xcfb bytes) 2024-05-11T23:02:10.368 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:10.368 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:10.368 INFO:teuthology.orchestra.run.smithi005.stdout:3232 write 0x53d027 thru 0x54b5ae (0xe588 bytes) 2024-05-11T23:02:10.368 INFO:teuthology.orchestra.run.smithi005.stdout:3233 write 0xb786ead thru 0xb78f65a (0x87ae bytes) 2024-05-11T23:02:10.368 INFO:teuthology.orchestra.run.smithi005.stdout:3234 writesame 0x1e21415 thru 0x1e2218d data_size 0x1(0xd79 bytes) 2024-05-11T23:02:10.369 INFO:teuthology.orchestra.run.smithi005.stdout:3235 read 0xd7fe363 thru 0xd80668d (0x832b bytes) 2024-05-11T23:02:10.369 INFO:teuthology.orchestra.run.smithi005.stdout:3236 punch from 0xa1f09be to 0xa2008a7, (0xfee9 bytes) 2024-05-11T23:02:10.369 INFO:teuthology.orchestra.run.smithi005.stdout:3238 write 0xa2546c1 thru 0xa255dbd (0x16fd bytes) 2024-05-11T23:02:10.369 INFO:teuthology.orchestra.run.smithi005.stdout:3239 writesame 0x3a6c711 thru 0x3a7a99e data_size 0x2f(0xe28e bytes) 2024-05-11T23:02:10.369 INFO:teuthology.orchestra.run.smithi005.stdout:3240 trunc from 0xeb079d1 to 0x5795f6b 2024-05-11T23:02:10.369 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:10.369 INFO:teuthology.orchestra.run.smithi005.stdout:3243 write 0xa6b01f3 thru 0xa6be82b (0xe639 bytes) 2024-05-11T23:02:10.369 INFO:teuthology.orchestra.run.smithi005.stdout:3244 read 0x3daeefb thru 0x3db193d (0x2a43 bytes) 2024-05-11T23:02:10.369 INFO:teuthology.orchestra.run.smithi005.stdout:3245 punch from 0x50657cd to 0x506b844, (0x6077 bytes) 2024-05-11T23:02:10.369 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:10.369 INFO:teuthology.orchestra.run.smithi005.stdout:3248 clone 25 order 20 su 8192 sc 2 2024-05-11T23:02:10.370 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:10.366+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:10.467 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:10.467+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:10.472 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24548 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.927949+0000", 2024-05-11T23:02:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.543447494999999, 2024-05-11T23:02:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037300396999999999, 2024-05-11T23:02:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.472 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24548 2024-05-11T23:02:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927949+0000", 2024-05-11T23:02:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927949+0000", 2024-05-11T23:02:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927950+0000", 2024-05-11T23:02:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9400000000000005e-07 2024-05-11T23:02:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927952+0000", 2024-05-11T23:02:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2330000000000001e-06 2024-05-11T23:02:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927959+0000", 2024-05-11T23:02:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8820000000000003e-06 2024-05-11T23:02:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927964+0000", 2024-05-11T23:02:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.899e-06 2024-05-11T23:02:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.959280+0000", 2024-05-11T23:02:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031315904999999998 2024-05-11T23:02:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.959348+0000", 2024-05-11T23:02:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8214999999999997e-05 2024-05-11T23:02:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965250+0000", 2024-05-11T23:02:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005901669 2024-05-11T23:02:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.476 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24555 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.927974+0000", 2024-05-11T23:02:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.543422228000001, 2024-05-11T23:02:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.03739746, 2024-05-11T23:02:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24555 2024-05-11T23:02:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927974+0000", 2024-05-11T23:02:10.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927974+0000", 2024-05-11T23:02:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927975+0000", 2024-05-11T23:02:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6899999999999998e-07 2024-05-11T23:02:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927977+0000", 2024-05-11T23:02:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.074e-06 2024-05-11T23:02:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927984+0000", 2024-05-11T23:02:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7309999999999996e-06 2024-05-11T23:02:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927989+0000", 2024-05-11T23:02:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.079e-06 2024-05-11T23:02:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965279+0000", 2024-05-11T23:02:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037290271999999999 2024-05-11T23:02:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965324+0000", 2024-05-11T23:02:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4638999999999997e-05 2024-05-11T23:02:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965372+0000", 2024-05-11T23:02:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8195999999999997e-05 2024-05-11T23:02:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.480 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24562 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.927995+0000", 2024-05-11T23:02:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.543401292999999, 2024-05-11T23:02:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037703408000000001, 2024-05-11T23:02:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.481 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24562 2024-05-11T23:02:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927995+0000", 2024-05-11T23:02:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927995+0000", 2024-05-11T23:02:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927996+0000", 2024-05-11T23:02:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9499999999999998e-07 2024-05-11T23:02:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927998+0000", 2024-05-11T23:02:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0889999999999998e-06 2024-05-11T23:02:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928004+0000", 2024-05-11T23:02:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4740000000000002e-06 2024-05-11T23:02:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928009+0000", 2024-05-11T23:02:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:02:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965549+0000", 2024-05-11T23:02:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037539826999999998 2024-05-11T23:02:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965616+0000", 2024-05-11T23:02:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7387999999999999e-05 2024-05-11T23:02:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965699+0000", 2024-05-11T23:02:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2347000000000003e-05 2024-05-11T23:02:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24568 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928016+0000", 2024-05-11T23:02:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.543381106999998, 2024-05-11T23:02:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037850338999999997, 2024-05-11T23:02:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.485 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24568 2024-05-11T23:02:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928016+0000", 2024-05-11T23:02:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928016+0000", 2024-05-11T23:02:10.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928016+0000", 2024-05-11T23:02:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-05-11T23:02:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928025+0000", 2024-05-11T23:02:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.363e-06 2024-05-11T23:02:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928032+0000", 2024-05-11T23:02:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.114e-06 2024-05-11T23:02:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928038+0000", 2024-05-11T23:02:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1250000000000001e-06 2024-05-11T23:02:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965720+0000", 2024-05-11T23:02:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037682488 2024-05-11T23:02:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965785+0000", 2024-05-11T23:02:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5180999999999998e-05 2024-05-11T23:02:10.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965866+0000", 2024-05-11T23:02:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0687999999999997e-05 2024-05-11T23:02:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24574 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 266240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928045+0000", 2024-05-11T23:02:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.543352129999999, 2024-05-11T23:02:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037990532, 2024-05-11T23:02:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.489 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24574 2024-05-11T23:02:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928045+0000", 2024-05-11T23:02:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928045+0000", 2024-05-11T23:02:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928045+0000", 2024-05-11T23:02:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:02:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928047+0000", 2024-05-11T23:02:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9649999999999998e-06 2024-05-11T23:02:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928053+0000", 2024-05-11T23:02:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1550000000000004e-06 2024-05-11T23:02:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928058+0000", 2024-05-11T23:02:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9060000000000001e-06 2024-05-11T23:02:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965890+0000", 2024-05-11T23:02:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037831754000000002 2024-05-11T23:02:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965957+0000", 2024-05-11T23:02:10.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6798999999999995e-05 2024-05-11T23:02:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966035+0000", 2024-05-11T23:02:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8532e-05 2024-05-11T23:02:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24581 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928064+0000", 2024-05-11T23:02:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.543332314000001, 2024-05-11T23:02:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038134135, 2024-05-11T23:02:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.493 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24581 2024-05-11T23:02:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928064+0000", 2024-05-11T23:02:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928064+0000", 2024-05-11T23:02:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928065+0000", 2024-05-11T23:02:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3099999999999998e-07 2024-05-11T23:02:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928067+0000", 2024-05-11T23:02:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9549999999999999e-06 2024-05-11T23:02:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928073+0000", 2024-05-11T23:02:10.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1929999999999998e-06 2024-05-11T23:02:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928078+0000", 2024-05-11T23:02:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9289999999999997e-06 2024-05-11T23:02:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966058+0000", 2024-05-11T23:02:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037980452999999997 2024-05-11T23:02:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966123+0000", 2024-05-11T23:02:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4832000000000005e-05 2024-05-11T23:02:10.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966198+0000", 2024-05-11T23:02:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5341999999999998e-05 2024-05-11T23:02:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24585 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928095+0000", 2024-05-11T23:02:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.543301794000001, 2024-05-11T23:02:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038282688000000002, 2024-05-11T23:02:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.497 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24585 2024-05-11T23:02:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928095+0000", 2024-05-11T23:02:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928095+0000", 2024-05-11T23:02:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928095+0000", 2024-05-11T23:02:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1900000000000003e-07 2024-05-11T23:02:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928098+0000", 2024-05-11T23:02:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1880000000000001e-06 2024-05-11T23:02:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928105+0000", 2024-05-11T23:02:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3690000000000001e-06 2024-05-11T23:02:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928120+0000", 2024-05-11T23:02:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4762e-05 2024-05-11T23:02:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966233+0000", 2024-05-11T23:02:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038113526000000002 2024-05-11T23:02:10.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966300+0000", 2024-05-11T23:02:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6637999999999995e-05 2024-05-11T23:02:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966378+0000", 2024-05-11T23:02:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7686000000000001e-05 2024-05-11T23:02:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.501 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24592 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 315392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928128+0000", 2024-05-11T23:02:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.543268488999999, 2024-05-11T23:02:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038417382, 2024-05-11T23:02:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.502 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24592 2024-05-11T23:02:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928128+0000", 2024-05-11T23:02:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928128+0000", 2024-05-11T23:02:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928129+0000", 2024-05-11T23:02:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.44e-07 2024-05-11T23:02:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928131+0000", 2024-05-11T23:02:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2910000000000002e-06 2024-05-11T23:02:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928138+0000", 2024-05-11T23:02:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9430000000000003e-06 2024-05-11T23:02:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928143+0000", 2024-05-11T23:02:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4739999999999996e-06 2024-05-11T23:02:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966400+0000", 2024-05-11T23:02:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038257001999999998 2024-05-11T23:02:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966465+0000", 2024-05-11T23:02:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4468999999999995e-05 2024-05-11T23:02:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966546+0000", 2024-05-11T23:02:10.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0659000000000001e-05 2024-05-11T23:02:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24598 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 331776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928150+0000", 2024-05-11T23:02:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.543246180000001, 2024-05-11T23:02:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038544469999999997, 2024-05-11T23:02:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.506 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24598 2024-05-11T23:02:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928150+0000", 2024-05-11T23:02:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928150+0000", 2024-05-11T23:02:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928151+0000", 2024-05-11T23:02:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999999e-07 2024-05-11T23:02:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928153+0000", 2024-05-11T23:02:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.21e-06 2024-05-11T23:02:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928160+0000", 2024-05-11T23:02:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5200000000000003e-06 2024-05-11T23:02:10.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928165+0000", 2024-05-11T23:02:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9269999999999996e-06 2024-05-11T23:02:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966569+0000", 2024-05-11T23:02:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038404595999999999 2024-05-11T23:02:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966634+0000", 2024-05-11T23:02:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5221000000000005e-05 2024-05-11T23:02:10.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966695+0000", 2024-05-11T23:02:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0547999999999997e-05 2024-05-11T23:02:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24604 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928171+0000", 2024-05-11T23:02:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.543225818, 2024-05-11T23:02:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038676227000000001, 2024-05-11T23:02:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.510 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24604 2024-05-11T23:02:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928171+0000", 2024-05-11T23:02:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928171+0000", 2024-05-11T23:02:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928171+0000", 2024-05-11T23:02:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-05-11T23:02:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928173+0000", 2024-05-11T23:02:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0619999999999999e-06 2024-05-11T23:02:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928180+0000", 2024-05-11T23:02:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-05-11T23:02:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928185+0000", 2024-05-11T23:02:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.04e-06 2024-05-11T23:02:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966718+0000", 2024-05-11T23:02:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038532759 2024-05-11T23:02:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966784+0000", 2024-05-11T23:02:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6865000000000004e-05 2024-05-11T23:02:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966847+0000", 2024-05-11T23:02:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2694000000000007e-05 2024-05-11T23:02:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24609 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 364544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928191+0000", 2024-05-11T23:02:10.514 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.543205749999998, 2024-05-11T23:02:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038806908000000001, 2024-05-11T23:02:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24609 2024-05-11T23:02:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928191+0000", 2024-05-11T23:02:10.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928191+0000", 2024-05-11T23:02:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928191+0000", 2024-05-11T23:02:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-05-11T23:02:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928198+0000", 2024-05-11T23:02:10.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2180000000000002e-06 2024-05-11T23:02:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928205+0000", 2024-05-11T23:02:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5899999999999996e-06 2024-05-11T23:02:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928210+0000", 2024-05-11T23:02:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.118e-06 2024-05-11T23:02:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966869+0000", 2024-05-11T23:02:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038659106999999998 2024-05-11T23:02:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966933+0000", 2024-05-11T23:02:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4158999999999993e-05 2024-05-11T23:02:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966998+0000", 2024-05-11T23:02:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4352999999999999e-05 2024-05-11T23:02:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.518 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24616 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 380928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928217+0000", 2024-05-11T23:02:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.543179606999999, 2024-05-11T23:02:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038928536999999999, 2024-05-11T23:02:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.519 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24616 2024-05-11T23:02:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928217+0000", 2024-05-11T23:02:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928217+0000", 2024-05-11T23:02:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928217+0000", 2024-05-11T23:02:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-05-11T23:02:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928219+0000", 2024-05-11T23:02:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0609999999999998e-06 2024-05-11T23:02:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928233+0000", 2024-05-11T23:02:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3715e-05 2024-05-11T23:02:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928238+0000", 2024-05-11T23:02:10.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1889999999999998e-06 2024-05-11T23:02:10.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967021+0000", 2024-05-11T23:02:10.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038782284 2024-05-11T23:02:10.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967085+0000", 2024-05-11T23:02:10.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4319000000000005e-05 2024-05-11T23:02:10.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967146+0000", 2024-05-11T23:02:10.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0579999999999999e-05 2024-05-11T23:02:10.523 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.523 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.523 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.523 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24622 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 397312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:10.523 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928262+0000", 2024-05-11T23:02:10.523 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.543134301999999, 2024-05-11T23:02:10.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039038392999999998, 2024-05-11T23:02:10.523 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.523 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.523 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.524 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.524 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.524 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.524 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24622 2024-05-11T23:02:10.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.524 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928262+0000", 2024-05-11T23:02:10.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928262+0000", 2024-05-11T23:02:10.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928263+0000", 2024-05-11T23:02:10.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0299999999999999e-07 2024-05-11T23:02:10.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928266+0000", 2024-05-11T23:02:10.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6690000000000002e-06 2024-05-11T23:02:10.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928275+0000", 2024-05-11T23:02:10.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1630000000000002e-06 2024-05-11T23:02:10.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928280+0000", 2024-05-11T23:02:10.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6579999999999999e-06 2024-05-11T23:02:10.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967169+0000", 2024-05-11T23:02:10.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038888569999999997 2024-05-11T23:02:10.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967240+0000", 2024-05-11T23:02:10.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0947999999999999e-05 2024-05-11T23:02:10.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967301+0000", 2024-05-11T23:02:10.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0782000000000002e-05 2024-05-11T23:02:10.527 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.527 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.527 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.527 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24628 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 413696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:10.527 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928288+0000", 2024-05-11T23:02:10.527 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.543108254, 2024-05-11T23:02:10.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039162792000000002, 2024-05-11T23:02:10.527 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.528 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.528 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.528 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.528 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.528 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.528 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24628 2024-05-11T23:02:10.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.528 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928288+0000", 2024-05-11T23:02:10.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928288+0000", 2024-05-11T23:02:10.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928289+0000", 2024-05-11T23:02:10.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7399999999999998e-07 2024-05-11T23:02:10.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928291+0000", 2024-05-11T23:02:10.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.221e-06 2024-05-11T23:02:10.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928298+0000", 2024-05-11T23:02:10.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0520000000000004e-06 2024-05-11T23:02:10.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928304+0000", 2024-05-11T23:02:10.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4149999999999998e-06 2024-05-11T23:02:10.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967324+0000", 2024-05-11T23:02:10.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039020436999999998 2024-05-11T23:02:10.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967390+0000", 2024-05-11T23:02:10.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5821000000000006e-05 2024-05-11T23:02:10.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967451+0000", 2024-05-11T23:02:10.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1371999999999994e-05 2024-05-11T23:02:10.531 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.531 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.531 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.531 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24636 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 430080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:10.531 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928310+0000", 2024-05-11T23:02:10.532 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.543086388999999, 2024-05-11T23:02:10.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039286303000000002, 2024-05-11T23:02:10.532 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.532 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.532 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.532 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24636 2024-05-11T23:02:10.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.532 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928310+0000", 2024-05-11T23:02:10.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928310+0000", 2024-05-11T23:02:10.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928311+0000", 2024-05-11T23:02:10.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:02:10.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928313+0000", 2024-05-11T23:02:10.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0499999999999999e-06 2024-05-11T23:02:10.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928319+0000", 2024-05-11T23:02:10.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4049999999999997e-06 2024-05-11T23:02:10.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928324+0000", 2024-05-11T23:02:10.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:02:10.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967473+0000", 2024-05-11T23:02:10.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039149138999999999 2024-05-11T23:02:10.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967537+0000", 2024-05-11T23:02:10.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4109000000000006e-05 2024-05-11T23:02:10.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967597+0000", 2024-05-11T23:02:10.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9407000000000002e-05 2024-05-11T23:02:10.535 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.535 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.535 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.535 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25144 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 28672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:10.536 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936035+0000", 2024-05-11T23:02:10.536 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.535361483999999, 2024-05-11T23:02:10.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036140510000000001, 2024-05-11T23:02:10.536 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.536 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.536 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.536 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.536 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.536 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.536 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25144 2024-05-11T23:02:10.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.536 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936035+0000", 2024-05-11T23:02:10.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936035+0000", 2024-05-11T23:02:10.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936036+0000", 2024-05-11T23:02:10.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-05-11T23:02:10.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936038+0000", 2024-05-11T23:02:10.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6910000000000002e-06 2024-05-11T23:02:10.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936045+0000", 2024-05-11T23:02:10.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1670000000000004e-06 2024-05-11T23:02:10.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936049+0000", 2024-05-11T23:02:10.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9609999999999996e-06 2024-05-11T23:02:10.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972024+0000", 2024-05-11T23:02:10.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.035974221000000001 2024-05-11T23:02:10.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972076+0000", 2024-05-11T23:02:10.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1918e-05 2024-05-11T23:02:10.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972176+0000", 2024-05-11T23:02:10.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010006200000000001 2024-05-11T23:02:10.539 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.540 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.540 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.540 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25150 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 45056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:10.540 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936063+0000", 2024-05-11T23:02:10.540 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.535333950999998, 2024-05-11T23:02:10.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036298174000000002, 2024-05-11T23:02:10.540 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.540 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.540 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.540 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.540 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.540 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.540 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25150 2024-05-11T23:02:10.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.541 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936063+0000", 2024-05-11T23:02:10.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936063+0000", 2024-05-11T23:02:10.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936063+0000", 2024-05-11T23:02:10.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.39e-07 2024-05-11T23:02:10.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936069+0000", 2024-05-11T23:02:10.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2770000000000002e-06 2024-05-11T23:02:10.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936088+0000", 2024-05-11T23:02:10.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8841000000000001e-05 2024-05-11T23:02:10.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936093+0000", 2024-05-11T23:02:10.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2190000000000003e-06 2024-05-11T23:02:10.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972213+0000", 2024-05-11T23:02:10.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036119039999999998 2024-05-11T23:02:10.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972293+0000", 2024-05-11T23:02:10.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0437000000000003e-05 2024-05-11T23:02:10.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972361+0000", 2024-05-11T23:02:10.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7921000000000002e-05 2024-05-11T23:02:10.544 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.544 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.544 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.544 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25156 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:10.544 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936107+0000", 2024-05-11T23:02:10.544 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.535289426999999, 2024-05-11T23:02:10.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036383785000000002, 2024-05-11T23:02:10.544 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.544 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.544 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.545 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25156 2024-05-11T23:02:10.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.545 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936107+0000", 2024-05-11T23:02:10.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936107+0000", 2024-05-11T23:02:10.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936108+0000", 2024-05-11T23:02:10.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3900000000000005e-07 2024-05-11T23:02:10.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936110+0000", 2024-05-11T23:02:10.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.435e-06 2024-05-11T23:02:10.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936118+0000", 2024-05-11T23:02:10.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2889999999999998e-06 2024-05-11T23:02:10.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936124+0000", 2024-05-11T23:02:10.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2850000000000002e-06 2024-05-11T23:02:10.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972378+0000", 2024-05-11T23:02:10.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036254020999999997 2024-05-11T23:02:10.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972420+0000", 2024-05-11T23:02:10.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2577999999999998e-05 2024-05-11T23:02:10.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972491+0000", 2024-05-11T23:02:10.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0637999999999997e-05 2024-05-11T23:02:10.548 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.548 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.548 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.548 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25164 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:10.548 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936132+0000", 2024-05-11T23:02:10.548 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.535264183999999, 2024-05-11T23:02:10.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036532028000000001, 2024-05-11T23:02:10.548 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.548 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.548 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.548 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.549 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25164 2024-05-11T23:02:10.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.549 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936132+0000", 2024-05-11T23:02:10.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936132+0000", 2024-05-11T23:02:10.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936133+0000", 2024-05-11T23:02:10.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-05-11T23:02:10.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936135+0000", 2024-05-11T23:02:10.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0660000000000002e-06 2024-05-11T23:02:10.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936146+0000", 2024-05-11T23:02:10.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.098e-05 2024-05-11T23:02:10.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936153+0000", 2024-05-11T23:02:10.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8619999999999997e-06 2024-05-11T23:02:10.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972532+0000", 2024-05-11T23:02:10.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036379702999999999 2024-05-11T23:02:10.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972577+0000", 2024-05-11T23:02:10.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4697999999999998e-05 2024-05-11T23:02:10.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972664+0000", 2024-05-11T23:02:10.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7338999999999998e-05 2024-05-11T23:02:10.552 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.552 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.552 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.552 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25168 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 94208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:10.552 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936160+0000", 2024-05-11T23:02:10.552 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.535236906000002, 2024-05-11T23:02:10.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036655488999999999, 2024-05-11T23:02:10.552 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.553 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.553 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.553 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.553 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.553 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.553 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25168 2024-05-11T23:02:10.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.553 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936160+0000", 2024-05-11T23:02:10.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936160+0000", 2024-05-11T23:02:10.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936160+0000", 2024-05-11T23:02:10.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3000000000000001e-07 2024-05-11T23:02:10.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936162+0000", 2024-05-11T23:02:10.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.108e-06 2024-05-11T23:02:10.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.937848+0000", 2024-05-11T23:02:10.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001685284 2024-05-11T23:02:10.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.937858+0000", 2024-05-11T23:02:10.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0222e-05 2024-05-11T23:02:10.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972704+0000", 2024-05-11T23:02:10.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034845736000000002 2024-05-11T23:02:10.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972746+0000", 2024-05-11T23:02:10.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2077999999999999e-05 2024-05-11T23:02:10.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972815+0000", 2024-05-11T23:02:10.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9630999999999998e-05 2024-05-11T23:02:10.556 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.556 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.556 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.556 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.556 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.556 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:10.557 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:02:10.567 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:10.568+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:10.662 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:10.663 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:10.663 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:10.663 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:10.668 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:10.669+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:10.671 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:02:10.769 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:10.770+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:10.776 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:10.776 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:10.776 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:10.776 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:10.776 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:10.785 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:02:10.870 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:10.869+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:10.891 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67817 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.2705649570000004, 2024-05-11T23:02:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044148845999999999, 2024-05-11T23:02:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.891 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67817 2024-05-11T23:02:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620227+0000", 2024-05-11T23:02:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00046792099999999999 2024-05-11T23:02:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620232+0000", 2024-05-11T23:02:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3050000000000003e-06 2024-05-11T23:02:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620243+0000", 2024-05-11T23:02:10.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1289e-05 2024-05-11T23:02:10.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620250+0000", 2024-05-11T23:02:10.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2130000000000002e-06 2024-05-11T23:02:10.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663745+0000", 2024-05-11T23:02:10.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043494865000000001 2024-05-11T23:02:10.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663815+0000", 2024-05-11T23:02:10.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0284999999999995e-05 2024-05-11T23:02:10.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663908+0000", 2024-05-11T23:02:10.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2967999999999995e-05 2024-05-11T23:02:10.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.895 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.895 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67820 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4902912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:10.895 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:10.895 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.2700154499999998, 2024-05-11T23:02:10.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044096781000000002, 2024-05-11T23:02:10.895 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.896 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.896 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.896 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67820 2024-05-11T23:02:10.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.896 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:10.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:10.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620310+0000", 2024-05-11T23:02:10.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9800000000000004e-07 2024-05-11T23:02:10.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620312+0000", 2024-05-11T23:02:10.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.272e-06 2024-05-11T23:02:10.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620319+0000", 2024-05-11T23:02:10.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9310000000000002e-06 2024-05-11T23:02:10.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620323+0000", 2024-05-11T23:02:10.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:02:10.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664284+0000", 2024-05-11T23:02:10.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043960908999999999 2024-05-11T23:02:10.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664327+0000", 2024-05-11T23:02:10.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2811000000000001e-05 2024-05-11T23:02:10.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664406+0000", 2024-05-11T23:02:10.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8588e-05 2024-05-11T23:02:10.899 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.899 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.899 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.899 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67821 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4927488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:10.899 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:10.899 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.2699950400000004, 2024-05-11T23:02:10.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044203340000000001, 2024-05-11T23:02:10.900 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.900 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.900 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.900 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.900 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.900 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.900 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67821 2024-05-11T23:02:10.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.900 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:10.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:10.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620330+0000", 2024-05-11T23:02:10.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:02:10.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620332+0000", 2024-05-11T23:02:10.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:02:10.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620338+0000", 2024-05-11T23:02:10.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6250000000000004e-06 2024-05-11T23:02:10.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620342+0000", 2024-05-11T23:02:10.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7609999999999998e-06 2024-05-11T23:02:10.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664433+0000", 2024-05-11T23:02:10.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044090455000000001 2024-05-11T23:02:10.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664476+0000", 2024-05-11T23:02:10.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2861999999999997e-05 2024-05-11T23:02:10.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664533+0000", 2024-05-11T23:02:10.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7156e-05 2024-05-11T23:02:10.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.903 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.903 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67822 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4976640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:10.903 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:10.904 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.2699760019999999, 2024-05-11T23:02:10.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044409416, 2024-05-11T23:02:10.904 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.904 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.904 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.904 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67822 2024-05-11T23:02:10.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.904 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:10.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:10.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620349+0000", 2024-05-11T23:02:10.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.51e-07 2024-05-11T23:02:10.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620351+0000", 2024-05-11T23:02:10.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.181e-06 2024-05-11T23:02:10.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620358+0000", 2024-05-11T23:02:10.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3060000000000003e-06 2024-05-11T23:02:10.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620362+0000", 2024-05-11T23:02:10.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7949999999999998e-06 2024-05-11T23:02:10.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664572+0000", 2024-05-11T23:02:10.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044209318999999997 2024-05-11T23:02:10.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664654+0000", 2024-05-11T23:02:10.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2367999999999995e-05 2024-05-11T23:02:10.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664758+0000", 2024-05-11T23:02:10.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103896 2024-05-11T23:02:10.907 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.907 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.907 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.908 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67823 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4952064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:10.908 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:10.908 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.2699559520000001, 2024-05-11T23:02:10.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044623293000000001, 2024-05-11T23:02:10.908 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.908 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.908 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.908 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67823 2024-05-11T23:02:10.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.908 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:10.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:10.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622228+0000", 2024-05-11T23:02:10.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018592280000000001 2024-05-11T23:02:10.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622232+0000", 2024-05-11T23:02:10.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.403e-06 2024-05-11T23:02:10.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622248+0000", 2024-05-11T23:02:10.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5449000000000001e-05 2024-05-11T23:02:10.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622254+0000", 2024-05-11T23:02:10.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5819999999999998e-06 2024-05-11T23:02:10.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664800+0000", 2024-05-11T23:02:10.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042546048000000003 2024-05-11T23:02:10.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664886+0000", 2024-05-11T23:02:10.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5525000000000005e-05 2024-05-11T23:02:10.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664992+0000", 2024-05-11T23:02:10.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010605800000000001 2024-05-11T23:02:10.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.912 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.912 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.912 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67833 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5099520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:10.912 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:10.912 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.2678494789999997, 2024-05-11T23:02:10.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044076606999999997, 2024-05-11T23:02:10.912 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.912 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.912 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.912 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67833 2024-05-11T23:02:10.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.913 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:10.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:10.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:10.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8099999999999998e-07 2024-05-11T23:02:10.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622477+0000", 2024-05-11T23:02:10.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0700000000000001e-06 2024-05-11T23:02:10.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622483+0000", 2024-05-11T23:02:10.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0780000000000003e-06 2024-05-11T23:02:10.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622487+0000", 2024-05-11T23:02:10.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.352e-06 2024-05-11T23:02:10.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666460+0000", 2024-05-11T23:02:10.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043973085000000002 2024-05-11T23:02:10.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666498+0000", 2024-05-11T23:02:10.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8441e-05 2024-05-11T23:02:10.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666552+0000", 2024-05-11T23:02:10.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3199999999999999e-05 2024-05-11T23:02:10.916 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.916 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.916 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.916 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67834 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3944448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:10.916 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:10.916 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.2678112009999998, 2024-05-11T23:02:10.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044166551999999998, 2024-05-11T23:02:10.916 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.916 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.916 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.916 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.916 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.916 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.916 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67834 2024-05-11T23:02:10.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.917 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:10.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:10.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622514+0000", 2024-05-11T23:02:10.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1100000000000002e-07 2024-05-11T23:02:10.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622515+0000", 2024-05-11T23:02:10.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:02:10.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622519+0000", 2024-05-11T23:02:10.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8260000000000003e-06 2024-05-11T23:02:10.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622522+0000", 2024-05-11T23:02:10.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8899999999999999e-06 2024-05-11T23:02:10.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666580+0000", 2024-05-11T23:02:10.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044058235000000001 2024-05-11T23:02:10.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666624+0000", 2024-05-11T23:02:10.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3501999999999998e-05 2024-05-11T23:02:10.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666680+0000", 2024-05-11T23:02:10.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6042999999999999e-05 2024-05-11T23:02:10.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.920 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.920 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67835 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:10.920 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:10.920 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.2677986649999999, 2024-05-11T23:02:10.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044272019000000003, 2024-05-11T23:02:10.920 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.920 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.920 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.921 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67835 2024-05-11T23:02:10.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.921 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:10.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:10.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:10.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1699999999999999e-07 2024-05-11T23:02:10.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622527+0000", 2024-05-11T23:02:10.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2109999999999999e-06 2024-05-11T23:02:10.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622531+0000", 2024-05-11T23:02:10.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4910000000000001e-06 2024-05-11T23:02:10.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622534+0000", 2024-05-11T23:02:10.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7080000000000002e-06 2024-05-11T23:02:10.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666703+0000", 2024-05-11T23:02:10.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044169377000000003 2024-05-11T23:02:10.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666745+0000", 2024-05-11T23:02:10.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1616000000000003e-05 2024-05-11T23:02:10.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666798+0000", 2024-05-11T23:02:10.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3298999999999999e-05 2024-05-11T23:02:10.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.924 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.924 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67836 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:10.924 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:10.924 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.2677880049999999, 2024-05-11T23:02:10.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044373692999999999, 2024-05-11T23:02:10.924 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.924 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.924 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.925 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67836 2024-05-11T23:02:10.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.925 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:10.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:10.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622537+0000", 2024-05-11T23:02:10.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:10.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622538+0000", 2024-05-11T23:02:10.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1510000000000001e-06 2024-05-11T23:02:10.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622541+0000", 2024-05-11T23:02:10.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:02:10.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622543+0000", 2024-05-11T23:02:10.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4870000000000001e-06 2024-05-11T23:02:10.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666821+0000", 2024-05-11T23:02:10.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044278038999999998 2024-05-11T23:02:10.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666859+0000", 2024-05-11T23:02:10.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7224000000000001e-05 2024-05-11T23:02:10.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666910+0000", 2024-05-11T23:02:10.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1629999999999999e-05 2024-05-11T23:02:10.928 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.928 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.928 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.928 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67837 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:10.928 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:10.928 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.2677782779999998, 2024-05-11T23:02:10.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044477883000000003, 2024-05-11T23:02:10.928 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.928 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.928 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.929 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.929 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.929 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67837 2024-05-11T23:02:10.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.929 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:10.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:10.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:10.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:10.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622548+0000", 2024-05-11T23:02:10.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:02:10.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622550+0000", 2024-05-11T23:02:10.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-05-11T23:02:10.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622553+0000", 2024-05-11T23:02:10.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.43e-06 2024-05-11T23:02:10.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666932+0000", 2024-05-11T23:02:10.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044379584 2024-05-11T23:02:10.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666971+0000", 2024-05-11T23:02:10.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8427000000000003e-05 2024-05-11T23:02:10.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667024+0000", 2024-05-11T23:02:10.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3226000000000002e-05 2024-05-11T23:02:10.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.932 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.932 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67838 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:10.932 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:10.932 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.2677687610000001, 2024-05-11T23:02:10.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044580374999999998, 2024-05-11T23:02:10.932 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.932 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.932 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.932 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.932 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.933 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67838 2024-05-11T23:02:10.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.933 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:10.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:10.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:10.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:02:10.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622557+0000", 2024-05-11T23:02:10.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.102e-06 2024-05-11T23:02:10.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622560+0000", 2024-05-11T23:02:10.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:02:10.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622562+0000", 2024-05-11T23:02:10.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:02:10.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667045+0000", 2024-05-11T23:02:10.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044482936000000001 2024-05-11T23:02:10.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667083+0000", 2024-05-11T23:02:10.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7443e-05 2024-05-11T23:02:10.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667136+0000", 2024-05-11T23:02:10.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3257000000000002e-05 2024-05-11T23:02:10.936 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.936 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.936 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.936 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67839 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5394432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:10.936 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:10.936 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.2677590189999997, 2024-05-11T23:02:10.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04470263, 2024-05-11T23:02:10.936 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.936 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.936 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.936 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67839 2024-05-11T23:02:10.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.937 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:10.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:10.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622566+0000", 2024-05-11T23:02:10.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-05-11T23:02:10.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622572+0000", 2024-05-11T23:02:10.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5610000000000004e-06 2024-05-11T23:02:10.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622576+0000", 2024-05-11T23:02:10.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3210000000000001e-06 2024-05-11T23:02:10.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622578+0000", 2024-05-11T23:02:10.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4710000000000001e-06 2024-05-11T23:02:10.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667160+0000", 2024-05-11T23:02:10.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044582220999999998 2024-05-11T23:02:10.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667205+0000", 2024-05-11T23:02:10.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5049000000000002e-05 2024-05-11T23:02:10.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667268+0000", 2024-05-11T23:02:10.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2788999999999998e-05 2024-05-11T23:02:10.939 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.939 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.940 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.940 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67840 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5419008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:10.940 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:10.940 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.2677433889999996, 2024-05-11T23:02:10.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044796006999999999, 2024-05-11T23:02:10.940 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.940 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.940 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.940 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67840 2024-05-11T23:02:10.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.940 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:10.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:10.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:10.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:02:10.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622583+0000", 2024-05-11T23:02:10.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4389999999999999e-06 2024-05-11T23:02:10.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622586+0000", 2024-05-11T23:02:10.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:02:10.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622588+0000", 2024-05-11T23:02:10.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4540000000000001e-06 2024-05-11T23:02:10.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667287+0000", 2024-05-11T23:02:10.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044698882000000002 2024-05-11T23:02:10.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667324+0000", 2024-05-11T23:02:10.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7196000000000001e-05 2024-05-11T23:02:10.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667377+0000", 2024-05-11T23:02:10.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2927999999999999e-05 2024-05-11T23:02:10.943 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.943 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.944 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.944 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67841 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5443584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:10.944 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:10.944 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.2677336160000001, 2024-05-11T23:02:10.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044907337999999998, 2024-05-11T23:02:10.944 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.944 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.944 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.944 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.944 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.944 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.944 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67841 2024-05-11T23:02:10.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.944 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:10.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:10.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:10.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:10.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622592+0000", 2024-05-11T23:02:10.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-05-11T23:02:10.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622595+0000", 2024-05-11T23:02:10.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:02:10.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622598+0000", 2024-05-11T23:02:10.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5909999999999998e-06 2024-05-11T23:02:10.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667401+0000", 2024-05-11T23:02:10.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044802980999999999 2024-05-11T23:02:10.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667439+0000", 2024-05-11T23:02:10.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8840000000000001e-05 2024-05-11T23:02:10.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667498+0000", 2024-05-11T23:02:10.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8709000000000003e-05 2024-05-11T23:02:10.947 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.947 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.947 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.948 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67842 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5468160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:10.948 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:10.948 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.2677240090000002, 2024-05-11T23:02:10.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045005876, 2024-05-11T23:02:10.948 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.948 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.948 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.948 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67842 2024-05-11T23:02:10.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.948 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:10.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:10.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622601+0000", 2024-05-11T23:02:10.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:10.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622602+0000", 2024-05-11T23:02:10.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0780000000000001e-06 2024-05-11T23:02:10.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622605+0000", 2024-05-11T23:02:10.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.914e-06 2024-05-11T23:02:10.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622607+0000", 2024-05-11T23:02:10.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:02:10.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667516+0000", 2024-05-11T23:02:10.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044909156999999998 2024-05-11T23:02:10.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667554+0000", 2024-05-11T23:02:10.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7920000000000003e-05 2024-05-11T23:02:10.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667606+0000", 2024-05-11T23:02:10.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2111000000000003e-05 2024-05-11T23:02:10.951 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.951 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.952 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.952 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67843 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4067328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:10.952 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:10.952 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.2677144829999998, 2024-05-11T23:02:10.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045119014999999998, 2024-05-11T23:02:10.952 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.952 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.952 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.952 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.952 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.952 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.952 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67843 2024-05-11T23:02:10.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.953 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:10.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:10.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624228+0000", 2024-05-11T23:02:10.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001617867 2024-05-11T23:02:10.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624232+0000", 2024-05-11T23:02:10.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-05-11T23:02:10.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624240+0000", 2024-05-11T23:02:10.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4589999999999998e-06 2024-05-11T23:02:10.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624247+0000", 2024-05-11T23:02:10.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-05-11T23:02:10.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667629+0000", 2024-05-11T23:02:10.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043382012999999997 2024-05-11T23:02:10.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667674+0000", 2024-05-11T23:02:10.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5172000000000001e-05 2024-05-11T23:02:10.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667729+0000", 2024-05-11T23:02:10.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5086000000000001e-05 2024-05-11T23:02:10.956 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.956 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.956 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.956 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67849 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4190208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:10.956 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:10.956 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.2659563619999998, 2024-05-11T23:02:10.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044061433999999997, 2024-05-11T23:02:10.956 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.956 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.956 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.956 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.956 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.957 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67849 2024-05-11T23:02:10.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.957 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:10.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:10.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624369+0000", 2024-05-11T23:02:10.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5900000000000002e-07 2024-05-11T23:02:10.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624371+0000", 2024-05-11T23:02:10.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0310000000000001e-06 2024-05-11T23:02:10.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624376+0000", 2024-05-11T23:02:10.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4029999999999998e-06 2024-05-11T23:02:10.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624381+0000", 2024-05-11T23:02:10.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4349999999999996e-06 2024-05-11T23:02:10.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668331+0000", 2024-05-11T23:02:10.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043949191999999998 2024-05-11T23:02:10.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668370+0000", 2024-05-11T23:02:10.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9227999999999998e-05 2024-05-11T23:02:10.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668430+0000", 2024-05-11T23:02:10.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9685999999999997e-05 2024-05-11T23:02:10.960 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.960 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.960 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.960 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67850 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:10.960 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:10.960 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.2659351299999999, 2024-05-11T23:02:10.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044151953000000001, 2024-05-11T23:02:10.960 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.960 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.960 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.961 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.961 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.961 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.961 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67850 2024-05-11T23:02:10.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.961 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:10.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:10.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624390+0000", 2024-05-11T23:02:10.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:02:10.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624392+0000", 2024-05-11T23:02:10.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1880000000000001e-06 2024-05-11T23:02:10.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624399+0000", 2024-05-11T23:02:10.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7390000000000002e-06 2024-05-11T23:02:10.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624405+0000", 2024-05-11T23:02:10.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1059999999999996e-06 2024-05-11T23:02:10.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668450+0000", 2024-05-11T23:02:10.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044045147999999999 2024-05-11T23:02:10.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668487+0000", 2024-05-11T23:02:10.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7450000000000002e-05 2024-05-11T23:02:10.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668541+0000", 2024-05-11T23:02:10.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3952000000000001e-05 2024-05-11T23:02:10.964 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.964 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.964 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.964 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67851 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5541888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:10.964 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:10.964 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.2659085479999996, 2024-05-11T23:02:10.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044264413000000002, 2024-05-11T23:02:10.964 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.964 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.965 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:10.965 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.965 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.965 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.965 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67851 2024-05-11T23:02:10.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.965 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:10.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:10.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:10.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-05-11T23:02:10.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624419+0000", 2024-05-11T23:02:10.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.21e-06 2024-05-11T23:02:10.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624424+0000", 2024-05-11T23:02:10.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9050000000000002e-06 2024-05-11T23:02:10.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624430+0000", 2024-05-11T23:02:10.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-05-11T23:02:10.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668566+0000", 2024-05-11T23:02:10.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044136208000000003 2024-05-11T23:02:10.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:10.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668614+0000", 2024-05-11T23:02:10.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8022000000000002e-05 2024-05-11T23:02:10.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668680+0000", 2024-05-11T23:02:10.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6181999999999998e-05 2024-05-11T23:02:10.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.968 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.968 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67852 3.16 3.1a4a4af6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:10.968 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:10.968 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.2658880809999999, 2024-05-11T23:02:10.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044278287999999999, 2024-05-11T23:02:10.969 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:10.969 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:10.969 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:10.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:10.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:10.969 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:10.969 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67852 2024-05-11T23:02:10.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.969 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:10.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:10.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:10.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:10.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:10.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:10.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:10.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626226+0000", 2024-05-11T23:02:10.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017893 2024-05-11T23:02:10.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:10.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626229+0000", 2024-05-11T23:02:10.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0709999999999999e-06 2024-05-11T23:02:10.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:10.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626239+0000", 2024-05-11T23:02:10.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0507e-05 2024-05-11T23:02:10.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:10.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626244+0000", 2024-05-11T23:02:10.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4880000000000001e-06 2024-05-11T23:02:10.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:10.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668706+0000", 2024-05-11T23:02:10.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042462054999999999 2024-05-11T23:02:10.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:10.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:10.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:10.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668715+0000", 2024-05-11T23:02:10.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.867e-06 2024-05-11T23:02:10.972 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.972 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.972 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.972 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:10.972 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:10.972 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:10.973 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-11T23:02:10.974 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:10.974+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:10.975 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:02:11.000 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone24 from 0xa6be82c (overlap 0x32b7af3) to 0x1449794 2024-05-11T23:02:11.075 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:11.075+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:11.128 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:11.129 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:11.129 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:11.129 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:11.134 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-05-11T23:02:11.138 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:02:11.139 INFO:teuthology.orchestra.run.smithi005.stdout:3249 punch from 0x4f7968e to 0x4f834a9, (0x9e1b bytes) 2024-05-11T23:02:11.141 INFO:teuthology.orchestra.run.smithi005.stdout:3253 write 0x2ecb1c8 thru 0x2ed4a05 (0x983e bytes) 2024-05-11T23:02:11.175 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:11.175+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:11.185 INFO:teuthology.orchestra.run.smithi005.stdout:3255 read 0x1a1eb27 thru 0x1a20a8d (0x1f67 bytes) 2024-05-11T23:02:11.188 INFO:teuthology.orchestra.run.smithi005.stdout:3256 trunc from 0xa6be82c to 0x84173ee 2024-05-11T23:02:11.197 INFO:teuthology.orchestra.run.smithi005.stdout:3257 read 0x956782 thru 0x95fa2d (0x92ac bytes) 2024-05-11T23:02:11.198 INFO:teuthology.orchestra.run.smithi005.stdout:3258 write 0x743755d thru 0x743fa8c (0x8530 bytes) 2024-05-11T23:02:11.202 INFO:teuthology.orchestra.run.smithi005.stdout:3259 punch from 0x163f56f to 0x16421e5, (0x2c76 bytes) 2024-05-11T23:02:11.202 INFO:teuthology.orchestra.run.smithi005.stdout:3260 read 0x8142fce thru 0x8147620 (0x4653 bytes) 2024-05-11T23:02:11.202 INFO:teuthology.orchestra.run.smithi005.stdout:3263 read 0x19a06e6 thru 0x19a1866 (0x1181 bytes) 2024-05-11T23:02:11.202 INFO:teuthology.orchestra.run.smithi005.stdout:3264 writesame 0x1e40728 thru 0x1e43950 data_size 0x1(0x3229 bytes) 2024-05-11T23:02:11.254 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:11.254 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:11.254 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:11.254 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:11.254 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:11.259 INFO:teuthology.orchestra.run.smithi005.stdout:3265 write 0xd5ddf22 thru 0xd5e0535 (0x2614 bytes) 2024-05-11T23:02:11.263 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:02:11.268 INFO:teuthology.orchestra.run.smithi005.stdout:3266 writesame 0xe4e2d0b thru 0xe4f0397 data_size 0x19(0xd68d bytes) 2024-05-11T23:02:11.279 INFO:teuthology.orchestra.run.smithi005.stdout:3267 trunc from 0xe4f0398 to 0x46bfa93 2024-05-11T23:02:11.293 INFO:teuthology.orchestra.run.smithi005.stdout:3268 read 0x90e913 thru 0x9198d7 (0xafc5 bytes) 2024-05-11T23:02:11.294 INFO:teuthology.orchestra.run.smithi005.stdout:3270 read 0x432faab thru 0x433091c (0xe72 bytes) 2024-05-11T23:02:11.299 INFO:teuthology.orchestra.run.smithi005.stdout:3271 write 0x4cc78db thru 0x4cd30c9 (0xb7ef bytes) 2024-05-11T23:02:11.303 INFO:teuthology.orchestra.run.smithi005.stdout:3272 write 0x2d466ca thru 0x2d4fc0f (0x9546 bytes) 2024-05-11T23:02:11.307 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:11.307+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:11.336 INFO:teuthology.orchestra.run.smithi005.stdout:3273 writesame 0xdce42da thru 0xdce5416 data_size 0x1(0x113d bytes) 2024-05-11T23:02:11.337 INFO:teuthology.orchestra.run.smithi005.stdout:3274 punch from 0x74888a0 to 0x748ea6e, (0x61ce bytes) 2024-05-11T23:02:11.337 INFO:teuthology.orchestra.run.smithi005.stdout:3276 read 0xadef19b thru 0xadf743a (0x82a0 bytes) 2024-05-11T23:02:11.337 INFO:teuthology.orchestra.run.smithi005.stdout:3277 read 0x106c907 thru 0x107937a (0xca74 bytes) 2024-05-11T23:02:11.338 INFO:teuthology.orchestra.run.smithi005.stdout:3279 trunc from 0xdce5417 to 0xcb2db39 2024-05-11T23:02:11.343 INFO:teuthology.orchestra.run.smithi005.stdout:3280 read 0x69bd39f thru 0x69c2772 (0x53d4 bytes) 2024-05-11T23:02:11.343 INFO:teuthology.orchestra.run.smithi005.stdout:3281 read 0x4aaf7a5 thru 0x4ab00e8 (0x944 bytes) 2024-05-11T23:02:11.343 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:11.343 INFO:teuthology.orchestra.run.smithi005.stdout:3283 punch from 0x7991253 to 0x79a017c, (0xef29 bytes) 2024-05-11T23:02:11.353 INFO:teuthology.orchestra.run.smithi005.stdout:3284 trunc from 0xcb2db39 to 0x379540b 2024-05-11T23:02:11.367 INFO:teuthology.orchestra.run.smithi005.stdout:3285 read 0x28d546a thru 0x28e16ba (0xc251 bytes) 2024-05-11T23:02:11.372 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.71823698, 2024-05-11T23:02:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:02:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:02:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:02:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:02:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:02:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:02:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:02:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:02:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:02:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:02:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:02:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:02:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:02:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:02:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:02:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:11.377 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.714855791000002, 2024-05-11T23:02:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:02:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:02:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:11.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:02:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:02:11.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:02:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:02:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:02:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:02:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:02:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:02:11.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:02:11.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:02:11.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:02:11.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:02:11.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:02:11.381 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.381 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.381 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.382 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.382 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:11.382 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.714837083999999, 2024-05-11T23:02:11.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:02:11.382 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.382 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.382 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:11.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.382 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:02:11.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.382 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:11.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:11.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:02:11.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:02:11.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:02:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:02:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:02:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:02:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:02:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:02:11.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:02:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:02:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:02:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:02:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:02:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:02:11.385 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.714818941000001, 2024-05-11T23:02:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:02:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:02:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:02:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:02:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:02:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:02:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:02:11.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:02:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:02:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:02:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:02:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:02:11.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:02:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:02:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:02:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.714791454, 2024-05-11T23:02:11.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:02:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:02:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:02:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:02:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:02:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:02:11.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:02:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:02:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:02:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:02:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:02:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:02:11.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:02:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:02:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:02:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:02:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.394 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.714770926, 2024-05-11T23:02:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:02:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:02:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:02:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:02:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:02:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:02:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:02:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:02:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:02:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:02:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:02:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:02:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:02:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:02:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:02:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.714751151000002, 2024-05-11T23:02:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:02:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:02:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.399 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:02:11.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:02:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:02:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:02:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:02:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:02:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:02:11.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:02:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:02:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:02:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:02:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:02:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:02:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:02:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.714732345000002, 2024-05-11T23:02:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:02:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.403 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:02:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:02:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:02:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:02:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:02:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:02:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:02:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:02:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:02:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:02:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:02:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:02:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:02:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:02:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:02:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:11.407 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.714713421999999, 2024-05-11T23:02:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:02:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:02:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:02:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:02:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:02:11.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:02:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:02:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:02:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:02:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:02:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:02:11.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:02:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:02:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:02:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:02:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.411 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.71469476, 2024-05-11T23:02:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:02:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:02:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:11.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:02:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:02:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:02:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:02:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:02:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:02:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:02:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:02:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:02:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:02:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:02:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:02:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:02:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.415 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.714349597999998, 2024-05-11T23:02:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:02:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:02:11.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:02:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:02:11.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:02:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:02:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:02:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:02:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:02:11.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:02:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:02:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:02:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:02:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:02:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:02:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:02:11.419 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.714324048000002, 2024-05-11T23:02:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:02:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:02:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:02:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:02:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:02:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:02:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:02:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:02:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:02:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:02:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:02:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:02:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:02:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:02:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:02:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.714241789999999, 2024-05-11T23:02:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:02:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.424 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:02:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:02:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:02:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:02:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:02:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:02:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:02:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:02:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:02:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:02:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:02:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:02:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:02:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:02:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:11.428 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.714219343, 2024-05-11T23:02:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:02:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:02:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:02:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:02:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:02:11.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:02:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:02:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:02:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:02:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:02:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:02:11.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:02:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:02:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:02:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:02:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:02:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.714199884999999, 2024-05-11T23:02:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:02:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:02:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:02:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:02:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:02:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:02:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:02:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:02:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:02:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:02:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:02:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:02:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:02:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:02:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:02:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.436 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.714179824999999, 2024-05-11T23:02:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:02:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.437 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:02:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:02:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:02:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:02:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:02:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:02:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:02:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:02:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:02:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:02:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:02:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:02:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:02:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:02:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.714160554999999, 2024-05-11T23:02:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:02:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.441 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:02:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:02:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:02:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:02:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:02:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:02:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:02:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:02:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:02:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:02:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:02:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:02:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:02:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:02:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:11.445 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.713614489000001, 2024-05-11T23:02:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:02:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:02:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:11.446 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:02:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:02:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:02:11.447 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:02:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:02:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:02:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:02:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:02:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:02:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:02:11.448 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:02:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:02:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:02:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:02:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.449 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.713582705, 2024-05-11T23:02:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:02:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:02:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.450 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:02:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.451 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:02:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:02:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:02:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:02:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:02:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:02:11.452 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:02:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:02:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:02:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:02:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:02:11.453 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:02:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:02:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.2763372410000002, 2024-05-11T23:02:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:02:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:11.454 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:02:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:11.455 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:02:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:02:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:02:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:02:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.456 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:02:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:02:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:02:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:02:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:02:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:02:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.457 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:02:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:02:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:02:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:02:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:02:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:02:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:02:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:11.458 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:02:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:02:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:02:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:02:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:02:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:02:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.459 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.460 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.460 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:11.462 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:02:11.462 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:11.461+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:11.464 INFO:teuthology.orchestra.run.smithi005.stdout:3287 write 0x7f650a9 thru 0x7f6dceb (0x8c43 bytes) 2024-05-11T23:02:11.464 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:11.464 INFO:teuthology.orchestra.run.smithi005.stdout:3289 punch from 0x380fc67 to 0x3819cba, (0xa053 bytes) 2024-05-11T23:02:11.464 INFO:teuthology.orchestra.run.smithi005.stdout:3290 read 0x62c28ba thru 0x62d0109 (0xd850 bytes) 2024-05-11T23:02:11.465 INFO:teuthology.orchestra.run.smithi005.stdout:3293 writesame 0x7749e43 thru 0x77528ea data_size 0x88(0x8aa8 bytes) 2024-05-11T23:02:11.465 INFO:teuthology.orchestra.run.smithi005.stdout:3294 write 0x7ddff3c thru 0x7de0299 (0x35e bytes) 2024-05-11T23:02:11.465 INFO:teuthology.orchestra.run.smithi005.stdout:3296 trunc from 0x7f6dcec to 0x5e9dc28 2024-05-11T23:02:11.465 INFO:teuthology.orchestra.run.smithi005.stdout:3298 read 0x245bd49 thru 0x2460f5a (0x5212 bytes) 2024-05-11T23:02:11.465 INFO:teuthology.orchestra.run.smithi005.stdout:3300 read 0x10eb898 thru 0x10ebc07 (0x370 bytes) 2024-05-11T23:02:11.465 INFO:teuthology.orchestra.run.smithi005.stdout:3302 trunc from 0x5e9dc28 to 0xce7bfed 2024-05-11T23:02:11.465 INFO:teuthology.orchestra.run.smithi005.stdout:3303 read 0xca6862c thru 0xca7679f (0xe174 bytes) 2024-05-11T23:02:11.465 INFO:teuthology.orchestra.run.smithi005.stdout:3304 write 0x7587508 thru 0x7596a7e (0xf577 bytes) 2024-05-11T23:02:11.465 INFO:teuthology.orchestra.run.smithi005.stdout:3305 write 0x9f3e723 thru 0x9f4363f (0x4f1d bytes) 2024-05-11T23:02:11.465 INFO:teuthology.orchestra.run.smithi005.stdout:3307 punch from 0x805ee34 to 0x806731d, (0x84e9 bytes) 2024-05-11T23:02:11.465 INFO:teuthology.orchestra.run.smithi005.stdout:3308 read 0x6dee431 thru 0x6df81fd (0x9dcd bytes) 2024-05-11T23:02:11.465 INFO:teuthology.orchestra.run.smithi005.stdout:3309 trunc from 0xce7bfed to 0xe1b684c 2024-05-11T23:02:11.465 INFO:teuthology.orchestra.run.smithi005.stdout:3310 read 0xb65b85d thru 0xb663ff3 (0x8797 bytes) 2024-05-11T23:02:11.465 INFO:teuthology.orchestra.run.smithi005.stdout:3311 punch from 0x867d4fa to 0x8687187, (0x9c8d bytes) 2024-05-11T23:02:11.465 INFO:teuthology.orchestra.run.smithi005.stdout:3312 read 0x4318e73 thru 0x4327cb9 (0xee47 bytes) 2024-05-11T23:02:11.466 INFO:teuthology.orchestra.run.smithi005.stdout:3313 writesame 0x5369902 thru 0x536d0f6 data_size 0xf(0x37f5 bytes) 2024-05-11T23:02:11.466 INFO:teuthology.orchestra.run.smithi005.stdout:3314 read 0x774edfc thru 0x77535a5 (0x47aa bytes) 2024-05-11T23:02:11.466 INFO:teuthology.orchestra.run.smithi005.stdout:3315 read 0x15373a2 thru 0x153ceec (0x5b4b bytes) 2024-05-11T23:02:11.466 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:11.466 INFO:teuthology.orchestra.run.smithi005.stdout:3317 read 0xa4446dc thru 0xa446114 (0x1a39 bytes) 2024-05-11T23:02:11.466 INFO:teuthology.orchestra.run.smithi005.stdout:3318 writesame 0x92f27fb thru 0x92fd66f data_size 0x1(0xae75 bytes) 2024-05-11T23:02:11.466 INFO:teuthology.orchestra.run.smithi005.stdout:3321 write 0x15cd2d8 thru 0x15dabf2 (0xd91b bytes) 2024-05-11T23:02:11.502 INFO:teuthology.orchestra.run.smithi005.stdout:3322 clone 26 order 21 su 262144 sc 12 2024-05-11T23:02:11.562 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:11.562+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:11.608 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:11.608 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:11.608 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:11.608 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:11.616 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:02:11.663 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:11.663+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:11.761 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:11.761 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:11.761 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:11.761 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:11.761 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:11.764 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:11.764+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:11.770 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:02:11.865 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:11.865+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:11.875 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:11.875 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:02:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 54.711706949000003, 2024-05-11T23:02:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:02:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:02:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:02:11.876 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:02:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:02:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:02:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:02:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:02:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:02:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:02:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:02:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:02:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:02:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:02:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:02:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:02:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:02:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:02:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:02:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:02:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:02:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:02:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:02:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:02:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:11.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:02:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:02:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:02:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:02:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:02:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:02:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:02:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:02:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31872 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506353+0000", 2024-05-11T23:02:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.368605728999999, 2024-05-11T23:02:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040469576, 2024-05-11T23:02:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.882 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31872 2024-05-11T23:02:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506353+0000", 2024-05-11T23:02:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506353+0000", 2024-05-11T23:02:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506353+0000", 2024-05-11T23:02:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:02:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506356+0000", 2024-05-11T23:02:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0339999999999999e-06 2024-05-11T23:02:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506362+0000", 2024-05-11T23:02:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:02:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506367+0000", 2024-05-11T23:02:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3129999999999998e-06 2024-05-11T23:02:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546813+0000", 2024-05-11T23:02:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040445980999999999 2024-05-11T23:02:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546823+0000", 2024-05-11T23:02:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7669999999999998e-06 2024-05-11T23:02:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31873 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506374+0000", 2024-05-11T23:02:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.368585190000001, 2024-05-11T23:02:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040466416999999998, 2024-05-11T23:02:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.886 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31873 2024-05-11T23:02:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506374+0000", 2024-05-11T23:02:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506374+0000", 2024-05-11T23:02:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506374+0000", 2024-05-11T23:02:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1600000000000002e-07 2024-05-11T23:02:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506376+0000", 2024-05-11T23:02:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.075e-06 2024-05-11T23:02:11.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506383+0000", 2024-05-11T23:02:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4189999999999999e-06 2024-05-11T23:02:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506388+0000", 2024-05-11T23:02:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.147e-06 2024-05-11T23:02:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546836+0000", 2024-05-11T23:02:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040448005000000002 2024-05-11T23:02:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546840+0000", 2024-05-11T23:02:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3549999999999998e-06 2024-05-11T23:02:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.890 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31874 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506394+0000", 2024-05-11T23:02:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.368564869, 2024-05-11T23:02:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040469948999999998, 2024-05-11T23:02:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31874 2024-05-11T23:02:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.891 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506394+0000", 2024-05-11T23:02:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506394+0000", 2024-05-11T23:02:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506394+0000", 2024-05-11T23:02:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0900000000000002e-07 2024-05-11T23:02:11.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506396+0000", 2024-05-11T23:02:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0669999999999999e-06 2024-05-11T23:02:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506403+0000", 2024-05-11T23:02:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4509999999999997e-06 2024-05-11T23:02:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506408+0000", 2024-05-11T23:02:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0810000000000001e-06 2024-05-11T23:02:11.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546855+0000", 2024-05-11T23:02:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040446825999999998 2024-05-11T23:02:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546864+0000", 2024-05-11T23:02:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.115e-06 2024-05-11T23:02:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.894 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31878 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506510+0000", 2024-05-11T23:02:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.368448982, 2024-05-11T23:02:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040458415999999997, 2024-05-11T23:02:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31878 2024-05-11T23:02:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506510+0000", 2024-05-11T23:02:11.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506510+0000", 2024-05-11T23:02:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506510+0000", 2024-05-11T23:02:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-07 2024-05-11T23:02:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506512+0000", 2024-05-11T23:02:11.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0470000000000001e-06 2024-05-11T23:02:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506519+0000", 2024-05-11T23:02:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3130000000000004e-06 2024-05-11T23:02:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506523+0000", 2024-05-11T23:02:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6999999999999999e-06 2024-05-11T23:02:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546959+0000", 2024-05-11T23:02:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040436110999999997 2024-05-11T23:02:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546968+0000", 2024-05-11T23:02:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.884e-06 2024-05-11T23:02:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31880 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506550+0000", 2024-05-11T23:02:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.368409107000002, 2024-05-11T23:02:11.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040454849000000001, 2024-05-11T23:02:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31880 2024-05-11T23:02:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506550+0000", 2024-05-11T23:02:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506550+0000", 2024-05-11T23:02:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506550+0000", 2024-05-11T23:02:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7300000000000002e-07 2024-05-11T23:02:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506552+0000", 2024-05-11T23:02:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.012e-06 2024-05-11T23:02:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506558+0000", 2024-05-11T23:02:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:02:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506563+0000", 2024-05-11T23:02:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0000000000000004e-06 2024-05-11T23:02:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546997+0000", 2024-05-11T23:02:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040433416 2024-05-11T23:02:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547005+0000", 2024-05-11T23:02:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9440000000000005e-06 2024-05-11T23:02:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31881 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506570+0000", 2024-05-11T23:02:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.368388896999999, 2024-05-11T23:02:11.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040460073999999999, 2024-05-11T23:02:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31881 2024-05-11T23:02:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506570+0000", 2024-05-11T23:02:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506570+0000", 2024-05-11T23:02:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506570+0000", 2024-05-11T23:02:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:02:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506577+0000", 2024-05-11T23:02:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0099999999999998e-06 2024-05-11T23:02:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506584+0000", 2024-05-11T23:02:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7689999999999999e-06 2024-05-11T23:02:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506589+0000", 2024-05-11T23:02:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2340000000000002e-06 2024-05-11T23:02:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547022+0000", 2024-05-11T23:02:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040432526000000003 2024-05-11T23:02:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547030+0000", 2024-05-11T23:02:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1650000000000006e-06 2024-05-11T23:02:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31882 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506596+0000", 2024-05-11T23:02:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.368362677, 2024-05-11T23:02:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040473955999999998, 2024-05-11T23:02:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:11.906 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31882 2024-05-11T23:02:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506596+0000", 2024-05-11T23:02:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506596+0000", 2024-05-11T23:02:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506597+0000", 2024-05-11T23:02:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5299999999999999e-07 2024-05-11T23:02:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506599+0000", 2024-05-11T23:02:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2900000000000001e-06 2024-05-11T23:02:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506605+0000", 2024-05-11T23:02:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5259999999999999e-06 2024-05-11T23:02:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506610+0000", 2024-05-11T23:02:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1030000000000001e-06 2024-05-11T23:02:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547061+0000", 2024-05-11T23:02:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040450451999999998 2024-05-11T23:02:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547070+0000", 2024-05-11T23:02:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.132e-06 2024-05-11T23:02:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31883 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506618+0000", 2024-05-11T23:02:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.368340864, 2024-05-11T23:02:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040494963000000002, 2024-05-11T23:02:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31883 2024-05-11T23:02:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.910 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506618+0000", 2024-05-11T23:02:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506618+0000", 2024-05-11T23:02:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506619+0000", 2024-05-11T23:02:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1799999999999995e-07 2024-05-11T23:02:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506621+0000", 2024-05-11T23:02:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2879999999999999e-06 2024-05-11T23:02:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506628+0000", 2024-05-11T23:02:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9600000000000003e-06 2024-05-11T23:02:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506633+0000", 2024-05-11T23:02:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4229999999999996e-06 2024-05-11T23:02:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547102+0000", 2024-05-11T23:02:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040469172999999997 2024-05-11T23:02:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547113+0000", 2024-05-11T23:02:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0501e-05 2024-05-11T23:02:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31884 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506640+0000", 2024-05-11T23:02:11.913 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.368318583000001, 2024-05-11T23:02:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040498244000000003, 2024-05-11T23:02:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31884 2024-05-11T23:02:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506640+0000", 2024-05-11T23:02:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506640+0000", 2024-05-11T23:02:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506641+0000", 2024-05-11T23:02:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3000000000000001e-07 2024-05-11T23:02:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506643+0000", 2024-05-11T23:02:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0200000000000001e-06 2024-05-11T23:02:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506649+0000", 2024-05-11T23:02:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1689999999999997e-06 2024-05-11T23:02:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506654+0000", 2024-05-11T23:02:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3800000000000002e-06 2024-05-11T23:02:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547131+0000", 2024-05-11T23:02:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040476810000000002 2024-05-11T23:02:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547138+0000", 2024-05-11T23:02:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.435e-06 2024-05-11T23:02:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31885 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506660+0000", 2024-05-11T23:02:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.368298408000001, 2024-05-11T23:02:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040504498, 2024-05-11T23:02:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.917 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31885 2024-05-11T23:02:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506660+0000", 2024-05-11T23:02:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506660+0000", 2024-05-11T23:02:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506661+0000", 2024-05-11T23:02:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4900000000000001e-07 2024-05-11T23:02:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506663+0000", 2024-05-11T23:02:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0660000000000002e-06 2024-05-11T23:02:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506669+0000", 2024-05-11T23:02:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3829999999999997e-06 2024-05-11T23:02:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506675+0000", 2024-05-11T23:02:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3750000000000002e-06 2024-05-11T23:02:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547156+0000", 2024-05-11T23:02:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040481389 2024-05-11T23:02:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547165+0000", 2024-05-11T23:02:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-05-11T23:02:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31886 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506681+0000", 2024-05-11T23:02:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.368277977999998, 2024-05-11T23:02:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040511004000000003, 2024-05-11T23:02:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31886 2024-05-11T23:02:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506681+0000", 2024-05-11T23:02:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506681+0000", 2024-05-11T23:02:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506681+0000", 2024-05-11T23:02:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:02:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506683+0000", 2024-05-11T23:02:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-05-11T23:02:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506703+0000", 2024-05-11T23:02:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0224000000000002e-05 2024-05-11T23:02:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506709+0000", 2024-05-11T23:02:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5799999999999999e-06 2024-05-11T23:02:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547183+0000", 2024-05-11T23:02:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040474060999999999 2024-05-11T23:02:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547192+0000", 2024-05-11T23:02:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6910000000000003e-06 2024-05-11T23:02:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31887 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506716+0000", 2024-05-11T23:02:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.368243154999998, 2024-05-11T23:02:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040522425000000001, 2024-05-11T23:02:11.924 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31887 2024-05-11T23:02:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506716+0000", 2024-05-11T23:02:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506716+0000", 2024-05-11T23:02:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506716+0000", 2024-05-11T23:02:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8799999999999998e-07 2024-05-11T23:02:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506721+0000", 2024-05-11T23:02:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5630000000000002e-06 2024-05-11T23:02:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506733+0000", 2024-05-11T23:02:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2418999999999999e-05 2024-05-11T23:02:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506739+0000", 2024-05-11T23:02:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5149999999999997e-06 2024-05-11T23:02:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547229+0000", 2024-05-11T23:02:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040490809000000003 2024-05-11T23:02:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547238+0000", 2024-05-11T23:02:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7309999999999999e-06 2024-05-11T23:02:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31888 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506745+0000", 2024-05-11T23:02:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.368213416, 2024-05-11T23:02:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040530374000000001, 2024-05-11T23:02:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31888 2024-05-11T23:02:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506745+0000", 2024-05-11T23:02:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506745+0000", 2024-05-11T23:02:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506746+0000", 2024-05-11T23:02:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2599999999999998e-07 2024-05-11T23:02:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506748+0000", 2024-05-11T23:02:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2800000000000002e-06 2024-05-11T23:02:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506769+0000", 2024-05-11T23:02:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.137e-05 2024-05-11T23:02:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506779+0000", 2024-05-11T23:02:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4909999999999994e-06 2024-05-11T23:02:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547270+0000", 2024-05-11T23:02:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040491335000000003 2024-05-11T23:02:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547276+0000", 2024-05-11T23:02:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4720000000000003e-06 2024-05-11T23:02:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31889 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506788+0000", 2024-05-11T23:02:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.368170706000001, 2024-05-11T23:02:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040514673000000001, 2024-05-11T23:02:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31889 2024-05-11T23:02:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506788+0000", 2024-05-11T23:02:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506788+0000", 2024-05-11T23:02:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506789+0000", 2024-05-11T23:02:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1099999999999996e-07 2024-05-11T23:02:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506791+0000", 2024-05-11T23:02:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7030000000000002e-06 2024-05-11T23:02:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506799+0000", 2024-05-11T23:02:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1899999999999998e-06 2024-05-11T23:02:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506804+0000", 2024-05-11T23:02:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2549999999999997e-06 2024-05-11T23:02:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547294+0000", 2024-05-11T23:02:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040489875000000002 2024-05-11T23:02:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547303+0000", 2024-05-11T23:02:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1390000000000001e-06 2024-05-11T23:02:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31890 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.935 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506811+0000", 2024-05-11T23:02:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.368148186999999, 2024-05-11T23:02:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040521258999999997, 2024-05-11T23:02:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31890 2024-05-11T23:02:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506811+0000", 2024-05-11T23:02:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506811+0000", 2024-05-11T23:02:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506811+0000", 2024-05-11T23:02:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999998e-07 2024-05-11T23:02:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506816+0000", 2024-05-11T23:02:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9459999999999997e-06 2024-05-11T23:02:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506823+0000", 2024-05-11T23:02:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.153e-06 2024-05-11T23:02:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506840+0000", 2024-05-11T23:02:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6793e-05 2024-05-11T23:02:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547319+0000", 2024-05-11T23:02:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040478675999999998 2024-05-11T23:02:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547332+0000", 2024-05-11T23:02:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3236e-05 2024-05-11T23:02:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31891 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506865+0000", 2024-05-11T23:02:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.368093763000001, 2024-05-11T23:02:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040491271000000002, 2024-05-11T23:02:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.939 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31891 2024-05-11T23:02:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506865+0000", 2024-05-11T23:02:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506865+0000", 2024-05-11T23:02:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506866+0000", 2024-05-11T23:02:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0399999999999996e-07 2024-05-11T23:02:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506868+0000", 2024-05-11T23:02:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6979999999999998e-06 2024-05-11T23:02:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506876+0000", 2024-05-11T23:02:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2169999999999994e-06 2024-05-11T23:02:11.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506889+0000", 2024-05-11T23:02:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2497000000000001e-05 2024-05-11T23:02:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547350+0000", 2024-05-11T23:02:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040460782000000001 2024-05-11T23:02:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547356+0000", 2024-05-11T23:02:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5729999999999996e-06 2024-05-11T23:02:11.942 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31892 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506897+0000", 2024-05-11T23:02:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.368061313999998, 2024-05-11T23:02:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040501916999999998, 2024-05-11T23:02:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31892 2024-05-11T23:02:11.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506897+0000", 2024-05-11T23:02:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506897+0000", 2024-05-11T23:02:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506898+0000", 2024-05-11T23:02:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0400000000000002e-07 2024-05-11T23:02:11.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506900+0000", 2024-05-11T23:02:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.26e-06 2024-05-11T23:02:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506910+0000", 2024-05-11T23:02:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0237999999999999e-05 2024-05-11T23:02:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506916+0000", 2024-05-11T23:02:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6849999999999998e-06 2024-05-11T23:02:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547388+0000", 2024-05-11T23:02:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040472373999999998 2024-05-11T23:02:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547399+0000", 2024-05-11T23:02:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0956e-05 2024-05-11T23:02:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31893 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.946 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506923+0000", 2024-05-11T23:02:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.368035833, 2024-05-11T23:02:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040510939000000003, 2024-05-11T23:02:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31893 2024-05-11T23:02:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.947 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506923+0000", 2024-05-11T23:02:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506923+0000", 2024-05-11T23:02:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506923+0000", 2024-05-11T23:02:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2300000000000002e-07 2024-05-11T23:02:11.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506936+0000", 2024-05-11T23:02:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2639e-05 2024-05-11T23:02:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506949+0000", 2024-05-11T23:02:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2866e-05 2024-05-11T23:02:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506955+0000", 2024-05-11T23:02:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1530000000000002e-06 2024-05-11T23:02:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547425+0000", 2024-05-11T23:02:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040469752999999997 2024-05-11T23:02:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547434+0000", 2024-05-11T23:02:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1050000000000001e-06 2024-05-11T23:02:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31894 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506964+0000", 2024-05-11T23:02:11.950 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.367995214, 2024-05-11T23:02:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040492497000000002, 2024-05-11T23:02:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31894 2024-05-11T23:02:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506964+0000", 2024-05-11T23:02:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506964+0000", 2024-05-11T23:02:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506964+0000", 2024-05-11T23:02:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0100000000000005e-07 2024-05-11T23:02:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506967+0000", 2024-05-11T23:02:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.475e-06 2024-05-11T23:02:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506979+0000", 2024-05-11T23:02:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1847e-05 2024-05-11T23:02:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506984+0000", 2024-05-11T23:02:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7599999999999999e-06 2024-05-11T23:02:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547448+0000", 2024-05-11T23:02:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040463784000000003 2024-05-11T23:02:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:11.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547456+0000", 2024-05-11T23:02:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0299999999999994e-06 2024-05-11T23:02:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:11.954 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:11.954 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:11.956 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-05-11T23:02:11.956 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:02:11.997 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:11.997+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:12.046 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:02:12.090 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:12.090+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:12.113 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:12.113 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:12.113 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:02:12.113 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:12.123 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:02:12.177 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:12.177 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:12.177 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:12.177+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:12.202 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:12.219 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:12.219 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:12.236 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:12.236 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:12.254 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:12.254 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:12.268 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:12.268+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:12.272 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:12.278 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:12.278 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:12.287 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:02:12.289 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:12.290 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:12.309 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:12.309 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:12.369 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:12.369+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:12.400 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:12.400 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:12.409 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:02:12.470 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:12.470+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:12.519 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:12.519 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:12.528 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:02:12.572 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:12.572+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:12.637 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:12.637 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:12.646 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:02:12.674 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:12.673+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:12.775 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:12.774+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:12.795 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:12.795 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:12.804 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:02:12.876 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:12.875+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:12.918 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:12.918 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:12.927 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:02:13.011 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:13.010+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:13.077 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:13.077 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:13.078 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:13.077+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:13.086 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:02:13.178 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:13.177+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:13.236 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:13.237 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:13.245 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:02:13.286 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:13.285+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:13.380 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:13.379+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:13.395 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:13.395 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:13.404 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:02:13.489 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:13.487+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:13.553 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:13.553 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:13.563 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:02:13.582 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:13.582+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:13.683 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:13.683+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:13.712 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:13.713 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:13.721 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:02:13.784 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:13.783+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:13.838 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:13.838 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:13.847 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:02:13.884 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:13.884+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:13.952 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone25 from 0xe1b684c (overlap 0x379540b) to 0x58eecd 2024-05-11T23:02:13.956 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-11T23:02:13.960 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:13.960 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:13.974 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:02:14.018 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:14.018+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:14.022 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7] out_osds: [3, 6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:02:14.022 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:02:14.022 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-05-11T23:02:14.022 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-11T23:02:14.022 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-11T23:02:14.022 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-05-11T23:02:14.086 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:14.085+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:14.144 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:14.144 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:14.155 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:02:14.187 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:14.186+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:14.201 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:14.201 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "" 2024-05-11T23:02:14.201 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:14.288 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:14.287+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:14.307 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:14.307 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:14.316 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:02:14.321 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:02:14.346 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-05-11T23:02:14.388 INFO:teuthology.orchestra.run.smithi005.stdout:3323 read 0x3857dd2 thru 0x385ba11 (0x3c40 bytes) 2024-05-11T23:02:14.388 INFO:teuthology.orchestra.run.smithi005.stdout:3324 read 0xcb6c0ab thru 0xcb7a622 (0xe578 bytes) 2024-05-11T23:02:14.389 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:14.388+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:14.389 INFO:teuthology.orchestra.run.smithi005.stdout:3325 read 0x51862f8 thru 0x5191eb8 (0xbbc1 bytes) 2024-05-11T23:02:14.389 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:14.389 INFO:teuthology.orchestra.run.smithi005.stdout:3327 write 0xd01786b thru 0xd01d942 (0x60d8 bytes) 2024-05-11T23:02:14.396 INFO:teuthology.orchestra.run.smithi005.stdout:3328 write 0x42fe5db thru 0x4303656 (0x507c bytes) 2024-05-11T23:02:14.413 INFO:teuthology.orchestra.run.smithi005.stdout:3330 write 0xc0810ad thru 0xc0842ad (0x3201 bytes) 2024-05-11T23:02:14.415 INFO:teuthology.orchestra.run.smithi005.stdout:3331 writesame 0x825ef7 thru 0x82858b data_size 0x11(0x2695 bytes) 2024-05-11T23:02:14.429 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:14.429 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:14.431 INFO:teuthology.orchestra.run.smithi005.stdout:3332 punch from 0x3da0a7b to 0x3da7969, (0x6eee bytes) 2024-05-11T23:02:14.431 INFO:teuthology.orchestra.run.smithi005.stdout:3333 clone 27 order 24 su 4194304 sc 6 2024-05-11T23:02:14.438 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:02:14.461 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:14.461 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:14.478 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:14.495 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:14.495 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:14.522 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:14.522+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:14.582 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:14.583 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:14.591 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:02:14.592 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:14.591+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:14.692 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:14.691+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:14.714 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:14.714 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:14.731 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:14.732 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:14.738 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:14.738 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:14.747 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:02:14.749 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:14.750 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:14.766 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:14.784 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:14.784 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:14.792 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:14.792+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:14.852 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:14.853 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:14.853 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:14.853 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:14.853 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:14.861 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:02:14.893 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:14.892+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:14.927 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-05-11T23:02:14.994 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:14.993+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:15.019 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.364423772999999, 2024-05-11T23:02:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:02:15.019 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:02:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:02:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:02:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:02:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:02:15.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:02:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:02:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:02:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:02:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:02:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:02:15.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:02:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:02:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:02:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.361042584, 2024-05-11T23:02:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:02:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.023 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:02:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:02:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:02:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:02:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:02:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:02:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:02:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:02:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:02:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:02:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:02:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:02:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:02:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:02:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.361023877000001, 2024-05-11T23:02:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:02:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.027 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:02:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:02:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:02:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:02:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:02:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:02:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:02:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:02:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:02:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:02:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:02:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:02:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:02:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:02:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:02:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.361005733999999, 2024-05-11T23:02:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:02:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.031 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:02:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:02:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:02:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:02:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:02:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:02:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:02:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:02:15.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:02:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:02:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:02:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:02:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:02:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:02:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.034 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.360978246999998, 2024-05-11T23:02:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:02:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:02:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.035 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:02:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:02:15.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:02:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:02:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:02:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:02:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:02:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:02:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:02:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:02:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:02:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:02:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:02:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:02:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.360957719000002, 2024-05-11T23:02:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:02:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:02:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.039 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:02:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:02:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:02:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:02:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:02:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:02:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:02:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:02:15.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:02:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:02:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:02:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:02:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:02:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.360937944, 2024-05-11T23:02:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:02:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:02:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:15.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:02:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:02:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:02:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:02:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:02:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:02:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:02:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:02:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:02:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:02:15.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:02:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:02:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:02:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:02:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.046 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.360919138, 2024-05-11T23:02:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:02:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:02:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:02:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:02:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:02:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:02:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:02:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:02:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:02:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:02:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:02:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:02:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:02:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:02:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:02:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:02:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.360900215000001, 2024-05-11T23:02:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:02:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:02:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:02:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:02:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:02:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:02:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:02:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:02:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:02:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:02:15.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:02:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:02:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:02:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:02:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:02:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.054 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.360881552999999, 2024-05-11T23:02:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:02:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:02:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:02:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:02:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:02:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:02:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:02:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:02:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:02:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:02:15.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:02:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:02:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:02:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:02:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:02:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.360536391, 2024-05-11T23:02:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:02:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:02:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:02:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:02:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:02:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:02:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:02:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:02:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:02:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:02:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:02:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:02:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:02:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:02:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:02:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:02:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.062 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.360510841, 2024-05-11T23:02:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:02:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:02:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.063 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:02:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:02:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:02:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:02:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:02:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:02:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:02:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:02:15.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:02:15.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:02:15.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:02:15.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:02:15.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:02:15.066 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.066 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.066 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.067 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.067 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:15.067 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.360428583000001, 2024-05-11T23:02:15.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:02:15.067 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.067 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.067 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:15.067 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.067 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.067 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.067 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:02:15.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.067 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:15.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:15.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:15.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:02:15.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:02:15.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:02:15.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:02:15.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:02:15.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:02:15.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:02:15.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:02:15.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:02:15.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:02:15.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:02:15.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:02:15.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:02:15.070 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.070 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.070 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.071 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.071 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:15.071 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.360406136000002, 2024-05-11T23:02:15.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:02:15.071 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.071 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.071 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:15.071 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.071 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.071 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.071 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:02:15.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.071 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:15.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:15.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:02:15.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:02:15.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:02:15.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:02:15.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:02:15.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:02:15.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:02:15.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:02:15.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:02:15.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:02:15.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:02:15.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:02:15.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:02:15.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:02:15.074 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.075 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.075 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.075 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.075 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:15.075 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.360386678000001, 2024-05-11T23:02:15.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:02:15.075 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.075 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.075 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:15.075 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.075 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.075 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.075 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:02:15.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.076 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:15.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:15.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:15.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:02:15.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:02:15.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:02:15.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:02:15.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:02:15.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:02:15.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:02:15.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:02:15.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:02:15.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:02:15.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:02:15.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:02:15.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:02:15.079 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.079 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.079 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.079 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.079 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:15.079 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.360366618, 2024-05-11T23:02:15.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:02:15.079 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.079 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.079 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:15.079 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.079 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.079 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.079 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:02:15.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.080 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:15.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:15.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:15.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:02:15.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:02:15.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:02:15.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:02:15.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:02:15.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:02:15.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:02:15.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:02:15.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:02:15.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:02:15.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:02:15.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:02:15.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:02:15.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.083 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.083 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.083 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:15.083 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.360347348000001, 2024-05-11T23:02:15.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:02:15.083 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.083 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.083 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:15.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.084 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:02:15.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.084 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:15.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:15.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:15.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:02:15.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:02:15.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:02:15.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:02:15.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:02:15.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:02:15.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:02:15.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:02:15.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:02:15.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:02:15.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:02:15.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:02:15.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:02:15.087 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.087 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.087 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.087 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.087 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:15.087 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.359801281999999, 2024-05-11T23:02:15.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:02:15.087 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.087 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.088 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:15.088 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.088 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.088 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.088 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:02:15.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.088 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:15.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:15.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:02:15.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:02:15.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:02:15.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:02:15.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:02:15.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:02:15.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:02:15.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:02:15.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:02:15.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:02:15.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:02:15.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:02:15.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:02:15.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:02:15.091 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.091 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.091 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.091 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.091 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:15.092 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.359769497999999, 2024-05-11T23:02:15.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:02:15.092 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.092 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.092 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:15.092 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.092 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.092 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.092 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:02:15.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.092 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:15.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:15.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:15.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:02:15.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:02:15.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:02:15.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:02:15.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:02:15.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:02:15.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:02:15.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:02:15.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:02:15.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:02:15.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:02:15.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:02:15.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:02:15.095 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.095 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.095 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.095 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:02:15.095 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:15.096 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.922524034, 2024-05-11T23:02:15.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:02:15.096 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.096 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.096 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:15.096 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.096 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.096 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.096 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:02:15.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.096 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:15.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:15.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:02:15.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:02:15.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:02:15.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:02:15.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:02:15.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:02:15.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:02:15.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:02:15.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:02:15.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:02:15.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:02:15.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:02:15.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:02:15.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:02:15.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:02:15.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:15.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:02:15.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:02:15.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:15.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:02:15.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:02:15.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:15.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:02:15.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:02:15.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:02:15.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:02:15.100 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.100 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.101 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.101 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:15.102 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:02:15.103 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:15.102+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:15.203 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:15.202+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:15.208 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:15.208 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:15.208 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:15.208 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:15.217 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:02:15.304 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:15.304+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:15.361 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:15.362 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:15.362 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:15.362 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:15.362 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:15.370 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:02:15.411 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:15.411+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:15.506 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:15.505+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:15.517 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:15.517 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:15.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:15.517 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:15.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.518 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:02:15.518 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:15.518 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 58.353911236999998, 2024-05-11T23:02:15.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:02:15.518 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.518 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.518 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:15.518 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.518 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:02:15.518 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:02:15.518 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:02:15.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.518 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:15.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:15.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:02:15.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:02:15.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:02:15.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:02:15.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:02:15.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:02:15.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:02:15.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:02:15.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:02:15.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:02:15.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:02:15.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:02:15.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:02:15.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:02:15.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:02:15.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:15.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:02:15.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:02:15.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:02:15.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:02:15.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:02:15.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:15.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:02:15.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:02:15.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:15.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:02:15.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:02:15.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:15.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:02:15.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:02:15.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:02:15.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:02:15.523 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.523 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.523 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.524 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31872 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.524 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506353+0000", 2024-05-11T23:02:15.524 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.010810017000001, 2024-05-11T23:02:15.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040469576, 2024-05-11T23:02:15.524 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.524 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.524 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:15.524 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.524 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.524 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.524 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31872 2024-05-11T23:02:15.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.524 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506353+0000", 2024-05-11T23:02:15.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506353+0000", 2024-05-11T23:02:15.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506353+0000", 2024-05-11T23:02:15.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:02:15.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506356+0000", 2024-05-11T23:02:15.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0339999999999999e-06 2024-05-11T23:02:15.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506362+0000", 2024-05-11T23:02:15.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:02:15.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506367+0000", 2024-05-11T23:02:15.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3129999999999998e-06 2024-05-11T23:02:15.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546813+0000", 2024-05-11T23:02:15.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040445980999999999 2024-05-11T23:02:15.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546823+0000", 2024-05-11T23:02:15.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.7669999999999998e-06 2024-05-11T23:02:15.527 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.527 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.527 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.527 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31873 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.528 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506374+0000", 2024-05-11T23:02:15.528 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.010789478, 2024-05-11T23:02:15.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040466416999999998, 2024-05-11T23:02:15.528 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.528 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.528 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:15.528 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.528 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.528 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.528 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31873 2024-05-11T23:02:15.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.528 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506374+0000", 2024-05-11T23:02:15.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506374+0000", 2024-05-11T23:02:15.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506374+0000", 2024-05-11T23:02:15.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1600000000000002e-07 2024-05-11T23:02:15.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506376+0000", 2024-05-11T23:02:15.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.075e-06 2024-05-11T23:02:15.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506383+0000", 2024-05-11T23:02:15.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4189999999999999e-06 2024-05-11T23:02:15.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506388+0000", 2024-05-11T23:02:15.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.147e-06 2024-05-11T23:02:15.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546836+0000", 2024-05-11T23:02:15.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040448005000000002 2024-05-11T23:02:15.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546840+0000", 2024-05-11T23:02:15.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3549999999999998e-06 2024-05-11T23:02:15.531 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.531 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.531 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.531 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31874 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.531 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506394+0000", 2024-05-11T23:02:15.531 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.010769156999999, 2024-05-11T23:02:15.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040469948999999998, 2024-05-11T23:02:15.531 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.532 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.532 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:15.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.532 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31874 2024-05-11T23:02:15.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.532 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506394+0000", 2024-05-11T23:02:15.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506394+0000", 2024-05-11T23:02:15.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506394+0000", 2024-05-11T23:02:15.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0900000000000002e-07 2024-05-11T23:02:15.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506396+0000", 2024-05-11T23:02:15.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0669999999999999e-06 2024-05-11T23:02:15.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506403+0000", 2024-05-11T23:02:15.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4509999999999997e-06 2024-05-11T23:02:15.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506408+0000", 2024-05-11T23:02:15.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0810000000000001e-06 2024-05-11T23:02:15.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546855+0000", 2024-05-11T23:02:15.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040446825999999998 2024-05-11T23:02:15.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546864+0000", 2024-05-11T23:02:15.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.115e-06 2024-05-11T23:02:15.534 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.535 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.535 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.535 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31878 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.535 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506510+0000", 2024-05-11T23:02:15.535 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.010653269999999, 2024-05-11T23:02:15.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040458415999999997, 2024-05-11T23:02:15.535 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.535 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.535 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:15.535 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.535 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.535 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.535 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31878 2024-05-11T23:02:15.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.536 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506510+0000", 2024-05-11T23:02:15.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506510+0000", 2024-05-11T23:02:15.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506510+0000", 2024-05-11T23:02:15.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6100000000000002e-07 2024-05-11T23:02:15.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506512+0000", 2024-05-11T23:02:15.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0470000000000001e-06 2024-05-11T23:02:15.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506519+0000", 2024-05-11T23:02:15.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3130000000000004e-06 2024-05-11T23:02:15.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506523+0000", 2024-05-11T23:02:15.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6999999999999999e-06 2024-05-11T23:02:15.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546959+0000", 2024-05-11T23:02:15.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040436110999999997 2024-05-11T23:02:15.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546968+0000", 2024-05-11T23:02:15.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.884e-06 2024-05-11T23:02:15.539 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.540 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.540 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.540 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31880 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.540 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506550+0000", 2024-05-11T23:02:15.540 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.010613395, 2024-05-11T23:02:15.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040454849000000001, 2024-05-11T23:02:15.540 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.540 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.540 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:15.540 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.540 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.540 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.540 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31880 2024-05-11T23:02:15.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.541 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506550+0000", 2024-05-11T23:02:15.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506550+0000", 2024-05-11T23:02:15.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506550+0000", 2024-05-11T23:02:15.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7300000000000002e-07 2024-05-11T23:02:15.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506552+0000", 2024-05-11T23:02:15.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.012e-06 2024-05-11T23:02:15.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506558+0000", 2024-05-11T23:02:15.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:02:15.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506563+0000", 2024-05-11T23:02:15.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0000000000000004e-06 2024-05-11T23:02:15.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.546997+0000", 2024-05-11T23:02:15.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040433416 2024-05-11T23:02:15.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547005+0000", 2024-05-11T23:02:15.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9440000000000005e-06 2024-05-11T23:02:15.543 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.543 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.543 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.543 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31881 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.544 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506570+0000", 2024-05-11T23:02:15.544 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.010593185000001, 2024-05-11T23:02:15.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040460073999999999, 2024-05-11T23:02:15.544 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.544 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.544 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:15.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.544 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31881 2024-05-11T23:02:15.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.544 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506570+0000", 2024-05-11T23:02:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506570+0000", 2024-05-11T23:02:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506570+0000", 2024-05-11T23:02:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:02:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506577+0000", 2024-05-11T23:02:15.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0099999999999998e-06 2024-05-11T23:02:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506584+0000", 2024-05-11T23:02:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7689999999999999e-06 2024-05-11T23:02:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506589+0000", 2024-05-11T23:02:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2340000000000002e-06 2024-05-11T23:02:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547022+0000", 2024-05-11T23:02:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040432526000000003 2024-05-11T23:02:15.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547030+0000", 2024-05-11T23:02:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1650000000000006e-06 2024-05-11T23:02:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31882 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506596+0000", 2024-05-11T23:02:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.010566964999999, 2024-05-11T23:02:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040473955999999998, 2024-05-11T23:02:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.547 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31882 2024-05-11T23:02:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506596+0000", 2024-05-11T23:02:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506596+0000", 2024-05-11T23:02:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506597+0000", 2024-05-11T23:02:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5299999999999999e-07 2024-05-11T23:02:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506599+0000", 2024-05-11T23:02:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2900000000000001e-06 2024-05-11T23:02:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506605+0000", 2024-05-11T23:02:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5259999999999999e-06 2024-05-11T23:02:15.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506610+0000", 2024-05-11T23:02:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1030000000000001e-06 2024-05-11T23:02:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547061+0000", 2024-05-11T23:02:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040450451999999998 2024-05-11T23:02:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547070+0000", 2024-05-11T23:02:15.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.132e-06 2024-05-11T23:02:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31883 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506618+0000", 2024-05-11T23:02:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.010545151999999, 2024-05-11T23:02:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040494963000000002, 2024-05-11T23:02:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.551 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31883 2024-05-11T23:02:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506618+0000", 2024-05-11T23:02:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506618+0000", 2024-05-11T23:02:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506619+0000", 2024-05-11T23:02:15.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1799999999999995e-07 2024-05-11T23:02:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506621+0000", 2024-05-11T23:02:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2879999999999999e-06 2024-05-11T23:02:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506628+0000", 2024-05-11T23:02:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9600000000000003e-06 2024-05-11T23:02:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506633+0000", 2024-05-11T23:02:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4229999999999996e-06 2024-05-11T23:02:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547102+0000", 2024-05-11T23:02:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040469172999999997 2024-05-11T23:02:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547113+0000", 2024-05-11T23:02:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0501e-05 2024-05-11T23:02:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31884 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506640+0000", 2024-05-11T23:02:15.554 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.010522870999999, 2024-05-11T23:02:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040498244000000003, 2024-05-11T23:02:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31884 2024-05-11T23:02:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506640+0000", 2024-05-11T23:02:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506640+0000", 2024-05-11T23:02:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506641+0000", 2024-05-11T23:02:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3000000000000001e-07 2024-05-11T23:02:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506643+0000", 2024-05-11T23:02:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0200000000000001e-06 2024-05-11T23:02:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506649+0000", 2024-05-11T23:02:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1689999999999997e-06 2024-05-11T23:02:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506654+0000", 2024-05-11T23:02:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3800000000000002e-06 2024-05-11T23:02:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547131+0000", 2024-05-11T23:02:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040476810000000002 2024-05-11T23:02:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547138+0000", 2024-05-11T23:02:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.435e-06 2024-05-11T23:02:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31885 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506660+0000", 2024-05-11T23:02:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.010502696, 2024-05-11T23:02:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040504498, 2024-05-11T23:02:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.558 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31885 2024-05-11T23:02:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506660+0000", 2024-05-11T23:02:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506660+0000", 2024-05-11T23:02:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506661+0000", 2024-05-11T23:02:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4900000000000001e-07 2024-05-11T23:02:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506663+0000", 2024-05-11T23:02:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0660000000000002e-06 2024-05-11T23:02:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506669+0000", 2024-05-11T23:02:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3829999999999997e-06 2024-05-11T23:02:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506675+0000", 2024-05-11T23:02:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3750000000000002e-06 2024-05-11T23:02:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547156+0000", 2024-05-11T23:02:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040481389 2024-05-11T23:02:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547165+0000", 2024-05-11T23:02:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8359999999999998e-06 2024-05-11T23:02:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31886 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506681+0000", 2024-05-11T23:02:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.010482266, 2024-05-11T23:02:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040511004000000003, 2024-05-11T23:02:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31886 2024-05-11T23:02:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506681+0000", 2024-05-11T23:02:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506681+0000", 2024-05-11T23:02:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506681+0000", 2024-05-11T23:02:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:02:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506683+0000", 2024-05-11T23:02:15.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.058e-06 2024-05-11T23:02:15.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506703+0000", 2024-05-11T23:02:15.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0224000000000002e-05 2024-05-11T23:02:15.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506709+0000", 2024-05-11T23:02:15.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5799999999999999e-06 2024-05-11T23:02:15.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547183+0000", 2024-05-11T23:02:15.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040474060999999999 2024-05-11T23:02:15.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547192+0000", 2024-05-11T23:02:15.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6910000000000003e-06 2024-05-11T23:02:15.565 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.565 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.565 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.565 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31887 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.565 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506716+0000", 2024-05-11T23:02:15.565 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.010447443, 2024-05-11T23:02:15.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040522425000000001, 2024-05-11T23:02:15.565 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.565 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.565 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:15.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.566 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31887 2024-05-11T23:02:15.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.566 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506716+0000", 2024-05-11T23:02:15.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506716+0000", 2024-05-11T23:02:15.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506716+0000", 2024-05-11T23:02:15.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8799999999999998e-07 2024-05-11T23:02:15.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506721+0000", 2024-05-11T23:02:15.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5630000000000002e-06 2024-05-11T23:02:15.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506733+0000", 2024-05-11T23:02:15.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2418999999999999e-05 2024-05-11T23:02:15.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506739+0000", 2024-05-11T23:02:15.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5149999999999997e-06 2024-05-11T23:02:15.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547229+0000", 2024-05-11T23:02:15.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040490809000000003 2024-05-11T23:02:15.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547238+0000", 2024-05-11T23:02:15.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7309999999999999e-06 2024-05-11T23:02:15.568 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.568 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.568 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.569 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31888 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.569 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506745+0000", 2024-05-11T23:02:15.569 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.010417704000002, 2024-05-11T23:02:15.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040530374000000001, 2024-05-11T23:02:15.569 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.569 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.569 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:15.569 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.569 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.569 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.569 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31888 2024-05-11T23:02:15.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.569 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506745+0000", 2024-05-11T23:02:15.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506745+0000", 2024-05-11T23:02:15.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506746+0000", 2024-05-11T23:02:15.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2599999999999998e-07 2024-05-11T23:02:15.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506748+0000", 2024-05-11T23:02:15.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2800000000000002e-06 2024-05-11T23:02:15.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506769+0000", 2024-05-11T23:02:15.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.137e-05 2024-05-11T23:02:15.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506779+0000", 2024-05-11T23:02:15.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4909999999999994e-06 2024-05-11T23:02:15.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547270+0000", 2024-05-11T23:02:15.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040491335000000003 2024-05-11T23:02:15.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547276+0000", 2024-05-11T23:02:15.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4720000000000003e-06 2024-05-11T23:02:15.572 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.572 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.572 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.572 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31889 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.572 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506788+0000", 2024-05-11T23:02:15.572 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.010374993999999, 2024-05-11T23:02:15.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040514673000000001, 2024-05-11T23:02:15.572 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.572 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.573 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:15.573 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.573 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.573 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.573 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31889 2024-05-11T23:02:15.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.573 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506788+0000", 2024-05-11T23:02:15.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506788+0000", 2024-05-11T23:02:15.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506789+0000", 2024-05-11T23:02:15.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1099999999999996e-07 2024-05-11T23:02:15.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506791+0000", 2024-05-11T23:02:15.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7030000000000002e-06 2024-05-11T23:02:15.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506799+0000", 2024-05-11T23:02:15.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1899999999999998e-06 2024-05-11T23:02:15.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506804+0000", 2024-05-11T23:02:15.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2549999999999997e-06 2024-05-11T23:02:15.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547294+0000", 2024-05-11T23:02:15.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040489875000000002 2024-05-11T23:02:15.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547303+0000", 2024-05-11T23:02:15.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1390000000000001e-06 2024-05-11T23:02:15.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.575 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.576 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.576 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31890 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.576 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506811+0000", 2024-05-11T23:02:15.576 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.010352475000001, 2024-05-11T23:02:15.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040521258999999997, 2024-05-11T23:02:15.576 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.576 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.576 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:15.576 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.576 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.576 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.576 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31890 2024-05-11T23:02:15.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.576 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506811+0000", 2024-05-11T23:02:15.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506811+0000", 2024-05-11T23:02:15.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506811+0000", 2024-05-11T23:02:15.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999998e-07 2024-05-11T23:02:15.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506816+0000", 2024-05-11T23:02:15.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9459999999999997e-06 2024-05-11T23:02:15.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506823+0000", 2024-05-11T23:02:15.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.153e-06 2024-05-11T23:02:15.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506840+0000", 2024-05-11T23:02:15.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6793e-05 2024-05-11T23:02:15.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547319+0000", 2024-05-11T23:02:15.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040478675999999998 2024-05-11T23:02:15.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547332+0000", 2024-05-11T23:02:15.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3236e-05 2024-05-11T23:02:15.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.579 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.579 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31891 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.579 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506865+0000", 2024-05-11T23:02:15.579 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.010298050999999, 2024-05-11T23:02:15.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040491271000000002, 2024-05-11T23:02:15.579 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.579 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.580 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:15.580 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.580 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.580 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.580 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31891 2024-05-11T23:02:15.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.580 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506865+0000", 2024-05-11T23:02:15.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506865+0000", 2024-05-11T23:02:15.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506866+0000", 2024-05-11T23:02:15.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0399999999999996e-07 2024-05-11T23:02:15.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506868+0000", 2024-05-11T23:02:15.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6979999999999998e-06 2024-05-11T23:02:15.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506876+0000", 2024-05-11T23:02:15.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2169999999999994e-06 2024-05-11T23:02:15.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506889+0000", 2024-05-11T23:02:15.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2497000000000001e-05 2024-05-11T23:02:15.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547350+0000", 2024-05-11T23:02:15.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040460782000000001 2024-05-11T23:02:15.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547356+0000", 2024-05-11T23:02:15.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5729999999999996e-06 2024-05-11T23:02:15.583 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.583 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.583 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.583 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31892 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.583 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506897+0000", 2024-05-11T23:02:15.583 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.010265602, 2024-05-11T23:02:15.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040501916999999998, 2024-05-11T23:02:15.583 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.583 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.583 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:15.583 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.583 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.583 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.583 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31892 2024-05-11T23:02:15.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.584 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506897+0000", 2024-05-11T23:02:15.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506897+0000", 2024-05-11T23:02:15.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506898+0000", 2024-05-11T23:02:15.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0400000000000002e-07 2024-05-11T23:02:15.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506900+0000", 2024-05-11T23:02:15.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.26e-06 2024-05-11T23:02:15.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506910+0000", 2024-05-11T23:02:15.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0237999999999999e-05 2024-05-11T23:02:15.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506916+0000", 2024-05-11T23:02:15.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6849999999999998e-06 2024-05-11T23:02:15.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547388+0000", 2024-05-11T23:02:15.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040472373999999998 2024-05-11T23:02:15.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547399+0000", 2024-05-11T23:02:15.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0956e-05 2024-05-11T23:02:15.586 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.586 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.587 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31893 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.587 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506923+0000", 2024-05-11T23:02:15.587 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.010240120999999, 2024-05-11T23:02:15.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040510939000000003, 2024-05-11T23:02:15.587 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.587 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.587 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:15.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.587 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31893 2024-05-11T23:02:15.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.588 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506923+0000", 2024-05-11T23:02:15.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506923+0000", 2024-05-11T23:02:15.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506923+0000", 2024-05-11T23:02:15.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2300000000000002e-07 2024-05-11T23:02:15.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506936+0000", 2024-05-11T23:02:15.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2639e-05 2024-05-11T23:02:15.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506949+0000", 2024-05-11T23:02:15.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2866e-05 2024-05-11T23:02:15.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506955+0000", 2024-05-11T23:02:15.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1530000000000002e-06 2024-05-11T23:02:15.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547425+0000", 2024-05-11T23:02:15.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040469752999999997 2024-05-11T23:02:15.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547434+0000", 2024-05-11T23:02:15.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1050000000000001e-06 2024-05-11T23:02:15.590 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.590 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.590 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.590 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:31894 3.f 3.3e04496f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:15.591 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.506964+0000", 2024-05-11T23:02:15.591 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.010199501999999, 2024-05-11T23:02:15.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040492497000000002, 2024-05-11T23:02:15.591 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:15.591 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:15.591 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:15.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:15.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:15.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:15.591 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 31894 2024-05-11T23:02:15.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.591 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:15.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:15.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506964+0000", 2024-05-11T23:02:15.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:15.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506964+0000", 2024-05-11T23:02:15.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:15.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:15.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506964+0000", 2024-05-11T23:02:15.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0100000000000005e-07 2024-05-11T23:02:15.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:15.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506967+0000", 2024-05-11T23:02:15.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.475e-06 2024-05-11T23:02:15.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:15.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506979+0000", 2024-05-11T23:02:15.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1847e-05 2024-05-11T23:02:15.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:15.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.506984+0000", 2024-05-11T23:02:15.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7599999999999999e-06 2024-05-11T23:02:15.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:15.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547448+0000", 2024-05-11T23:02:15.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.040463784000000003 2024-05-11T23:02:15.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:15.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:15.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:15.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.547456+0000", 2024-05-11T23:02:15.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0299999999999994e-06 2024-05-11T23:02:15.594 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.594 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.594 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.594 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:15.594 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:15.594 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:15.595 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:02:15.640 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:15.639+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:15.708 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:15.707+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:15.715 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:15.716 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:15.716 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:02:15.716 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:15.725 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:02:15.808 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:15.808+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:15.876 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:15.876 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:15.876 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:15.876 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:15.876 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:15.885 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:02:15.928 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:15.927+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:15.932 INFO:teuthology.orchestra.run.smithi005.stdout:flattening image image_client.0-clone26 2024-05-11T23:02:16.010 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:16.008+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:16.047 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:16.047 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:16.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:16.048 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:16.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.048 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71307 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:16.048 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.146873+0000", 2024-05-11T23:02:16.048 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.8995329600000002, 2024-05-11T23:02:16.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030836779000000002, 2024-05-11T23:02:16.048 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.048 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.048 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:16.048 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.048 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.048 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.048 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71307 2024-05-11T23:02:16.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.048 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146873+0000", 2024-05-11T23:02:16.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146873+0000", 2024-05-11T23:02:16.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146873+0000", 2024-05-11T23:02:16.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-05-11T23:02:16.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146874+0000", 2024-05-11T23:02:16.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7300000000000004e-07 2024-05-11T23:02:16.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146877+0000", 2024-05-11T23:02:16.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1250000000000001e-06 2024-05-11T23:02:16.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146880+0000", 2024-05-11T23:02:16.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5119999999999998e-06 2024-05-11T23:02:16.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177704+0000", 2024-05-11T23:02:16.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030824444999999999 2024-05-11T23:02:16.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177710+0000", 2024-05-11T23:02:16.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5130000000000004e-06 2024-05-11T23:02:16.051 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.052 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.052 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.052 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71308 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:16.052 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.146883+0000", 2024-05-11T23:02:16.052 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.8995228429999997, 2024-05-11T23:02:16.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030851522999999999, 2024-05-11T23:02:16.053 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.053 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:16.053 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.053 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.053 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.053 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71308 2024-05-11T23:02:16.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.053 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146883+0000", 2024-05-11T23:02:16.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146883+0000", 2024-05-11T23:02:16.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146884+0000", 2024-05-11T23:02:16.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3399999999999999e-07 2024-05-11T23:02:16.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146893+0000", 2024-05-11T23:02:16.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2159999999999995e-06 2024-05-11T23:02:16.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146899+0000", 2024-05-11T23:02:16.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.302e-06 2024-05-11T23:02:16.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146905+0000", 2024-05-11T23:02:16.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9120000000000003e-06 2024-05-11T23:02:16.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177730+0000", 2024-05-11T23:02:16.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030824449 2024-05-11T23:02:16.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177735+0000", 2024-05-11T23:02:16.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1100000000000002e-06 2024-05-11T23:02:16.056 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.056 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.056 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.056 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71309 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:16.056 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.146921+0000", 2024-05-11T23:02:16.056 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.8994845810000003, 2024-05-11T23:02:16.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030854467999999999, 2024-05-11T23:02:16.056 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.056 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.056 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:16.056 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.057 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.057 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.057 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71309 2024-05-11T23:02:16.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.057 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146921+0000", 2024-05-11T23:02:16.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146921+0000", 2024-05-11T23:02:16.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146922+0000", 2024-05-11T23:02:16.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1799999999999995e-07 2024-05-11T23:02:16.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146924+0000", 2024-05-11T23:02:16.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5009999999999999e-06 2024-05-11T23:02:16.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146938+0000", 2024-05-11T23:02:16.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3286e-05 2024-05-11T23:02:16.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146944+0000", 2024-05-11T23:02:16.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7839999999999998e-06 2024-05-11T23:02:16.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177766+0000", 2024-05-11T23:02:16.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030821988000000002 2024-05-11T23:02:16.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177776+0000", 2024-05-11T23:02:16.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3910000000000004e-06 2024-05-11T23:02:16.060 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.060 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.060 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.060 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71310 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:16.060 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.146954+0000", 2024-05-11T23:02:16.061 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.8994522439999999, 2024-05-11T23:02:16.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030837251999999999, 2024-05-11T23:02:16.061 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.061 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.061 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:16.061 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.061 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.061 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.061 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71310 2024-05-11T23:02:16.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.061 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146954+0000", 2024-05-11T23:02:16.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146954+0000", 2024-05-11T23:02:16.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146954+0000", 2024-05-11T23:02:16.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:02:16.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146957+0000", 2024-05-11T23:02:16.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.463e-06 2024-05-11T23:02:16.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146963+0000", 2024-05-11T23:02:16.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6460000000000004e-06 2024-05-11T23:02:16.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146968+0000", 2024-05-11T23:02:16.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0250000000000002e-06 2024-05-11T23:02:16.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177787+0000", 2024-05-11T23:02:16.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030818680000000001 2024-05-11T23:02:16.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177791+0000", 2024-05-11T23:02:16.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0090000000000001e-06 2024-05-11T23:02:16.064 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.064 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.064 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.064 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71313 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:16.064 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147016+0000", 2024-05-11T23:02:16.065 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.8993895619999996, 2024-05-11T23:02:16.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030841770000000001, 2024-05-11T23:02:16.065 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.065 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.065 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:16.065 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.065 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.065 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.065 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71313 2024-05-11T23:02:16.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.065 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147016+0000", 2024-05-11T23:02:16.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147016+0000", 2024-05-11T23:02:16.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147017+0000", 2024-05-11T23:02:16.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8599999999999999e-07 2024-05-11T23:02:16.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147019+0000", 2024-05-11T23:02:16.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0329999999999998e-06 2024-05-11T23:02:16.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147025+0000", 2024-05-11T23:02:16.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0299999999999999e-06 2024-05-11T23:02:16.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147029+0000", 2024-05-11T23:02:16.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5979999999999999e-06 2024-05-11T23:02:16.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177848+0000", 2024-05-11T23:02:16.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030818662 2024-05-11T23:02:16.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177858+0000", 2024-05-11T23:02:16.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0061000000000001e-05 2024-05-11T23:02:16.068 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.068 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.068 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.068 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71327 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:16.068 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147299+0000", 2024-05-11T23:02:16.068 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.8991068740000001, 2024-05-11T23:02:16.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030857026999999999, 2024-05-11T23:02:16.069 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.069 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.069 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:16.069 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.069 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.069 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.069 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71327 2024-05-11T23:02:16.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.069 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147299+0000", 2024-05-11T23:02:16.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147299+0000", 2024-05-11T23:02:16.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147299+0000", 2024-05-11T23:02:16.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1500000000000001e-07 2024-05-11T23:02:16.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147303+0000", 2024-05-11T23:02:16.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1459999999999998e-06 2024-05-11T23:02:16.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147312+0000", 2024-05-11T23:02:16.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3299999999999999e-06 2024-05-11T23:02:16.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147321+0000", 2024-05-11T23:02:16.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7420000000000003e-06 2024-05-11T23:02:16.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178145+0000", 2024-05-11T23:02:16.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030824852999999999 2024-05-11T23:02:16.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178156+0000", 2024-05-11T23:02:16.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0740999999999999e-05 2024-05-11T23:02:16.072 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.072 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.072 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.072 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71328 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:16.072 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147326+0000", 2024-05-11T23:02:16.072 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.8990803920000001, 2024-05-11T23:02:16.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030846630999999999, 2024-05-11T23:02:16.072 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.072 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.072 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:16.073 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.073 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.073 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.073 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71328 2024-05-11T23:02:16.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.073 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147326+0000", 2024-05-11T23:02:16.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147326+0000", 2024-05-11T23:02:16.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147326+0000", 2024-05-11T23:02:16.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5400000000000002e-07 2024-05-11T23:02:16.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147327+0000", 2024-05-11T23:02:16.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2500000000000001e-06 2024-05-11T23:02:16.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147331+0000", 2024-05-11T23:02:16.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5499999999999999e-06 2024-05-11T23:02:16.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147333+0000", 2024-05-11T23:02:16.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5450000000000002e-06 2024-05-11T23:02:16.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178168+0000", 2024-05-11T23:02:16.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030834934000000001 2024-05-11T23:02:16.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178172+0000", 2024-05-11T23:02:16.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0980000000000004e-06 2024-05-11T23:02:16.075 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.075 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.075 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.076 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71329 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:16.076 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147336+0000", 2024-05-11T23:02:16.076 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.899069538, 2024-05-11T23:02:16.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030865317999999999, 2024-05-11T23:02:16.076 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.076 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.076 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:16.076 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.076 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.076 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.076 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71329 2024-05-11T23:02:16.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.076 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147336+0000", 2024-05-11T23:02:16.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147336+0000", 2024-05-11T23:02:16.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147337+0000", 2024-05-11T23:02:16.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:16.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147338+0000", 2024-05-11T23:02:16.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5090000000000001e-06 2024-05-11T23:02:16.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147344+0000", 2024-05-11T23:02:16.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.66e-06 2024-05-11T23:02:16.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147348+0000", 2024-05-11T23:02:16.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2980000000000002e-06 2024-05-11T23:02:16.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178192+0000", 2024-05-11T23:02:16.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030843425000000001 2024-05-11T23:02:16.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178202+0000", 2024-05-11T23:02:16.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0217000000000001e-05 2024-05-11T23:02:16.079 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.079 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.079 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.079 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71330 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:16.079 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147355+0000", 2024-05-11T23:02:16.079 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.8990513839999998, 2024-05-11T23:02:16.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030862031000000002, 2024-05-11T23:02:16.080 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.080 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.080 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:16.080 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.080 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.080 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.080 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71330 2024-05-11T23:02:16.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.080 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147355+0000", 2024-05-11T23:02:16.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147355+0000", 2024-05-11T23:02:16.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147355+0000", 2024-05-11T23:02:16.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.39e-07 2024-05-11T23:02:16.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147365+0000", 2024-05-11T23:02:16.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7140000000000005e-06 2024-05-11T23:02:16.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147370+0000", 2024-05-11T23:02:16.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2979999999999999e-06 2024-05-11T23:02:16.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147373+0000", 2024-05-11T23:02:16.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.41e-06 2024-05-11T23:02:16.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178213+0000", 2024-05-11T23:02:16.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030839325000000001 2024-05-11T23:02:16.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178217+0000", 2024-05-11T23:02:16.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9450000000000003e-06 2024-05-11T23:02:16.083 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.083 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.083 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.083 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71331 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:16.083 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147391+0000", 2024-05-11T23:02:16.083 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.8990144630000003, 2024-05-11T23:02:16.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030863854999999999, 2024-05-11T23:02:16.083 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.083 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.083 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:16.084 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.084 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.084 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71331 2024-05-11T23:02:16.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.084 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147391+0000", 2024-05-11T23:02:16.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147391+0000", 2024-05-11T23:02:16.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147392+0000", 2024-05-11T23:02:16.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.68e-07 2024-05-11T23:02:16.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147395+0000", 2024-05-11T23:02:16.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.503e-06 2024-05-11T23:02:16.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147401+0000", 2024-05-11T23:02:16.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7889999999999997e-06 2024-05-11T23:02:16.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147406+0000", 2024-05-11T23:02:16.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.831e-06 2024-05-11T23:02:16.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178245+0000", 2024-05-11T23:02:16.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030838884 2024-05-11T23:02:16.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178255+0000", 2024-05-11T23:02:16.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.028e-05 2024-05-11T23:02:16.087 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.087 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.087 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.087 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71332 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:16.087 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147414+0000", 2024-05-11T23:02:16.087 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.8989924120000001, 2024-05-11T23:02:16.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030874374, 2024-05-11T23:02:16.087 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.087 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.087 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:16.087 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.087 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.087 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.088 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71332 2024-05-11T23:02:16.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.088 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147414+0000", 2024-05-11T23:02:16.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147414+0000", 2024-05-11T23:02:16.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147414+0000", 2024-05-11T23:02:16.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6899999999999998e-07 2024-05-11T23:02:16.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147416+0000", 2024-05-11T23:02:16.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:02:16.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147428+0000", 2024-05-11T23:02:16.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1428e-05 2024-05-11T23:02:16.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147436+0000", 2024-05-11T23:02:16.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9689999999999994e-06 2024-05-11T23:02:16.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178283+0000", 2024-05-11T23:02:16.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030847764 2024-05-11T23:02:16.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178288+0000", 2024-05-11T23:02:16.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6430000000000003e-06 2024-05-11T23:02:16.090 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.090 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.090 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.091 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71333 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:16.091 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147444+0000", 2024-05-11T23:02:16.091 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.8989624310000002, 2024-05-11T23:02:16.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.03086906, 2024-05-11T23:02:16.091 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.091 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.091 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:16.091 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.091 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.091 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.091 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71333 2024-05-11T23:02:16.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.091 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147444+0000", 2024-05-11T23:02:16.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147444+0000", 2024-05-11T23:02:16.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147444+0000", 2024-05-11T23:02:16.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:02:16.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147446+0000", 2024-05-11T23:02:16.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3590000000000002e-06 2024-05-11T23:02:16.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147453+0000", 2024-05-11T23:02:16.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7580000000000003e-06 2024-05-11T23:02:16.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147460+0000", 2024-05-11T23:02:16.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5010000000000001e-06 2024-05-11T23:02:16.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178308+0000", 2024-05-11T23:02:16.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030848179 2024-05-11T23:02:16.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178313+0000", 2024-05-11T23:02:16.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8219999999999997e-06 2024-05-11T23:02:16.094 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.094 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.094 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.094 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71334 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:16.094 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147466+0000", 2024-05-11T23:02:16.094 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.8989397610000003, 2024-05-11T23:02:16.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030868219999999998, 2024-05-11T23:02:16.095 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.095 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.095 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:16.095 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.095 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.095 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.095 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71334 2024-05-11T23:02:16.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.095 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147466+0000", 2024-05-11T23:02:16.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147466+0000", 2024-05-11T23:02:16.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147467+0000", 2024-05-11T23:02:16.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9099999999999999e-07 2024-05-11T23:02:16.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147469+0000", 2024-05-11T23:02:16.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9769999999999999e-06 2024-05-11T23:02:16.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147475+0000", 2024-05-11T23:02:16.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1369999999999999e-06 2024-05-11T23:02:16.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147479+0000", 2024-05-11T23:02:16.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6639999999999997e-06 2024-05-11T23:02:16.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178330+0000", 2024-05-11T23:02:16.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030850776 2024-05-11T23:02:16.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178334+0000", 2024-05-11T23:02:16.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2749999999999997e-06 2024-05-11T23:02:16.098 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.098 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.098 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.098 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71335 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:16.098 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147489+0000", 2024-05-11T23:02:16.098 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.898917204, 2024-05-11T23:02:16.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030866874999999998, 2024-05-11T23:02:16.098 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.098 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.098 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:16.098 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.098 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.099 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.099 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71335 2024-05-11T23:02:16.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.099 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147489+0000", 2024-05-11T23:02:16.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147489+0000", 2024-05-11T23:02:16.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147489+0000", 2024-05-11T23:02:16.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7899999999999999e-07 2024-05-11T23:02:16.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147491+0000", 2024-05-11T23:02:16.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0600000000000002e-06 2024-05-11T23:02:16.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147498+0000", 2024-05-11T23:02:16.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3339999999999998e-06 2024-05-11T23:02:16.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147502+0000", 2024-05-11T23:02:16.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8629999999999999e-06 2024-05-11T23:02:16.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178350+0000", 2024-05-11T23:02:16.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030847795000000001 2024-05-11T23:02:16.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178356+0000", 2024-05-11T23:02:16.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4439999999999999e-06 2024-05-11T23:02:16.101 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.101 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.101 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.102 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71336 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:16.102 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147508+0000", 2024-05-11T23:02:16.102 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.898897463, 2024-05-11T23:02:16.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030867196999999999, 2024-05-11T23:02:16.102 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.102 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.102 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:16.102 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.102 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.102 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.102 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71336 2024-05-11T23:02:16.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.102 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147508+0000", 2024-05-11T23:02:16.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147508+0000", 2024-05-11T23:02:16.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147509+0000", 2024-05-11T23:02:16.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:02:16.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147511+0000", 2024-05-11T23:02:16.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.88e-06 2024-05-11T23:02:16.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147517+0000", 2024-05-11T23:02:16.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7849999999999997e-06 2024-05-11T23:02:16.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147521+0000", 2024-05-11T23:02:16.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4240000000000003e-06 2024-05-11T23:02:16.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178371+0000", 2024-05-11T23:02:16.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030850452 2024-05-11T23:02:16.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178376+0000", 2024-05-11T23:02:16.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2560000000000004e-06 2024-05-11T23:02:16.105 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.105 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.105 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.105 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71337 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:16.105 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147527+0000", 2024-05-11T23:02:16.106 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.8988788660000004, 2024-05-11T23:02:16.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030869829000000001, 2024-05-11T23:02:16.106 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.106 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:16.106 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.106 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.106 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.106 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71337 2024-05-11T23:02:16.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.106 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147527+0000", 2024-05-11T23:02:16.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147527+0000", 2024-05-11T23:02:16.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147527+0000", 2024-05-11T23:02:16.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3099999999999999e-07 2024-05-11T23:02:16.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147529+0000", 2024-05-11T23:02:16.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8870000000000001e-06 2024-05-11T23:02:16.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147536+0000", 2024-05-11T23:02:16.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2910000000000004e-06 2024-05-11T23:02:16.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147540+0000", 2024-05-11T23:02:16.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8189999999999999e-06 2024-05-11T23:02:16.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178393+0000", 2024-05-11T23:02:16.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030852118000000001 2024-05-11T23:02:16.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178397+0000", 2024-05-11T23:02:16.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3830000000000002e-06 2024-05-11T23:02:16.109 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.109 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.109 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.109 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71338 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:16.109 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147546+0000", 2024-05-11T23:02:16.109 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.8988596070000003, 2024-05-11T23:02:16.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030872143000000001, 2024-05-11T23:02:16.109 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.109 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:16.110 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.110 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.110 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.110 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71338 2024-05-11T23:02:16.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.110 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147546+0000", 2024-05-11T23:02:16.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147546+0000", 2024-05-11T23:02:16.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147547+0000", 2024-05-11T23:02:16.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7500000000000001e-07 2024-05-11T23:02:16.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147549+0000", 2024-05-11T23:02:16.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0040000000000002e-06 2024-05-11T23:02:16.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147555+0000", 2024-05-11T23:02:16.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1959999999999996e-06 2024-05-11T23:02:16.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147559+0000", 2024-05-11T23:02:16.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3329999999999998e-06 2024-05-11T23:02:16.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178414+0000", 2024-05-11T23:02:16.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030854837999999999 2024-05-11T23:02:16.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178418+0000", 2024-05-11T23:02:16.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3970000000000004e-06 2024-05-11T23:02:16.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.113 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.113 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.113 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71339 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:16.113 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147570+0000", 2024-05-11T23:02:16.113 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.8988356089999998, 2024-05-11T23:02:16.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030868237, 2024-05-11T23:02:16.113 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:16.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.113 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71339 2024-05-11T23:02:16.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.114 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147570+0000", 2024-05-11T23:02:16.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147570+0000", 2024-05-11T23:02:16.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147571+0000", 2024-05-11T23:02:16.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2399999999999999e-07 2024-05-11T23:02:16.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147573+0000", 2024-05-11T23:02:16.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.43e-06 2024-05-11T23:02:16.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147589+0000", 2024-05-11T23:02:16.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5977999999999998e-05 2024-05-11T23:02:16.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147595+0000", 2024-05-11T23:02:16.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.463e-06 2024-05-11T23:02:16.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178434+0000", 2024-05-11T23:02:16.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030839582000000001 2024-05-11T23:02:16.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178439+0000", 2024-05-11T23:02:16.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3599999999999998e-06 2024-05-11T23:02:16.116 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.116 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.116 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.117 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71340 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:16.117 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147614+0000", 2024-05-11T23:02:16.117 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.8987915160000002, 2024-05-11T23:02:16.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030844255000000001, 2024-05-11T23:02:16.117 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.117 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:16.117 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.117 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.117 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.117 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71340 2024-05-11T23:02:16.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.117 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147614+0000", 2024-05-11T23:02:16.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147614+0000", 2024-05-11T23:02:16.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147615+0000", 2024-05-11T23:02:16.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4200000000000001e-07 2024-05-11T23:02:16.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147617+0000", 2024-05-11T23:02:16.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5150000000000001e-06 2024-05-11T23:02:16.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147626+0000", 2024-05-11T23:02:16.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1950000000000003e-06 2024-05-11T23:02:16.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147631+0000", 2024-05-11T23:02:16.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3369999999999999e-06 2024-05-11T23:02:16.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178454+0000", 2024-05-11T23:02:16.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030823481999999999 2024-05-11T23:02:16.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178459+0000", 2024-05-11T23:02:16.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.284e-06 2024-05-11T23:02:16.120 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.120 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.120 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.120 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71341 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:16.120 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147639+0000", 2024-05-11T23:02:16.120 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.8987674329999997, 2024-05-11T23:02:16.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030842306999999999, 2024-05-11T23:02:16.120 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.120 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.120 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71341 2024-05-11T23:02:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147639+0000", 2024-05-11T23:02:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147639+0000", 2024-05-11T23:02:16.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147639+0000", 2024-05-11T23:02:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6600000000000002e-07 2024-05-11T23:02:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147646+0000", 2024-05-11T23:02:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8870000000000003e-06 2024-05-11T23:02:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147650+0000", 2024-05-11T23:02:16.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4510000000000002e-06 2024-05-11T23:02:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147653+0000", 2024-05-11T23:02:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8700000000000001e-06 2024-05-11T23:02:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178476+0000", 2024-05-11T23:02:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030823350999999999 2024-05-11T23:02:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178481+0000", 2024-05-11T23:02:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3819999999999997e-06 2024-05-11T23:02:16.123 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.124 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.124 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.124 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.124 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.124 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:16.125 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:02:16.126 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:16.125+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:16.226 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:16.225+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:16.281 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:16.281 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:16.281 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:02:16.281 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:16.288 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-05-11T23:02:16.292 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:02:16.336 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:16.335+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:16.362 INFO:teuthology.orchestra.run.smithi005.stdout:3334 write 0xe99f0c1 thru 0xe9a2999 (0x38d9 bytes) 2024-05-11T23:02:16.365 INFO:teuthology.orchestra.run.smithi005.stdout:3335 read 0x1e4c3c2 thru 0x1e5581d (0x945c bytes) 2024-05-11T23:02:16.365 INFO:teuthology.orchestra.run.smithi005.stdout:3336 punch from 0xad90383 to 0xad9daaa, (0xd727 bytes) 2024-05-11T23:02:16.366 INFO:teuthology.orchestra.run.smithi005.stdout:3337 writesame 0x2681541 thru 0x268f4af data_size 0x1(0xdf6f bytes) 2024-05-11T23:02:16.407 INFO:teuthology.orchestra.run.smithi005.stdout:3338 punch from 0x703f38c to 0x704e99c, (0xf610 bytes) 2024-05-11T23:02:16.411 INFO:teuthology.orchestra.run.smithi005.stdout:3340 trunc from 0xe9a299a to 0xb05c7b9 2024-05-11T23:02:16.423 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:16.423 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:16.423 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:16.423 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:16.423 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:16.426 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:16.426+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:16.427 INFO:teuthology.orchestra.run.smithi005.stdout:3341 punch from 0x45141d9 to 0x4515e70, (0x1c97 bytes) 2024-05-11T23:02:16.427 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:16.427 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:16.427 INFO:teuthology.orchestra.run.smithi005.stdout:3344 punch from 0x608950a to 0x608ebcc, (0x56c2 bytes) 2024-05-11T23:02:16.428 INFO:teuthology.orchestra.run.smithi005.stdout:3345 write 0x7c55df0 thru 0x7c644e5 (0xe6f6 bytes) 2024-05-11T23:02:16.431 INFO:teuthology.orchestra.run.smithi005.stdout:3346 read 0x5f5e5d5 thru 0x5f68952 (0xa37e bytes) 2024-05-11T23:02:16.431 INFO:teuthology.orchestra.run.smithi005.stdout:3349 read 0x9576bf4 thru 0x957864d (0x1a5a bytes) 2024-05-11T23:02:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:3351 read 0xe7a49b thru 0xe88442 (0xdfa8 bytes) 2024-05-11T23:02:16.433 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:02:16.433 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:16.443 INFO:teuthology.orchestra.run.smithi005.stdout:3354 trunc from 0xb05c7b9 to 0x1395106 2024-05-11T23:02:16.488 INFO:teuthology.orchestra.run.smithi005.stdout:3355 writesame 0x83d7eb5 thru 0x83e540d data_size 0x1(0xd559 bytes) 2024-05-11T23:02:16.488 INFO:teuthology.orchestra.run.smithi005.stdout:3357 punch from 0x8561ab to 0x8643be, (0xe213 bytes) 2024-05-11T23:02:16.498 INFO:teuthology.orchestra.run.smithi005.stdout:3359 writesame 0xbcf267f thru 0xbcf3c49 data_size 0x1(0x15cb bytes) 2024-05-11T23:02:16.498 INFO:teuthology.orchestra.run.smithi005.stdout:3360 write 0x9861859 thru 0x987118b (0xf933 bytes) 2024-05-11T23:02:16.501 INFO:teuthology.orchestra.run.smithi005.stdout:3361 writesame 0x647cf55 thru 0x648504c data_size 0x4(0x80f8 bytes) 2024-05-11T23:02:16.506 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:16.506 INFO:teuthology.orchestra.run.smithi005.stdout:3363 read 0x6cde5f6 thru 0x6cec05e (0xda69 bytes) 2024-05-11T23:02:16.517 INFO:teuthology.orchestra.run.smithi005.stdout:3364 write 0xc22ec13 thru 0xc2337e5 (0x4bd3 bytes) 2024-05-11T23:02:16.527 INFO:teuthology.orchestra.run.smithi005.stdout:3367 writesame 0xd79dc0d thru 0xd7a9684 data_size 0x99(0xba78 bytes) 2024-05-11T23:02:16.527 INFO:teuthology.orchestra.run.smithi005.stdout:3368 read 0x31fa4d1 thru 0x31fe2e6 (0x3e16 bytes) 2024-05-11T23:02:16.528 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:16.527+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:16.528 INFO:teuthology.orchestra.run.smithi005.stdout:3372 read 0x30d4443 thru 0x30da0ef (0x5cad bytes) 2024-05-11T23:02:16.528 INFO:teuthology.orchestra.run.smithi005.stdout:3374 read 0x33b91e8 thru 0x33c3866 (0xa67f bytes) 2024-05-11T23:02:16.528 INFO:teuthology.orchestra.run.smithi005.stdout:3376 read 0x960dc0a thru 0x960eba7 (0xf9e bytes) 2024-05-11T23:02:16.528 INFO:teuthology.orchestra.run.smithi005.stdout:3377 write 0x67debdf thru 0x67e0414 (0x1836 bytes) 2024-05-11T23:02:16.529 INFO:teuthology.orchestra.run.smithi005.stdout:3379 write 0x4bc3517 thru 0x4bd1f31 (0xea1b bytes) 2024-05-11T23:02:16.532 INFO:teuthology.orchestra.run.smithi005.stdout:3381 trunc from 0xd7a9685 to 0xc6def3a 2024-05-11T23:02:16.538 INFO:teuthology.orchestra.run.smithi005.stdout:3382 read 0xa5000c6 thru 0xa509ec7 (0x9e02 bytes) 2024-05-11T23:02:16.538 INFO:teuthology.orchestra.run.smithi005.stdout:3383 read 0x79a72d4 thru 0x79abebf (0x4bec bytes) 2024-05-11T23:02:16.538 INFO:teuthology.orchestra.run.smithi005.stdout:3384 write 0x108dc8a thru 0x109d049 (0xf3c0 bytes) 2024-05-11T23:02:16.539 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:16.539 INFO:teuthology.orchestra.run.smithi005.stdout:3386 read 0x79cd488 thru 0x79d9f19 (0xca92 bytes) 2024-05-11T23:02:16.540 INFO:teuthology.orchestra.run.smithi005.stdout:3387 write 0x5fa3689 thru 0x5fa65e3 (0x2f5b bytes) 2024-05-11T23:02:16.543 INFO:teuthology.orchestra.run.smithi005.stdout:3388 trunc from 0xc6def3a to 0xbc7791d 2024-05-11T23:02:16.557 INFO:teuthology.orchestra.run.smithi005.stdout:3389 read 0x41ab512 thru 0x41b1908 (0x63f7 bytes) 2024-05-11T23:02:16.557 INFO:teuthology.orchestra.run.smithi005.stdout:3390 writesame 0x567af4a thru 0x5680c9f data_size 0xd(0x5d56 bytes) 2024-05-11T23:02:16.564 INFO:teuthology.orchestra.run.smithi005.stdout:3392 write 0xdcbd68a thru 0xdcbd716 (0x8d bytes) 2024-05-11T23:02:16.566 INFO:teuthology.orchestra.run.smithi005.stdout:3394 punch from 0x1b4eeda to 0x1b5318c, (0x42b2 bytes) 2024-05-11T23:02:16.567 INFO:teuthology.orchestra.run.smithi005.stdout:3396 write 0x6894eaa thru 0x689509b (0x1f2 bytes) 2024-05-11T23:02:16.567 INFO:teuthology.orchestra.run.smithi005.stdout:3397 read 0x22bb27a thru 0x22bd777 (0x24fe bytes) 2024-05-11T23:02:16.568 INFO:teuthology.orchestra.run.smithi005.stdout:3398 writesame 0xb6c621c thru 0xb6d0e1b data_size 0x80(0xac00 bytes) 2024-05-11T23:02:16.571 INFO:teuthology.orchestra.run.smithi005.stdout:3402 write 0xb42ebde thru 0xb43b70b (0xcb2e bytes) 2024-05-11T23:02:16.572 INFO:teuthology.orchestra.run.smithi005.stdout:3404 writesame 0xadbd46f thru 0xadbd61e data_size 0x9(0x1b0 bytes) 2024-05-11T23:02:16.574 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:16.574 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:16.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:16.574 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:16.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.574 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28445 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.492349014999999, 2024-05-11T23:02:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031719429, 2024-05-11T23:02:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28445 2024-05-11T23:02:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-05-11T23:02:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081234+0000", 2024-05-11T23:02:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.578e-06 2024-05-11T23:02:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081237+0000", 2024-05-11T23:02:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3340000000000002e-06 2024-05-11T23:02:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081240+0000", 2024-05-11T23:02:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7429999999999998e-06 2024-05-11T23:02:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112828+0000", 2024-05-11T23:02:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031587941000000001 2024-05-11T23:02:16.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112878+0000", 2024-05-11T23:02:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0535999999999999e-05 2024-05-11T23:02:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112951+0000", 2024-05-11T23:02:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3072999999999996e-05 2024-05-11T23:02:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28448 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 507904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:02:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.492316513999999, 2024-05-11T23:02:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031916923, 2024-05-11T23:02:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28448 2024-05-11T23:02:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:02:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:02:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081265+0000", 2024-05-11T23:02:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2899999999999999e-07 2024-05-11T23:02:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081267+0000", 2024-05-11T23:02:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6749999999999998e-06 2024-05-11T23:02:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081273+0000", 2024-05-11T23:02:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4049999999999999e-06 2024-05-11T23:02:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081278+0000", 2024-05-11T23:02:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0139999999999998e-06 2024-05-11T23:02:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112996+0000", 2024-05-11T23:02:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031718283 2024-05-11T23:02:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113080+0000", 2024-05-11T23:02:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4334999999999998e-05 2024-05-11T23:02:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113181+0000", 2024-05-11T23:02:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000100882 2024-05-11T23:02:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28451 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 540672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.492288011999999, 2024-05-11T23:02:16.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032054412999999997, 2024-05-11T23:02:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28451 2024-05-11T23:02:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2499999999999995e-07 2024-05-11T23:02:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081300+0000", 2024-05-11T23:02:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0929999999999999e-06 2024-05-11T23:02:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081306+0000", 2024-05-11T23:02:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6290000000000004e-06 2024-05-11T23:02:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081311+0000", 2024-05-11T23:02:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9289999999999997e-06 2024-05-11T23:02:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113208+0000", 2024-05-11T23:02:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031897162 2024-05-11T23:02:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113277+0000", 2024-05-11T23:02:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8811000000000002e-05 2024-05-11T23:02:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113347+0000", 2024-05-11T23:02:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0264000000000004e-05 2024-05-11T23:02:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28454 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 573440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:16.586 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.492213391, 2024-05-11T23:02:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032210564999999997, 2024-05-11T23:02:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28454 2024-05-11T23:02:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.99e-07 2024-05-11T23:02:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081371+0000", 2024-05-11T23:02:16.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1200000000000002e-06 2024-05-11T23:02:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081378+0000", 2024-05-11T23:02:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6089999999999997e-06 2024-05-11T23:02:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081384+0000", 2024-05-11T23:02:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0859999999999998e-06 2024-05-11T23:02:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113388+0000", 2024-05-11T23:02:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032004248999999999 2024-05-11T23:02:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113470+0000", 2024-05-11T23:02:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1871000000000003e-05 2024-05-11T23:02:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113578+0000", 2024-05-11T23:02:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000108031 2024-05-11T23:02:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.590 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28457 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 606208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.492183167, 2024-05-11T23:02:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032410130000000002, 2024-05-11T23:02:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28457 2024-05-11T23:02:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.32e-07 2024-05-11T23:02:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081400+0000", 2024-05-11T23:02:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9120000000000001e-06 2024-05-11T23:02:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081404+0000", 2024-05-11T23:02:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7900000000000001e-06 2024-05-11T23:02:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081407+0000", 2024-05-11T23:02:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8839999999999998e-06 2024-05-11T23:02:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113621+0000", 2024-05-11T23:02:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032213841 2024-05-11T23:02:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113702+0000", 2024-05-11T23:02:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1574000000000002e-05 2024-05-11T23:02:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113808+0000", 2024-05-11T23:02:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000105697 2024-05-11T23:02:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.594 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28460 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 638976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.492170336000001, 2024-05-11T23:02:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032616804999999999, 2024-05-11T23:02:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28460 2024-05-11T23:02:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5900000000000003e-07 2024-05-11T23:02:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081412+0000", 2024-05-11T23:02:16.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1990000000000001e-06 2024-05-11T23:02:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081415+0000", 2024-05-11T23:02:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.971e-06 2024-05-11T23:02:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081418+0000", 2024-05-11T23:02:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-05-11T23:02:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113853+0000", 2024-05-11T23:02:16.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032435155 2024-05-11T23:02:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113936+0000", 2024-05-11T23:02:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3441000000000003e-05 2024-05-11T23:02:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114027+0000", 2024-05-11T23:02:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1108999999999998e-05 2024-05-11T23:02:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28463 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 671744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.492153414000001, 2024-05-11T23:02:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032814958999999998, 2024-05-11T23:02:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28463 2024-05-11T23:02:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4999999999999998e-07 2024-05-11T23:02:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081430+0000", 2024-05-11T23:02:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6109999999999999e-06 2024-05-11T23:02:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081433+0000", 2024-05-11T23:02:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5590000000000001e-06 2024-05-11T23:02:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081437+0000", 2024-05-11T23:02:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3950000000000001e-06 2024-05-11T23:02:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114061+0000", 2024-05-11T23:02:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032624092 2024-05-11T23:02:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114142+0000", 2024-05-11T23:02:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0893e-05 2024-05-11T23:02:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114242+0000", 2024-05-11T23:02:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010095899999999999 2024-05-11T23:02:16.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28466 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 704512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.492140880000001, 2024-05-11T23:02:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032933658999999997, 2024-05-11T23:02:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.603 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28466 2024-05-11T23:02:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:16.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4700000000000002e-07 2024-05-11T23:02:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081442+0000", 2024-05-11T23:02:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.296e-06 2024-05-11T23:02:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081445+0000", 2024-05-11T23:02:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9009999999999998e-06 2024-05-11T23:02:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081447+0000", 2024-05-11T23:02:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4200000000000001e-06 2024-05-11T23:02:16.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114280+0000", 2024-05-11T23:02:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032832667000000003 2024-05-11T23:02:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114325+0000", 2024-05-11T23:02:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5018000000000002e-05 2024-05-11T23:02:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114374+0000", 2024-05-11T23:02:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.901e-05 2024-05-11T23:02:16.606 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28468 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 737280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.492120875000001, 2024-05-11T23:02:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033118776000000003, 2024-05-11T23:02:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.607 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28468 2024-05-11T23:02:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:16.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3799999999999998e-07 2024-05-11T23:02:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081462+0000", 2024-05-11T23:02:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4899999999999999e-06 2024-05-11T23:02:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081465+0000", 2024-05-11T23:02:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4079999999999998e-06 2024-05-11T23:02:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081468+0000", 2024-05-11T23:02:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7e-06 2024-05-11T23:02:16.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114415+0000", 2024-05-11T23:02:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032946766000000002 2024-05-11T23:02:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114497+0000", 2024-05-11T23:02:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1854000000000006e-05 2024-05-11T23:02:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114579+0000", 2024-05-11T23:02:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2120000000000007e-05 2024-05-11T23:02:16.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28471 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 770048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.492106167999999, 2024-05-11T23:02:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033234970000000003, 2024-05-11T23:02:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28471 2024-05-11T23:02:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-05-11T23:02:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081476+0000", 2024-05-11T23:02:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3120000000000001e-06 2024-05-11T23:02:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081480+0000", 2024-05-11T23:02:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1470000000000001e-06 2024-05-11T23:02:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081482+0000", 2024-05-11T23:02:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7480000000000002e-06 2024-05-11T23:02:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114616+0000", 2024-05-11T23:02:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033133322 2024-05-11T23:02:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114662+0000", 2024-05-11T23:02:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6020999999999999e-05 2024-05-11T23:02:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114710+0000", 2024-05-11T23:02:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8059999999999997e-05 2024-05-11T23:02:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28474 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.492090094000002, 2024-05-11T23:02:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033336431, 2024-05-11T23:02:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.615 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28474 2024-05-11T23:02:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.15e-07 2024-05-11T23:02:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081493+0000", 2024-05-11T23:02:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4950000000000001e-06 2024-05-11T23:02:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081496+0000", 2024-05-11T23:02:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.309e-06 2024-05-11T23:02:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081499+0000", 2024-05-11T23:02:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4570000000000001e-06 2024-05-11T23:02:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114740+0000", 2024-05-11T23:02:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033240619999999999 2024-05-11T23:02:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114781+0000", 2024-05-11T23:02:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1106999999999998e-05 2024-05-11T23:02:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114827+0000", 2024-05-11T23:02:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6028e-05 2024-05-11T23:02:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28477 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 835584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.492073080000001, 2024-05-11T23:02:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033527225000000001, 2024-05-11T23:02:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.619 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28477 2024-05-11T23:02:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1300000000000001e-07 2024-05-11T23:02:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081510+0000", 2024-05-11T23:02:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4610000000000001e-06 2024-05-11T23:02:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081513+0000", 2024-05-11T23:02:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2559999999999998e-06 2024-05-11T23:02:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081516+0000", 2024-05-11T23:02:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7099999999999999e-06 2024-05-11T23:02:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114869+0000", 2024-05-11T23:02:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033353718999999997 2024-05-11T23:02:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114954+0000", 2024-05-11T23:02:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4969000000000006e-05 2024-05-11T23:02:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115035+0000", 2024-05-11T23:02:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0697000000000003e-05 2024-05-11T23:02:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28480 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 868352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.49205611, 2024-05-11T23:02:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033695939000000001, 2024-05-11T23:02:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:16.623 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28480 2024-05-11T23:02:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.46e-07 2024-05-11T23:02:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081526+0000", 2024-05-11T23:02:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.347e-06 2024-05-11T23:02:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081530+0000", 2024-05-11T23:02:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.348e-06 2024-05-11T23:02:16.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081533+0000", 2024-05-11T23:02:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4369999999999999e-06 2024-05-11T23:02:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115078+0000", 2024-05-11T23:02:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033545063 2024-05-11T23:02:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115164+0000", 2024-05-11T23:02:16.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5662999999999996e-05 2024-05-11T23:02:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115221+0000", 2024-05-11T23:02:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6734999999999998e-05 2024-05-11T23:02:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28483 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.492038937, 2024-05-11T23:02:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033803466999999997, 2024-05-11T23:02:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28483 2024-05-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:16.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4999999999999998e-07 2024-05-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081544+0000", 2024-05-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081547+0000", 2024-05-11T23:02:16.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3139999999999999e-06 2024-05-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081550+0000", 2024-05-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7779999999999999e-06 2024-05-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115253+0000", 2024-05-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033703516000000003 2024-05-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115302+0000", 2024-05-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8869999999999998e-05 2024-05-11T23:02:16.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115345+0000", 2024-05-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3340999999999997e-05 2024-05-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28487 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 933888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.492027736000001, 2024-05-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033903918999999998, 2024-05-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.631 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28487 2024-05-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:16.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9999999999999999e-07 2024-05-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081559+0000", 2024-05-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-05-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081563+0000", 2024-05-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2370000000000003e-06 2024-05-11T23:02:16.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081566+0000", 2024-05-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6199999999999999e-06 2024-05-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115375+0000", 2024-05-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033808887000000003 2024-05-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115413+0000", 2024-05-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8309000000000002e-05 2024-05-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115457+0000", 2024-05-11T23:02:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3809e-05 2024-05-11T23:02:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28490 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 966656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.492011212000001, 2024-05-11T23:02:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034114903000000002, 2024-05-11T23:02:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:16.635 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28490 2024-05-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4400000000000001e-07 2024-05-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081571+0000", 2024-05-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2130000000000001e-06 2024-05-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081574+0000", 2024-05-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8270000000000002e-06 2024-05-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081577+0000", 2024-05-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-05-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115507+0000", 2024-05-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033929925999999999 2024-05-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115599+0000", 2024-05-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1571999999999997e-05 2024-05-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115685+0000", 2024-05-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6071999999999999e-05 2024-05-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28493 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 999424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.491999108999998, 2024-05-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034232944000000001, 2024-05-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.639 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28493 2024-05-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1300000000000001e-07 2024-05-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081584+0000", 2024-05-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9479999999999998e-06 2024-05-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081590+0000", 2024-05-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.728e-06 2024-05-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081595+0000", 2024-05-11T23:02:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8579999999999999e-06 2024-05-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115723+0000", 2024-05-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034128473999999999 2024-05-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115768+0000", 2024-05-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5108999999999998e-05 2024-05-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115815+0000", 2024-05-11T23:02:16.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6514000000000002e-05 2024-05-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28496 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.491979650000001, 2024-05-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034427754999999997, 2024-05-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.643 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28496 2024-05-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081602+0000", 2024-05-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081604+0000", 2024-05-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3999999999999999e-06 2024-05-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081610+0000", 2024-05-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.224e-06 2024-05-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081615+0000", 2024-05-11T23:02:16.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8269999999999997e-06 2024-05-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115859+0000", 2024-05-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034243498999999997 2024-05-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115946+0000", 2024-05-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7122999999999999e-05 2024-05-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116029+0000", 2024-05-11T23:02:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3332999999999996e-05 2024-05-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28498 3.0 3:05d5189e:::rbd_object_map.111e5e4d8275.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.490460998, 2024-05-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.035035108000000002, 2024-05-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28498 2024-05-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083121+0000", 2024-05-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5700000000000001e-07 2024-05-11T23:02:16.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083126+0000", 2024-05-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8860000000000003e-06 2024-05-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083135+0000", 2024-05-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8369999999999992e-06 2024-05-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083142+0000", 2024-05-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4779999999999996e-06 2024-05-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116072+0000", 2024-05-11T23:02:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032929991999999998 2024-05-11T23:02:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116158+0000", 2024-05-11T23:02:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6077999999999998e-05 2024-05-11T23:02:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-05-11T23:02:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116513+0000", 2024-05-11T23:02:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00035508300000000002 2024-05-11T23:02:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:16.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118025+0000", 2024-05-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015118919999999999 2024-05-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118039+0000", 2024-05-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4188999999999999e-05 2024-05-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118094+0000", 2024-05-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4929000000000002e-05 2024-05-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118155+0000", 2024-05-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0887000000000002e-05 2024-05-11T23:02:16.651 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:64049 3.8 3:11d98727:::rbd_object_map.111e8974a681:head [read 0~38] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-05-11T23:02:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:02:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 15.793699688, 2024-05-11T23:02:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99884604399999999, 2024-05-11T23:02:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 64049 2024-05-11T23:02:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:16.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:02:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:02:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779883+0000", 2024-05-11T23:02:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.57e-06 2024-05-11T23:02:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:16.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779887+0000", 2024-05-11T23:02:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5709999999999999e-06 2024-05-11T23:02:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779898+0000", 2024-05-11T23:02:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.042e-05 2024-05-11T23:02:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779905+0000", 2024-05-11T23:02:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0110000000000003e-06 2024-05-11T23:02:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779962+0000", 2024-05-11T23:02:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7395999999999997e-05 2024-05-11T23:02:16.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:02:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779969+0000", 2024-05-11T23:02:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2459999999999997e-06 2024-05-11T23:02:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778379+0000", 2024-05-11T23:02:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99840986300000001 2024-05-11T23:02:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778564+0000", 2024-05-11T23:02:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018420900000000001 2024-05-11T23:02:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:16.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:16.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:16.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778727+0000", 2024-05-11T23:02:16.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016375800000000001 2024-05-11T23:02:16.656 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.656 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.656 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.656 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:16.656 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:16.656 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:16.657 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:16.657 INFO:teuthology.orchestra.run.smithi005.stdout:3406 writesame 0x9b5c9c5 thru 0x9b6b38a data_size 0x17(0xe9c6 bytes) 2024-05-11T23:02:16.657 INFO:teuthology.orchestra.run.smithi005.stdout:3407 punch from 0x2556470 to 0x2558bb6, (0x2746 bytes) 2024-05-11T23:02:16.657 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:16.657 INFO:teuthology.orchestra.run.smithi005.stdout:3409 writesame 0x3206227 thru 0x320ef04 data_size 0x26(0x8cde bytes) 2024-05-11T23:02:16.657 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:16.657 INFO:teuthology.orchestra.run.smithi005.stdout:3413 read 0x4b4cef8 thru 0x4b56af9 (0x9c02 bytes) 2024-05-11T23:02:16.657 INFO:teuthology.orchestra.run.smithi005.stdout:3414 write 0x17a6459 thru 0x17b12e0 (0xae88 bytes) 2024-05-11T23:02:16.657 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:16.657 INFO:teuthology.orchestra.run.smithi005.stdout:3416 read 0xc951168 thru 0xc9558e3 (0x477c bytes) 2024-05-11T23:02:16.657 INFO:teuthology.orchestra.run.smithi005.stdout:3417 write 0x3cbda30 thru 0x3cc5750 (0x7d21 bytes) 2024-05-11T23:02:16.658 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:02:16.660 INFO:teuthology.orchestra.run.smithi005.stdout:3418 trunc from 0xdcbd717 to 0xcc3d48 2024-05-11T23:02:16.660 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:16.660 INFO:teuthology.orchestra.run.smithi005.stdout:3422 writesame 0x64fbcd8 thru 0x64fedd2 data_size 0x1(0x30fb bytes) 2024-05-11T23:02:16.660 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:16.660 INFO:teuthology.orchestra.run.smithi005.stdout:3425 writesame 0x5b782cb thru 0x5b7b7e6 data_size 0x21(0x351c bytes) 2024-05-11T23:02:16.660 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:16.660 INFO:teuthology.orchestra.run.smithi005.stdout:3428 write 0xc36a9e5 thru 0xc375fba (0xb5d6 bytes) 2024-05-11T23:02:16.660 INFO:teuthology.orchestra.run.smithi005.stdout:3429 writesame 0x366ceef thru 0x366f863 data_size 0x1(0x2975 bytes) 2024-05-11T23:02:16.661 INFO:teuthology.orchestra.run.smithi005.stdout:3431 write 0x6b95ed5 thru 0x6b9d027 (0x7153 bytes) 2024-05-11T23:02:16.661 INFO:teuthology.orchestra.run.smithi005.stdout:3432 write 0x93306a5 thru 0x9337c56 (0x75b2 bytes) 2024-05-11T23:02:16.661 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:16.661 INFO:teuthology.orchestra.run.smithi005.stdout:3434 punch from 0x158bd9d to 0x159595c, (0x9bbf bytes) 2024-05-11T23:02:16.661 INFO:teuthology.orchestra.run.smithi005.stdout:3435 writesame 0xc07face thru 0xc0893e3 data_size 0x5(0x9916 bytes) 2024-05-11T23:02:16.661 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:16.660+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:16.662 INFO:teuthology.orchestra.run.smithi005.stdout:3436 writesame 0xc623b0f thru 0xc624251 data_size 0xb(0x743 bytes) 2024-05-11T23:02:16.662 INFO:teuthology.orchestra.run.smithi005.stdout:3437 read 0xc9ffad thru 0xca190e (0x1962 bytes) 2024-05-11T23:02:16.662 INFO:teuthology.orchestra.run.smithi005.stdout:3438 read 0x177b88a thru 0x177e93e (0x30b5 bytes) 2024-05-11T23:02:16.668 INFO:teuthology.orchestra.run.smithi005.stdout:3440 write 0xea6cbaf thru 0xea731b3 (0x6605 bytes) 2024-05-11T23:02:16.671 INFO:teuthology.orchestra.run.smithi005.stdout:3441 read 0x4c1ac4a thru 0x4c1c883 (0x1c3a bytes) 2024-05-11T23:02:16.672 INFO:teuthology.orchestra.run.smithi005.stdout:3442 punch from 0x4f39875 to 0x4f39f72, (0x6fd bytes) 2024-05-11T23:02:16.672 INFO:teuthology.orchestra.run.smithi005.stdout:3443 punch from 0xb7a838b to 0xb7b0238, (0x7ead bytes) 2024-05-11T23:02:16.672 INFO:teuthology.orchestra.run.smithi005.stdout:3445 read 0x3278ccc thru 0x3286b2a (0xde5f bytes) 2024-05-11T23:02:16.672 INFO:teuthology.orchestra.run.smithi005.stdout:3447 punch from 0x661bd86 to 0x66298d9, (0xdb53 bytes) 2024-05-11T23:02:16.672 INFO:teuthology.orchestra.run.smithi005.stdout:3448 punch from 0x5633357 to 0x5641aff, (0xe7a8 bytes) 2024-05-11T23:02:16.673 INFO:teuthology.orchestra.run.smithi005.stdout:3449 writesame 0x4a99a38 thru 0x4aa7348 data_size 0x1(0xd911 bytes) 2024-05-11T23:02:16.696 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:16.697 INFO:teuthology.orchestra.run.smithi005.stdout:3452 writesame 0x3a83468 thru 0x3a8ef5a data_size 0x81(0xbaf3 bytes) 2024-05-11T23:02:16.698 INFO:teuthology.orchestra.run.smithi005.stdout:3453 read 0x4335dee thru 0x434093a (0xab4d bytes) 2024-05-11T23:02:16.698 INFO:teuthology.orchestra.run.smithi005.stdout:3454 read 0x52353a6 thru 0x523842d (0x3088 bytes) 2024-05-11T23:02:16.699 INFO:teuthology.orchestra.run.smithi005.stdout:3455 read 0x715a17f thru 0x71696a2 (0xf524 bytes) 2024-05-11T23:02:16.699 INFO:teuthology.orchestra.run.smithi005.stdout:3456 clone 28 order 23 su 65536 sc 10 2024-05-11T23:02:16.763 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:16.762+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:16.772 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:02:16.773 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:16.782 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:02:16.795 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:02:16.863 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:16.863+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:16.936 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:16.936 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:16.936 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:16.936 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:16.936 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:16.945 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-11T23:02:16.946 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:02:16.955 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:16.956 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:16.966 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:16.966+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:16.977 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:16.997 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:16.998 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:17.001 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone27 from 0xea731b4 (overlap 0xcc3d48) to 0x0 2024-05-11T23:02:17.033 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:17.033 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:17.058 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-05-11T23:02:17.066 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:17.066+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:17.099 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:17.100 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:17.122 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:02:17.167 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:17.166+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:17.253 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:17.253 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:17.267 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:17.267+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:17.271 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:17.271 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:17.277 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:17.277 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:17.287 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:02:17.290 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:17.290 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:17.308 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:17.308 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:17.322 INFO:teuthology.orchestra.run.smithi005.stdout:3457 writesame 0x6065ca9 thru 0x6066b65 data_size 0xb(0xebd bytes) 2024-05-11T23:02:17.334 INFO:teuthology.orchestra.run.smithi005.stdout:3458 write 0x13ee1e2 thru 0x13f6b24 (0x8943 bytes) 2024-05-11T23:02:17.350 INFO:teuthology.orchestra.run.smithi005.stdout:3463 write 0xad1d5ca thru 0xad2d40a (0xfe41 bytes) 2024-05-11T23:02:17.359 INFO:teuthology.orchestra.run.smithi005.stdout:3464 read 0xa63432e thru 0xa63b4f4 (0x71c7 bytes) 2024-05-11T23:02:17.360 INFO:teuthology.orchestra.run.smithi005.stdout:3465 writesame 0x9be8e97 thru 0x9beec00 data_size 0xb(0x5d6a bytes) 2024-05-11T23:02:17.368 INFO:teuthology.orchestra.run.smithi005.stdout:3466 write 0xdd128c7 thru 0xdd1b699 (0x8dd3 bytes) 2024-05-11T23:02:17.368 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:17.368+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:17.374 INFO:teuthology.orchestra.run.smithi005.stdout:3467 write 0x4e595dd thru 0x4e65f1a (0xc93e bytes) 2024-05-11T23:02:17.392 INFO:teuthology.orchestra.run.smithi005.stdout:3468 write 0x7bec133 thru 0x7bf32c0 (0x718e bytes) 2024-05-11T23:02:17.399 INFO:teuthology.orchestra.run.smithi005.stdout:3469 writesame 0xe24e52b thru 0xe25281b data_size 0x1(0x42f1 bytes) 2024-05-11T23:02:17.409 INFO:teuthology.orchestra.run.smithi005.stdout:3470 write 0x19bd4ba thru 0x19c3d56 (0x689d bytes) 2024-05-11T23:02:17.424 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:17.424 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:17.424 INFO:teuthology.orchestra.run.smithi005.stdout:3471 punch from 0x6898c91 to 0x68a2c37, (0x9fa6 bytes) 2024-05-11T23:02:17.424 INFO:teuthology.orchestra.run.smithi005.stdout:3472 punch from 0xe152b9 to 0xe176f3, (0x243a bytes) 2024-05-11T23:02:17.425 INFO:teuthology.orchestra.run.smithi005.stdout:3473 punch from 0x3d0b30 to 0x3dafa2, (0xa472 bytes) 2024-05-11T23:02:17.425 INFO:teuthology.orchestra.run.smithi005.stdout:3474 write 0xb581118 thru 0xb58331e (0x2207 bytes) 2024-05-11T23:02:17.430 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:17.430 INFO:teuthology.orchestra.run.smithi005.stdout:3477 write 0xc2758ee thru 0xc285585 (0xfc98 bytes) 2024-05-11T23:02:17.435 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:02:17.436 INFO:teuthology.orchestra.run.smithi005.stdout:3479 write 0x90489dd thru 0x904a14a (0x176e bytes) 2024-05-11T23:02:17.443 INFO:teuthology.orchestra.run.smithi005.stdout:3480 read 0xbc6521a thru 0xbc70d2e (0xbb15 bytes) 2024-05-11T23:02:17.447 INFO:teuthology.orchestra.run.smithi005.stdout:3481 trunc from 0xea731b4 to 0xb6e4703 2024-05-11T23:02:17.469 INFO:teuthology.orchestra.run.smithi005.stdout:3482 read 0x91935fc thru 0x919d645 (0xa04a bytes) 2024-05-11T23:02:17.469 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:17.469+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:17.469 INFO:teuthology.orchestra.run.smithi005.stdout:3484 write 0x5fac4ab thru 0x5fae9e2 (0x2538 bytes) 2024-05-11T23:02:17.476 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:17.476 INFO:teuthology.orchestra.run.smithi005.stdout:3486 punch from 0x7df4b95 to 0x7df8b4d, (0x3fb8 bytes) 2024-05-11T23:02:17.489 INFO:teuthology.orchestra.run.smithi005.stdout:3488 trunc from 0xb6e4703 to 0x519e4 2024-05-11T23:02:17.517 INFO:teuthology.orchestra.run.smithi005.stdout:3489 read 0x1d2b3 thru 0x20696 (0x33e4 bytes) 2024-05-11T23:02:17.518 INFO:teuthology.orchestra.run.smithi005.stdout:3491 read 0x45337 thru 0x519e3 (0xc6ad bytes) 2024-05-11T23:02:17.529 INFO:teuthology.orchestra.run.smithi005.stdout:3492 write 0x4a845a8 thru 0x4a87894 (0x32ed bytes) 2024-05-11T23:02:17.545 INFO:teuthology.orchestra.run.smithi005.stdout:3493 write 0xdf13ce1 thru 0xdf14898 (0xbb8 bytes) 2024-05-11T23:02:17.548 INFO:teuthology.orchestra.run.smithi005.stdout:3495 read 0x1ce4637 thru 0x1ce6f40 (0x290a bytes) 2024-05-11T23:02:17.548 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:17.548 INFO:teuthology.orchestra.run.smithi005.stdout:3497 write 0x85f61b6 thru 0x8606038 (0xfe83 bytes) 2024-05-11T23:02:17.552 INFO:teuthology.orchestra.run.smithi005.stdout:3498 read 0x29e7cc8 thru 0x29f014f (0x8488 bytes) 2024-05-11T23:02:17.558 INFO:teuthology.orchestra.run.smithi005.stdout:3500 write 0xea5e72a thru 0xea66bb9 (0x8490 bytes) 2024-05-11T23:02:17.560 INFO:teuthology.orchestra.run.smithi005.stdout:3502 read 0x78b76e2 thru 0x78c2bd3 (0xb4f2 bytes) 2024-05-11T23:02:17.561 INFO:teuthology.orchestra.run.smithi005.stdout:3503 writesame 0x904159 thru 0x9059d6 data_size 0x21(0x187e bytes) 2024-05-11T23:02:17.562 INFO:teuthology.orchestra.run.smithi005.stdout:3504 write 0x6e85bf3 thru 0x6e8f0c2 (0x94d0 bytes) 2024-05-11T23:02:17.565 INFO:teuthology.orchestra.run.smithi005.stdout:3506 write 0xc4db807 thru 0xc4e1d49 (0x6543 bytes) 2024-05-11T23:02:17.566 INFO:teuthology.orchestra.run.smithi005.stdout:3507 read 0xe96b0ed thru 0xe9789a7 (0xd8bb bytes) 2024-05-11T23:02:17.567 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:17.567 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:17.567 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:17.570 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:17.570 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:17.570 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:17.570+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:17.583 INFO:teuthology.orchestra.run.smithi005.stdout:3514 trunc from 0xea66bba to 0x205e752 2024-05-11T23:02:17.586 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:02:17.607 INFO:teuthology.orchestra.run.smithi005.stdout:3515 write 0x2748a2d thru 0x274b079 (0x264d bytes) 2024-05-11T23:02:17.609 INFO:teuthology.orchestra.run.smithi005.stdout:3516 punch from 0x113e92c to 0x113f160, (0x834 bytes) 2024-05-11T23:02:17.624 INFO:teuthology.orchestra.run.smithi005.stdout:3518 write 0xa656e9f thru 0xa662721 (0xb883 bytes) 2024-05-11T23:02:17.627 INFO:teuthology.orchestra.run.smithi005.stdout:3519 writesame 0x182e868 thru 0x183873d data_size 0x1b(0x9ed6 bytes) 2024-05-11T23:02:17.631 INFO:teuthology.orchestra.run.smithi005.stdout:3520 write 0x215cc2a thru 0x215e6bf (0x1a96 bytes) 2024-05-11T23:02:17.641 INFO:teuthology.orchestra.run.smithi005.stdout:3521 writesame 0xc5812a4 thru 0xc587535 data_size 0x3e(0x6292 bytes) 2024-05-11T23:02:17.641 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:17.641 INFO:teuthology.orchestra.run.smithi005.stdout:3523 punch from 0x13bd56d to 0x13c0765, (0x31f8 bytes) 2024-05-11T23:02:17.641 INFO:teuthology.orchestra.run.smithi005.stdout:3524 write 0x2c2381d thru 0x2c240ef (0x8d3 bytes) 2024-05-11T23:02:17.647 INFO:teuthology.orchestra.run.smithi005.stdout:3526 writesame 0xc643bf9 thru 0xc64c567 data_size 0x1(0x896f bytes) 2024-05-11T23:02:17.648 INFO:teuthology.orchestra.run.smithi005.stdout:3527 read 0x864ad59 thru 0x864bd74 (0x101c bytes) 2024-05-11T23:02:17.652 INFO:teuthology.orchestra.run.smithi005.stdout:3528 trunc from 0xc64c568 to 0x8beb63e 2024-05-11T23:02:17.670 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:17.670+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:17.676 INFO:teuthology.orchestra.run.smithi005.stdout:3530 write 0xe3953aa thru 0xe39bc29 (0x6880 bytes) 2024-05-11T23:02:17.688 INFO:teuthology.orchestra.run.smithi005.stdout:3532 trunc from 0xe39bc2a to 0x5006537 2024-05-11T23:02:17.699 INFO:teuthology.orchestra.run.smithi005.stdout:3533 trunc from 0x5006537 to 0x502283f 2024-05-11T23:02:17.701 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:17.702 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:17.711 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:02:17.720 INFO:teuthology.orchestra.run.smithi005.stdout:3534 write 0xbe6f767 thru 0xbe796a2 (0x9f3c bytes) 2024-05-11T23:02:17.728 INFO:teuthology.orchestra.run.smithi005.stdout:3535 writesame 0xc54c9c4 thru 0xc555184 data_size 0x1(0x87c1 bytes) 2024-05-11T23:02:17.729 INFO:teuthology.orchestra.run.smithi005.stdout:3536 punch from 0x2b30e5d to 0x2b36107, (0x52aa bytes) 2024-05-11T23:02:17.730 INFO:teuthology.orchestra.run.smithi005.stdout:3539 read 0x4c7fe61 thru 0x4c8d30e (0xd4ae bytes) 2024-05-11T23:02:17.730 INFO:teuthology.orchestra.run.smithi005.stdout:3544 clone 29 order 18 su 4096 sc 3 2024-05-11T23:02:17.771 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:17.772+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:17.856 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:17.856 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:17.865 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:02:17.872 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:17.872+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:17.936 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-05-11T23:02:17.973 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:17.973+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:17.974 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:17.974 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:17.986 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:02:18.003 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone28 from 0xc555185 (overlap 0x519e4) to 0x28cf2 2024-05-11T23:02:18.074 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:18.074+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:18.078 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-05-11T23:02:18.080 INFO:teuthology.orchestra.run.smithi005.stdout:3545 write 0x3227adf thru 0x323608b (0xe5ad bytes) 2024-05-11T23:02:18.096 INFO:teuthology.orchestra.run.smithi005.stdout:3547 write 0x4c4bfdb thru 0x4c5b95e (0xf984 bytes) 2024-05-11T23:02:18.112 INFO:teuthology.orchestra.run.smithi005.stdout:3549 read 0x7836bc6 thru 0x7846b26 (0xff61 bytes) 2024-05-11T23:02:18.112 INFO:teuthology.orchestra.run.smithi005.stdout:3550 read 0x1ec86dd thru 0x1ecdc19 (0x553d bytes) 2024-05-11T23:02:18.114 INFO:teuthology.orchestra.run.smithi005.stdout:3553 write 0x9ed961 thru 0x9ef53d (0x1bdd bytes) 2024-05-11T23:02:18.129 INFO:teuthology.orchestra.run.smithi005.stdout:3554 punch from 0x72dd3a6 to 0x72e36a2, (0x62fc bytes) 2024-05-11T23:02:18.130 INFO:teuthology.orchestra.run.smithi005.stdout:3555 read 0x96cd567 thru 0x96d4a81 (0x751b bytes) 2024-05-11T23:02:18.130 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:18.130 INFO:teuthology.orchestra.run.smithi005.stdout:3558 write 0x45969a6 thru 0x459869e (0x1cf9 bytes) 2024-05-11T23:02:18.143 INFO:teuthology.orchestra.run.smithi005.stdout:3559 read 0x68a8a1e thru 0x68b8331 (0xf914 bytes) 2024-05-11T23:02:18.146 INFO:teuthology.orchestra.run.smithi005.stdout:3560 trunc from 0xc555185 to 0x96daa3b 2024-05-11T23:02:18.155 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:18.155 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:18.156 INFO:teuthology.orchestra.run.smithi005.stdout:3563 write 0x145f365 thru 0x145f8b3 (0x54f bytes) 2024-05-11T23:02:18.165 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:02:18.167 INFO:teuthology.orchestra.run.smithi005.stdout:3564 trunc from 0x96daa3b to 0x5b5ab9e 2024-05-11T23:02:18.175 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:18.175+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:18.194 INFO:teuthology.orchestra.run.smithi005.stdout:3565 write 0xd87a732 thru 0xd885a89 (0xb358 bytes) 2024-05-11T23:02:18.204 INFO:teuthology.orchestra.run.smithi005.stdout:3567 read 0x66337bf thru 0x663d022 (0x9864 bytes) 2024-05-11T23:02:18.204 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:18.205 INFO:teuthology.orchestra.run.smithi005.stdout:3569 trunc from 0xd885a8a to 0xcdedf83 2024-05-11T23:02:18.213 INFO:teuthology.orchestra.run.smithi005.stdout:3570 read 0x805c6f thru 0x80ae4f (0x51e1 bytes) 2024-05-11T23:02:18.215 INFO:teuthology.orchestra.run.smithi005.stdout:3573 write 0x3d716fe thru 0x3d771ee (0x5af1 bytes) 2024-05-11T23:02:18.231 INFO:teuthology.orchestra.run.smithi005.stdout:3574 read 0x12bc3ba thru 0x12cb8e6 (0xf52d bytes) 2024-05-11T23:02:18.233 INFO:teuthology.orchestra.run.smithi005.stdout:3575 read 0xb43ceb thru 0xb4469f (0x9b5 bytes) 2024-05-11T23:02:18.247 INFO:teuthology.orchestra.run.smithi005.stdout:3576 trunc from 0xcdedf83 to 0x194ed5e 2024-05-11T23:02:18.273 INFO:teuthology.orchestra.run.smithi005.stdout:3578 read 0x185bc08 thru 0x1863a36 (0x7e2f bytes) 2024-05-11T23:02:18.274 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:18.274 INFO:teuthology.orchestra.run.smithi005.stdout:3580 writesame 0xf7f5c1 thru 0xf8b584 data_size 0x6(0xbfc4 bytes) 2024-05-11T23:02:18.275 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:18.275+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:18.295 INFO:teuthology.orchestra.run.smithi005.stdout:3581 writesame 0x24ed601 thru 0x24f29e6 data_size 0x1(0x53e6 bytes) 2024-05-11T23:02:18.310 INFO:teuthology.orchestra.run.smithi005.stdout:3582 write 0x90f392b thru 0x90f5d85 (0x245b bytes) 2024-05-11T23:02:18.314 INFO:teuthology.orchestra.run.smithi005.stdout:3583 trunc from 0x90f5d86 to 0x9641925 2024-05-11T23:02:18.318 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:18.319 INFO:teuthology.orchestra.run.smithi005.stdout:3585 punch from 0x8b3634b to 0x8b3cb73, (0x6828 bytes) 2024-05-11T23:02:18.325 INFO:teuthology.orchestra.run.smithi005.stdout:3586 write 0xabf540e thru 0xabf6336 (0xf29 bytes) 2024-05-11T23:02:18.329 INFO:teuthology.orchestra.run.smithi005.stdout:3588 write 0x414d1e2 thru 0x415c6e7 (0xf506 bytes) 2024-05-11T23:02:18.332 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:18.332 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:18.342 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:02:18.343 INFO:teuthology.orchestra.run.smithi005.stdout:3589 write 0xba5be5f thru 0xba667f0 (0xa992 bytes) 2024-05-11T23:02:18.352 INFO:teuthology.orchestra.run.smithi005.stdout:3590 write 0xc655cec thru 0xc65e68e (0x89a3 bytes) 2024-05-11T23:02:18.364 INFO:teuthology.orchestra.run.smithi005.stdout:3591 writesame 0xe1d1aa6 thru 0xe1d57b7 data_size 0x1(0x3d12 bytes) 2024-05-11T23:02:18.364 INFO:teuthology.orchestra.run.smithi005.stdout:3592 read 0x8498f4c thru 0x84a8473 (0xf528 bytes) 2024-05-11T23:02:18.364 INFO:teuthology.orchestra.run.smithi005.stdout:3593 punch from 0x3838d55 to 0x384232a, (0x95d5 bytes) 2024-05-11T23:02:18.365 INFO:teuthology.orchestra.run.smithi005.stdout:3594 writesame 0xa0e09cb thru 0xa0e4d91 data_size 0x1(0x43c7 bytes) 2024-05-11T23:02:18.372 INFO:teuthology.orchestra.run.smithi005.stdout:3595 write 0x11f933d thru 0x1201a33 (0x86f7 bytes) 2024-05-11T23:02:18.383 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:18.383+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:18.395 INFO:teuthology.orchestra.run.smithi005.stdout:3596 write 0x1137dcb thru 0x1143cde (0xbf14 bytes) 2024-05-11T23:02:18.414 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:18.426 INFO:teuthology.orchestra.run.smithi005.stdout:3598 trunc from 0xe1d57b8 to 0x39f5306 2024-05-11T23:02:18.451 INFO:teuthology.orchestra.run.smithi005.stdout:3599 writesame 0x79844e6 thru 0x798871e data_size 0x1(0x4239 bytes) 2024-05-11T23:02:18.458 INFO:teuthology.orchestra.run.smithi005.stdout:3601 writesame 0x8969f42 thru 0x8973ddb data_size 0x86(0x9e9a bytes) 2024-05-11T23:02:18.458 INFO:teuthology.orchestra.run.smithi005.stdout:3607 punch from 0x203ddcf to 0x204d140, (0xf371 bytes) 2024-05-11T23:02:18.459 INFO:teuthology.orchestra.run.smithi005.stdout:3608 read 0x49acc4d thru 0x49b22ed (0x56a1 bytes) 2024-05-11T23:02:18.459 INFO:teuthology.orchestra.run.smithi005.stdout:3609 read 0x2e3fbec thru 0x2e46475 (0x688a bytes) 2024-05-11T23:02:18.459 INFO:teuthology.orchestra.run.smithi005.stdout:3610 read 0x5299fd4 thru 0x52a76b5 (0xd6e2 bytes) 2024-05-11T23:02:18.459 INFO:teuthology.orchestra.run.smithi005.stdout:3611 punch from 0x256f80b to 0x257be52, (0xc647 bytes) 2024-05-11T23:02:18.459 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:18.459 INFO:teuthology.orchestra.run.smithi005.stdout:3615 punch from 0x5f4696d to 0x5f513b4, (0xaa47 bytes) 2024-05-11T23:02:18.459 INFO:teuthology.orchestra.run.smithi005.stdout:3616 writesame 0x84607df thru 0x8462ff6 data_size 0x4(0x2818 bytes) 2024-05-11T23:02:18.464 INFO:teuthology.orchestra.run.smithi005.stdout:3617 read 0x833868c thru 0x83477ff (0xf174 bytes) 2024-05-11T23:02:18.464 INFO:teuthology.orchestra.run.smithi005.stdout:3619 clone 30 order 25 su 262144 sc 4 2024-05-11T23:02:18.476 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:18.476+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:18.500 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:18.500 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:18.509 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:02:18.578 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:18.578+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:18.613 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:18.614 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:18.623 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:02:18.679 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:18.679+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:18.728 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:18.728 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:18.736 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:02:18.780 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:18.780+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:18.842 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:18.842 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:18.850 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:02:18.891 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:18.891+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:18.982 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:18.982+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:19.002 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:19.003 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:19.013 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:02:19.021 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone29 from 0x8973ddc (overlap 0x194ed5e) to 0xa1f88c 2024-05-11T23:02:19.083 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:19.083+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:19.184 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:19.183+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:19.212 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7] out_osds: [3, 6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:02:19.212 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:02:19.212 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-05-11T23:02:19.213 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-11T23:02:19.215 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:19.215 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:19.225 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:02:19.268 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-05-11T23:02:19.271 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:19.271 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:19.271 INFO:teuthology.orchestra.run.smithi005.stdout:3623 trunc from 0x8973ddc to 0x93b31fa 2024-05-11T23:02:19.280 INFO:teuthology.orchestra.run.smithi005.stdout:3624 write 0x5645185 thru 0x564fa7e (0xa8fa bytes) 2024-05-11T23:02:19.284 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:19.284+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:19.321 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:02:19.386 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:19.386+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:19.410 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:19.410 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:19.420 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:02:19.443 INFO:teuthology.orchestra.run.smithi005.stdout:3626 write 0xec80ba8 thru 0xec86e29 (0x6282 bytes) 2024-05-11T23:02:19.447 INFO:teuthology.orchestra.run.smithi005.stdout:3627 write 0xa9867ea thru 0xa98d58b (0x6da2 bytes) 2024-05-11T23:02:19.448 INFO:teuthology.orchestra.run.smithi005.stdout:3628 writesame 0x1bdce60 thru 0x1be81e9 data_size 0xe(0xb38a bytes) 2024-05-11T23:02:19.487 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:19.486+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:19.546 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:19.546 INFO:teuthology.orchestra.run.smithi005.stdout:3630 write 0x5a45d9a thru 0x5a46fa5 (0x120c bytes) 2024-05-11T23:02:19.548 INFO:teuthology.orchestra.run.smithi005.stdout:3632 write 0x66eed64 thru 0x66f2912 (0x3baf bytes) 2024-05-11T23:02:19.549 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:19.549 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:19.549 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:19.549 INFO:teuthology.orchestra.run.smithi005.stdout:3635 writesame 0x20ccc6c thru 0x20dc352 data_size 0x9(0xf6e7 bytes) 2024-05-11T23:02:19.553 INFO:teuthology.orchestra.run.smithi005.stdout:3636 read 0xebc998d thru 0xebd435f (0xa9d3 bytes) 2024-05-11T23:02:19.553 INFO:teuthology.orchestra.run.smithi005.stdout:3637 read 0x48ca74e thru 0x48d5f58 (0xb80b bytes) 2024-05-11T23:02:19.553 INFO:teuthology.orchestra.run.smithi005.stdout:3638 read 0x8745967 thru 0x875188f (0xbf29 bytes) 2024-05-11T23:02:19.554 INFO:teuthology.orchestra.run.smithi005.stdout:3640 flatten 2024-05-11T23:02:19.558 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:02:19.558 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:19.558 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 65, 2024-05-11T23:02:19.558 INFO:teuthology.orchestra.run.smithi005.stdout: "fsid": "34b20022-f11b-4169-a5f6-6386270a2f8f", 2024-05-11T23:02:19.558 INFO:teuthology.orchestra.run.smithi005.stdout: "created": "2024-05-11T23:01:05.977693+0000", 2024-05-11T23:02:19.558 INFO:teuthology.orchestra.run.smithi005.stdout: "modified": "2024-05-11T23:02:18.937250+0000", 2024-05-11T23:02:19.558 INFO:teuthology.orchestra.run.smithi005.stdout: "last_up_change": "2024-05-11T23:01:17.148916+0000", 2024-05-11T23:02:19.558 INFO:teuthology.orchestra.run.smithi005.stdout: "last_in_change": "2024-05-11T23:02:03.871950+0000", 2024-05-11T23:02:19.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-11T23:02:19.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_num": 5806080, 2024-05-11T23:02:19.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_set": [ 2024-05-11T23:02:19.559 INFO:teuthology.orchestra.run.smithi005.stdout: "nodeep-scrub", 2024-05-11T23:02:19.559 INFO:teuthology.orchestra.run.smithi005.stdout: "noscrub", 2024-05-11T23:02:19.559 INFO:teuthology.orchestra.run.smithi005.stdout: "pglog_hardlimit", 2024-05-11T23:02:19.559 INFO:teuthology.orchestra.run.smithi005.stdout: "purged_snapdirs", 2024-05-11T23:02:19.559 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_deletes", 2024-05-11T23:02:19.559 INFO:teuthology.orchestra.run.smithi005.stdout: "sortbitwise" 2024-05-11T23:02:19.559 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:02:19.559 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_version": 5, 2024-05-11T23:02:19.559 INFO:teuthology.orchestra.run.smithi005.stdout: "full_ratio": 0.94999998807907104, 2024-05-11T23:02:19.559 INFO:teuthology.orchestra.run.smithi005.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-11T23:02:19.559 INFO:teuthology.orchestra.run.smithi005.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-11T23:02:19.560 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_snapshot": "", 2024-05-11T23:02:19.560 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_max": 3, 2024-05-11T23:02:19.560 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd": 8, 2024-05-11T23:02:19.560 INFO:teuthology.orchestra.run.smithi005.stdout: "require_min_compat_client": "luminous", 2024-05-11T23:02:19.560 INFO:teuthology.orchestra.run.smithi005.stdout: "min_compat_client": "luminous", 2024-05-11T23:02:19.560 INFO:teuthology.orchestra.run.smithi005.stdout: "require_osd_release": "squid", 2024-05-11T23:02:19.560 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_crimson": false, 2024-05-11T23:02:19.560 INFO:teuthology.orchestra.run.smithi005.stdout: "pools": [ 2024-05-11T23:02:19.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.560 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 1, 2024-05-11T23:02:19.560 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": ".mgr", 2024-05-11T23:02:19.560 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-05-11T23:01:15.187614+0000", 2024-05-11T23:02:19.560 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-11T23:02:19.560 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool", 2024-05-11T23:02:19.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-05-11T23:02:19.561 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-05-11T23:02:19.561 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-05-11T23:02:19.561 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-05-11T23:02:19.561 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-05-11T23:02:19.561 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-05-11T23:02:19.561 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-05-11T23:02:19.561 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-11T23:02:19.561 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-05-11T23:02:19.561 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-05-11T23:02:19.561 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 1, 2024-05-11T23:02:19.561 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 1, 2024-05-11T23:02:19.561 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 1, 2024-05-11T23:02:19.561 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 1, 2024-05-11T23:02:19.561 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 1, 2024-05-11T23:02:19.561 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-05-11T23:02:19.562 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-05-11T23:02:19.562 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-05-11T23:02:19.562 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-05-11T23:02:19.562 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-05-11T23:02:19.562 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-05-11T23:02:19.562 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-05-11T23:02:19.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.562 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "17", 2024-05-11T23:02:19.562 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-05-11T23:02:19.562 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-11T23:02:19.562 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-05-11T23:02:19.562 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-05-11T23:02:19.562 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-05-11T23:02:19.562 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 0, 2024-05-11T23:02:19.562 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 0, 2024-05-11T23:02:19.563 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-05-11T23:02:19.563 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-05-11T23:02:19.563 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-05-11T23:02:19.563 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-05-11T23:02:19.563 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-05-11T23:02:19.563 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-05-11T23:02:19.563 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-05-11T23:02:19.563 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-05-11T23:02:19.563 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-05-11T23:02:19.563 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-05-11T23:02:19.563 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-05-11T23:02:19.563 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-11T23:02:19.563 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-11T23:02:19.563 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-11T23:02:19.563 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-05-11T23:02:19.564 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-05-11T23:02:19.564 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-05-11T23:02:19.564 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-05-11T23:02:19.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-05-11T23:02:19.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.564 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-05-11T23:02:19.564 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-05-11T23:02:19.564 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-05-11T23:02:19.564 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-05-11T23:02:19.564 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-05-11T23:02:19.564 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-05-11T23:02:19.564 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-05-11T23:02:19.564 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-05-11T23:02:19.564 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-05-11T23:02:19.564 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-05-11T23:02:19.565 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-05-11T23:02:19.565 INFO:teuthology.orchestra.run.smithi005.stdout: "options": { 2024-05-11T23:02:19.565 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_max": 32, 2024-05-11T23:02:19.565 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_min": 1 2024-05-11T23:02:19.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.565 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-05-11T23:02:19.565 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": {} 2024-05-11T23:02:19.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.565 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-05-11T23:02:19.565 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-05-11T23:02:19.565 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 8, 2024-05-11T23:02:19.565 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 8, 2024-05-11T23:02:19.565 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 0.25, 2024-05-11T23:02:19.565 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2, 2024-05-11T23:02:19.565 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2, 2024-05-11T23:02:19.566 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 1, 2024-05-11T23:02:19.566 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 1, 2024-05-11T23:02:19.566 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 1 2024-05-11T23:02:19.566 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.566 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 2, 2024-05-11T23:02:19.566 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "rbd", 2024-05-11T23:02:19.566 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-05-11T23:01:17.689285+0000", 2024-05-11T23:02:19.566 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-11T23:02:19.566 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool", 2024-05-11T23:02:19.566 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-05-11T23:02:19.566 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-05-11T23:02:19.566 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-05-11T23:02:19.567 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-05-11T23:02:19.567 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-05-11T23:02:19.567 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-05-11T23:02:19.567 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-05-11T23:02:19.567 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-11T23:02:19.567 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-05-11T23:02:19.567 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-05-11T23:02:19.567 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 8, 2024-05-11T23:02:19.567 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 8, 2024-05-11T23:02:19.567 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 8, 2024-05-11T23:02:19.567 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 8, 2024-05-11T23:02:19.567 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 8, 2024-05-11T23:02:19.567 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-05-11T23:02:19.567 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-05-11T23:02:19.568 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-05-11T23:02:19.568 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-05-11T23:02:19.568 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-05-11T23:02:19.568 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-05-11T23:02:19.568 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-05-11T23:02:19.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.568 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "18", 2024-05-11T23:02:19.568 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-05-11T23:02:19.568 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-11T23:02:19.568 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-05-11T23:02:19.568 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-05-11T23:02:19.568 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-05-11T23:02:19.568 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 0, 2024-05-11T23:02:19.568 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 0, 2024-05-11T23:02:19.568 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-05-11T23:02:19.569 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-05-11T23:02:19.569 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-05-11T23:02:19.569 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-05-11T23:02:19.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-05-11T23:02:19.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-05-11T23:02:19.569 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-05-11T23:02:19.569 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-05-11T23:02:19.569 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-05-11T23:02:19.569 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-05-11T23:02:19.569 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-05-11T23:02:19.569 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-11T23:02:19.569 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-11T23:02:19.569 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-11T23:02:19.569 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-05-11T23:02:19.569 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-05-11T23:02:19.570 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-05-11T23:02:19.570 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-05-11T23:02:19.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-05-11T23:02:19.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.570 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-05-11T23:02:19.570 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-05-11T23:02:19.570 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-05-11T23:02:19.570 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-05-11T23:02:19.570 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-05-11T23:02:19.570 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-05-11T23:02:19.570 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-05-11T23:02:19.570 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-05-11T23:02:19.570 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-05-11T23:02:19.570 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-05-11T23:02:19.571 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-05-11T23:02:19.571 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-05-11T23:02:19.571 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-05-11T23:02:19.571 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-05-11T23:02:19.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.571 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-05-11T23:02:19.571 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-05-11T23:02:19.571 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 3, 2024-05-11T23:02:19.571 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 3, 2024-05-11T23:02:19.571 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 0.75, 2024-05-11T23:02:19.571 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2.25, 2024-05-11T23:02:19.571 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2.25, 2024-05-11T23:02:19.571 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 1, 2024-05-11T23:02:19.571 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 1, 2024-05-11T23:02:19.571 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 1 2024-05-11T23:02:19.572 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.572 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 3, 2024-05-11T23:02:19.572 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "pool_client.0", 2024-05-11T23:02:19.572 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-05-11T23:01:25.743347+0000", 2024-05-11T23:02:19.572 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-05-11T23:02:19.572 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-11T23:02:19.572 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-05-11T23:02:19.572 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-05-11T23:02:19.572 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-05-11T23:02:19.572 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-05-11T23:02:19.572 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-05-11T23:02:19.572 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-05-11T23:02:19.572 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-05-11T23:02:19.573 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-11T23:02:19.573 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-05-11T23:02:19.573 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-05-11T23:02:19.573 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 32, 2024-05-11T23:02:19.573 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 32, 2024-05-11T23:02:19.573 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 32, 2024-05-11T23:02:19.573 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 32, 2024-05-11T23:02:19.573 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 32, 2024-05-11T23:02:19.573 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-05-11T23:02:19.573 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-05-11T23:02:19.573 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-05-11T23:02:19.573 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-05-11T23:02:19.573 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-05-11T23:02:19.573 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-05-11T23:02:19.573 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-05-11T23:02:19.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.574 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "65", 2024-05-11T23:02:19.574 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-05-11T23:02:19.574 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-11T23:02:19.574 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-05-11T23:02:19.574 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-05-11T23:02:19.574 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-05-11T23:02:19.574 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 32, 2024-05-11T23:02:19.574 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 65, 2024-05-11T23:02:19.574 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-05-11T23:02:19.574 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-05-11T23:02:19.574 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-05-11T23:02:19.574 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-05-11T23:02:19.574 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-05-11T23:02:19.575 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-05-11T23:02:19.575 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-05-11T23:02:19.575 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-05-11T23:02:19.575 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-05-11T23:02:19.575 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-05-11T23:02:19.575 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-05-11T23:02:19.575 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-11T23:02:19.575 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-11T23:02:19.575 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-11T23:02:19.575 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-05-11T23:02:19.575 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-05-11T23:02:19.575 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-05-11T23:02:19.575 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-05-11T23:02:19.575 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-05-11T23:02:19.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.576 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-05-11T23:02:19.576 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-05-11T23:02:19.576 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-05-11T23:02:19.576 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-05-11T23:02:19.576 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-05-11T23:02:19.576 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-05-11T23:02:19.576 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-05-11T23:02:19.576 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-05-11T23:02:19.576 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-05-11T23:02:19.576 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-05-11T23:02:19.576 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-05-11T23:02:19.576 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-05-11T23:02:19.576 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-05-11T23:02:19.576 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-05-11T23:02:19.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.576 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-05-11T23:02:19.577 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-05-11T23:02:19.577 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 2, 2024-05-11T23:02:19.577 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 2, 2024-05-11T23:02:19.577 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 0.75, 2024-05-11T23:02:19.577 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 1.5, 2024-05-11T23:02:19.577 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 1.5, 2024-05-11T23:02:19.577 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 1, 2024-05-11T23:02:19.577 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 1, 2024-05-11T23:02:19.577 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 1 2024-05-11T23:02:19.577 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.577 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.577 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:02:19.577 INFO:teuthology.orchestra.run.smithi005.stdout: "osds": [ 2024-05-11T23:02:19.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.578 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-05-11T23:02:19.578 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "57ab1c70-9478-4a26-bb41-d753b4f3eb79", 2024-05-11T23:02:19.578 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:02:19.578 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:02:19.578 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 0.95001220703125, 2024-05-11T23:02:19.578 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:02:19.578 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:02:19.578 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:02:19.578 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 13, 2024-05-11T23:02:19.578 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 60, 2024-05-11T23:02:19.578 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:02:19.578 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:02:19.578 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:02:19.578 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.579 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-05-11T23:02:19.579 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:02:19.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.579 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-05-11T23:02:19.579 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:02:19.579 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.579 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.579 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:02:19.579 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.580 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.580 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6802", 2024-05-11T23:02:19.580 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:02:19.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.580 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.580 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6803", 2024-05-11T23:02:19.580 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:02:19.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.580 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.580 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:02:19.580 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.580 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.581 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6806", 2024-05-11T23:02:19.581 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:02:19.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.581 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6808", 2024-05-11T23:02:19.581 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:02:19.581 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.581 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.581 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:02:19.581 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.581 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6804", 2024-05-11T23:02:19.582 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:02:19.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.582 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6805", 2024-05-11T23:02:19.582 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:02:19.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.582 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.582 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6801/3452725643", 2024-05-11T23:02:19.582 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6803/3452725643", 2024-05-11T23:02:19.582 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6808/3452725643", 2024-05-11T23:02:19.582 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6805/3452725643", 2024-05-11T23:02:19.582 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:02:19.583 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:02:19.583 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:02:19.583 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.583 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-05-11T23:02:19.583 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "05ece110-4a99-46f0-b061-5980f7ef2a55", 2024-05-11T23:02:19.583 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:02:19.583 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:02:19.583 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 0.95001220703125, 2024-05-11T23:02:19.583 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:02:19.583 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:02:19.583 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:02:19.583 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 14, 2024-05-11T23:02:19.583 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 50, 2024-05-11T23:02:19.583 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:02:19.584 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:02:19.584 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:02:19.584 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.584 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6824", 2024-05-11T23:02:19.584 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:02:19.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.584 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6825", 2024-05-11T23:02:19.584 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:02:19.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.584 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.585 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:02:19.585 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.585 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6826", 2024-05-11T23:02:19.585 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:02:19.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.585 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6827", 2024-05-11T23:02:19.585 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:02:19.585 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.585 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.585 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:02:19.586 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.586 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6830", 2024-05-11T23:02:19.586 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:02:19.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.586 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6831", 2024-05-11T23:02:19.586 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:02:19.586 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.586 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.586 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:02:19.586 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.587 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6828", 2024-05-11T23:02:19.587 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:02:19.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.587 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6829", 2024-05-11T23:02:19.587 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:02:19.587 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.587 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.587 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6825/4028727138", 2024-05-11T23:02:19.587 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6827/4028727138", 2024-05-11T23:02:19.587 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6831/4028727138", 2024-05-11T23:02:19.588 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6829/4028727138", 2024-05-11T23:02:19.588 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:02:19.588 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:02:19.588 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:02:19.588 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.588 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-05-11T23:02:19.588 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5", 2024-05-11T23:02:19.588 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:02:19.588 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:02:19.588 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:02:19.588 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:02:19.588 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:02:19.588 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:02:19.589 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 13, 2024-05-11T23:02:19.589 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 61, 2024-05-11T23:02:19.589 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:02:19.589 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:02:19.589 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:02:19.589 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.589 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6807", 2024-05-11T23:02:19.589 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:02:19.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.589 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6809", 2024-05-11T23:02:19.589 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:02:19.589 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.590 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.590 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:02:19.590 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.590 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6810", 2024-05-11T23:02:19.590 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:02:19.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.590 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6811", 2024-05-11T23:02:19.590 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:02:19.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.591 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.591 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:02:19.591 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.591 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6814", 2024-05-11T23:02:19.591 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:02:19.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.591 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6815", 2024-05-11T23:02:19.591 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:02:19.591 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.592 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.592 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:02:19.592 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.592 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.592 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6812", 2024-05-11T23:02:19.592 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:02:19.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.592 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.592 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6813", 2024-05-11T23:02:19.592 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:02:19.592 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.592 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.593 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6809/2059959509", 2024-05-11T23:02:19.593 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6811/2059959509", 2024-05-11T23:02:19.593 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6815/2059959509", 2024-05-11T23:02:19.593 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6813/2059959509", 2024-05-11T23:02:19.593 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:02:19.593 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:02:19.593 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:02:19.593 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.593 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-05-11T23:02:19.593 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a", 2024-05-11T23:02:19.593 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:02:19.593 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 0, 2024-05-11T23:02:19.594 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 0, 2024-05-11T23:02:19.594 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:02:19.594 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:02:19.594 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:02:19.594 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 14, 2024-05-11T23:02:19.594 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 19, 2024-05-11T23:02:19.594 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:02:19.594 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:02:19.594 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:02:19.594 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.594 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6816", 2024-05-11T23:02:19.594 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:02:19.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.595 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6817", 2024-05-11T23:02:19.595 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:02:19.595 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.595 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.595 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:02:19.595 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.595 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6818", 2024-05-11T23:02:19.595 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:02:19.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.596 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6819", 2024-05-11T23:02:19.596 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:02:19.596 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.596 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.596 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:02:19.596 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.596 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6822", 2024-05-11T23:02:19.596 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:02:19.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.597 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.597 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6823", 2024-05-11T23:02:19.597 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:02:19.597 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.597 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.597 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:02:19.597 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.597 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.597 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6820", 2024-05-11T23:02:19.597 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:02:19.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.598 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6821", 2024-05-11T23:02:19.598 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:02:19.598 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.598 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.598 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6817/296930146", 2024-05-11T23:02:19.598 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6819/296930146", 2024-05-11T23:02:19.598 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6823/296930146", 2024-05-11T23:02:19.598 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6821/296930146", 2024-05-11T23:02:19.598 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:02:19.598 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:02:19.598 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:02:19.599 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.599 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-05-11T23:02:19.599 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "4e87e4f0-288e-4ee6-919e-9337925a5d5e", 2024-05-11T23:02:19.599 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:02:19.599 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:02:19.599 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:02:19.599 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:02:19.599 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:02:19.599 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:02:19.599 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 15, 2024-05-11T23:02:19.599 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 53, 2024-05-11T23:02:19.599 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:02:19.599 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:02:19.600 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:02:19.600 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.600 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6800", 2024-05-11T23:02:19.600 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:02:19.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.600 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6801", 2024-05-11T23:02:19.600 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:02:19.600 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.600 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.601 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:02:19.601 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.601 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6802", 2024-05-11T23:02:19.601 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:02:19.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.601 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6803", 2024-05-11T23:02:19.601 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:02:19.601 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.601 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.601 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:02:19.601 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.602 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6806", 2024-05-11T23:02:19.602 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:02:19.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.602 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6807", 2024-05-11T23:02:19.602 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:02:19.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.602 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.602 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:02:19.602 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.603 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6804", 2024-05-11T23:02:19.603 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:02:19.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.603 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6805", 2024-05-11T23:02:19.603 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:02:19.603 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.603 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.603 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.115:6801/495703077", 2024-05-11T23:02:19.603 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.115:6803/495703077", 2024-05-11T23:02:19.603 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.115:6807/495703077", 2024-05-11T23:02:19.603 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.115:6805/495703077", 2024-05-11T23:02:19.603 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:02:19.604 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:02:19.604 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:02:19.604 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.604 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-05-11T23:02:19.604 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5", 2024-05-11T23:02:19.604 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:02:19.604 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:02:19.604 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:02:19.604 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:02:19.604 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:02:19.604 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:02:19.604 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 15, 2024-05-11T23:02:19.604 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 53, 2024-05-11T23:02:19.605 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:02:19.605 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:02:19.605 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:02:19.605 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.605 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.605 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6811", 2024-05-11T23:02:19.605 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:02:19.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.605 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.605 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6813", 2024-05-11T23:02:19.605 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:02:19.605 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.605 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.606 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:02:19.606 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.606 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6815", 2024-05-11T23:02:19.606 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:02:19.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.606 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6817", 2024-05-11T23:02:19.606 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:02:19.606 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.606 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.606 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:02:19.607 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.607 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6822", 2024-05-11T23:02:19.607 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:02:19.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.607 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6823", 2024-05-11T23:02:19.607 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:02:19.607 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.607 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.607 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:02:19.607 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.608 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6819", 2024-05-11T23:02:19.608 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:02:19.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.608 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6821", 2024-05-11T23:02:19.608 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:02:19.608 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.608 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.608 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.115:6813/902573489", 2024-05-11T23:02:19.608 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.115:6817/902573489", 2024-05-11T23:02:19.608 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.115:6823/902573489", 2024-05-11T23:02:19.609 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.115:6821/902573489", 2024-05-11T23:02:19.609 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:02:19.609 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:02:19.609 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:02:19.609 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.609 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 6, 2024-05-11T23:02:19.609 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "edb686ec-e1a2-40d6-975a-e32e0e7760ff", 2024-05-11T23:02:19.609 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:02:19.609 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 0, 2024-05-11T23:02:19.609 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 0, 2024-05-11T23:02:19.609 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:02:19.609 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:02:19.609 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:02:19.609 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 15, 2024-05-11T23:02:19.610 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 47, 2024-05-11T23:02:19.610 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:02:19.610 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:02:19.610 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:02:19.610 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.610 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.610 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6824", 2024-05-11T23:02:19.610 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:02:19.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.610 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.610 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6825", 2024-05-11T23:02:19.610 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:02:19.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.611 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.611 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:02:19.611 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.611 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6826", 2024-05-11T23:02:19.611 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:02:19.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.611 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6827", 2024-05-11T23:02:19.611 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:02:19.612 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.612 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.612 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:02:19.612 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.612 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6830", 2024-05-11T23:02:19.612 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:02:19.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.612 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6831", 2024-05-11T23:02:19.612 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:02:19.613 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.613 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.613 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:02:19.613 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.613 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6828", 2024-05-11T23:02:19.613 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:02:19.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.613 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6829", 2024-05-11T23:02:19.613 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:02:19.614 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.614 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.614 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.115:6825/2749332272", 2024-05-11T23:02:19.614 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.115:6827/2749332272", 2024-05-11T23:02:19.614 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.115:6831/2749332272", 2024-05-11T23:02:19.614 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.115:6829/2749332272", 2024-05-11T23:02:19.614 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:02:19.614 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:02:19.614 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:02:19.614 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.614 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 7, 2024-05-11T23:02:19.615 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "7aa14dbb-c6cb-4540-b160-3fd346f6a046", 2024-05-11T23:02:19.615 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:02:19.615 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:02:19.615 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:02:19.615 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:02:19.615 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:02:19.615 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:02:19.615 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 15, 2024-05-11T23:02:19.615 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 61, 2024-05-11T23:02:19.615 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:02:19.615 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:02:19.615 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:02:19.615 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.616 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6808", 2024-05-11T23:02:19.616 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:02:19.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.616 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6809", 2024-05-11T23:02:19.616 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:02:19.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.616 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.616 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:02:19.616 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.616 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6810", 2024-05-11T23:02:19.617 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:02:19.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.617 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6812", 2024-05-11T23:02:19.617 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:02:19.617 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.617 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.617 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:02:19.617 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.617 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6818", 2024-05-11T23:02:19.617 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:02:19.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.618 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.618 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6820", 2024-05-11T23:02:19.618 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:02:19.618 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.618 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.618 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:02:19.618 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:19.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.618 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:19.618 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6814", 2024-05-11T23:02:19.618 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:02:19.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:19.619 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6816", 2024-05-11T23:02:19.619 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:02:19.619 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.619 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.619 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.115:6809/1043826337", 2024-05-11T23:02:19.619 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.115:6812/1043826337", 2024-05-11T23:02:19.619 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.115:6820/1043826337", 2024-05-11T23:02:19.619 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.115:6816/1043826337", 2024-05-11T23:02:19.619 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:02:19.619 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:02:19.619 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:02:19.620 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.620 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:02:19.620 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_xinfo": [ 2024-05-11T23:02:19.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.620 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-05-11T23:02:19.620 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:02:19.620 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:02:19.620 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:02:19.620 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:02:19.620 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:02:19.620 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.113232+0000", 2024-05-11T23:02:19.620 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:02:19.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.621 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-05-11T23:02:19.621 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:02:19.621 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:02:19.621 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:02:19.621 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:02:19.621 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:02:19.621 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.174679+0000", 2024-05-11T23:02:19.621 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:02:19.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.621 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-05-11T23:02:19.621 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:02:19.621 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:02:19.621 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:02:19.621 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:02:19.622 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:02:19.622 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.131118+0000", 2024-05-11T23:02:19.622 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:02:19.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.622 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-05-11T23:02:19.622 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:02:19.622 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:02:19.622 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:02:19.622 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:02:19.622 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 58984, 2024-05-11T23:02:19.622 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.207668+0000", 2024-05-11T23:02:19.622 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:02:19.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.623 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-05-11T23:02:19.623 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:02:19.623 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:02:19.623 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:02:19.623 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:02:19.623 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:02:19.623 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.124542+0000", 2024-05-11T23:02:19.623 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:02:19.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.623 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-05-11T23:02:19.623 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:02:19.623 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:02:19.623 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:02:19.624 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:02:19.624 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:02:19.624 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.200832+0000", 2024-05-11T23:02:19.624 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:02:19.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.625 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 6, 2024-05-11T23:02:19.625 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:02:19.625 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:02:19.625 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:02:19.625 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:02:19.625 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 65536, 2024-05-11T23:02:19.625 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.169560+0000", 2024-05-11T23:02:19.626 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:02:19.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.626 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 7, 2024-05-11T23:02:19.626 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:02:19.626 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:02:19.626 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:02:19.626 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:02:19.626 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:02:19.626 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.182465+0000", 2024-05-11T23:02:19.626 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:02:19.626 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.626 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:02:19.626 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap": [], 2024-05-11T23:02:19.626 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_items": [ 2024-05-11T23:02:19.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.627 INFO:teuthology.orchestra.run.smithi005.stdout: "pgid": "3.3", 2024-05-11T23:02:19.627 INFO:teuthology.orchestra.run.smithi005.stdout: "mappings": [ 2024-05-11T23:02:19.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.627 INFO:teuthology.orchestra.run.smithi005.stdout: "from": 4, 2024-05-11T23:02:19.627 INFO:teuthology.orchestra.run.smithi005.stdout: "to": 2 2024-05-11T23:02:19.627 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.627 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.627 INFO:teuthology.orchestra.run.smithi005.stdout: "pgid": "3.11", 2024-05-11T23:02:19.627 INFO:teuthology.orchestra.run.smithi005.stdout: "mappings": [ 2024-05-11T23:02:19.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.627 INFO:teuthology.orchestra.run.smithi005.stdout: "from": 4, 2024-05-11T23:02:19.627 INFO:teuthology.orchestra.run.smithi005.stdout: "to": 2 2024-05-11T23:02:19.628 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.628 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.628 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.628 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:02:19.628 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_primaries": [], 2024-05-11T23:02:19.628 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_temp": [], 2024-05-11T23:02:19.628 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_temp": [], 2024-05-11T23:02:19.628 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklist": {}, 2024-05-11T23:02:19.628 INFO:teuthology.orchestra.run.smithi005.stdout: "range_blocklist": {}, 2024-05-11T23:02:19.628 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profiles": { 2024-05-11T23:02:19.628 INFO:teuthology.orchestra.run.smithi005.stdout: "default": { 2024-05-11T23:02:19.628 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-failure-domain": "osd", 2024-05-11T23:02:19.628 INFO:teuthology.orchestra.run.smithi005.stdout: "k": "2", 2024-05-11T23:02:19.628 INFO:teuthology.orchestra.run.smithi005.stdout: "m": "1", 2024-05-11T23:02:19.629 INFO:teuthology.orchestra.run.smithi005.stdout: "plugin": "jerasure", 2024-05-11T23:02:19.629 INFO:teuthology.orchestra.run.smithi005.stdout: "technique": "reed_sol_van" 2024-05-11T23:02:19.629 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.629 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps_queue": [], 2024-05-11T23:02:19.629 INFO:teuthology.orchestra.run.smithi005.stdout: "new_removed_snaps": [], 2024-05-11T23:02:19.629 INFO:teuthology.orchestra.run.smithi005.stdout: "new_purged_snaps": [], 2024-05-11T23:02:19.629 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_node_flags": {}, 2024-05-11T23:02:19.629 INFO:teuthology.orchestra.run.smithi005.stdout: "device_class_flags": {}, 2024-05-11T23:02:19.629 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode": { 2024-05-11T23:02:19.629 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_enabled": false, 2024-05-11T23:02:19.629 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_bucket_count": 0, 2024-05-11T23:02:19.629 INFO:teuthology.orchestra.run.smithi005.stdout: "degraded_stretch_mode": 0, 2024-05-11T23:02:19.629 INFO:teuthology.orchestra.run.smithi005.stdout: "recovering_stretch_mode": 0, 2024-05-11T23:02:19.629 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_bucket": 0 2024-05-11T23:02:19.630 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.630 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:19.631 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:02:19.633 INFO:tasks.thrashosds.thrasher:j is {'epoch': 65, 'fsid': '34b20022-f11b-4169-a5f6-6386270a2f8f', 'created': '2024-05-11T23:01:05.977693+0000', 'modified': '2024-05-11T23:02:18.937250+0000', 'last_up_change': '2024-05-11T23:01:17.148916+0000', 'last_in_change': '2024-05-11T23:02:03.871950+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-11T23:01:15.187614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-05-11T23:01:17.689285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 0.75, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-11T23:01:25.743347+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 32, 'snap_epoch': 65, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 0.75, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '57ab1c70-9478-4a26-bb41-d753b4f3eb79', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6800', 'nonce': 3452725643}, {'type': 'v1', 'addr': '172.21.15.5:6801', 'nonce': 3452725643}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6802', 'nonce': 3452725643}, {'type': 'v1', 'addr': '172.21.15.5:6803', 'nonce': 3452725643}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6806', 'nonce': 3452725643}, {'type': 'v1', 'addr': '172.21.15.5:6808', 'nonce': 3452725643}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6804', 'nonce': 3452725643}, {'type': 'v1', 'addr': '172.21.15.5:6805', 'nonce': 3452725643}]}, 'public_addr': '172.21.15.5:6801/3452725643', 'cluster_addr': '172.21.15.5:6803/3452725643', 'heartbeat_back_addr': '172.21.15.5:6808/3452725643', 'heartbeat_front_addr': '172.21.15.5:6805/3452725643', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '05ece110-4a99-46f0-b061-5980f7ef2a55', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6824', 'nonce': 4028727138}, {'type': 'v1', 'addr': '172.21.15.5:6825', 'nonce': 4028727138}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6826', 'nonce': 4028727138}, {'type': 'v1', 'addr': '172.21.15.5:6827', 'nonce': 4028727138}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6830', 'nonce': 4028727138}, {'type': 'v1', 'addr': '172.21.15.5:6831', 'nonce': 4028727138}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6828', 'nonce': 4028727138}, {'type': 'v1', 'addr': '172.21.15.5:6829', 'nonce': 4028727138}]}, 'public_addr': '172.21.15.5:6825/4028727138', 'cluster_addr': '172.21.15.5:6827/4028727138', 'heartbeat_back_addr': '172.21.15.5:6831/4028727138', 'heartbeat_front_addr': '172.21.15.5:6829/4028727138', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 61, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6807', 'nonce': 2059959509}, {'type': 'v1', 'addr': '172.21.15.5:6809', 'nonce': 2059959509}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6810', 'nonce': 2059959509}, {'type': 'v1', 'addr': '172.21.15.5:6811', 'nonce': 2059959509}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6814', 'nonce': 2059959509}, {'type': 'v1', 'addr': '172.21.15.5:6815', 'nonce': 2059959509}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6812', 'nonce': 2059959509}, {'type': 'v1', 'addr': '172.21.15.5:6813', 'nonce': 2059959509}]}, 'public_addr': '172.21.15.5:6809/2059959509', 'cluster_addr': '172.21.15.5:6811/2059959509', 'heartbeat_back_addr': '172.21.15.5:6815/2059959509', 'heartbeat_front_addr': '172.21.15.5:6813/2059959509', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6816', 'nonce': 296930146}, {'type': 'v1', 'addr': '172.21.15.5:6817', 'nonce': 296930146}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6818', 'nonce': 296930146}, {'type': 'v1', 'addr': '172.21.15.5:6819', 'nonce': 296930146}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6822', 'nonce': 296930146}, {'type': 'v1', 'addr': '172.21.15.5:6823', 'nonce': 296930146}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6820', 'nonce': 296930146}, {'type': 'v1', 'addr': '172.21.15.5:6821', 'nonce': 296930146}]}, 'public_addr': '172.21.15.5:6817/296930146', 'cluster_addr': '172.21.15.5:6819/296930146', 'heartbeat_back_addr': '172.21.15.5:6823/296930146', 'heartbeat_front_addr': '172.21.15.5:6821/296930146', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '4e87e4f0-288e-4ee6-919e-9337925a5d5e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 495703077}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 495703077}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 495703077}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 495703077}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 495703077}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 495703077}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 495703077}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 495703077}]}, 'public_addr': '172.21.15.115:6801/495703077', 'cluster_addr': '172.21.15.115:6803/495703077', 'heartbeat_back_addr': '172.21.15.115:6807/495703077', 'heartbeat_front_addr': '172.21.15.115:6805/495703077', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6811', 'nonce': 902573489}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 902573489}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6815', 'nonce': 902573489}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 902573489}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 902573489}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 902573489}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6819', 'nonce': 902573489}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 902573489}]}, 'public_addr': '172.21.15.115:6813/902573489', 'cluster_addr': '172.21.15.115:6817/902573489', 'heartbeat_back_addr': '172.21.15.115:6823/902573489', 'heartbeat_front_addr': '172.21.15.115:6821/902573489', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'edb686ec-e1a2-40d6-975a-e32e0e7760ff', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 47, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6824', 'nonce': 2749332272}, {'type': 'v1', 'addr': '172.21.15.115:6825', 'nonce': 2749332272}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 2749332272}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 2749332272}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 2749332272}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 2749332272}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 2749332272}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 2749332272}]}, 'public_addr': '172.21.15.115:6825/2749332272', 'cluster_addr': '172.21.15.115:6827/2749332272', 'heartbeat_back_addr': '172.21.15.115:6831/2749332272', 'heartbeat_front_addr': '172.21.15.115:6829/2749332272', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '7aa14dbb-c6cb-4540-b160-3fd346f6a046', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 61, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 1043826337}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 1043826337}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 1043826337}, {'type': 'v1', 'addr': '172.21.15.115:6812', 'nonce': 1043826337}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 1043826337}, {'type': 'v1', 'addr': '172.21.15.115:6820', 'nonce': 1043826337}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 1043826337}, {'type': 'v1', 'addr': '172.21.15.115:6816', 'nonce': 1043826337}]}, 'public_addr': '172.21.15.115:6809/1043826337', 'cluster_addr': '172.21.15.115:6812/1043826337', 'heartbeat_back_addr': '172.21.15.115:6820/1043826337', 'heartbeat_front_addr': '172.21.15.115:6816/1043826337', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.113232+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.174679+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.131118+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 58984, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.207668+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.124542+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.200832+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.169560+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.182465+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.3', 'mappings': [{'from': 4, 'to': 2}]}, {'pgid': '3.11', 'mappings': [{'from': 4, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-11T23:02:19.633 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.11 2024-05-11T23:02:19.633 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap-items 3.11 2024-05-11T23:02:19.634 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:19.630+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:19.666 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:19.667 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:19.689 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:19.690 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:19.714 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:19.715 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:19.732 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:19.732+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:19.732 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:19.745 INFO:teuthology.orchestra.run.smithi005.stdout:3641 read 0x6f91a20 thru 0x6f98706 (0x6ce7 bytes) 2024-05-11T23:02:19.751 INFO:teuthology.orchestra.run.smithi005.stdout:3642 trunc from 0xec86e2a to 0xa3ee955 2024-05-11T23:02:19.752 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:19.752 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:19.759 INFO:teuthology.orchestra.run.smithi005.stdout:3643 read 0x12e7707 thru 0x12e948c (0x1d86 bytes) 2024-05-11T23:02:19.760 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:19.760 INFO:teuthology.orchestra.run.smithi005.stdout:3645 write 0x8a0d103 thru 0x8a0fec9 (0x2dc7 bytes) 2024-05-11T23:02:19.763 INFO:teuthology.orchestra.run.smithi005.stdout:3646 clone 31 order 24 su 262144 sc 7 2024-05-11T23:02:19.772 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:19.772 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:19.789 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:19.790 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:19.797 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:19.797 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:19.797 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:19.797 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:19.797 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:19.806 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:02:19.808 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:19.833 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:19.833+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:19.919 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:19.919 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:19.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:19.920 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:19.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.920 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71308 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:19.920 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.146883+0000", 2024-05-11T23:02:19.920 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7716872410000004, 2024-05-11T23:02:19.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030851522999999999, 2024-05-11T23:02:19.920 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:19.920 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:19.920 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:19.920 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:19.920 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:19.920 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:19.920 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71308 2024-05-11T23:02:19.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.921 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:19.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:19.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146883+0000", 2024-05-11T23:02:19.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:19.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146883+0000", 2024-05-11T23:02:19.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:19.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146884+0000", 2024-05-11T23:02:19.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3399999999999999e-07 2024-05-11T23:02:19.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:19.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146893+0000", 2024-05-11T23:02:19.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2159999999999995e-06 2024-05-11T23:02:19.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:19.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146899+0000", 2024-05-11T23:02:19.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.302e-06 2024-05-11T23:02:19.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:19.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146905+0000", 2024-05-11T23:02:19.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9120000000000003e-06 2024-05-11T23:02:19.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:19.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177730+0000", 2024-05-11T23:02:19.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030824449 2024-05-11T23:02:19.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:19.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177735+0000", 2024-05-11T23:02:19.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1100000000000002e-06 2024-05-11T23:02:19.924 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.924 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.924 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.925 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71309 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:19.925 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.146921+0000", 2024-05-11T23:02:19.925 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7716489790000001, 2024-05-11T23:02:19.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030854467999999999, 2024-05-11T23:02:19.925 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:19.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:19.925 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:19.925 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:19.925 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:19.925 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:19.925 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71309 2024-05-11T23:02:19.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.926 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:19.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:19.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146921+0000", 2024-05-11T23:02:19.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:19.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146921+0000", 2024-05-11T23:02:19.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:19.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146922+0000", 2024-05-11T23:02:19.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1799999999999995e-07 2024-05-11T23:02:19.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:19.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146924+0000", 2024-05-11T23:02:19.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5009999999999999e-06 2024-05-11T23:02:19.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:19.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146938+0000", 2024-05-11T23:02:19.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3286e-05 2024-05-11T23:02:19.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:19.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146944+0000", 2024-05-11T23:02:19.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7839999999999998e-06 2024-05-11T23:02:19.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:19.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177766+0000", 2024-05-11T23:02:19.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030821988000000002 2024-05-11T23:02:19.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:19.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177776+0000", 2024-05-11T23:02:19.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3910000000000004e-06 2024-05-11T23:02:19.929 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.929 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.929 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.929 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71310 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:19.929 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.146954+0000", 2024-05-11T23:02:19.929 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7716166419999997, 2024-05-11T23:02:19.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030837251999999999, 2024-05-11T23:02:19.929 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:19.929 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:19.930 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:19.930 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:19.930 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:19.930 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:19.930 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71310 2024-05-11T23:02:19.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.930 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:19.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:19.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146954+0000", 2024-05-11T23:02:19.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:19.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146954+0000", 2024-05-11T23:02:19.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:19.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146954+0000", 2024-05-11T23:02:19.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:02:19.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:19.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146957+0000", 2024-05-11T23:02:19.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.463e-06 2024-05-11T23:02:19.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:19.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146963+0000", 2024-05-11T23:02:19.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6460000000000004e-06 2024-05-11T23:02:19.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:19.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146968+0000", 2024-05-11T23:02:19.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0250000000000002e-06 2024-05-11T23:02:19.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:19.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177787+0000", 2024-05-11T23:02:19.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030818680000000001 2024-05-11T23:02:19.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:19.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177791+0000", 2024-05-11T23:02:19.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0090000000000001e-06 2024-05-11T23:02:19.933 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.933 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.933 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.933 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71313 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:19.933 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147016+0000", 2024-05-11T23:02:19.933 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7715539600000003, 2024-05-11T23:02:19.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030841770000000001, 2024-05-11T23:02:19.933 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:19.933 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:19.933 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:19.933 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:19.933 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:19.934 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:19.934 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71313 2024-05-11T23:02:19.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.934 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:19.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:19.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147016+0000", 2024-05-11T23:02:19.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:19.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147016+0000", 2024-05-11T23:02:19.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:19.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147017+0000", 2024-05-11T23:02:19.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8599999999999999e-07 2024-05-11T23:02:19.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:19.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147019+0000", 2024-05-11T23:02:19.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0329999999999998e-06 2024-05-11T23:02:19.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:19.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147025+0000", 2024-05-11T23:02:19.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0299999999999999e-06 2024-05-11T23:02:19.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:19.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147029+0000", 2024-05-11T23:02:19.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5979999999999999e-06 2024-05-11T23:02:19.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:19.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177848+0000", 2024-05-11T23:02:19.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030818662 2024-05-11T23:02:19.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:19.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177858+0000", 2024-05-11T23:02:19.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0061000000000001e-05 2024-05-11T23:02:19.937 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.937 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.937 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.938 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71327 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:19.938 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147299+0000", 2024-05-11T23:02:19.938 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7712712719999999, 2024-05-11T23:02:19.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030857026999999999, 2024-05-11T23:02:19.938 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:19.938 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:19.938 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:19.938 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:19.939 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:19.939 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:19.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71327 2024-05-11T23:02:19.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.939 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:19.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:19.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147299+0000", 2024-05-11T23:02:19.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:19.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147299+0000", 2024-05-11T23:02:19.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:19.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147299+0000", 2024-05-11T23:02:19.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1500000000000001e-07 2024-05-11T23:02:19.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:19.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147303+0000", 2024-05-11T23:02:19.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1459999999999998e-06 2024-05-11T23:02:19.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:19.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147312+0000", 2024-05-11T23:02:19.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3299999999999999e-06 2024-05-11T23:02:19.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:19.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147321+0000", 2024-05-11T23:02:19.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7420000000000003e-06 2024-05-11T23:02:19.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:19.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178145+0000", 2024-05-11T23:02:19.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030824852999999999 2024-05-11T23:02:19.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:19.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178156+0000", 2024-05-11T23:02:19.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0740999999999999e-05 2024-05-11T23:02:19.942 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.942 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.942 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.942 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71328 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:19.942 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147326+0000", 2024-05-11T23:02:19.942 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7712447900000008, 2024-05-11T23:02:19.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030846630999999999, 2024-05-11T23:02:19.942 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:19.942 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:19.942 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:19.943 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:19.943 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:19.943 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:19.943 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71328 2024-05-11T23:02:19.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.943 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:19.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:19.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147326+0000", 2024-05-11T23:02:19.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:19.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147326+0000", 2024-05-11T23:02:19.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:19.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147326+0000", 2024-05-11T23:02:19.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5400000000000002e-07 2024-05-11T23:02:19.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:19.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147327+0000", 2024-05-11T23:02:19.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2500000000000001e-06 2024-05-11T23:02:19.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:19.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147331+0000", 2024-05-11T23:02:19.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5499999999999999e-06 2024-05-11T23:02:19.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:19.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147333+0000", 2024-05-11T23:02:19.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5450000000000002e-06 2024-05-11T23:02:19.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:19.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178168+0000", 2024-05-11T23:02:19.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030834934000000001 2024-05-11T23:02:19.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:19.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178172+0000", 2024-05-11T23:02:19.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0980000000000004e-06 2024-05-11T23:02:19.945 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.946 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.946 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71329 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:19.946 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147336+0000", 2024-05-11T23:02:19.946 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7712339359999998, 2024-05-11T23:02:19.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030865317999999999, 2024-05-11T23:02:19.946 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:19.946 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:19.946 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:19.946 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:19.946 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:19.946 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:19.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71329 2024-05-11T23:02:19.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.947 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:19.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:19.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147336+0000", 2024-05-11T23:02:19.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:19.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147336+0000", 2024-05-11T23:02:19.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:19.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147337+0000", 2024-05-11T23:02:19.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:19.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:19.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147338+0000", 2024-05-11T23:02:19.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5090000000000001e-06 2024-05-11T23:02:19.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:19.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147344+0000", 2024-05-11T23:02:19.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.66e-06 2024-05-11T23:02:19.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:19.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147348+0000", 2024-05-11T23:02:19.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2980000000000002e-06 2024-05-11T23:02:19.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:19.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178192+0000", 2024-05-11T23:02:19.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030843425000000001 2024-05-11T23:02:19.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:19.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178202+0000", 2024-05-11T23:02:19.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0217000000000001e-05 2024-05-11T23:02:19.950 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.950 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.950 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.950 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71330 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:19.950 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147355+0000", 2024-05-11T23:02:19.951 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7712157820000005, 2024-05-11T23:02:19.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030862031000000002, 2024-05-11T23:02:19.951 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:19.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:19.951 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:19.951 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:19.951 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:19.951 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:19.951 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71330 2024-05-11T23:02:19.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.951 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:19.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:19.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147355+0000", 2024-05-11T23:02:19.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:19.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147355+0000", 2024-05-11T23:02:19.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:19.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147355+0000", 2024-05-11T23:02:19.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.39e-07 2024-05-11T23:02:19.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:19.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147365+0000", 2024-05-11T23:02:19.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7140000000000005e-06 2024-05-11T23:02:19.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:19.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147370+0000", 2024-05-11T23:02:19.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2979999999999999e-06 2024-05-11T23:02:19.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:19.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147373+0000", 2024-05-11T23:02:19.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.41e-06 2024-05-11T23:02:19.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:19.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178213+0000", 2024-05-11T23:02:19.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030839325000000001 2024-05-11T23:02:19.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:19.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178217+0000", 2024-05-11T23:02:19.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9450000000000003e-06 2024-05-11T23:02:19.955 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.955 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.955 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.955 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71331 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:19.955 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147391+0000", 2024-05-11T23:02:19.955 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7711788609999992, 2024-05-11T23:02:19.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030863854999999999, 2024-05-11T23:02:19.956 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:19.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:19.956 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:19.956 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:19.956 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:19.956 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:19.956 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71331 2024-05-11T23:02:19.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.956 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:19.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:19.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147391+0000", 2024-05-11T23:02:19.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:19.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147391+0000", 2024-05-11T23:02:19.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:19.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147392+0000", 2024-05-11T23:02:19.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.68e-07 2024-05-11T23:02:19.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:19.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147395+0000", 2024-05-11T23:02:19.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.503e-06 2024-05-11T23:02:19.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:19.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147401+0000", 2024-05-11T23:02:19.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7889999999999997e-06 2024-05-11T23:02:19.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:19.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147406+0000", 2024-05-11T23:02:19.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.831e-06 2024-05-11T23:02:19.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:19.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178245+0000", 2024-05-11T23:02:19.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030838884 2024-05-11T23:02:19.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:19.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178255+0000", 2024-05-11T23:02:19.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.028e-05 2024-05-11T23:02:19.960 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.960 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.960 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.960 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71332 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:19.960 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147414+0000", 2024-05-11T23:02:19.960 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7711568100000008, 2024-05-11T23:02:19.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030874374, 2024-05-11T23:02:19.960 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:19.960 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:19.960 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:19.960 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:19.960 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71332 2024-05-11T23:02:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147414+0000", 2024-05-11T23:02:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147414+0000", 2024-05-11T23:02:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:19.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147414+0000", 2024-05-11T23:02:19.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6899999999999998e-07 2024-05-11T23:02:19.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:19.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147416+0000", 2024-05-11T23:02:19.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:02:19.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:19.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147428+0000", 2024-05-11T23:02:19.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1428e-05 2024-05-11T23:02:19.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:19.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147436+0000", 2024-05-11T23:02:19.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9689999999999994e-06 2024-05-11T23:02:19.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:19.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178283+0000", 2024-05-11T23:02:19.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030847764 2024-05-11T23:02:19.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:19.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178288+0000", 2024-05-11T23:02:19.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6430000000000003e-06 2024-05-11T23:02:19.964 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.964 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.964 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.964 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71333 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:19.964 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147444+0000", 2024-05-11T23:02:19.964 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.771126829, 2024-05-11T23:02:19.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.03086906, 2024-05-11T23:02:19.964 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:19.964 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:19.964 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:19.964 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:19.965 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:19.965 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:19.965 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71333 2024-05-11T23:02:19.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.965 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:19.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:19.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147444+0000", 2024-05-11T23:02:19.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:19.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147444+0000", 2024-05-11T23:02:19.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:19.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147444+0000", 2024-05-11T23:02:19.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:02:19.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:19.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147446+0000", 2024-05-11T23:02:19.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3590000000000002e-06 2024-05-11T23:02:19.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:19.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147453+0000", 2024-05-11T23:02:19.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7580000000000003e-06 2024-05-11T23:02:19.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:19.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147460+0000", 2024-05-11T23:02:19.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5010000000000001e-06 2024-05-11T23:02:19.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:19.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178308+0000", 2024-05-11T23:02:19.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030848179 2024-05-11T23:02:19.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:19.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178313+0000", 2024-05-11T23:02:19.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8219999999999997e-06 2024-05-11T23:02:19.968 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.968 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.968 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.968 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71334 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:19.968 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147466+0000", 2024-05-11T23:02:19.968 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7711041590000001, 2024-05-11T23:02:19.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030868219999999998, 2024-05-11T23:02:19.968 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:19.969 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:19.969 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:19.969 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:19.969 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:19.969 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:19.969 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71334 2024-05-11T23:02:19.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.969 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:19.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:19.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147466+0000", 2024-05-11T23:02:19.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:19.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147466+0000", 2024-05-11T23:02:19.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:19.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147467+0000", 2024-05-11T23:02:19.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9099999999999999e-07 2024-05-11T23:02:19.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:19.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147469+0000", 2024-05-11T23:02:19.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9769999999999999e-06 2024-05-11T23:02:19.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:19.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147475+0000", 2024-05-11T23:02:19.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1369999999999999e-06 2024-05-11T23:02:19.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:19.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147479+0000", 2024-05-11T23:02:19.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6639999999999997e-06 2024-05-11T23:02:19.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:19.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178330+0000", 2024-05-11T23:02:19.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030850776 2024-05-11T23:02:19.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:19.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178334+0000", 2024-05-11T23:02:19.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2749999999999997e-06 2024-05-11T23:02:19.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.972 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.972 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71335 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:19.972 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147489+0000", 2024-05-11T23:02:19.972 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7710816020000006, 2024-05-11T23:02:19.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030866874999999998, 2024-05-11T23:02:19.973 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:19.973 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:19.973 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:19.973 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:19.973 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:19.973 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:19.973 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71335 2024-05-11T23:02:19.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.973 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:19.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:19.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147489+0000", 2024-05-11T23:02:19.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:19.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147489+0000", 2024-05-11T23:02:19.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:19.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147489+0000", 2024-05-11T23:02:19.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7899999999999999e-07 2024-05-11T23:02:19.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:19.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147491+0000", 2024-05-11T23:02:19.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0600000000000002e-06 2024-05-11T23:02:19.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:19.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147498+0000", 2024-05-11T23:02:19.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3339999999999998e-06 2024-05-11T23:02:19.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:19.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147502+0000", 2024-05-11T23:02:19.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8629999999999999e-06 2024-05-11T23:02:19.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:19.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178350+0000", 2024-05-11T23:02:19.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030847795000000001 2024-05-11T23:02:19.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:19.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178356+0000", 2024-05-11T23:02:19.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4439999999999999e-06 2024-05-11T23:02:19.976 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.976 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.976 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.976 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71336 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:19.976 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147508+0000", 2024-05-11T23:02:19.976 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7710618609999997, 2024-05-11T23:02:19.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030867196999999999, 2024-05-11T23:02:19.976 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:19.976 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:19.977 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:19.977 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:19.977 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:19.977 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:19.977 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71336 2024-05-11T23:02:19.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.977 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:19.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:19.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147508+0000", 2024-05-11T23:02:19.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:19.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147508+0000", 2024-05-11T23:02:19.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:19.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147509+0000", 2024-05-11T23:02:19.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:02:19.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:19.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147511+0000", 2024-05-11T23:02:19.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.88e-06 2024-05-11T23:02:19.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:19.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147517+0000", 2024-05-11T23:02:19.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7849999999999997e-06 2024-05-11T23:02:19.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:19.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147521+0000", 2024-05-11T23:02:19.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4240000000000003e-06 2024-05-11T23:02:19.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:19.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178371+0000", 2024-05-11T23:02:19.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030850452 2024-05-11T23:02:19.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:19.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178376+0000", 2024-05-11T23:02:19.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2560000000000004e-06 2024-05-11T23:02:19.980 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.980 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.980 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.980 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71337 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:19.980 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147527+0000", 2024-05-11T23:02:19.980 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7710432639999993, 2024-05-11T23:02:19.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030869829000000001, 2024-05-11T23:02:19.980 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:19.980 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:19.980 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:19.980 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:19.980 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:19.981 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:19.981 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71337 2024-05-11T23:02:19.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.981 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:19.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:19.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147527+0000", 2024-05-11T23:02:19.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:19.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147527+0000", 2024-05-11T23:02:19.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:19.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147527+0000", 2024-05-11T23:02:19.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3099999999999999e-07 2024-05-11T23:02:19.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:19.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147529+0000", 2024-05-11T23:02:19.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8870000000000001e-06 2024-05-11T23:02:19.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:19.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147536+0000", 2024-05-11T23:02:19.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2910000000000004e-06 2024-05-11T23:02:19.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:19.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147540+0000", 2024-05-11T23:02:19.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8189999999999999e-06 2024-05-11T23:02:19.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:19.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178393+0000", 2024-05-11T23:02:19.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030852118000000001 2024-05-11T23:02:19.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:19.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178397+0000", 2024-05-11T23:02:19.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3830000000000002e-06 2024-05-11T23:02:19.983 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.983 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.984 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.984 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71338 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:19.984 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147546+0000", 2024-05-11T23:02:19.984 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7710240049999992, 2024-05-11T23:02:19.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030872143000000001, 2024-05-11T23:02:19.984 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:19.984 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:19.984 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:19.984 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:19.984 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:19.984 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:19.985 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71338 2024-05-11T23:02:19.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.985 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:19.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:19.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147546+0000", 2024-05-11T23:02:19.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:19.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147546+0000", 2024-05-11T23:02:19.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:19.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147547+0000", 2024-05-11T23:02:19.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7500000000000001e-07 2024-05-11T23:02:19.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:19.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147549+0000", 2024-05-11T23:02:19.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0040000000000002e-06 2024-05-11T23:02:19.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:19.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147555+0000", 2024-05-11T23:02:19.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1959999999999996e-06 2024-05-11T23:02:19.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:19.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147559+0000", 2024-05-11T23:02:19.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3329999999999998e-06 2024-05-11T23:02:19.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:19.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178414+0000", 2024-05-11T23:02:19.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030854837999999999 2024-05-11T23:02:19.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:19.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178418+0000", 2024-05-11T23:02:19.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3970000000000004e-06 2024-05-11T23:02:19.990 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.990 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.990 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.990 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71339 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:19.990 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147570+0000", 2024-05-11T23:02:19.991 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7710000069999996, 2024-05-11T23:02:19.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030868237, 2024-05-11T23:02:19.991 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:19.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:19.991 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:19.991 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:19.992 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:19.992 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:19.992 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71339 2024-05-11T23:02:19.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.992 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:19.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:19.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147570+0000", 2024-05-11T23:02:19.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:19.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147570+0000", 2024-05-11T23:02:19.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:19.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:19.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147571+0000", 2024-05-11T23:02:19.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2399999999999999e-07 2024-05-11T23:02:19.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:19.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147573+0000", 2024-05-11T23:02:19.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.43e-06 2024-05-11T23:02:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147589+0000", 2024-05-11T23:02:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5977999999999998e-05 2024-05-11T23:02:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:19.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147595+0000", 2024-05-11T23:02:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.463e-06 2024-05-11T23:02:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178434+0000", 2024-05-11T23:02:19.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030839582000000001 2024-05-11T23:02:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178439+0000", 2024-05-11T23:02:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3599999999999998e-06 2024-05-11T23:02:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.998 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71340 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147614+0000", 2024-05-11T23:02:19.999 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.770955914, 2024-05-11T23:02:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030844255000000001, 2024-05-11T23:02:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.000 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71340 2024-05-11T23:02:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147614+0000", 2024-05-11T23:02:20.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147614+0000", 2024-05-11T23:02:20.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147615+0000", 2024-05-11T23:02:20.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4200000000000001e-07 2024-05-11T23:02:20.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147617+0000", 2024-05-11T23:02:20.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5150000000000001e-06 2024-05-11T23:02:20.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147626+0000", 2024-05-11T23:02:20.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1950000000000003e-06 2024-05-11T23:02:20.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147631+0000", 2024-05-11T23:02:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3369999999999999e-06 2024-05-11T23:02:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178454+0000", 2024-05-11T23:02:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030823481999999999 2024-05-11T23:02:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.007 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178459+0000", 2024-05-11T23:02:20.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.284e-06 2024-05-11T23:02:20.008 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.008 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.008 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.009 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71341 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:20.009 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147639+0000", 2024-05-11T23:02:20.009 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.7709318310000004, 2024-05-11T23:02:20.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030842306999999999, 2024-05-11T23:02:20.010 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.010 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.010 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:20.011 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.011 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.011 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.011 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71341 2024-05-11T23:02:20.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.011 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147639+0000", 2024-05-11T23:02:20.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147639+0000", 2024-05-11T23:02:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147639+0000", 2024-05-11T23:02:20.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6600000000000002e-07 2024-05-11T23:02:20.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147646+0000", 2024-05-11T23:02:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8870000000000003e-06 2024-05-11T23:02:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147650+0000", 2024-05-11T23:02:20.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4510000000000002e-06 2024-05-11T23:02:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147653+0000", 2024-05-11T23:02:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8700000000000001e-06 2024-05-11T23:02:20.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178476+0000", 2024-05-11T23:02:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030823350999999999 2024-05-11T23:02:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178481+0000", 2024-05-11T23:02:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3819999999999997e-06 2024-05-11T23:02:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:81250 3.b 3:d7828050:::rbd_data.111e7ffeb370.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:16.371363+0000", 2024-05-11T23:02:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 3.547207475, 2024-05-11T23:02:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.037265637999999997, 2024-05-11T23:02:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.019 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 81250 2024-05-11T23:02:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.371363+0000", 2024-05-11T23:02:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.371363+0000", 2024-05-11T23:02:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.371365+0000", 2024-05-11T23:02:20.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1330000000000002e-06 2024-05-11T23:02:20.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.372770+0000", 2024-05-11T23:02:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0014046110000000001 2024-05-11T23:02:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.372790+0000", 2024-05-11T23:02:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0530000000000002e-05 2024-05-11T23:02:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.372799+0000", 2024-05-11T23:02:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3000000000000007e-06 2024-05-11T23:02:20.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.372883+0000", 2024-05-11T23:02:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3709999999999996e-05 2024-05-11T23:02:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.373076+0000", 2024-05-11T23:02:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019249499999999999 2024-05-11T23:02:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-05-11T23:02:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.380091+0000", 2024-05-11T23:02:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.007015533 2024-05-11T23:02:20.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.408542+0000", 2024-05-11T23:02:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028451026000000001 2024-05-11T23:02:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:02:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.408555+0000", 2024-05-11T23:02:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3151e-05 2024-05-11T23:02:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.408592+0000", 2024-05-11T23:02:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6463e-05 2024-05-11T23:02:20.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.408629+0000", 2024-05-11T23:02:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6686000000000001e-05 2024-05-11T23:02:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.026 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.026 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:20.027 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-11T23:02:20.028 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:20.027+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:20.029 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:02:20.129 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:20.129+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:20.134 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-05-11T23:02:20.194 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:20.195 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:20.195 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:02:20.195 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:20.205 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:02:20.230 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:20.229+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:20.274 INFO:teuthology.orchestra.run.smithi005.stdout:3647 clone 32 order 23 su 1048576 sc 11 2024-05-11T23:02:20.331 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:20.331+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:20.357 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:20.357 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:20.357 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:20.357 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:20.357 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:20.366 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:02:20.450 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:20.450+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:20.517 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:20.517 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:20.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:20.517 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:20.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.517 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28445 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:20.517 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:20.517 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.435091301, 2024-05-11T23:02:20.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031719429, 2024-05-11T23:02:20.518 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.518 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.518 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:20.518 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.518 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.518 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.518 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28445 2024-05-11T23:02:20.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.518 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:20.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:20.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:20.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-05-11T23:02:20.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081234+0000", 2024-05-11T23:02:20.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.578e-06 2024-05-11T23:02:20.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081237+0000", 2024-05-11T23:02:20.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3340000000000002e-06 2024-05-11T23:02:20.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081240+0000", 2024-05-11T23:02:20.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7429999999999998e-06 2024-05-11T23:02:20.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112828+0000", 2024-05-11T23:02:20.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031587941000000001 2024-05-11T23:02:20.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112878+0000", 2024-05-11T23:02:20.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0535999999999999e-05 2024-05-11T23:02:20.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112951+0000", 2024-05-11T23:02:20.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3072999999999996e-05 2024-05-11T23:02:20.521 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.521 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.521 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.522 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28448 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 507904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:20.522 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:02:20.522 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.4350588, 2024-05-11T23:02:20.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031916923, 2024-05-11T23:02:20.522 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.522 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.522 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:20.522 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.522 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.522 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.522 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28448 2024-05-11T23:02:20.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.522 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:02:20.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:02:20.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081265+0000", 2024-05-11T23:02:20.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2899999999999999e-07 2024-05-11T23:02:20.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081267+0000", 2024-05-11T23:02:20.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6749999999999998e-06 2024-05-11T23:02:20.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081273+0000", 2024-05-11T23:02:20.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4049999999999999e-06 2024-05-11T23:02:20.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081278+0000", 2024-05-11T23:02:20.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0139999999999998e-06 2024-05-11T23:02:20.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112996+0000", 2024-05-11T23:02:20.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031718283 2024-05-11T23:02:20.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113080+0000", 2024-05-11T23:02:20.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4334999999999998e-05 2024-05-11T23:02:20.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113181+0000", 2024-05-11T23:02:20.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000100882 2024-05-11T23:02:20.526 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.526 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.526 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.526 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28451 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 540672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:20.527 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:20.527 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.435030298000001, 2024-05-11T23:02:20.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032054412999999997, 2024-05-11T23:02:20.527 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.527 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.527 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:20.527 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.527 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.527 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.527 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28451 2024-05-11T23:02:20.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.527 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:20.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:20.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:20.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2499999999999995e-07 2024-05-11T23:02:20.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081300+0000", 2024-05-11T23:02:20.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0929999999999999e-06 2024-05-11T23:02:20.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081306+0000", 2024-05-11T23:02:20.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6290000000000004e-06 2024-05-11T23:02:20.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081311+0000", 2024-05-11T23:02:20.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9289999999999997e-06 2024-05-11T23:02:20.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113208+0000", 2024-05-11T23:02:20.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031897162 2024-05-11T23:02:20.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113277+0000", 2024-05-11T23:02:20.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8811000000000002e-05 2024-05-11T23:02:20.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113347+0000", 2024-05-11T23:02:20.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0264000000000004e-05 2024-05-11T23:02:20.530 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.530 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.530 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.530 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28454 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 573440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:20.530 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:20.531 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.434955677000001, 2024-05-11T23:02:20.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032210564999999997, 2024-05-11T23:02:20.531 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.531 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.531 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:20.531 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.531 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.531 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28454 2024-05-11T23:02:20.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.531 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:20.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:20.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:20.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.99e-07 2024-05-11T23:02:20.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081371+0000", 2024-05-11T23:02:20.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1200000000000002e-06 2024-05-11T23:02:20.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081378+0000", 2024-05-11T23:02:20.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6089999999999997e-06 2024-05-11T23:02:20.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081384+0000", 2024-05-11T23:02:20.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0859999999999998e-06 2024-05-11T23:02:20.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113388+0000", 2024-05-11T23:02:20.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032004248999999999 2024-05-11T23:02:20.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113470+0000", 2024-05-11T23:02:20.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1871000000000003e-05 2024-05-11T23:02:20.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113578+0000", 2024-05-11T23:02:20.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000108031 2024-05-11T23:02:20.535 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.535 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.535 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.535 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28457 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 606208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:20.535 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:20.535 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.434925453000002, 2024-05-11T23:02:20.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032410130000000002, 2024-05-11T23:02:20.535 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.535 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.535 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:20.535 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.535 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.535 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.536 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28457 2024-05-11T23:02:20.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.536 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:20.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:20.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:20.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.32e-07 2024-05-11T23:02:20.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081400+0000", 2024-05-11T23:02:20.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9120000000000001e-06 2024-05-11T23:02:20.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081404+0000", 2024-05-11T23:02:20.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7900000000000001e-06 2024-05-11T23:02:20.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081407+0000", 2024-05-11T23:02:20.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8839999999999998e-06 2024-05-11T23:02:20.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113621+0000", 2024-05-11T23:02:20.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032213841 2024-05-11T23:02:20.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113702+0000", 2024-05-11T23:02:20.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1574000000000002e-05 2024-05-11T23:02:20.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113808+0000", 2024-05-11T23:02:20.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000105697 2024-05-11T23:02:20.539 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.539 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.539 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.539 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28460 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 638976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:20.539 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:20.539 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.434912621999999, 2024-05-11T23:02:20.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032616804999999999, 2024-05-11T23:02:20.540 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.540 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.540 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:20.540 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.540 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.540 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.540 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28460 2024-05-11T23:02:20.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.540 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:20.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:20.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:20.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5900000000000003e-07 2024-05-11T23:02:20.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081412+0000", 2024-05-11T23:02:20.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1990000000000001e-06 2024-05-11T23:02:20.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081415+0000", 2024-05-11T23:02:20.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.971e-06 2024-05-11T23:02:20.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081418+0000", 2024-05-11T23:02:20.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-05-11T23:02:20.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113853+0000", 2024-05-11T23:02:20.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032435155 2024-05-11T23:02:20.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113936+0000", 2024-05-11T23:02:20.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3441000000000003e-05 2024-05-11T23:02:20.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114027+0000", 2024-05-11T23:02:20.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1108999999999998e-05 2024-05-11T23:02:20.543 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.543 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.543 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.543 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28463 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 671744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:20.543 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:20.543 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.434895699999998, 2024-05-11T23:02:20.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032814958999999998, 2024-05-11T23:02:20.544 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.544 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.544 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:20.544 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.544 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.544 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.544 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28463 2024-05-11T23:02:20.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.544 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:20.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:20.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:20.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4999999999999998e-07 2024-05-11T23:02:20.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081430+0000", 2024-05-11T23:02:20.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6109999999999999e-06 2024-05-11T23:02:20.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081433+0000", 2024-05-11T23:02:20.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5590000000000001e-06 2024-05-11T23:02:20.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081437+0000", 2024-05-11T23:02:20.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3950000000000001e-06 2024-05-11T23:02:20.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114061+0000", 2024-05-11T23:02:20.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032624092 2024-05-11T23:02:20.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114142+0000", 2024-05-11T23:02:20.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0893e-05 2024-05-11T23:02:20.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114242+0000", 2024-05-11T23:02:20.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010095899999999999 2024-05-11T23:02:20.548 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.548 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.548 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.548 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28466 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 704512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:20.548 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:20.548 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.434883165999999, 2024-05-11T23:02:20.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032933658999999997, 2024-05-11T23:02:20.548 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.548 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:20.548 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.548 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.548 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.548 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28466 2024-05-11T23:02:20.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.549 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:20.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:20.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:20.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4700000000000002e-07 2024-05-11T23:02:20.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081442+0000", 2024-05-11T23:02:20.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.296e-06 2024-05-11T23:02:20.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081445+0000", 2024-05-11T23:02:20.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9009999999999998e-06 2024-05-11T23:02:20.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081447+0000", 2024-05-11T23:02:20.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4200000000000001e-06 2024-05-11T23:02:20.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114280+0000", 2024-05-11T23:02:20.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032832667000000003 2024-05-11T23:02:20.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114325+0000", 2024-05-11T23:02:20.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5018000000000002e-05 2024-05-11T23:02:20.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114374+0000", 2024-05-11T23:02:20.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.901e-05 2024-05-11T23:02:20.552 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.552 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.552 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.552 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28468 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 737280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:20.552 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:20.552 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.434863160999999, 2024-05-11T23:02:20.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033118776000000003, 2024-05-11T23:02:20.552 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:20.552 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.552 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.553 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.553 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28468 2024-05-11T23:02:20.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.553 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:20.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:20.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:20.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3799999999999998e-07 2024-05-11T23:02:20.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081462+0000", 2024-05-11T23:02:20.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4899999999999999e-06 2024-05-11T23:02:20.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081465+0000", 2024-05-11T23:02:20.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4079999999999998e-06 2024-05-11T23:02:20.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081468+0000", 2024-05-11T23:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7e-06 2024-05-11T23:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114415+0000", 2024-05-11T23:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032946766000000002 2024-05-11T23:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114497+0000", 2024-05-11T23:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1854000000000006e-05 2024-05-11T23:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114579+0000", 2024-05-11T23:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2120000000000007e-05 2024-05-11T23:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28471 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 770048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.434848454000001, 2024-05-11T23:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033234970000000003, 2024-05-11T23:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28471 2024-05-11T23:02:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-05-11T23:02:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081476+0000", 2024-05-11T23:02:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3120000000000001e-06 2024-05-11T23:02:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081480+0000", 2024-05-11T23:02:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1470000000000001e-06 2024-05-11T23:02:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081482+0000", 2024-05-11T23:02:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7480000000000002e-06 2024-05-11T23:02:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114616+0000", 2024-05-11T23:02:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033133322 2024-05-11T23:02:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114662+0000", 2024-05-11T23:02:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6020999999999999e-05 2024-05-11T23:02:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114710+0000", 2024-05-11T23:02:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8059999999999997e-05 2024-05-11T23:02:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28474 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.43483238, 2024-05-11T23:02:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033336431, 2024-05-11T23:02:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28474 2024-05-11T23:02:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.15e-07 2024-05-11T23:02:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081493+0000", 2024-05-11T23:02:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4950000000000001e-06 2024-05-11T23:02:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081496+0000", 2024-05-11T23:02:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.309e-06 2024-05-11T23:02:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081499+0000", 2024-05-11T23:02:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4570000000000001e-06 2024-05-11T23:02:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114740+0000", 2024-05-11T23:02:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033240619999999999 2024-05-11T23:02:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114781+0000", 2024-05-11T23:02:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1106999999999998e-05 2024-05-11T23:02:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114827+0000", 2024-05-11T23:02:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6028e-05 2024-05-11T23:02:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28477 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 835584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.434815365999999, 2024-05-11T23:02:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033527225000000001, 2024-05-11T23:02:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28477 2024-05-11T23:02:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1300000000000001e-07 2024-05-11T23:02:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081510+0000", 2024-05-11T23:02:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4610000000000001e-06 2024-05-11T23:02:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081513+0000", 2024-05-11T23:02:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2559999999999998e-06 2024-05-11T23:02:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081516+0000", 2024-05-11T23:02:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7099999999999999e-06 2024-05-11T23:02:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114869+0000", 2024-05-11T23:02:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033353718999999997 2024-05-11T23:02:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114954+0000", 2024-05-11T23:02:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4969000000000006e-05 2024-05-11T23:02:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115035+0000", 2024-05-11T23:02:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0697000000000003e-05 2024-05-11T23:02:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28480 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 868352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.434798396000001, 2024-05-11T23:02:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033695939000000001, 2024-05-11T23:02:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28480 2024-05-11T23:02:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.46e-07 2024-05-11T23:02:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081526+0000", 2024-05-11T23:02:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.347e-06 2024-05-11T23:02:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081530+0000", 2024-05-11T23:02:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.348e-06 2024-05-11T23:02:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081533+0000", 2024-05-11T23:02:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4369999999999999e-06 2024-05-11T23:02:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115078+0000", 2024-05-11T23:02:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033545063 2024-05-11T23:02:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115164+0000", 2024-05-11T23:02:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5662999999999996e-05 2024-05-11T23:02:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115221+0000", 2024-05-11T23:02:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6734999999999998e-05 2024-05-11T23:02:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28483 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.434781223000002, 2024-05-11T23:02:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033803466999999997, 2024-05-11T23:02:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28483 2024-05-11T23:02:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4999999999999998e-07 2024-05-11T23:02:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081544+0000", 2024-05-11T23:02:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:02:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081547+0000", 2024-05-11T23:02:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3139999999999999e-06 2024-05-11T23:02:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081550+0000", 2024-05-11T23:02:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7779999999999999e-06 2024-05-11T23:02:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115253+0000", 2024-05-11T23:02:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033703516000000003 2024-05-11T23:02:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115302+0000", 2024-05-11T23:02:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8869999999999998e-05 2024-05-11T23:02:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115345+0000", 2024-05-11T23:02:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3340999999999997e-05 2024-05-11T23:02:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28487 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 933888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.434770021999999, 2024-05-11T23:02:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033903918999999998, 2024-05-11T23:02:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28487 2024-05-11T23:02:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9999999999999999e-07 2024-05-11T23:02:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081559+0000", 2024-05-11T23:02:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-05-11T23:02:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081563+0000", 2024-05-11T23:02:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2370000000000003e-06 2024-05-11T23:02:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081566+0000", 2024-05-11T23:02:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6199999999999999e-06 2024-05-11T23:02:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115375+0000", 2024-05-11T23:02:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033808887000000003 2024-05-11T23:02:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115413+0000", 2024-05-11T23:02:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8309000000000002e-05 2024-05-11T23:02:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115457+0000", 2024-05-11T23:02:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3809e-05 2024-05-11T23:02:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28490 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 966656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.434753497999999, 2024-05-11T23:02:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034114903000000002, 2024-05-11T23:02:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28490 2024-05-11T23:02:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4400000000000001e-07 2024-05-11T23:02:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081571+0000", 2024-05-11T23:02:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2130000000000001e-06 2024-05-11T23:02:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081574+0000", 2024-05-11T23:02:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8270000000000002e-06 2024-05-11T23:02:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081577+0000", 2024-05-11T23:02:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-05-11T23:02:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115507+0000", 2024-05-11T23:02:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033929925999999999 2024-05-11T23:02:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115599+0000", 2024-05-11T23:02:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1571999999999997e-05 2024-05-11T23:02:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115685+0000", 2024-05-11T23:02:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6071999999999999e-05 2024-05-11T23:02:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28493 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 999424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.434741395, 2024-05-11T23:02:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034232944000000001, 2024-05-11T23:02:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28493 2024-05-11T23:02:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1300000000000001e-07 2024-05-11T23:02:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081584+0000", 2024-05-11T23:02:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9479999999999998e-06 2024-05-11T23:02:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081590+0000", 2024-05-11T23:02:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.728e-06 2024-05-11T23:02:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081595+0000", 2024-05-11T23:02:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8579999999999999e-06 2024-05-11T23:02:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115723+0000", 2024-05-11T23:02:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034128473999999999 2024-05-11T23:02:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115768+0000", 2024-05-11T23:02:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5108999999999998e-05 2024-05-11T23:02:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115815+0000", 2024-05-11T23:02:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6514000000000002e-05 2024-05-11T23:02:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28496 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:02:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.434721935999999, 2024-05-11T23:02:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034427754999999997, 2024-05-11T23:02:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28496 2024-05-11T23:02:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:02:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:02:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081602+0000", 2024-05-11T23:02:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:02:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081604+0000", 2024-05-11T23:02:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3999999999999999e-06 2024-05-11T23:02:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081610+0000", 2024-05-11T23:02:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.224e-06 2024-05-11T23:02:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081615+0000", 2024-05-11T23:02:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8269999999999997e-06 2024-05-11T23:02:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115859+0000", 2024-05-11T23:02:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034243498999999997 2024-05-11T23:02:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115946+0000", 2024-05-11T23:02:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7122999999999999e-05 2024-05-11T23:02:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116029+0000", 2024-05-11T23:02:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3332999999999996e-05 2024-05-11T23:02:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28498 3.0 3:05d5189e:::rbd_object_map.111e5e4d8275.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:02:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.433203284000001, 2024-05-11T23:02:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.035035108000000002, 2024-05-11T23:02:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28498 2024-05-11T23:02:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:02:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:02:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083121+0000", 2024-05-11T23:02:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5700000000000001e-07 2024-05-11T23:02:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083126+0000", 2024-05-11T23:02:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8860000000000003e-06 2024-05-11T23:02:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083135+0000", 2024-05-11T23:02:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8369999999999992e-06 2024-05-11T23:02:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083142+0000", 2024-05-11T23:02:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4779999999999996e-06 2024-05-11T23:02:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116072+0000", 2024-05-11T23:02:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032929991999999998 2024-05-11T23:02:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116158+0000", 2024-05-11T23:02:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6077999999999998e-05 2024-05-11T23:02:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-05-11T23:02:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116513+0000", 2024-05-11T23:02:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00035508300000000002 2024-05-11T23:02:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118025+0000", 2024-05-11T23:02:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015118919999999999 2024-05-11T23:02:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:02:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118039+0000", 2024-05-11T23:02:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4188999999999999e-05 2024-05-11T23:02:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118094+0000", 2024-05-11T23:02:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4929000000000002e-05 2024-05-11T23:02:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118155+0000", 2024-05-11T23:02:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0887000000000002e-05 2024-05-11T23:02:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:64049 3.8 3:11d98727:::rbd_object_map.111e8974a681:head [read 0~38] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-05-11T23:02:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:02:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.736441974000002, 2024-05-11T23:02:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99884604399999999, 2024-05-11T23:02:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 64049 2024-05-11T23:02:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:02:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:02:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779883+0000", 2024-05-11T23:02:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.57e-06 2024-05-11T23:02:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779887+0000", 2024-05-11T23:02:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5709999999999999e-06 2024-05-11T23:02:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779898+0000", 2024-05-11T23:02:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.042e-05 2024-05-11T23:02:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779905+0000", 2024-05-11T23:02:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0110000000000003e-06 2024-05-11T23:02:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779962+0000", 2024-05-11T23:02:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7395999999999997e-05 2024-05-11T23:02:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:02:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779969+0000", 2024-05-11T23:02:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2459999999999997e-06 2024-05-11T23:02:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778379+0000", 2024-05-11T23:02:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99840986300000001 2024-05-11T23:02:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778564+0000", 2024-05-11T23:02:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018420900000000001 2024-05-11T23:02:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778727+0000", 2024-05-11T23:02:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016375800000000001 2024-05-11T23:02:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:20.602 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:20.604 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:02:20.604 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:20.603+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:20.704 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:20.704+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:20.755 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:20.755 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:20.755 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:02:20.756 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:20.764 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:02:20.804 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:20.804+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:20.873 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:20.873 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:20.873 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:20.873 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:20.873 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:20.881 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:02:20.926 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:20.926+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:clear 3.11 pg_upmap_items mapping 2024-05-11T23:02:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-05-11T23:02:20.998 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:20.998 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:20.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:20.998 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:20.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.998 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60471 3.4 e43/19 3:2079ea20:::rbd_data.111e1df04b77.0000000000000000:head v 43'84, mlcod=42'81)", 2024-05-11T23:02:20.998 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:20.998 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.070352603, 2024-05-11T23:02:20.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0079043050000000004, 2024-05-11T23:02:20.999 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:20.999 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:20.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:20.999 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:20.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:20.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:20.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:20.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:20.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:20.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:21.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:21.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-05-11T23:02:21.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926880+0000", 2024-05-11T23:02:21.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:02:21.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926893+0000", 2024-05-11T23:02:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3256e-05 2024-05-11T23:02:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926899+0000", 2024-05-11T23:02:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9850000000000003e-06 2024-05-11T23:02:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926982+0000", 2024-05-11T23:02:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2997000000000005e-05 2024-05-11T23:02:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927005+0000", 2024-05-11T23:02:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3428e-05 2024-05-11T23:02:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.934747+0000", 2024-05-11T23:02:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0077415820000000003 2024-05-11T23:02:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.934778+0000", 2024-05-11T23:02:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1479999999999997e-05 2024-05-11T23:02:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.002 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60472 3.9 e43/19 3:94f86c79:::rbd_data.111e1df04b77.0000000000000001:head v 43'122, mlcod=42'119)", 2024-05-11T23:02:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.927124+0000", 2024-05-11T23:02:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.070102902999999, 2024-05-11T23:02:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0062295900000000001, 2024-05-11T23:02:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927124+0000", 2024-05-11T23:02:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927124+0000", 2024-05-11T23:02:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927124+0000", 2024-05-11T23:02:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927128+0000", 2024-05-11T23:02:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3390000000000003e-06 2024-05-11T23:02:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927139+0000", 2024-05-11T23:02:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1161e-05 2024-05-11T23:02:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927147+0000", 2024-05-11T23:02:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1910000000000003e-06 2024-05-11T23:02:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927199+0000", 2024-05-11T23:02:21.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.24e-05 2024-05-11T23:02:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927223+0000", 2024-05-11T23:02:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3811999999999998e-05 2024-05-11T23:02:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.933287+0000", 2024-05-11T23:02:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.006063723 2024-05-11T23:02:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.933353+0000", 2024-05-11T23:02:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6744999999999999e-05 2024-05-11T23:02:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:64399 3.1e e48/47 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head v 48'297, mlcod=0'0)", 2024-05-11T23:02:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:02.104619+0000", 2024-05-11T23:02:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.89260749, 2024-05-11T23:02:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014840949000000001, 2024-05-11T23:02:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.007 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.104619+0000", 2024-05-11T23:02:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.104619+0000", 2024-05-11T23:02:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.104620+0000", 2024-05-11T23:02:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.299e-06 2024-05-11T23:02:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.112866+0000", 2024-05-11T23:02:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082450949999999992 2024-05-11T23:02:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.112893+0000", 2024-05-11T23:02:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7131000000000002e-05 2024-05-11T23:02:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.112902+0000", 2024-05-11T23:02:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4089999999999992e-06 2024-05-11T23:02:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.112980+0000", 2024-05-11T23:02:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8375999999999996e-05 2024-05-11T23:02:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.113009+0000", 2024-05-11T23:02:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8713000000000001e-05 2024-05-11T23:02:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.119403+0000", 2024-05-11T23:02:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0063940580000000002 2024-05-11T23:02:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.119460+0000", 2024-05-11T23:02:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6867999999999998e-05 2024-05-11T23:02:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65750 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 5242880~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.980128+0000", 2024-05-11T23:02:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 15.017098388000001, 2024-05-11T23:02:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0045445240000000003, 2024-05-11T23:02:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.011 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65750 2024-05-11T23:02:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.980128+0000", 2024-05-11T23:02:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.980128+0000", 2024-05-11T23:02:21.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.980128+0000", 2024-05-11T23:02:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.22e-07 2024-05-11T23:02:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.980130+0000", 2024-05-11T23:02:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1009999999999999e-06 2024-05-11T23:02:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.980133+0000", 2024-05-11T23:02:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9890000000000001e-06 2024-05-11T23:02:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.980135+0000", 2024-05-11T23:02:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.43e-06 2024-05-11T23:02:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.984329+0000", 2024-05-11T23:02:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0041943270000000003 2024-05-11T23:02:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.984401+0000", 2024-05-11T23:02:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1268999999999998e-05 2024-05-11T23:02:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.984673+0000", 2024-05-11T23:02:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000272186 2024-05-11T23:02:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65755 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.981205+0000", 2024-05-11T23:02:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 15.01602185, 2024-05-11T23:02:21.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.004494077, 2024-05-11T23:02:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65755 2024-05-11T23:02:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.981205+0000", 2024-05-11T23:02:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.981205+0000", 2024-05-11T23:02:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.981205+0000", 2024-05-11T23:02:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9999999999999998e-07 2024-05-11T23:02:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.981209+0000", 2024-05-11T23:02:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5209999999999998e-06 2024-05-11T23:02:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.981221+0000", 2024-05-11T23:02:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2407e-05 2024-05-11T23:02:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.981227+0000", 2024-05-11T23:02:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2110000000000003e-06 2024-05-11T23:02:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.985298+0000", 2024-05-11T23:02:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.004070415 2024-05-11T23:02:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.985369+0000", 2024-05-11T23:02:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1191000000000003e-05 2024-05-11T23:02:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.985699+0000", 2024-05-11T23:02:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032983200000000001 2024-05-11T23:02:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.019 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65759 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 5767168~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.982321+0000", 2024-05-11T23:02:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 15.014905568, 2024-05-11T23:02:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0043249059999999999, 2024-05-11T23:02:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65759 2024-05-11T23:02:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.982321+0000", 2024-05-11T23:02:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.982321+0000", 2024-05-11T23:02:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.982322+0000", 2024-05-11T23:02:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5199999999999997e-07 2024-05-11T23:02:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.982325+0000", 2024-05-11T23:02:21.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.224e-06 2024-05-11T23:02:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.982332+0000", 2024-05-11T23:02:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.063e-06 2024-05-11T23:02:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.982346+0000", 2024-05-11T23:02:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3715e-05 2024-05-11T23:02:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.986221+0000", 2024-05-11T23:02:21.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0038758460000000001 2024-05-11T23:02:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.986320+0000", 2024-05-11T23:02:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8892000000000001e-05 2024-05-11T23:02:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.986646+0000", 2024-05-11T23:02:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000325614 2024-05-11T23:02:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65788 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 7995392~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.989762+0000", 2024-05-11T23:02:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 15.00746457, 2024-05-11T23:02:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0043592930000000002, 2024-05-11T23:02:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65788 2024-05-11T23:02:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.989762+0000", 2024-05-11T23:02:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.989762+0000", 2024-05-11T23:02:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.989763+0000", 2024-05-11T23:02:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.37e-07 2024-05-11T23:02:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.989765+0000", 2024-05-11T23:02:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5940000000000001e-06 2024-05-11T23:02:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.989773+0000", 2024-05-11T23:02:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5549999999999997e-06 2024-05-11T23:02:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.989778+0000", 2024-05-11T23:02:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4750000000000001e-06 2024-05-11T23:02:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993696+0000", 2024-05-11T23:02:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0039173639999999996 2024-05-11T23:02:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993770+0000", 2024-05-11T23:02:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4307999999999994e-05 2024-05-11T23:02:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.994121+0000", 2024-05-11T23:02:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00035146 2024-05-11T23:02:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65791 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 6815744~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.990491+0000", 2024-05-11T23:02:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 15.006735834000001, 2024-05-11T23:02:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.004670258, 2024-05-11T23:02:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65791 2024-05-11T23:02:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990491+0000", 2024-05-11T23:02:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990491+0000", 2024-05-11T23:02:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990492+0000", 2024-05-11T23:02:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.026e-06 2024-05-11T23:02:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990496+0000", 2024-05-11T23:02:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.481e-06 2024-05-11T23:02:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990507+0000", 2024-05-11T23:02:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0689e-05 2024-05-11T23:02:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990515+0000", 2024-05-11T23:02:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8990000000000001e-06 2024-05-11T23:02:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.994180+0000", 2024-05-11T23:02:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0036648710000000001 2024-05-11T23:02:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.994308+0000", 2024-05-11T23:02:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012834700000000001 2024-05-11T23:02:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.995161+0000", 2024-05-11T23:02:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00085294500000000003 2024-05-11T23:02:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65792 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 8257536~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.990528+0000", 2024-05-11T23:02:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 15.00669882, 2024-05-11T23:02:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0053202220000000003, 2024-05-11T23:02:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65792 2024-05-11T23:02:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990528+0000", 2024-05-11T23:02:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990528+0000", 2024-05-11T23:02:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990528+0000", 2024-05-11T23:02:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.13e-07 2024-05-11T23:02:21.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990539+0000", 2024-05-11T23:02:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0152999999999999e-05 2024-05-11T23:02:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990546+0000", 2024-05-11T23:02:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1779999999999997e-06 2024-05-11T23:02:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990551+0000", 2024-05-11T23:02:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.147e-06 2024-05-11T23:02:21.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.995187+0000", 2024-05-11T23:02:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.004636215 2024-05-11T23:02:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.995276+0000", 2024-05-11T23:02:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9086999999999996e-05 2024-05-11T23:02:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.995848+0000", 2024-05-11T23:02:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00057182900000000004 2024-05-11T23:02:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.035 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65794 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 6684672~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.990959+0000", 2024-05-11T23:02:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 15.006267985999999, 2024-05-11T23:02:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.005377672, 2024-05-11T23:02:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65794 2024-05-11T23:02:21.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990959+0000", 2024-05-11T23:02:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990959+0000", 2024-05-11T23:02:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990959+0000", 2024-05-11T23:02:21.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7199999999999997e-07 2024-05-11T23:02:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990964+0000", 2024-05-11T23:02:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.639e-06 2024-05-11T23:02:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990976+0000", 2024-05-11T23:02:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1420000000000001e-05 2024-05-11T23:02:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990983+0000", 2024-05-11T23:02:21.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7719999999999994e-06 2024-05-11T23:02:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.995887+0000", 2024-05-11T23:02:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0049036239999999997 2024-05-11T23:02:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.996005+0000", 2024-05-11T23:02:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118387 2024-05-11T23:02:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.996336+0000", 2024-05-11T23:02:21.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00033095800000000001 2024-05-11T23:02:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65795 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 8126464~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.990996+0000", 2024-05-11T23:02:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 15.006230753000001, 2024-05-11T23:02:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0057710749999999996, 2024-05-11T23:02:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.040 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65795 2024-05-11T23:02:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990996+0000", 2024-05-11T23:02:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990996+0000", 2024-05-11T23:02:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990996+0000", 2024-05-11T23:02:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5600000000000001e-07 2024-05-11T23:02:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.991000+0000", 2024-05-11T23:02:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2550000000000002e-06 2024-05-11T23:02:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.991006+0000", 2024-05-11T23:02:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4080000000000003e-06 2024-05-11T23:02:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.991011+0000", 2024-05-11T23:02:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1749999999999996e-06 2024-05-11T23:02:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.996368+0000", 2024-05-11T23:02:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0053564040000000004 2024-05-11T23:02:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.996461+0000", 2024-05-11T23:02:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3435000000000002e-05 2024-05-11T23:02:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.996767+0000", 2024-05-11T23:02:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00030594200000000002 2024-05-11T23:02:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65799 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 7077888~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.992390+0000", 2024-05-11T23:02:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 15.004836403000001, 2024-05-11T23:02:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0050808049999999999, 2024-05-11T23:02:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.044 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65799 2024-05-11T23:02:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.992390+0000", 2024-05-11T23:02:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.992390+0000", 2024-05-11T23:02:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.992392+0000", 2024-05-11T23:02:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.314e-06 2024-05-11T23:02:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.992397+0000", 2024-05-11T23:02:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.023e-06 2024-05-11T23:02:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.992411+0000", 2024-05-11T23:02:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4681000000000001e-05 2024-05-11T23:02:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.992434+0000", 2024-05-11T23:02:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2518999999999999e-05 2024-05-11T23:02:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.996799+0000", 2024-05-11T23:02:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0043649090000000001 2024-05-11T23:02:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.996904+0000", 2024-05-11T23:02:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010525499999999999 2024-05-11T23:02:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.997471+0000", 2024-05-11T23:02:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00056710400000000002 2024-05-11T23:02:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.048 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65803 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 7340032~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.993407+0000", 2024-05-11T23:02:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 15.003819844000001, 2024-05-11T23:02:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0045385190000000004, 2024-05-11T23:02:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.050 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65803 2024-05-11T23:02:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993407+0000", 2024-05-11T23:02:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993407+0000", 2024-05-11T23:02:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993408+0000", 2024-05-11T23:02:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3600000000000003e-07 2024-05-11T23:02:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993411+0000", 2024-05-11T23:02:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9859999999999996e-06 2024-05-11T23:02:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993424+0000", 2024-05-11T23:02:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2308e-05 2024-05-11T23:02:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993431+0000", 2024-05-11T23:02:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2099999999999996e-06 2024-05-11T23:02:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.997506+0000", 2024-05-11T23:02:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0040752749999999997 2024-05-11T23:02:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.997613+0000", 2024-05-11T23:02:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106895 2024-05-11T23:02:21.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.997945+0000", 2024-05-11T23:02:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00033210899999999997 2024-05-11T23:02:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65807 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 7208960~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.993534+0000", 2024-05-11T23:02:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 15.003692267, 2024-05-11T23:02:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0048977109999999999, 2024-05-11T23:02:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.054 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65807 2024-05-11T23:02:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993534+0000", 2024-05-11T23:02:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993534+0000", 2024-05-11T23:02:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993535+0000", 2024-05-11T23:02:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9699999999999996e-07 2024-05-11T23:02:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993538+0000", 2024-05-11T23:02:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9840000000000001e-06 2024-05-11T23:02:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993545+0000", 2024-05-11T23:02:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8009999999999998e-06 2024-05-11T23:02:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993549+0000", 2024-05-11T23:02:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0029999999999996e-06 2024-05-11T23:02:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.997983+0000", 2024-05-11T23:02:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0044348269999999997 2024-05-11T23:02:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.998093+0000", 2024-05-11T23:02:21.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000109992 2024-05-11T23:02:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.998432+0000", 2024-05-11T23:02:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00033860699999999999 2024-05-11T23:02:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:66044 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [sparse-read 0~8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:06.114668+0000", 2024-05-11T23:02:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.882558802, 2024-05-11T23:02:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0062482759999999997, 2024-05-11T23:02:21.058 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 66044 2024-05-11T23:02:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114668+0000", 2024-05-11T23:02:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114668+0000", 2024-05-11T23:02:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114669+0000", 2024-05-11T23:02:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.299e-06 2024-05-11T23:02:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114675+0000", 2024-05-11T23:02:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-05-11T23:02:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114690+0000", 2024-05-11T23:02:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5349000000000002e-05 2024-05-11T23:02:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114697+0000", 2024-05-11T23:02:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6279999999999999e-06 2024-05-11T23:02:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114749+0000", 2024-05-11T23:02:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1958e-05 2024-05-11T23:02:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114828+0000", 2024-05-11T23:02:21.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9728999999999995e-05 2024-05-11T23:02:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.120916+0000", 2024-05-11T23:02:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0060877919999999999 2024-05-11T23:02:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "MOSDPGPushReply(3.1e 52/51 [PushReplyOp(3:7943f929:::rbd_data.111ef7a115a1.0000000000000050:head)])", 2024-05-11T23:02:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:06.114699+0000", 2024-05-11T23:02:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.882527725999999, 2024-05-11T23:02:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0064609460000000004, 2024-05-11T23:02:21.062 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114699+0000", 2024-05-11T23:02:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114699+0000", 2024-05-11T23:02:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114700+0000", 2024-05-11T23:02:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0780000000000001e-06 2024-05-11T23:02:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114706+0000", 2024-05-11T23:02:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4200000000000004e-06 2024-05-11T23:02:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114724+0000", 2024-05-11T23:02:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7142e-05 2024-05-11T23:02:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114732+0000", 2024-05-11T23:02:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7630000000000006e-06 2024-05-11T23:02:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.120983+0000", 2024-05-11T23:02:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0062507250000000004 2024-05-11T23:02:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.121160+0000", 2024-05-11T23:02:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000176818 2024-05-11T23:02:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.065 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:77128 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_get in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-11T23:02:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:02:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.9812249409999998, 2024-05-11T23:02:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91737855899999998, 2024-05-11T23:02:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 77128 2024-05-11T23:02:21.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:02:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:02:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016003+0000", 2024-05-11T23:02:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5379999999999999e-06 2024-05-11T23:02:21.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016011+0000", 2024-05-11T23:02:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6410000000000002e-06 2024-05-11T23:02:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016026+0000", 2024-05-11T23:02:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4779e-05 2024-05-11T23:02:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016032+0000", 2024-05-11T23:02:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.066e-06 2024-05-11T23:02:21.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016093+0000", 2024-05-11T23:02:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0878000000000002e-05 2024-05-11T23:02:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:02:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016100+0000", 2024-05-11T23:02:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6799999999999993e-06 2024-05-11T23:02:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933302+0000", 2024-05-11T23:02:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91720135300000005 2024-05-11T23:02:21.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933380+0000", 2024-05-11T23:02:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8623999999999998e-05 2024-05-11T23:02:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:78890 3.3 3:c289f39e:::rbd_data.111ea74f268d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:16.008252+0000", 2024-05-11T23:02:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.9889748010000003, 2024-05-11T23:02:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0070432100000000003, 2024-05-11T23:02:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.070 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 78890 2024-05-11T23:02:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.008252+0000", 2024-05-11T23:02:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.008252+0000", 2024-05-11T23:02:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.008253+0000", 2024-05-11T23:02:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-05-11T23:02:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009306+0000", 2024-05-11T23:02:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001053327 2024-05-11T23:02:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009329+0000", 2024-05-11T23:02:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3113999999999999e-05 2024-05-11T23:02:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009345+0000", 2024-05-11T23:02:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5136e-05 2024-05-11T23:02:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009447+0000", 2024-05-11T23:02:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010199799999999999 2024-05-11T23:02:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009640+0000", 2024-05-11T23:02:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019346000000000001 2024-05-11T23:02:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-05-11T23:02:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009832+0000", 2024-05-11T23:02:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019212799999999999 2024-05-11T23:02:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:02:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.014184+0000", 2024-05-11T23:02:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0043513909999999996 2024-05-11T23:02:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.015200+0000", 2024-05-11T23:02:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001016926 2024-05-11T23:02:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.015250+0000", 2024-05-11T23:02:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9162999999999997e-05 2024-05-11T23:02:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.015295+0000", 2024-05-11T23:02:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5425e-05 2024-05-11T23:02:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:78891 3.3 3:c289f39e:::rbd_data.111ea74f268d.0000000000000001:head [write 0~0] snapc 1d=[1d] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:16.009376+0000", 2024-05-11T23:02:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.9878501809999998, 2024-05-11T23:02:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0061637829999999999, 2024-05-11T23:02:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 78891 2024-05-11T23:02:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009376+0000", 2024-05-11T23:02:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009376+0000", 2024-05-11T23:02:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009378+0000", 2024-05-11T23:02:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.328e-06 2024-05-11T23:02:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009381+0000", 2024-05-11T23:02:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9519999999999999e-06 2024-05-11T23:02:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009385+0000", 2024-05-11T23:02:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5279999999999997e-06 2024-05-11T23:02:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009389+0000", 2024-05-11T23:02:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3139999999999999e-06 2024-05-11T23:02:21.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.012644+0000", 2024-05-11T23:02:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0032556439999999998 2024-05-11T23:02:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.012730+0000", 2024-05-11T23:02:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6063000000000006e-05 2024-05-11T23:02:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-05-11T23:02:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.012891+0000", 2024-05-11T23:02:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016067100000000001 2024-05-11T23:02:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:02:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.014464+0000", 2024-05-11T23:02:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015728949999999999 2024-05-11T23:02:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.015477+0000", 2024-05-11T23:02:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010127650000000001 2024-05-11T23:02:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.015507+0000", 2024-05-11T23:02:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0550999999999999e-05 2024-05-11T23:02:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.015540+0000", 2024-05-11T23:02:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3071999999999998e-05 2024-05-11T23:02:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:78892 3.5 e62/44 3:a1776d60:::rbd_data.111ea74f268d.0000000000000004:head v 62'156, mlcod=60'154)", 2024-05-11T23:02:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:16.012888+0000", 2024-05-11T23:02:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.984338975, 2024-05-11T23:02:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0057186540000000001, 2024-05-11T23:02:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:21.081 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.012888+0000", 2024-05-11T23:02:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.012888+0000", 2024-05-11T23:02:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.012888+0000", 2024-05-11T23:02:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6899999999999998e-07 2024-05-11T23:02:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.016107+0000", 2024-05-11T23:02:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0032186150000000002 2024-05-11T23:02:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.016117+0000", 2024-05-11T23:02:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0305e-05 2024-05-11T23:02:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.016124+0000", 2024-05-11T23:02:21.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7100000000000001e-06 2024-05-11T23:02:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.016184+0000", 2024-05-11T23:02:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0764000000000003e-05 2024-05-11T23:02:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.016210+0000", 2024-05-11T23:02:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5740000000000001e-05 2024-05-11T23:02:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.018572+0000", 2024-05-11T23:02:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0023623519999999999 2024-05-11T23:02:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.018606+0000", 2024-05-11T23:02:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3798999999999999e-05 2024-05-11T23:02:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.085 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.085 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:21.086 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:02:21.087 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:21.086+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:21.187 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:21.187+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:21.238 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:21.238 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:21.238 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:02:21.238 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:21.247 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:02:21.288 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:21.288+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:21.388 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:21.388+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:21.396 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:21.396 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:21.396 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:21.396 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:21.397 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:21.405 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:02:21.490 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:21.490+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:21.558 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:21.558 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:21.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:21.558 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:21.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.558 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36828 3.b 3:d385bc8e:::rbd_data.111e5e4d8275.0000000000000020:11 [read 512000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:21.558 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:21.558 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.624049243000002, 2024-05-11T23:02:21.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051481218000000002, 2024-05-11T23:02:21.558 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.559 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36828 2024-05-11T23:02:21.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.559 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:21.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:21.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934915+0000", 2024-05-11T23:02:21.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00181814 2024-05-11T23:02:21.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934919+0000", 2024-05-11T23:02:21.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6739999999999996e-06 2024-05-11T23:02:21.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934934+0000", 2024-05-11T23:02:21.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4415e-05 2024-05-11T23:02:21.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934942+0000", 2024-05-11T23:02:21.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8619999999999992e-06 2024-05-11T23:02:21.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984433+0000", 2024-05-11T23:02:21.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049490658999999999 2024-05-11T23:02:21.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984508+0000", 2024-05-11T23:02:21.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4394999999999995e-05 2024-05-11T23:02:21.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984578+0000", 2024-05-11T23:02:21.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0073000000000005e-05 2024-05-11T23:02:21.562 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.562 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.562 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.563 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36950 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 520192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:21.563 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:21.563 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.619743477, 2024-05-11T23:02:21.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051450856000000003, 2024-05-11T23:02:21.563 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.563 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.563 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.563 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.563 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.563 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.563 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36950 2024-05-11T23:02:21.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.563 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:21.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:21.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938914+0000", 2024-05-11T23:02:21.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015121480000000001 2024-05-11T23:02:21.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938918+0000", 2024-05-11T23:02:21.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4149999999999999e-06 2024-05-11T23:02:21.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938925+0000", 2024-05-11T23:02:21.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0779999999999998e-06 2024-05-11T23:02:21.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938929+0000", 2024-05-11T23:02:21.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2930000000000002e-06 2024-05-11T23:02:21.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988675+0000", 2024-05-11T23:02:21.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049746394999999999 2024-05-11T23:02:21.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988756+0000", 2024-05-11T23:02:21.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0499000000000003e-05 2024-05-11T23:02:21.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988853+0000", 2024-05-11T23:02:21.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7027999999999993e-05 2024-05-11T23:02:21.566 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.566 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.567 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.567 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36957 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 778240~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:21.567 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:21.567 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.618121687999999, 2024-05-11T23:02:21.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051328273000000001, 2024-05-11T23:02:21.567 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.567 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.567 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.567 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.567 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.567 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.567 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36957 2024-05-11T23:02:21.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.568 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:21.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:21.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:21.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2099999999999998e-07 2024-05-11T23:02:21.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939026+0000", 2024-05-11T23:02:21.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5069999999999999e-06 2024-05-11T23:02:21.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939030+0000", 2024-05-11T23:02:21.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7919999999999999e-06 2024-05-11T23:02:21.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939032+0000", 2024-05-11T23:02:21.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9129999999999999e-06 2024-05-11T23:02:21.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990178+0000", 2024-05-11T23:02:21.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051145373000000001 2024-05-11T23:02:21.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990255+0000", 2024-05-11T23:02:21.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7195999999999997e-05 2024-05-11T23:02:21.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990352+0000", 2024-05-11T23:02:21.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7170999999999995e-05 2024-05-11T23:02:21.571 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.571 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.571 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.571 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36958 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 815104~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:21.571 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:21.571 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.618109368999999, 2024-05-11T23:02:21.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051531625999999997, 2024-05-11T23:02:21.571 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.571 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.571 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.572 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.572 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.572 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.572 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36958 2024-05-11T23:02:21.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.572 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:21.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:21.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939037+0000", 2024-05-11T23:02:21.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0400000000000002e-07 2024-05-11T23:02:21.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939038+0000", 2024-05-11T23:02:21.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.164e-06 2024-05-11T23:02:21.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939041+0000", 2024-05-11T23:02:21.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1109999999999999e-06 2024-05-11T23:02:21.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939043+0000", 2024-05-11T23:02:21.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.554e-06 2024-05-11T23:02:21.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990382+0000", 2024-05-11T23:02:21.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051338953999999999 2024-05-11T23:02:21.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990465+0000", 2024-05-11T23:02:21.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2461999999999998e-05 2024-05-11T23:02:21.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990568+0000", 2024-05-11T23:02:21.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000103077 2024-05-11T23:02:21.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.575 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.575 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36959 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 851968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:21.575 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:21.575 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.618097472999999, 2024-05-11T23:02:21.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051758936999999998, 2024-05-11T23:02:21.576 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.576 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.576 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.576 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.576 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.576 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.576 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36959 2024-05-11T23:02:21.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.576 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:21.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:21.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:21.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.16e-07 2024-05-11T23:02:21.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939050+0000", 2024-05-11T23:02:21.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.176e-06 2024-05-11T23:02:21.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939053+0000", 2024-05-11T23:02:21.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1140000000000002e-06 2024-05-11T23:02:21.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939055+0000", 2024-05-11T23:02:21.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.621e-06 2024-05-11T23:02:21.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990610+0000", 2024-05-11T23:02:21.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051554244999999999 2024-05-11T23:02:21.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990700+0000", 2024-05-11T23:02:21.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0284999999999994e-05 2024-05-11T23:02:21.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990807+0000", 2024-05-11T23:02:21.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010728 2024-05-11T23:02:21.579 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.579 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.579 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.579 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36960 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 888832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:21.580 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:21.580 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.618087238000001, 2024-05-11T23:02:21.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051886831000000001, 2024-05-11T23:02:21.580 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.580 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.580 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.580 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.580 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.580 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36960 2024-05-11T23:02:21.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.580 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:21.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:21.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940913+0000", 2024-05-11T23:02:21.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001855104 2024-05-11T23:02:21.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940916+0000", 2024-05-11T23:02:21.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0000000000000001e-06 2024-05-11T23:02:21.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940924+0000", 2024-05-11T23:02:21.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0999999999999998e-06 2024-05-11T23:02:21.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940927+0000", 2024-05-11T23:02:21.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7160000000000001e-06 2024-05-11T23:02:21.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990826+0000", 2024-05-11T23:02:21.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049898832999999997 2024-05-11T23:02:21.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990872+0000", 2024-05-11T23:02:21.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5717000000000003e-05 2024-05-11T23:02:21.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990945+0000", 2024-05-11T23:02:21.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3361000000000005e-05 2024-05-11T23:02:21.583 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.583 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.584 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36986 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 528384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:21.584 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:21.584 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.616081019999999, 2024-05-11T23:02:21.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051341053999999997, 2024-05-11T23:02:21.584 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.584 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.584 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.584 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.584 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.584 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36986 2024-05-11T23:02:21.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.585 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:21.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:21.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:02:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941066+0000", 2024-05-11T23:02:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1629999999999999e-06 2024-05-11T23:02:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941069+0000", 2024-05-11T23:02:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.118e-06 2024-05-11T23:02:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941072+0000", 2024-05-11T23:02:21.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5720000000000001e-06 2024-05-11T23:02:21.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992292+0000", 2024-05-11T23:02:21.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051220620000000001 2024-05-11T23:02:21.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992344+0000", 2024-05-11T23:02:21.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1737000000000003e-05 2024-05-11T23:02:21.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992406+0000", 2024-05-11T23:02:21.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1623000000000003e-05 2024-05-11T23:02:21.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.588 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.588 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36987 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 565248~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:21.588 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:21.588 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.616070899, 2024-05-11T23:02:21.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051568725000000003, 2024-05-11T23:02:21.588 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.588 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.588 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36987 2024-05-11T23:02:21.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.589 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:21.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:21.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:21.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.91e-07 2024-05-11T23:02:21.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941076+0000", 2024-05-11T23:02:21.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1379999999999999e-06 2024-05-11T23:02:21.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941079+0000", 2024-05-11T23:02:21.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9739999999999998e-06 2024-05-11T23:02:21.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941082+0000", 2024-05-11T23:02:21.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-05-11T23:02:21.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992446+0000", 2024-05-11T23:02:21.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051364058999999997 2024-05-11T23:02:21.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992536+0000", 2024-05-11T23:02:21.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0085000000000002e-05 2024-05-11T23:02:21.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992643+0000", 2024-05-11T23:02:21.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107377 2024-05-11T23:02:21.592 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.592 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.592 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.592 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36988 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 602112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:21.592 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:21.592 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.616060529999999, 2024-05-11T23:02:21.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051700007999999999, 2024-05-11T23:02:21.593 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.593 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.593 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.593 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.593 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.593 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36988 2024-05-11T23:02:21.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.593 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:21.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:21.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:21.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.05e-07 2024-05-11T23:02:21.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941086+0000", 2024-05-11T23:02:21.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.125e-06 2024-05-11T23:02:21.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941089+0000", 2024-05-11T23:02:21.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0599999999999999e-06 2024-05-11T23:02:21.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941092+0000", 2024-05-11T23:02:21.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6529999999999998e-06 2024-05-11T23:02:21.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992667+0000", 2024-05-11T23:02:21.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051574587999999998 2024-05-11T23:02:21.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992723+0000", 2024-05-11T23:02:21.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6682999999999999e-05 2024-05-11T23:02:21.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992785+0000", 2024-05-11T23:02:21.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1693999999999996e-05 2024-05-11T23:02:21.596 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.596 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.596 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.596 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36989 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 638976~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:21.597 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:21.597 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.616050341000001, 2024-05-11T23:02:21.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05192811, 2024-05-11T23:02:21.597 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.597 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.597 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.597 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.597 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.597 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.597 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36989 2024-05-11T23:02:21.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.597 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:21.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:21.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942916+0000", 2024-05-11T23:02:21.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001820835 2024-05-11T23:02:21.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942920+0000", 2024-05-11T23:02:21.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5310000000000002e-06 2024-05-11T23:02:21.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942931+0000", 2024-05-11T23:02:21.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1839e-05 2024-05-11T23:02:21.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942940+0000", 2024-05-11T23:02:21.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2309999999999996e-06 2024-05-11T23:02:21.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992820+0000", 2024-05-11T23:02:21.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049880751000000001 2024-05-11T23:02:21.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992918+0000", 2024-05-11T23:02:21.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7051000000000003e-05 2024-05-11T23:02:21.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.993023+0000", 2024-05-11T23:02:21.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010587200000000001 2024-05-11T23:02:21.600 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.600 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.600 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.601 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36997 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 933888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:21.601 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:21.601 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.614016505999999, 2024-05-11T23:02:21.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051390602000000001, 2024-05-11T23:02:21.601 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.601 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.601 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.601 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.601 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.601 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36997 2024-05-11T23:02:21.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.601 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:21.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:21.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943130+0000", 2024-05-11T23:02:21.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0200000000000003e-07 2024-05-11T23:02:21.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943132+0000", 2024-05-11T23:02:21.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.305e-06 2024-05-11T23:02:21.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943138+0000", 2024-05-11T23:02:21.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4219999999999997e-06 2024-05-11T23:02:21.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943143+0000", 2024-05-11T23:02:21.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0610000000000003e-06 2024-05-11T23:02:21.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994323+0000", 2024-05-11T23:02:21.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051179592000000003 2024-05-11T23:02:21.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994412+0000", 2024-05-11T23:02:21.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8709999999999996e-05 2024-05-11T23:02:21.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994520+0000", 2024-05-11T23:02:21.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010811000000000001 2024-05-11T23:02:21.605 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.605 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.605 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.605 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36998 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 970752~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:21.605 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:21.605 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.613995627000001, 2024-05-11T23:02:21.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051517669000000002, 2024-05-11T23:02:21.605 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.605 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.605 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.605 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.605 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.605 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.606 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36998 2024-05-11T23:02:21.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.606 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:21.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:21.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:21.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:02:21.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943153+0000", 2024-05-11T23:02:21.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.306e-06 2024-05-11T23:02:21.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943160+0000", 2024-05-11T23:02:21.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.452e-06 2024-05-11T23:02:21.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943165+0000", 2024-05-11T23:02:21.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.152e-06 2024-05-11T23:02:21.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994561+0000", 2024-05-11T23:02:21.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051395765000000003 2024-05-11T23:02:21.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994607+0000", 2024-05-11T23:02:21.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6415000000000002e-05 2024-05-11T23:02:21.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994668+0000", 2024-05-11T23:02:21.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0179000000000001e-05 2024-05-11T23:02:21.609 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.609 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.609 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.609 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36999 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 1007616~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:21.609 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:21.609 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.613950746, 2024-05-11T23:02:21.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051696919000000001, 2024-05-11T23:02:21.609 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.610 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.610 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.610 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.610 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.610 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36999 2024-05-11T23:02:21.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.610 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:21.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:21.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:21.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:02:21.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943197+0000", 2024-05-11T23:02:21.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1280000000000002e-06 2024-05-11T23:02:21.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943202+0000", 2024-05-11T23:02:21.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8910000000000003e-06 2024-05-11T23:02:21.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943206+0000", 2024-05-11T23:02:21.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3550000000000001e-06 2024-05-11T23:02:21.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994703+0000", 2024-05-11T23:02:21.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051497045999999998 2024-05-11T23:02:21.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994789+0000", 2024-05-11T23:02:21.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6458000000000005e-05 2024-05-11T23:02:21.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994892+0000", 2024-05-11T23:02:21.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102705 2024-05-11T23:02:21.613 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.613 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.613 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.613 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37000 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 1044480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:21.613 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:21.613 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.613934800999999, 2024-05-11T23:02:21.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051842371999999998, 2024-05-11T23:02:21.614 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.614 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.614 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.614 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.614 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.614 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.614 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37000 2024-05-11T23:02:21.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.614 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:21.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:21.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944916+0000", 2024-05-11T23:02:21.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001705097 2024-05-11T23:02:21.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944921+0000", 2024-05-11T23:02:21.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5809999999999998e-06 2024-05-11T23:02:21.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944930+0000", 2024-05-11T23:02:21.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5529999999999998e-06 2024-05-11T23:02:21.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944938+0000", 2024-05-11T23:02:21.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2519999999999999e-06 2024-05-11T23:02:21.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994911+0000", 2024-05-11T23:02:21.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049973113999999999 2024-05-11T23:02:21.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994982+0000", 2024-05-11T23:02:21.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0544999999999996e-05 2024-05-11T23:02:21.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.995053+0000", 2024-05-11T23:02:21.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1229999999999994e-05 2024-05-11T23:02:21.617 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.617 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.617 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.618 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37041 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 618496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:21.618 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:21.618 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.612090821999999, 2024-05-11T23:02:21.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051352768999999999, 2024-05-11T23:02:21.618 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.618 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.618 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.618 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.618 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.618 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.618 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37041 2024-05-11T23:02:21.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.618 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:21.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:21.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:21.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:02:21.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945056+0000", 2024-05-11T23:02:21.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:02:21.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945059+0000", 2024-05-11T23:02:21.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.965e-06 2024-05-11T23:02:21.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945062+0000", 2024-05-11T23:02:21.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:02:21.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996213+0000", 2024-05-11T23:02:21.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051151532999999999 2024-05-11T23:02:21.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996301+0000", 2024-05-11T23:02:21.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8143e-05 2024-05-11T23:02:21.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996408+0000", 2024-05-11T23:02:21.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010623700000000001 2024-05-11T23:02:21.621 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.622 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.622 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.622 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37042 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 655360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:21.622 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:21.622 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.612080837000001, 2024-05-11T23:02:21.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051462075000000003, 2024-05-11T23:02:21.622 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.622 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.622 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.622 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.622 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.623 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.623 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37042 2024-05-11T23:02:21.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.623 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:21.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:21.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:21.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0599999999999999e-07 2024-05-11T23:02:21.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945066+0000", 2024-05-11T23:02:21.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-05-11T23:02:21.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945069+0000", 2024-05-11T23:02:21.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0079999999999998e-06 2024-05-11T23:02:21.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945072+0000", 2024-05-11T23:02:21.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6740000000000001e-06 2024-05-11T23:02:21.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996422+0000", 2024-05-11T23:02:21.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051349999 2024-05-11T23:02:21.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996467+0000", 2024-05-11T23:02:21.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.528e-05 2024-05-11T23:02:21.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996527+0000", 2024-05-11T23:02:21.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9759999999999997e-05 2024-05-11T23:02:21.626 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.626 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.626 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.626 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37043 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 692224~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:21.626 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:21.626 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.612068764, 2024-05-11T23:02:21.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051679535999999998, 2024-05-11T23:02:21.626 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.626 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.627 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.627 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.627 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.627 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37043 2024-05-11T23:02:21.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.627 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:21.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:21.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:21.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8999999999999997e-07 2024-05-11T23:02:21.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945079+0000", 2024-05-11T23:02:21.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.001e-06 2024-05-11T23:02:21.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945087+0000", 2024-05-11T23:02:21.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1749999999999999e-06 2024-05-11T23:02:21.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945093+0000", 2024-05-11T23:02:21.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9639999999999997e-06 2024-05-11T23:02:21.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996563+0000", 2024-05-11T23:02:21.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051469050000000002 2024-05-11T23:02:21.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996651+0000", 2024-05-11T23:02:21.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8356000000000006e-05 2024-05-11T23:02:21.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996756+0000", 2024-05-11T23:02:21.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0001055 2024-05-11T23:02:21.630 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.630 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.630 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.630 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37044 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 729088~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:21.630 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:21.630 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.612034852000001, 2024-05-11T23:02:21.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051763445999999998, 2024-05-11T23:02:21.630 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.631 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.631 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.631 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.631 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.631 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.631 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37044 2024-05-11T23:02:21.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.631 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:21.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:21.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946915+0000", 2024-05-11T23:02:21.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018042760000000001 2024-05-11T23:02:21.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946918+0000", 2024-05-11T23:02:21.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.36e-06 2024-05-11T23:02:21.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946926+0000", 2024-05-11T23:02:21.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3130000000000001e-06 2024-05-11T23:02:21.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946930+0000", 2024-05-11T23:02:21.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5249999999999999e-06 2024-05-11T23:02:21.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996772+0000", 2024-05-11T23:02:21.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049841868999999997 2024-05-11T23:02:21.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996817+0000", 2024-05-11T23:02:21.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5351e-05 2024-05-11T23:02:21.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996874+0000", 2024-05-11T23:02:21.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6752000000000001e-05 2024-05-11T23:02:21.634 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.634 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.634 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.634 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37052 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 1024000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:21.634 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:21.635 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.610128001, 2024-05-11T23:02:21.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051326140999999999, 2024-05-11T23:02:21.635 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.635 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.635 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.635 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.635 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.635 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.635 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37052 2024-05-11T23:02:21.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.635 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:21.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:21.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:21.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:21.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947019+0000", 2024-05-11T23:02:21.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.218e-06 2024-05-11T23:02:21.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947022+0000", 2024-05-11T23:02:21.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1889999999999999e-06 2024-05-11T23:02:21.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947025+0000", 2024-05-11T23:02:21.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.464e-06 2024-05-11T23:02:21.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998237+0000", 2024-05-11T23:02:21.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051212494999999997 2024-05-11T23:02:21.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998284+0000", 2024-05-11T23:02:21.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7246000000000001e-05 2024-05-11T23:02:21.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998344+0000", 2024-05-11T23:02:21.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9320000000000001e-05 2024-05-11T23:02:21.638 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.638 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.638 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.639 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37071 3.b 3:d385bc8e:::rbd_data.111e5e4d8275.0000000000000020:11 [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:21.639 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:21.639 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.610117792, 2024-05-11T23:02:21.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051528796000000002, 2024-05-11T23:02:21.639 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:21.639 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:21.639 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:21.639 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:21.639 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:21.639 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:21.639 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37071 2024-05-11T23:02:21.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.640 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:21.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:21.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:21.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:21.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:21.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:21.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:21.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948913+0000", 2024-05-11T23:02:21.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018854970000000001 2024-05-11T23:02:21.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:21.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948917+0000", 2024-05-11T23:02:21.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8089999999999999e-06 2024-05-11T23:02:21.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:21.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948926+0000", 2024-05-11T23:02:21.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8389999999999996e-06 2024-05-11T23:02:21.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:21.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948930+0000", 2024-05-11T23:02:21.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4519999999999999e-06 2024-05-11T23:02:21.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:21.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998379+0000", 2024-05-11T23:02:21.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049449011000000001 2024-05-11T23:02:21.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:21.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998462+0000", 2024-05-11T23:02:21.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2739999999999997e-05 2024-05-11T23:02:21.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:21.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:21.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:21.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998557+0000", 2024-05-11T23:02:21.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4448000000000001e-05 2024-05-11T23:02:21.642 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.643 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.643 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.643 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:21.643 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:21.643 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:21.644 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:02:21.645 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:21.644+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:21.745 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:21.745+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:21.762 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:21.762 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:21.762 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:21.762 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:21.771 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:02:21.821 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:02:21.846 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:21.845+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:21.877 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:21.878 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:21.886 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:02:21.947 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:21.947+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:21.948 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:21.948 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:21.968 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:21.968 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:21.985 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:22.002 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:22.002 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:22.020 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:22.020 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:22.034 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:22.034 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:22.037 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:22.038 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:22.043 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:02:22.055 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:22.055 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:22.073 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:22.073 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:22.086 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:22.086+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:22.148 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:22.148 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:22.148 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:22.148+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:22.156 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:02:22.249 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:22.248+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:22.302 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:22.302 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:22.311 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:02:22.350 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:22.350+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:22.451 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:22.450+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:22.454 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:22.454 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:22.463 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:02:22.552 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:22.552+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:22.608 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:22.608 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:22.617 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:02:22.660 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:22.660+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:22.721 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:22.721 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:22.730 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:02:22.755 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:22.755+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:22.856 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:22.856+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:22.875 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:22.875 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:22.885 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:02:22.957 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:22.957+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:22.970 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-11T23:02:22.992 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:22.993 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:23.002 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:02:23.058 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:23.058+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:23.115 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone31 from 0xa3ee955 (overlap 0xa3ee955) to 0x1064a88 2024-05-11T23:02:23.117 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:23.117 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:23.130 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:02:23.174 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:23.174+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:23.246 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-05-11T23:02:23.257 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:23.257 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:23.259 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:23.259+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:23.266 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:02:23.361 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:23.360+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:23.387 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:23.387 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:23.397 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:02:23.437 INFO:teuthology.orchestra.run.smithi005.stdout:3648 write 0x1d0ccf9 thru 0x1d13da3 (0x70ab bytes) 2024-05-11T23:02:23.457 INFO:teuthology.orchestra.run.smithi005.stdout:3649 writesame 0x35520ee thru 0x355b013 data_size 0x49(0x8f26 bytes) 2024-05-11T23:02:23.461 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:23.461+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:23.479 INFO:teuthology.orchestra.run.smithi005.stdout:3650 punch from 0x4ddab9e to 0x4ddbd76, (0x11d8 bytes) 2024-05-11T23:02:23.479 INFO:teuthology.orchestra.run.smithi005.stdout:3652 write 0x6d51ce0 thru 0x6d5b048 (0x9369 bytes) 2024-05-11T23:02:23.484 INFO:teuthology.orchestra.run.smithi005.stdout:3653 clone 33 order 18 su 8192 sc 8 2024-05-11T23:02:23.514 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:23.514 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:23.523 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:02:23.564 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:23.564+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:23.663 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:23.663+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:23.672 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:23.672 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:23.681 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:02:23.766 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:23.766+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:23.832 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:23.832 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:23.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:02:23.864 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:23.864+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:23.963 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-05-11T23:02:23.965 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:23.965+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:24.002 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:24.002 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:24.011 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:02:24.066 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:24.066+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:24.085 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:02:24.162 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:24.162 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:24.167 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:24.167+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:24.171 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:02:24.222 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:24.222 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:24.240 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:24.241 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:24.258 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:24.258 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:24.268 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:24.268+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:24.276 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:24.276 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:24.295 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:24.295 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:24.313 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:24.313 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:24.323 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:24.323 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:24.323 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:02:24.323 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:24.331 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:24.331 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:24.332 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:02:24.348 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:24.349 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:24.401 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:24.401+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:24.458 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:24.459 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:24.459 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:24.459 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:24.459 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:24.467 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:02:24.508 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:24.508+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:24.571 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:24.571+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:24.618 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:24.618 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:24.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:24.618 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:24.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.618 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60471 3.4 e43/19 3:2079ea20:::rbd_data.111e1df04b77.0000000000000000:head v 43'84, mlcod=42'81)", 2024-05-11T23:02:24.618 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:24.618 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.690317253, 2024-05-11T23:02:24.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0079043050000000004, 2024-05-11T23:02:24.619 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:24.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:24.619 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:24.619 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:24.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:24.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:24.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:24.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:24.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:24.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:24.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-05-11T23:02:24.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:24.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926880+0000", 2024-05-11T23:02:24.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:02:24.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:24.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926893+0000", 2024-05-11T23:02:24.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3256e-05 2024-05-11T23:02:24.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:24.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926899+0000", 2024-05-11T23:02:24.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9850000000000003e-06 2024-05-11T23:02:24.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926982+0000", 2024-05-11T23:02:24.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2997000000000005e-05 2024-05-11T23:02:24.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:24.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927005+0000", 2024-05-11T23:02:24.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3428e-05 2024-05-11T23:02:24.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:24.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.934747+0000", 2024-05-11T23:02:24.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0077415820000000003 2024-05-11T23:02:24.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:24.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.934778+0000", 2024-05-11T23:02:24.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1479999999999997e-05 2024-05-11T23:02:24.622 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.622 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.622 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.623 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60472 3.9 e43/19 3:94f86c79:::rbd_data.111e1df04b77.0000000000000001:head v 43'122, mlcod=42'119)", 2024-05-11T23:02:24.623 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.927124+0000", 2024-05-11T23:02:24.623 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.690067552999999, 2024-05-11T23:02:24.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0062295900000000001, 2024-05-11T23:02:24.623 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:24.623 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:24.623 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:24.623 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:24.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:24.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927124+0000", 2024-05-11T23:02:24.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:24.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927124+0000", 2024-05-11T23:02:24.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:24.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927124+0000", 2024-05-11T23:02:24.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:24.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:24.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927128+0000", 2024-05-11T23:02:24.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3390000000000003e-06 2024-05-11T23:02:24.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:24.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927139+0000", 2024-05-11T23:02:24.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1161e-05 2024-05-11T23:02:24.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:24.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927147+0000", 2024-05-11T23:02:24.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1910000000000003e-06 2024-05-11T23:02:24.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927199+0000", 2024-05-11T23:02:24.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.24e-05 2024-05-11T23:02:24.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:24.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927223+0000", 2024-05-11T23:02:24.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3811999999999998e-05 2024-05-11T23:02:24.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:24.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.933287+0000", 2024-05-11T23:02:24.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.006063723 2024-05-11T23:02:24.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:24.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.933353+0000", 2024-05-11T23:02:24.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6744999999999999e-05 2024-05-11T23:02:24.626 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.627 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.627 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.627 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:64399 3.1e e48/47 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head v 48'297, mlcod=0'0)", 2024-05-11T23:02:24.627 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:02.104619+0000", 2024-05-11T23:02:24.627 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.51257214, 2024-05-11T23:02:24.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014840949000000001, 2024-05-11T23:02:24.627 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:24.627 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:24.627 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:24.627 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:24.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:24.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.104619+0000", 2024-05-11T23:02:24.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:24.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.104619+0000", 2024-05-11T23:02:24.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:24.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.104620+0000", 2024-05-11T23:02:24.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.299e-06 2024-05-11T23:02:24.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:24.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.112866+0000", 2024-05-11T23:02:24.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082450949999999992 2024-05-11T23:02:24.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:24.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.112893+0000", 2024-05-11T23:02:24.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7131000000000002e-05 2024-05-11T23:02:24.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:24.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.112902+0000", 2024-05-11T23:02:24.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4089999999999992e-06 2024-05-11T23:02:24.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.112980+0000", 2024-05-11T23:02:24.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8375999999999996e-05 2024-05-11T23:02:24.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:24.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.113009+0000", 2024-05-11T23:02:24.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8713000000000001e-05 2024-05-11T23:02:24.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:24.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.119403+0000", 2024-05-11T23:02:24.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0063940580000000002 2024-05-11T23:02:24.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:24.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.119460+0000", 2024-05-11T23:02:24.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6867999999999998e-05 2024-05-11T23:02:24.631 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.631 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.631 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.631 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65750 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 5242880~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:24.631 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.980128+0000", 2024-05-11T23:02:24.631 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.637063038000001, 2024-05-11T23:02:24.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0045445240000000003, 2024-05-11T23:02:24.631 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:24.631 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:24.631 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:24.631 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:24.631 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:24.632 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:24.632 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65750 2024-05-11T23:02:24.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.632 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:24.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:24.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.980128+0000", 2024-05-11T23:02:24.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:24.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.980128+0000", 2024-05-11T23:02:24.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:24.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.980128+0000", 2024-05-11T23:02:24.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.22e-07 2024-05-11T23:02:24.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:24.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.980130+0000", 2024-05-11T23:02:24.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1009999999999999e-06 2024-05-11T23:02:24.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:24.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.980133+0000", 2024-05-11T23:02:24.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9890000000000001e-06 2024-05-11T23:02:24.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:24.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.980135+0000", 2024-05-11T23:02:24.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.43e-06 2024-05-11T23:02:24.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.984329+0000", 2024-05-11T23:02:24.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0041943270000000003 2024-05-11T23:02:24.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:24.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.984401+0000", 2024-05-11T23:02:24.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1268999999999998e-05 2024-05-11T23:02:24.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:24.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.984673+0000", 2024-05-11T23:02:24.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000272186 2024-05-11T23:02:24.635 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.635 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.635 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.635 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65755 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 5636096~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:24.635 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.981205+0000", 2024-05-11T23:02:24.635 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.635986500000001, 2024-05-11T23:02:24.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.004494077, 2024-05-11T23:02:24.636 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:24.636 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:24.636 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:24.636 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:24.636 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:24.636 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:24.636 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65755 2024-05-11T23:02:24.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.636 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:24.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:24.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.981205+0000", 2024-05-11T23:02:24.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:24.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.981205+0000", 2024-05-11T23:02:24.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:24.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.981205+0000", 2024-05-11T23:02:24.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9999999999999998e-07 2024-05-11T23:02:24.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:24.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.981209+0000", 2024-05-11T23:02:24.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5209999999999998e-06 2024-05-11T23:02:24.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:24.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.981221+0000", 2024-05-11T23:02:24.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2407e-05 2024-05-11T23:02:24.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:24.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.981227+0000", 2024-05-11T23:02:24.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2110000000000003e-06 2024-05-11T23:02:24.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.985298+0000", 2024-05-11T23:02:24.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.004070415 2024-05-11T23:02:24.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:24.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.985369+0000", 2024-05-11T23:02:24.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1191000000000003e-05 2024-05-11T23:02:24.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:24.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.985699+0000", 2024-05-11T23:02:24.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00032983200000000001 2024-05-11T23:02:24.639 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.640 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.640 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65759 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 5767168~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:24.640 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.982321+0000", 2024-05-11T23:02:24.640 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.634870218, 2024-05-11T23:02:24.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0043249059999999999, 2024-05-11T23:02:24.640 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:24.640 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:24.640 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:24.640 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:24.640 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:24.640 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:24.640 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65759 2024-05-11T23:02:24.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.641 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:24.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:24.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.982321+0000", 2024-05-11T23:02:24.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:24.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.982321+0000", 2024-05-11T23:02:24.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:24.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.982322+0000", 2024-05-11T23:02:24.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5199999999999997e-07 2024-05-11T23:02:24.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:24.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.982325+0000", 2024-05-11T23:02:24.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.224e-06 2024-05-11T23:02:24.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:24.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.982332+0000", 2024-05-11T23:02:24.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.063e-06 2024-05-11T23:02:24.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:24.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.982346+0000", 2024-05-11T23:02:24.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3715e-05 2024-05-11T23:02:24.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.986221+0000", 2024-05-11T23:02:24.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0038758460000000001 2024-05-11T23:02:24.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:24.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.986320+0000", 2024-05-11T23:02:24.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8892000000000001e-05 2024-05-11T23:02:24.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:24.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.986646+0000", 2024-05-11T23:02:24.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000325614 2024-05-11T23:02:24.644 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.644 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.644 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.644 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65788 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 7995392~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:24.644 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.989762+0000", 2024-05-11T23:02:24.644 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.62742922, 2024-05-11T23:02:24.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0043592930000000002, 2024-05-11T23:02:24.644 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:24.644 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:24.644 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:24.644 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:24.644 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:24.645 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:24.645 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65788 2024-05-11T23:02:24.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.645 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:24.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:24.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.989762+0000", 2024-05-11T23:02:24.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:24.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.989762+0000", 2024-05-11T23:02:24.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:24.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.989763+0000", 2024-05-11T23:02:24.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.37e-07 2024-05-11T23:02:24.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:24.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.989765+0000", 2024-05-11T23:02:24.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5940000000000001e-06 2024-05-11T23:02:24.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:24.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.989773+0000", 2024-05-11T23:02:24.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5549999999999997e-06 2024-05-11T23:02:24.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:24.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.989778+0000", 2024-05-11T23:02:24.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4750000000000001e-06 2024-05-11T23:02:24.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993696+0000", 2024-05-11T23:02:24.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0039173639999999996 2024-05-11T23:02:24.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:24.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993770+0000", 2024-05-11T23:02:24.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4307999999999994e-05 2024-05-11T23:02:24.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:24.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.994121+0000", 2024-05-11T23:02:24.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00035146 2024-05-11T23:02:24.648 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.648 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.649 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.649 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65791 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 6815744~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:24.649 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.990491+0000", 2024-05-11T23:02:24.649 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.626700484000001, 2024-05-11T23:02:24.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.004670258, 2024-05-11T23:02:24.649 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:24.649 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:24.649 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:24.650 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:24.650 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:24.650 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:24.650 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65791 2024-05-11T23:02:24.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.650 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:24.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:24.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990491+0000", 2024-05-11T23:02:24.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:24.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990491+0000", 2024-05-11T23:02:24.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:24.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990492+0000", 2024-05-11T23:02:24.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.026e-06 2024-05-11T23:02:24.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:24.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990496+0000", 2024-05-11T23:02:24.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.481e-06 2024-05-11T23:02:24.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:24.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990507+0000", 2024-05-11T23:02:24.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0689e-05 2024-05-11T23:02:24.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:24.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990515+0000", 2024-05-11T23:02:24.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8990000000000001e-06 2024-05-11T23:02:24.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.994180+0000", 2024-05-11T23:02:24.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0036648710000000001 2024-05-11T23:02:24.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:24.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.994308+0000", 2024-05-11T23:02:24.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00012834700000000001 2024-05-11T23:02:24.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:24.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.995161+0000", 2024-05-11T23:02:24.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00085294500000000003 2024-05-11T23:02:24.653 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.653 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.653 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.653 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65792 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 8257536~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:24.653 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.990528+0000", 2024-05-11T23:02:24.653 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.62666347, 2024-05-11T23:02:24.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0053202220000000003, 2024-05-11T23:02:24.654 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:24.654 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:24.654 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:24.654 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:24.654 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:24.654 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:24.654 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65792 2024-05-11T23:02:24.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.654 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:24.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:24.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990528+0000", 2024-05-11T23:02:24.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:24.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990528+0000", 2024-05-11T23:02:24.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:24.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990528+0000", 2024-05-11T23:02:24.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.13e-07 2024-05-11T23:02:24.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:24.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990539+0000", 2024-05-11T23:02:24.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0152999999999999e-05 2024-05-11T23:02:24.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:24.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990546+0000", 2024-05-11T23:02:24.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1779999999999997e-06 2024-05-11T23:02:24.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:24.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990551+0000", 2024-05-11T23:02:24.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.147e-06 2024-05-11T23:02:24.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.995187+0000", 2024-05-11T23:02:24.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.004636215 2024-05-11T23:02:24.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:24.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.995276+0000", 2024-05-11T23:02:24.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9086999999999996e-05 2024-05-11T23:02:24.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:24.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.995848+0000", 2024-05-11T23:02:24.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00057182900000000004 2024-05-11T23:02:24.657 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.657 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.657 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.657 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65794 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 6684672~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:24.657 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.990959+0000", 2024-05-11T23:02:24.658 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.626232636000001, 2024-05-11T23:02:24.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.005377672, 2024-05-11T23:02:24.658 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:24.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:24.658 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:24.658 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:24.658 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:24.658 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:24.658 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65794 2024-05-11T23:02:24.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.658 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:24.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:24.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990959+0000", 2024-05-11T23:02:24.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:24.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990959+0000", 2024-05-11T23:02:24.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:24.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990959+0000", 2024-05-11T23:02:24.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7199999999999997e-07 2024-05-11T23:02:24.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:24.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990964+0000", 2024-05-11T23:02:24.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.639e-06 2024-05-11T23:02:24.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:24.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990976+0000", 2024-05-11T23:02:24.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1420000000000001e-05 2024-05-11T23:02:24.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:24.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990983+0000", 2024-05-11T23:02:24.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7719999999999994e-06 2024-05-11T23:02:24.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.995887+0000", 2024-05-11T23:02:24.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0049036239999999997 2024-05-11T23:02:24.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:24.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.996005+0000", 2024-05-11T23:02:24.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118387 2024-05-11T23:02:24.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:24.662 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.996336+0000", 2024-05-11T23:02:24.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00033095800000000001 2024-05-11T23:02:24.662 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.662 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.662 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.662 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65795 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 8126464~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:24.662 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.990996+0000", 2024-05-11T23:02:24.662 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.626195403000001, 2024-05-11T23:02:24.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0057710749999999996, 2024-05-11T23:02:24.662 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:24.662 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:24.662 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:24.663 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:24.663 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:24.663 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:24.663 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65795 2024-05-11T23:02:24.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.663 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:24.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:24.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990996+0000", 2024-05-11T23:02:24.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:24.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990996+0000", 2024-05-11T23:02:24.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:24.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.990996+0000", 2024-05-11T23:02:24.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5600000000000001e-07 2024-05-11T23:02:24.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:24.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.991000+0000", 2024-05-11T23:02:24.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2550000000000002e-06 2024-05-11T23:02:24.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:24.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.991006+0000", 2024-05-11T23:02:24.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4080000000000003e-06 2024-05-11T23:02:24.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:24.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.991011+0000", 2024-05-11T23:02:24.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1749999999999996e-06 2024-05-11T23:02:24.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.996368+0000", 2024-05-11T23:02:24.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0053564040000000004 2024-05-11T23:02:24.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:24.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.996461+0000", 2024-05-11T23:02:24.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3435000000000002e-05 2024-05-11T23:02:24.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:24.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.996767+0000", 2024-05-11T23:02:24.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00030594200000000002 2024-05-11T23:02:24.666 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.666 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.666 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.666 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65799 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 7077888~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:24.666 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.992390+0000", 2024-05-11T23:02:24.666 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.624801052999999, 2024-05-11T23:02:24.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0050808049999999999, 2024-05-11T23:02:24.666 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:24.667 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:24.667 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:24.667 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:24.667 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:24.667 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:24.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65799 2024-05-11T23:02:24.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.667 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:24.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:24.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.992390+0000", 2024-05-11T23:02:24.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:24.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.992390+0000", 2024-05-11T23:02:24.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:24.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.992392+0000", 2024-05-11T23:02:24.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.314e-06 2024-05-11T23:02:24.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:24.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.992397+0000", 2024-05-11T23:02:24.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.023e-06 2024-05-11T23:02:24.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:24.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.992411+0000", 2024-05-11T23:02:24.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4681000000000001e-05 2024-05-11T23:02:24.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:24.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.992434+0000", 2024-05-11T23:02:24.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2518999999999999e-05 2024-05-11T23:02:24.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.996799+0000", 2024-05-11T23:02:24.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0043649090000000001 2024-05-11T23:02:24.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:24.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.996904+0000", 2024-05-11T23:02:24.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010525499999999999 2024-05-11T23:02:24.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:24.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.997471+0000", 2024-05-11T23:02:24.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00056710400000000002 2024-05-11T23:02:24.670 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.670 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.670 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.670 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65803 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 7340032~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:24.670 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.993407+0000", 2024-05-11T23:02:24.671 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.623784493999999, 2024-05-11T23:02:24.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0045385190000000004, 2024-05-11T23:02:24.671 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:24.671 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:24.671 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:24.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:24.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:24.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:24.671 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65803 2024-05-11T23:02:24.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.671 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:24.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:24.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993407+0000", 2024-05-11T23:02:24.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:24.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993407+0000", 2024-05-11T23:02:24.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:24.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993408+0000", 2024-05-11T23:02:24.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3600000000000003e-07 2024-05-11T23:02:24.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:24.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993411+0000", 2024-05-11T23:02:24.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9859999999999996e-06 2024-05-11T23:02:24.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:24.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993424+0000", 2024-05-11T23:02:24.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2308e-05 2024-05-11T23:02:24.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:24.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993431+0000", 2024-05-11T23:02:24.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2099999999999996e-06 2024-05-11T23:02:24.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.997506+0000", 2024-05-11T23:02:24.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0040752749999999997 2024-05-11T23:02:24.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:24.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.997613+0000", 2024-05-11T23:02:24.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000106895 2024-05-11T23:02:24.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:24.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.997945+0000", 2024-05-11T23:02:24.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00033210899999999997 2024-05-11T23:02:24.675 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.675 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.675 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.675 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:65807 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:17 [sparse-read 7208960~131072] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:24.675 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:05.993534+0000", 2024-05-11T23:02:24.675 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.623656917000002, 2024-05-11T23:02:24.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0048977109999999999, 2024-05-11T23:02:24.675 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:24.675 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:24.675 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:24.675 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:24.676 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:24.676 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:24.676 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 65807 2024-05-11T23:02:24.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.676 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:24.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:24.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993534+0000", 2024-05-11T23:02:24.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:24.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993534+0000", 2024-05-11T23:02:24.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:24.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993535+0000", 2024-05-11T23:02:24.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9699999999999996e-07 2024-05-11T23:02:24.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:24.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993538+0000", 2024-05-11T23:02:24.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9840000000000001e-06 2024-05-11T23:02:24.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:24.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993545+0000", 2024-05-11T23:02:24.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8009999999999998e-06 2024-05-11T23:02:24.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:24.678 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.993549+0000", 2024-05-11T23:02:24.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0029999999999996e-06 2024-05-11T23:02:24.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.678 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.997983+0000", 2024-05-11T23:02:24.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0044348269999999997 2024-05-11T23:02:24.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:24.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.998093+0000", 2024-05-11T23:02:24.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000109992 2024-05-11T23:02:24.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:24.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:05.998432+0000", 2024-05-11T23:02:24.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00033860699999999999 2024-05-11T23:02:24.679 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.679 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.679 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.679 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:66044 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [sparse-read 0~8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:24.679 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:06.114668+0000", 2024-05-11T23:02:24.680 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.502523451999998, 2024-05-11T23:02:24.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0062482759999999997, 2024-05-11T23:02:24.680 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:24.680 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:24.680 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:24.680 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:24.680 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:24.680 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:24.680 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 66044 2024-05-11T23:02:24.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.680 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:24.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:24.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114668+0000", 2024-05-11T23:02:24.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:24.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114668+0000", 2024-05-11T23:02:24.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:24.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114669+0000", 2024-05-11T23:02:24.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.299e-06 2024-05-11T23:02:24.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:24.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114675+0000", 2024-05-11T23:02:24.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5210000000000002e-06 2024-05-11T23:02:24.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:24.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114690+0000", 2024-05-11T23:02:24.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5349000000000002e-05 2024-05-11T23:02:24.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:24.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114697+0000", 2024-05-11T23:02:24.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6279999999999999e-06 2024-05-11T23:02:24.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114749+0000", 2024-05-11T23:02:24.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1958e-05 2024-05-11T23:02:24.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:24.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114828+0000", 2024-05-11T23:02:24.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9728999999999995e-05 2024-05-11T23:02:24.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:24.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.120916+0000", 2024-05-11T23:02:24.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0060877919999999999 2024-05-11T23:02:24.684 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.684 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.684 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.684 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "MOSDPGPushReply(3.1e 52/51 [PushReplyOp(3:7943f929:::rbd_data.111ef7a115a1.0000000000000050:head)])", 2024-05-11T23:02:24.684 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:06.114699+0000", 2024-05-11T23:02:24.684 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.502492375999999, 2024-05-11T23:02:24.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0064609460000000004, 2024-05-11T23:02:24.685 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:24.685 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:24.685 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:24.685 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:24.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:24.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114699+0000", 2024-05-11T23:02:24.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:24.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114699+0000", 2024-05-11T23:02:24.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:24.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114700+0000", 2024-05-11T23:02:24.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0780000000000001e-06 2024-05-11T23:02:24.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:24.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114706+0000", 2024-05-11T23:02:24.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4200000000000004e-06 2024-05-11T23:02:24.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:24.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114724+0000", 2024-05-11T23:02:24.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7142e-05 2024-05-11T23:02:24.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:24.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114732+0000", 2024-05-11T23:02:24.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7630000000000006e-06 2024-05-11T23:02:24.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.120983+0000", 2024-05-11T23:02:24.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0062507250000000004 2024-05-11T23:02:24.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:24.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.121160+0000", 2024-05-11T23:02:24.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000176818 2024-05-11T23:02:24.688 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.688 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.688 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.688 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:77128 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_get in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-11T23:02:24.688 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:02:24.689 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.6011895910000007, 2024-05-11T23:02:24.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91737855899999998, 2024-05-11T23:02:24.689 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:24.689 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:24.689 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:24.689 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:24.689 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:24.689 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:24.689 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 77128 2024-05-11T23:02:24.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.689 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:24.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:24.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:02:24.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:24.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:02:24.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:24.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016003+0000", 2024-05-11T23:02:24.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5379999999999999e-06 2024-05-11T23:02:24.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:24.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016011+0000", 2024-05-11T23:02:24.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6410000000000002e-06 2024-05-11T23:02:24.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:24.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016026+0000", 2024-05-11T23:02:24.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4779e-05 2024-05-11T23:02:24.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:24.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016032+0000", 2024-05-11T23:02:24.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.066e-06 2024-05-11T23:02:24.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016093+0000", 2024-05-11T23:02:24.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0878000000000002e-05 2024-05-11T23:02:24.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:02:24.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016100+0000", 2024-05-11T23:02:24.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6799999999999993e-06 2024-05-11T23:02:24.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933302+0000", 2024-05-11T23:02:24.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91720135300000005 2024-05-11T23:02:24.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:24.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933380+0000", 2024-05-11T23:02:24.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8623999999999998e-05 2024-05-11T23:02:24.693 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.693 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.693 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.693 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:78890 3.3 3:c289f39e:::rbd_data.111ea74f268d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:24.693 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:16.008252+0000", 2024-05-11T23:02:24.693 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.6089394509999995, 2024-05-11T23:02:24.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0070432100000000003, 2024-05-11T23:02:24.693 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:24.693 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:24.694 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:24.694 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:24.694 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:24.694 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:24.694 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 78890 2024-05-11T23:02:24.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.694 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:24.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:24.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.008252+0000", 2024-05-11T23:02:24.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:24.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.008252+0000", 2024-05-11T23:02:24.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:24.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.008253+0000", 2024-05-11T23:02:24.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-05-11T23:02:24.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:24.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009306+0000", 2024-05-11T23:02:24.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001053327 2024-05-11T23:02:24.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:24.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009329+0000", 2024-05-11T23:02:24.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3113999999999999e-05 2024-05-11T23:02:24.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:24.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009345+0000", 2024-05-11T23:02:24.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5136e-05 2024-05-11T23:02:24.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009447+0000", 2024-05-11T23:02:24.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010199799999999999 2024-05-11T23:02:24.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:24.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009640+0000", 2024-05-11T23:02:24.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019346000000000001 2024-05-11T23:02:24.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-05-11T23:02:24.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009832+0000", 2024-05-11T23:02:24.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019212799999999999 2024-05-11T23:02:24.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:02:24.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.014184+0000", 2024-05-11T23:02:24.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0043513909999999996 2024-05-11T23:02:24.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:24.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.015200+0000", 2024-05-11T23:02:24.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001016926 2024-05-11T23:02:24.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:24.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.015250+0000", 2024-05-11T23:02:24.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9162999999999997e-05 2024-05-11T23:02:24.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:24.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.015295+0000", 2024-05-11T23:02:24.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5425e-05 2024-05-11T23:02:24.698 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.698 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.699 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.699 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:78891 3.3 3:c289f39e:::rbd_data.111ea74f268d.0000000000000001:head [write 0~0] snapc 1d=[1d] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:24.699 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:16.009376+0000", 2024-05-11T23:02:24.699 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.6078148310000007, 2024-05-11T23:02:24.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0061637829999999999, 2024-05-11T23:02:24.699 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:24.699 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:24.699 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:24.699 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:24.699 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:24.699 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:24.699 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 78891 2024-05-11T23:02:24.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.700 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:24.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:24.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009376+0000", 2024-05-11T23:02:24.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:24.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009376+0000", 2024-05-11T23:02:24.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:24.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009378+0000", 2024-05-11T23:02:24.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.328e-06 2024-05-11T23:02:24.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:24.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009381+0000", 2024-05-11T23:02:24.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9519999999999999e-06 2024-05-11T23:02:24.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:24.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009385+0000", 2024-05-11T23:02:24.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5279999999999997e-06 2024-05-11T23:02:24.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:24.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009389+0000", 2024-05-11T23:02:24.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3139999999999999e-06 2024-05-11T23:02:24.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.012644+0000", 2024-05-11T23:02:24.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0032556439999999998 2024-05-11T23:02:24.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:24.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.012730+0000", 2024-05-11T23:02:24.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6063000000000006e-05 2024-05-11T23:02:24.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-05-11T23:02:24.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.012891+0000", 2024-05-11T23:02:24.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016067100000000001 2024-05-11T23:02:24.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:02:24.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.014464+0000", 2024-05-11T23:02:24.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015728949999999999 2024-05-11T23:02:24.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:24.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.015477+0000", 2024-05-11T23:02:24.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010127650000000001 2024-05-11T23:02:24.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:24.704 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.015507+0000", 2024-05-11T23:02:24.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0550999999999999e-05 2024-05-11T23:02:24.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:24.704 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.015540+0000", 2024-05-11T23:02:24.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3071999999999998e-05 2024-05-11T23:02:24.704 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.704 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.704 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.705 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:78892 3.5 e62/44 3:a1776d60:::rbd_data.111ea74f268d.0000000000000004:head v 62'156, mlcod=60'154)", 2024-05-11T23:02:24.705 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:16.012888+0000", 2024-05-11T23:02:24.705 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.604303625, 2024-05-11T23:02:24.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0057186540000000001, 2024-05-11T23:02:24.705 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:24.705 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:24.705 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:24.705 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:24.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:24.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.012888+0000", 2024-05-11T23:02:24.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:24.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.012888+0000", 2024-05-11T23:02:24.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:24.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:24.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.012888+0000", 2024-05-11T23:02:24.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6899999999999998e-07 2024-05-11T23:02:24.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.016107+0000", 2024-05-11T23:02:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0032186150000000002 2024-05-11T23:02:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.016117+0000", 2024-05-11T23:02:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0305e-05 2024-05-11T23:02:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.016124+0000", 2024-05-11T23:02:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7100000000000001e-06 2024-05-11T23:02:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:24.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.016184+0000", 2024-05-11T23:02:24.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0764000000000003e-05 2024-05-11T23:02:24.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:24.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.016210+0000", 2024-05-11T23:02:24.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5740000000000001e-05 2024-05-11T23:02:24.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:24.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.018572+0000", 2024-05-11T23:02:24.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0023623519999999999 2024-05-11T23:02:24.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:24.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:24.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:24.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.018606+0000", 2024-05-11T23:02:24.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3798999999999999e-05 2024-05-11T23:02:24.709 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.709 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.709 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.709 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:24.709 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:24.709 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:24.711 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:02:24.711 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:24.710+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:24.811 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:24.811+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:24.861 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:24.861 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:24.862 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:02:24.862 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:24.870 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:02:24.911 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:24.911+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:25.013 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:25.013+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:25.014 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone32 from 0xa3ee955 (overlap 0xa3ee955) to 0x9389ecc 2024-05-11T23:02:25.040 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:25.041 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:25.041 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:25.041 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:25.041 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:25.056 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:02:25.114 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:25.113+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:25.177 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:25.177 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:25.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:25.177 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:25.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.177 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36828 3.b 3:d385bc8e:::rbd_data.111e5e4d8275.0000000000000020:11 [read 512000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.178 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:25.178 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.243361419999999, 2024-05-11T23:02:25.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051481218000000002, 2024-05-11T23:02:25.178 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:25.178 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:25.178 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:25.178 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:25.178 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:25.178 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.178 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36828 2024-05-11T23:02:25.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.178 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:25.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:25.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:25.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:25.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:25.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:25.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934915+0000", 2024-05-11T23:02:25.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00181814 2024-05-11T23:02:25.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:25.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934919+0000", 2024-05-11T23:02:25.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6739999999999996e-06 2024-05-11T23:02:25.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:25.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934934+0000", 2024-05-11T23:02:25.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4415e-05 2024-05-11T23:02:25.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934942+0000", 2024-05-11T23:02:25.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8619999999999992e-06 2024-05-11T23:02:25.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:25.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984433+0000", 2024-05-11T23:02:25.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049490658999999999 2024-05-11T23:02:25.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:25.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984508+0000", 2024-05-11T23:02:25.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4394999999999995e-05 2024-05-11T23:02:25.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:25.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984578+0000", 2024-05-11T23:02:25.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0073000000000005e-05 2024-05-11T23:02:25.182 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.182 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.182 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.182 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36950 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 520192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.182 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:25.182 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.239055653999998, 2024-05-11T23:02:25.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051450856000000003, 2024-05-11T23:02:25.182 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:25.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:25.182 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:25.182 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:25.182 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:25.182 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.183 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36950 2024-05-11T23:02:25.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.183 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:25.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:25.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:25.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:25.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:25.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:25.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938914+0000", 2024-05-11T23:02:25.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015121480000000001 2024-05-11T23:02:25.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:25.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938918+0000", 2024-05-11T23:02:25.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4149999999999999e-06 2024-05-11T23:02:25.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:25.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938925+0000", 2024-05-11T23:02:25.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0779999999999998e-06 2024-05-11T23:02:25.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938929+0000", 2024-05-11T23:02:25.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2930000000000002e-06 2024-05-11T23:02:25.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:25.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988675+0000", 2024-05-11T23:02:25.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049746394999999999 2024-05-11T23:02:25.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:25.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988756+0000", 2024-05-11T23:02:25.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0499000000000003e-05 2024-05-11T23:02:25.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:25.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988853+0000", 2024-05-11T23:02:25.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7027999999999993e-05 2024-05-11T23:02:25.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.186 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.186 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36957 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 778240~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.186 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:25.186 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.237433865, 2024-05-11T23:02:25.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051328273000000001, 2024-05-11T23:02:25.186 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:25.186 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:25.186 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:25.186 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:25.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:25.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.187 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36957 2024-05-11T23:02:25.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.187 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:25.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:25.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:25.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:25.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:25.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:25.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:25.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2099999999999998e-07 2024-05-11T23:02:25.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:25.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939026+0000", 2024-05-11T23:02:25.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5069999999999999e-06 2024-05-11T23:02:25.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:25.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939030+0000", 2024-05-11T23:02:25.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7919999999999999e-06 2024-05-11T23:02:25.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939032+0000", 2024-05-11T23:02:25.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9129999999999999e-06 2024-05-11T23:02:25.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:25.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990178+0000", 2024-05-11T23:02:25.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051145373000000001 2024-05-11T23:02:25.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:25.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990255+0000", 2024-05-11T23:02:25.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7195999999999997e-05 2024-05-11T23:02:25.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:25.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990352+0000", 2024-05-11T23:02:25.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7170999999999995e-05 2024-05-11T23:02:25.190 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.190 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.190 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.190 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36958 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 815104~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.190 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:25.191 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.237421546, 2024-05-11T23:02:25.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051531625999999997, 2024-05-11T23:02:25.191 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:25.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:25.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:25.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:25.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:25.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36958 2024-05-11T23:02:25.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.191 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:25.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:25.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:25.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:25.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:25.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:25.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939037+0000", 2024-05-11T23:02:25.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0400000000000002e-07 2024-05-11T23:02:25.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:25.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939038+0000", 2024-05-11T23:02:25.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.164e-06 2024-05-11T23:02:25.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:25.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939041+0000", 2024-05-11T23:02:25.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1109999999999999e-06 2024-05-11T23:02:25.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939043+0000", 2024-05-11T23:02:25.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.554e-06 2024-05-11T23:02:25.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:25.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990382+0000", 2024-05-11T23:02:25.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051338953999999999 2024-05-11T23:02:25.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:25.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990465+0000", 2024-05-11T23:02:25.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2461999999999998e-05 2024-05-11T23:02:25.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:25.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990568+0000", 2024-05-11T23:02:25.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000103077 2024-05-11T23:02:25.195 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.195 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.195 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.195 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36959 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 851968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.195 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:25.195 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.237409649999996, 2024-05-11T23:02:25.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051758936999999998, 2024-05-11T23:02:25.195 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:25.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:25.195 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:25.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:25.196 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:25.196 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.196 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36959 2024-05-11T23:02:25.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.196 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:25.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:25.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:25.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:25.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:25.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:25.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:25.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.16e-07 2024-05-11T23:02:25.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:25.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939050+0000", 2024-05-11T23:02:25.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.176e-06 2024-05-11T23:02:25.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:25.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939053+0000", 2024-05-11T23:02:25.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1140000000000002e-06 2024-05-11T23:02:25.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939055+0000", 2024-05-11T23:02:25.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.621e-06 2024-05-11T23:02:25.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:25.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990610+0000", 2024-05-11T23:02:25.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051554244999999999 2024-05-11T23:02:25.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:25.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990700+0000", 2024-05-11T23:02:25.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0284999999999994e-05 2024-05-11T23:02:25.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:25.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990807+0000", 2024-05-11T23:02:25.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010728 2024-05-11T23:02:25.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.199 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.199 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36960 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 888832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.199 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:25.199 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.237399414999999, 2024-05-11T23:02:25.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051886831000000001, 2024-05-11T23:02:25.199 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:25.200 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:25.200 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:25.200 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:25.200 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:25.200 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.200 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36960 2024-05-11T23:02:25.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.200 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:25.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:25.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:25.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:25.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:25.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:25.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940913+0000", 2024-05-11T23:02:25.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001855104 2024-05-11T23:02:25.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:25.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940916+0000", 2024-05-11T23:02:25.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0000000000000001e-06 2024-05-11T23:02:25.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:25.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940924+0000", 2024-05-11T23:02:25.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0999999999999998e-06 2024-05-11T23:02:25.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940927+0000", 2024-05-11T23:02:25.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7160000000000001e-06 2024-05-11T23:02:25.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:25.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990826+0000", 2024-05-11T23:02:25.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049898832999999997 2024-05-11T23:02:25.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:25.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990872+0000", 2024-05-11T23:02:25.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5717000000000003e-05 2024-05-11T23:02:25.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:25.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990945+0000", 2024-05-11T23:02:25.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3361000000000005e-05 2024-05-11T23:02:25.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.204 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.204 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36986 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 528384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.204 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:25.204 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.235393197, 2024-05-11T23:02:25.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051341053999999997, 2024-05-11T23:02:25.204 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:25.204 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:25.204 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:25.204 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:25.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:25.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36986 2024-05-11T23:02:25.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.205 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:25.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:25.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:25.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:25.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:25.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:25.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:25.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:02:25.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:25.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941066+0000", 2024-05-11T23:02:25.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1629999999999999e-06 2024-05-11T23:02:25.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:25.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941069+0000", 2024-05-11T23:02:25.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.118e-06 2024-05-11T23:02:25.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941072+0000", 2024-05-11T23:02:25.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5720000000000001e-06 2024-05-11T23:02:25.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:25.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992292+0000", 2024-05-11T23:02:25.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051220620000000001 2024-05-11T23:02:25.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:25.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992344+0000", 2024-05-11T23:02:25.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1737000000000003e-05 2024-05-11T23:02:25.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:25.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992406+0000", 2024-05-11T23:02:25.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1623000000000003e-05 2024-05-11T23:02:25.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.208 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.208 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36987 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 565248~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.208 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:25.208 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.235383075999998, 2024-05-11T23:02:25.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051568725000000003, 2024-05-11T23:02:25.209 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:25.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:25.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:25.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:25.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:25.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36987 2024-05-11T23:02:25.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.209 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:25.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.91e-07 2024-05-11T23:02:25.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941076+0000", 2024-05-11T23:02:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1379999999999999e-06 2024-05-11T23:02:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941079+0000", 2024-05-11T23:02:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9739999999999998e-06 2024-05-11T23:02:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941082+0000", 2024-05-11T23:02:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-05-11T23:02:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992446+0000", 2024-05-11T23:02:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051364058999999997 2024-05-11T23:02:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992536+0000", 2024-05-11T23:02:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0085000000000002e-05 2024-05-11T23:02:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992643+0000", 2024-05-11T23:02:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107377 2024-05-11T23:02:25.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36988 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 602112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.235372707000003, 2024-05-11T23:02:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051700007999999999, 2024-05-11T23:02:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.213 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36988 2024-05-11T23:02:25.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.214 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:25.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:25.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:25.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:25.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:25.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:25.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:25.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.05e-07 2024-05-11T23:02:25.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:25.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941086+0000", 2024-05-11T23:02:25.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.125e-06 2024-05-11T23:02:25.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:25.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941089+0000", 2024-05-11T23:02:25.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0599999999999999e-06 2024-05-11T23:02:25.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941092+0000", 2024-05-11T23:02:25.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6529999999999998e-06 2024-05-11T23:02:25.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:25.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992667+0000", 2024-05-11T23:02:25.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051574587999999998 2024-05-11T23:02:25.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:25.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992723+0000", 2024-05-11T23:02:25.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6682999999999999e-05 2024-05-11T23:02:25.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:25.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992785+0000", 2024-05-11T23:02:25.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1693999999999996e-05 2024-05-11T23:02:25.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.217 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.217 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36989 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 638976~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.217 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:25.217 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.235362518000002, 2024-05-11T23:02:25.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05192811, 2024-05-11T23:02:25.217 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:25.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:25.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:25.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:25.218 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:25.218 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.218 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36989 2024-05-11T23:02:25.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.218 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:25.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:25.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:25.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:25.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:25.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:25.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942916+0000", 2024-05-11T23:02:25.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001820835 2024-05-11T23:02:25.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:25.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942920+0000", 2024-05-11T23:02:25.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5310000000000002e-06 2024-05-11T23:02:25.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:25.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942931+0000", 2024-05-11T23:02:25.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1839e-05 2024-05-11T23:02:25.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942940+0000", 2024-05-11T23:02:25.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2309999999999996e-06 2024-05-11T23:02:25.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:25.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992820+0000", 2024-05-11T23:02:25.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049880751000000001 2024-05-11T23:02:25.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:25.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992918+0000", 2024-05-11T23:02:25.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7051000000000003e-05 2024-05-11T23:02:25.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:25.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.993023+0000", 2024-05-11T23:02:25.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010587200000000001 2024-05-11T23:02:25.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.221 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.221 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36997 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 933888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.221 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:25.221 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.233328683000003, 2024-05-11T23:02:25.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051390602000000001, 2024-05-11T23:02:25.221 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:25.221 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:25.222 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:25.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:25.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:25.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36997 2024-05-11T23:02:25.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.222 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:25.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:25.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:25.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:25.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:25.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:25.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943130+0000", 2024-05-11T23:02:25.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0200000000000003e-07 2024-05-11T23:02:25.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:25.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943132+0000", 2024-05-11T23:02:25.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.305e-06 2024-05-11T23:02:25.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:25.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943138+0000", 2024-05-11T23:02:25.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4219999999999997e-06 2024-05-11T23:02:25.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943143+0000", 2024-05-11T23:02:25.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0610000000000003e-06 2024-05-11T23:02:25.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:25.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994323+0000", 2024-05-11T23:02:25.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051179592000000003 2024-05-11T23:02:25.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:25.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994412+0000", 2024-05-11T23:02:25.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8709999999999996e-05 2024-05-11T23:02:25.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:25.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994520+0000", 2024-05-11T23:02:25.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010811000000000001 2024-05-11T23:02:25.225 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.225 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.225 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.225 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36998 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 970752~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.226 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:25.226 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.233307803999999, 2024-05-11T23:02:25.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051517669000000002, 2024-05-11T23:02:25.226 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:25.226 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:25.226 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:25.226 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:25.226 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:25.226 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.226 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36998 2024-05-11T23:02:25.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.226 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:25.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:25.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:25.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:25.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:25.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:25.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:25.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:02:25.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:25.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943153+0000", 2024-05-11T23:02:25.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.306e-06 2024-05-11T23:02:25.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:25.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943160+0000", 2024-05-11T23:02:25.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.452e-06 2024-05-11T23:02:25.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943165+0000", 2024-05-11T23:02:25.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.152e-06 2024-05-11T23:02:25.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:25.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994561+0000", 2024-05-11T23:02:25.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051395765000000003 2024-05-11T23:02:25.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:25.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994607+0000", 2024-05-11T23:02:25.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6415000000000002e-05 2024-05-11T23:02:25.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:25.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994668+0000", 2024-05-11T23:02:25.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0179000000000001e-05 2024-05-11T23:02:25.229 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.229 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.230 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.230 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36999 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 1007616~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.230 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:25.230 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.233262922999998, 2024-05-11T23:02:25.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051696919000000001, 2024-05-11T23:02:25.230 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:25.230 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:25.230 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:25.230 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:25.230 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:25.230 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.231 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36999 2024-05-11T23:02:25.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.231 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:25.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:25.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:25.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:25.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:25.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:25.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:25.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:02:25.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:25.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943197+0000", 2024-05-11T23:02:25.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1280000000000002e-06 2024-05-11T23:02:25.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:25.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943202+0000", 2024-05-11T23:02:25.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8910000000000003e-06 2024-05-11T23:02:25.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943206+0000", 2024-05-11T23:02:25.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3550000000000001e-06 2024-05-11T23:02:25.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:25.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994703+0000", 2024-05-11T23:02:25.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051497045999999998 2024-05-11T23:02:25.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:25.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994789+0000", 2024-05-11T23:02:25.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6458000000000005e-05 2024-05-11T23:02:25.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:25.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994892+0000", 2024-05-11T23:02:25.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102705 2024-05-11T23:02:25.234 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.234 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.234 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.234 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37000 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 1044480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.234 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:25.234 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.233246977999997, 2024-05-11T23:02:25.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051842371999999998, 2024-05-11T23:02:25.234 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:25.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:25.235 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:25.235 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:25.235 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:25.235 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.235 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37000 2024-05-11T23:02:25.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.235 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:25.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:25.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:25.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:25.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:25.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:25.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944916+0000", 2024-05-11T23:02:25.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001705097 2024-05-11T23:02:25.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:25.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944921+0000", 2024-05-11T23:02:25.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5809999999999998e-06 2024-05-11T23:02:25.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:25.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944930+0000", 2024-05-11T23:02:25.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5529999999999998e-06 2024-05-11T23:02:25.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944938+0000", 2024-05-11T23:02:25.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2519999999999999e-06 2024-05-11T23:02:25.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:25.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994911+0000", 2024-05-11T23:02:25.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049973113999999999 2024-05-11T23:02:25.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:25.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994982+0000", 2024-05-11T23:02:25.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0544999999999996e-05 2024-05-11T23:02:25.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:25.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.995053+0000", 2024-05-11T23:02:25.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1229999999999994e-05 2024-05-11T23:02:25.239 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.239 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.239 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.239 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37041 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 618496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.239 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:25.239 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.231402998999997, 2024-05-11T23:02:25.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051352768999999999, 2024-05-11T23:02:25.239 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:25.239 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:25.239 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:25.239 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:25.240 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:25.240 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.240 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37041 2024-05-11T23:02:25.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.240 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:25.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:25.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:25.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:25.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:25.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:25.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:25.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:02:25.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:25.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945056+0000", 2024-05-11T23:02:25.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:02:25.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:25.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945059+0000", 2024-05-11T23:02:25.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.965e-06 2024-05-11T23:02:25.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945062+0000", 2024-05-11T23:02:25.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:02:25.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:25.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996213+0000", 2024-05-11T23:02:25.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051151532999999999 2024-05-11T23:02:25.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:25.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996301+0000", 2024-05-11T23:02:25.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8143e-05 2024-05-11T23:02:25.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:25.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996408+0000", 2024-05-11T23:02:25.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010623700000000001 2024-05-11T23:02:25.243 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.243 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.243 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.243 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37042 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 655360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.243 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:25.243 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.231393013999998, 2024-05-11T23:02:25.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051462075000000003, 2024-05-11T23:02:25.243 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:25.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:25.244 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:25.244 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:25.244 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:25.244 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.244 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37042 2024-05-11T23:02:25.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.244 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:25.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:25.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:25.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:25.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:25.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:25.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:25.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0599999999999999e-07 2024-05-11T23:02:25.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:25.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945066+0000", 2024-05-11T23:02:25.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-05-11T23:02:25.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:25.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945069+0000", 2024-05-11T23:02:25.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0079999999999998e-06 2024-05-11T23:02:25.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945072+0000", 2024-05-11T23:02:25.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6740000000000001e-06 2024-05-11T23:02:25.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:25.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996422+0000", 2024-05-11T23:02:25.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051349999 2024-05-11T23:02:25.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:25.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996467+0000", 2024-05-11T23:02:25.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.528e-05 2024-05-11T23:02:25.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:25.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996527+0000", 2024-05-11T23:02:25.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9759999999999997e-05 2024-05-11T23:02:25.247 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.247 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.247 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.247 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37043 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 692224~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.247 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:25.247 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.231380940999998, 2024-05-11T23:02:25.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051679535999999998, 2024-05-11T23:02:25.248 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:25.248 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:25.248 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:25.248 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:25.248 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:25.248 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.248 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37043 2024-05-11T23:02:25.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.248 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:25.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:25.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:25.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:25.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:25.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:25.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:25.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8999999999999997e-07 2024-05-11T23:02:25.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:25.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945079+0000", 2024-05-11T23:02:25.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.001e-06 2024-05-11T23:02:25.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:25.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945087+0000", 2024-05-11T23:02:25.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1749999999999999e-06 2024-05-11T23:02:25.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945093+0000", 2024-05-11T23:02:25.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9639999999999997e-06 2024-05-11T23:02:25.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:25.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996563+0000", 2024-05-11T23:02:25.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051469050000000002 2024-05-11T23:02:25.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:25.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996651+0000", 2024-05-11T23:02:25.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8356000000000006e-05 2024-05-11T23:02:25.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:25.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996756+0000", 2024-05-11T23:02:25.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0001055 2024-05-11T23:02:25.251 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.251 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.251 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.251 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37044 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 729088~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.251 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:25.252 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.231347028999998, 2024-05-11T23:02:25.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051763445999999998, 2024-05-11T23:02:25.252 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:25.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:25.252 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:25.252 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:25.252 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:25.252 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.252 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37044 2024-05-11T23:02:25.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.252 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:25.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:25.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:25.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:25.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:25.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:25.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946915+0000", 2024-05-11T23:02:25.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018042760000000001 2024-05-11T23:02:25.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:25.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946918+0000", 2024-05-11T23:02:25.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.36e-06 2024-05-11T23:02:25.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:25.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946926+0000", 2024-05-11T23:02:25.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3130000000000001e-06 2024-05-11T23:02:25.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946930+0000", 2024-05-11T23:02:25.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5249999999999999e-06 2024-05-11T23:02:25.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:25.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996772+0000", 2024-05-11T23:02:25.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049841868999999997 2024-05-11T23:02:25.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:25.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996817+0000", 2024-05-11T23:02:25.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5351e-05 2024-05-11T23:02:25.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:25.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996874+0000", 2024-05-11T23:02:25.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6752000000000001e-05 2024-05-11T23:02:25.255 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.255 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.255 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.255 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37052 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 1024000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.255 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:25.256 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.229440177999997, 2024-05-11T23:02:25.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051326140999999999, 2024-05-11T23:02:25.256 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:25.256 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:25.256 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:25.256 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:25.256 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:25.256 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.256 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37052 2024-05-11T23:02:25.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.256 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:25.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:25.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:25.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:25.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:25.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:25.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:25.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:25.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:25.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947019+0000", 2024-05-11T23:02:25.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.218e-06 2024-05-11T23:02:25.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:25.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947022+0000", 2024-05-11T23:02:25.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1889999999999999e-06 2024-05-11T23:02:25.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947025+0000", 2024-05-11T23:02:25.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.464e-06 2024-05-11T23:02:25.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:25.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998237+0000", 2024-05-11T23:02:25.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051212494999999997 2024-05-11T23:02:25.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:25.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998284+0000", 2024-05-11T23:02:25.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7246000000000001e-05 2024-05-11T23:02:25.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:25.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998344+0000", 2024-05-11T23:02:25.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9320000000000001e-05 2024-05-11T23:02:25.259 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.259 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.259 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.260 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37071 3.b 3:d385bc8e:::rbd_data.111e5e4d8275.0000000000000020:11 [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.260 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:25.260 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.229429969000002, 2024-05-11T23:02:25.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051528796000000002, 2024-05-11T23:02:25.260 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:25.260 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:25.260 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:25.260 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:25.260 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:25.260 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.260 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37071 2024-05-11T23:02:25.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.260 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:25.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:25.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:25.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:25.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:25.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:25.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:25.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948913+0000", 2024-05-11T23:02:25.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018854970000000001 2024-05-11T23:02:25.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:25.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948917+0000", 2024-05-11T23:02:25.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8089999999999999e-06 2024-05-11T23:02:25.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:25.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948926+0000", 2024-05-11T23:02:25.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8389999999999996e-06 2024-05-11T23:02:25.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948930+0000", 2024-05-11T23:02:25.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4519999999999999e-06 2024-05-11T23:02:25.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998379+0000", 2024-05-11T23:02:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049449011000000001 2024-05-11T23:02:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998462+0000", 2024-05-11T23:02:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2739999999999997e-05 2024-05-11T23:02:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998557+0000", 2024-05-11T23:02:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4448000000000001e-05 2024-05-11T23:02:25.263 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.264 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.264 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.264 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:25.264 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:25.264 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:25.266 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:02:25.266 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-05-11T23:02:25.267 INFO:teuthology.orchestra.run.smithi005.stdout:3654 trunc from 0xa3ee955 to 0x24d4d9d 2024-05-11T23:02:25.267 INFO:teuthology.orchestra.run.smithi005.stdout:3655 read 0x9d6da4 thru 0x9dbe1e (0x507b bytes) 2024-05-11T23:02:25.267 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:25.267 INFO:teuthology.orchestra.run.smithi005.stdout:3657 clone 34 order 24 su 8388608 sc 12 2024-05-11T23:02:25.267 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:25.265+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:25.366 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:25.366+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:25.371 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:25.371 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:25.371 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:25.371 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:25.380 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:02:25.467 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:25.467+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:25.525 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:25.525 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:25.525 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:25.525 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:25.525 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:25.534 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:02:25.578 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:25.578+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:25.640 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:25.640 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:25.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:25.640 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:25.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.640 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24548 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.640 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.927949+0000", 2024-05-11T23:02:25.641 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.711165076, 2024-05-11T23:02:25.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037300396999999999, 2024-05-11T23:02:25.641 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:25.641 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:25.641 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:25.641 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:25.641 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:25.641 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.641 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24548 2024-05-11T23:02:25.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.641 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:25.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:25.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927949+0000", 2024-05-11T23:02:25.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:25.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927949+0000", 2024-05-11T23:02:25.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:25.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927950+0000", 2024-05-11T23:02:25.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9400000000000005e-07 2024-05-11T23:02:25.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:25.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927952+0000", 2024-05-11T23:02:25.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2330000000000001e-06 2024-05-11T23:02:25.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:25.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927959+0000", 2024-05-11T23:02:25.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8820000000000003e-06 2024-05-11T23:02:25.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927964+0000", 2024-05-11T23:02:25.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.899e-06 2024-05-11T23:02:25.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:25.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.959280+0000", 2024-05-11T23:02:25.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031315904999999998 2024-05-11T23:02:25.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:25.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.959348+0000", 2024-05-11T23:02:25.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8214999999999997e-05 2024-05-11T23:02:25.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:25.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965250+0000", 2024-05-11T23:02:25.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005901669 2024-05-11T23:02:25.645 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.645 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.645 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.645 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24555 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.645 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.927974+0000", 2024-05-11T23:02:25.646 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.711139809000002, 2024-05-11T23:02:25.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.03739746, 2024-05-11T23:02:25.646 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:25.646 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:25.646 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:25.646 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:25.646 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:25.646 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.646 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24555 2024-05-11T23:02:25.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.647 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:25.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:25.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927974+0000", 2024-05-11T23:02:25.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:25.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927974+0000", 2024-05-11T23:02:25.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:25.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927975+0000", 2024-05-11T23:02:25.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6899999999999998e-07 2024-05-11T23:02:25.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:25.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927977+0000", 2024-05-11T23:02:25.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.074e-06 2024-05-11T23:02:25.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:25.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927984+0000", 2024-05-11T23:02:25.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7309999999999996e-06 2024-05-11T23:02:25.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927989+0000", 2024-05-11T23:02:25.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.079e-06 2024-05-11T23:02:25.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:25.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965279+0000", 2024-05-11T23:02:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037290271999999999 2024-05-11T23:02:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965324+0000", 2024-05-11T23:02:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4638999999999997e-05 2024-05-11T23:02:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965372+0000", 2024-05-11T23:02:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8195999999999997e-05 2024-05-11T23:02:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24562 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.927995+0000", 2024-05-11T23:02:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.711118874, 2024-05-11T23:02:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037703408000000001, 2024-05-11T23:02:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24562 2024-05-11T23:02:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927995+0000", 2024-05-11T23:02:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927995+0000", 2024-05-11T23:02:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:25.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927996+0000", 2024-05-11T23:02:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9499999999999998e-07 2024-05-11T23:02:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927998+0000", 2024-05-11T23:02:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0889999999999998e-06 2024-05-11T23:02:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:25.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928004+0000", 2024-05-11T23:02:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4740000000000002e-06 2024-05-11T23:02:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928009+0000", 2024-05-11T23:02:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:02:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965549+0000", 2024-05-11T23:02:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037539826999999998 2024-05-11T23:02:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:25.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965616+0000", 2024-05-11T23:02:25.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7387999999999999e-05 2024-05-11T23:02:25.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:25.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965699+0000", 2024-05-11T23:02:25.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2347000000000003e-05 2024-05-11T23:02:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24568 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928016+0000", 2024-05-11T23:02:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.711098688, 2024-05-11T23:02:25.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037850338999999997, 2024-05-11T23:02:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24568 2024-05-11T23:02:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928016+0000", 2024-05-11T23:02:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928016+0000", 2024-05-11T23:02:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928016+0000", 2024-05-11T23:02:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-05-11T23:02:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928025+0000", 2024-05-11T23:02:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.363e-06 2024-05-11T23:02:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928032+0000", 2024-05-11T23:02:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.114e-06 2024-05-11T23:02:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928038+0000", 2024-05-11T23:02:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1250000000000001e-06 2024-05-11T23:02:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965720+0000", 2024-05-11T23:02:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037682488 2024-05-11T23:02:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965785+0000", 2024-05-11T23:02:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5180999999999998e-05 2024-05-11T23:02:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965866+0000", 2024-05-11T23:02:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0687999999999997e-05 2024-05-11T23:02:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24574 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 266240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928045+0000", 2024-05-11T23:02:25.660 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.711069711, 2024-05-11T23:02:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037990532, 2024-05-11T23:02:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24574 2024-05-11T23:02:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928045+0000", 2024-05-11T23:02:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928045+0000", 2024-05-11T23:02:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928045+0000", 2024-05-11T23:02:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:02:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928047+0000", 2024-05-11T23:02:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9649999999999998e-06 2024-05-11T23:02:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928053+0000", 2024-05-11T23:02:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1550000000000004e-06 2024-05-11T23:02:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928058+0000", 2024-05-11T23:02:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9060000000000001e-06 2024-05-11T23:02:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:25.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965890+0000", 2024-05-11T23:02:25.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037831754000000002 2024-05-11T23:02:25.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:25.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965957+0000", 2024-05-11T23:02:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6798999999999995e-05 2024-05-11T23:02:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966035+0000", 2024-05-11T23:02:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8532e-05 2024-05-11T23:02:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24581 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928064+0000", 2024-05-11T23:02:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.711049895000002, 2024-05-11T23:02:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038134135, 2024-05-11T23:02:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24581 2024-05-11T23:02:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:25.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928064+0000", 2024-05-11T23:02:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928064+0000", 2024-05-11T23:02:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928065+0000", 2024-05-11T23:02:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3099999999999998e-07 2024-05-11T23:02:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928067+0000", 2024-05-11T23:02:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9549999999999999e-06 2024-05-11T23:02:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928073+0000", 2024-05-11T23:02:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1929999999999998e-06 2024-05-11T23:02:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928078+0000", 2024-05-11T23:02:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9289999999999997e-06 2024-05-11T23:02:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966058+0000", 2024-05-11T23:02:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037980452999999997 2024-05-11T23:02:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966123+0000", 2024-05-11T23:02:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4832000000000005e-05 2024-05-11T23:02:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966198+0000", 2024-05-11T23:02:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5341999999999998e-05 2024-05-11T23:02:25.669 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24585 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928095+0000", 2024-05-11T23:02:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.711019374999999, 2024-05-11T23:02:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038282688000000002, 2024-05-11T23:02:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.670 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24585 2024-05-11T23:02:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928095+0000", 2024-05-11T23:02:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928095+0000", 2024-05-11T23:02:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928095+0000", 2024-05-11T23:02:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1900000000000003e-07 2024-05-11T23:02:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928098+0000", 2024-05-11T23:02:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1880000000000001e-06 2024-05-11T23:02:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928105+0000", 2024-05-11T23:02:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3690000000000001e-06 2024-05-11T23:02:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928120+0000", 2024-05-11T23:02:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4762e-05 2024-05-11T23:02:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966233+0000", 2024-05-11T23:02:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038113526000000002 2024-05-11T23:02:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966300+0000", 2024-05-11T23:02:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6637999999999995e-05 2024-05-11T23:02:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966378+0000", 2024-05-11T23:02:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7686000000000001e-05 2024-05-11T23:02:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24592 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 315392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928128+0000", 2024-05-11T23:02:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.710986069999997, 2024-05-11T23:02:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038417382, 2024-05-11T23:02:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:25.674 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24592 2024-05-11T23:02:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928128+0000", 2024-05-11T23:02:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928128+0000", 2024-05-11T23:02:25.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928129+0000", 2024-05-11T23:02:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.44e-07 2024-05-11T23:02:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928131+0000", 2024-05-11T23:02:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2910000000000002e-06 2024-05-11T23:02:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928138+0000", 2024-05-11T23:02:25.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9430000000000003e-06 2024-05-11T23:02:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928143+0000", 2024-05-11T23:02:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4739999999999996e-06 2024-05-11T23:02:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966400+0000", 2024-05-11T23:02:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038257001999999998 2024-05-11T23:02:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:25.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966465+0000", 2024-05-11T23:02:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4468999999999995e-05 2024-05-11T23:02:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966546+0000", 2024-05-11T23:02:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0659000000000001e-05 2024-05-11T23:02:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24598 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 331776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928150+0000", 2024-05-11T23:02:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.710963761000002, 2024-05-11T23:02:25.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038544469999999997, 2024-05-11T23:02:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24598 2024-05-11T23:02:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928150+0000", 2024-05-11T23:02:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928150+0000", 2024-05-11T23:02:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928151+0000", 2024-05-11T23:02:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999999e-07 2024-05-11T23:02:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928153+0000", 2024-05-11T23:02:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.21e-06 2024-05-11T23:02:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928160+0000", 2024-05-11T23:02:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5200000000000003e-06 2024-05-11T23:02:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928165+0000", 2024-05-11T23:02:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9269999999999996e-06 2024-05-11T23:02:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966569+0000", 2024-05-11T23:02:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038404595999999999 2024-05-11T23:02:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966634+0000", 2024-05-11T23:02:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5221000000000005e-05 2024-05-11T23:02:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966695+0000", 2024-05-11T23:02:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0547999999999997e-05 2024-05-11T23:02:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24604 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928171+0000", 2024-05-11T23:02:25.682 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.710943399000001, 2024-05-11T23:02:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038676227000000001, 2024-05-11T23:02:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24604 2024-05-11T23:02:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928171+0000", 2024-05-11T23:02:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928171+0000", 2024-05-11T23:02:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928171+0000", 2024-05-11T23:02:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-05-11T23:02:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928173+0000", 2024-05-11T23:02:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0619999999999999e-06 2024-05-11T23:02:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928180+0000", 2024-05-11T23:02:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-05-11T23:02:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928185+0000", 2024-05-11T23:02:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.04e-06 2024-05-11T23:02:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966718+0000", 2024-05-11T23:02:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038532759 2024-05-11T23:02:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966784+0000", 2024-05-11T23:02:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6865000000000004e-05 2024-05-11T23:02:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966847+0000", 2024-05-11T23:02:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2694000000000007e-05 2024-05-11T23:02:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24609 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 364544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.686 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928191+0000", 2024-05-11T23:02:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.710923331000004, 2024-05-11T23:02:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038806908000000001, 2024-05-11T23:02:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24609 2024-05-11T23:02:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928191+0000", 2024-05-11T23:02:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:25.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928191+0000", 2024-05-11T23:02:25.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:25.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928191+0000", 2024-05-11T23:02:25.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-05-11T23:02:25.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928198+0000", 2024-05-11T23:02:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2180000000000002e-06 2024-05-11T23:02:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928205+0000", 2024-05-11T23:02:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5899999999999996e-06 2024-05-11T23:02:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928210+0000", 2024-05-11T23:02:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.118e-06 2024-05-11T23:02:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966869+0000", 2024-05-11T23:02:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038659106999999998 2024-05-11T23:02:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966933+0000", 2024-05-11T23:02:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4158999999999993e-05 2024-05-11T23:02:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966998+0000", 2024-05-11T23:02:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4352999999999999e-05 2024-05-11T23:02:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.692 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24616 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 380928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928217+0000", 2024-05-11T23:02:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.710897187999997, 2024-05-11T23:02:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038928536999999999, 2024-05-11T23:02:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24616 2024-05-11T23:02:25.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928217+0000", 2024-05-11T23:02:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928217+0000", 2024-05-11T23:02:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928217+0000", 2024-05-11T23:02:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-05-11T23:02:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928219+0000", 2024-05-11T23:02:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0609999999999998e-06 2024-05-11T23:02:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928233+0000", 2024-05-11T23:02:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3715e-05 2024-05-11T23:02:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928238+0000", 2024-05-11T23:02:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1889999999999998e-06 2024-05-11T23:02:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967021+0000", 2024-05-11T23:02:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038782284 2024-05-11T23:02:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967085+0000", 2024-05-11T23:02:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4319000000000005e-05 2024-05-11T23:02:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:25.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967146+0000", 2024-05-11T23:02:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0579999999999999e-05 2024-05-11T23:02:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24622 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 397312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928262+0000", 2024-05-11T23:02:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.710851882999997, 2024-05-11T23:02:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039038392999999998, 2024-05-11T23:02:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:25.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24622 2024-05-11T23:02:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928262+0000", 2024-05-11T23:02:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928262+0000", 2024-05-11T23:02:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928263+0000", 2024-05-11T23:02:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0299999999999999e-07 2024-05-11T23:02:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928266+0000", 2024-05-11T23:02:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6690000000000002e-06 2024-05-11T23:02:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928275+0000", 2024-05-11T23:02:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1630000000000002e-06 2024-05-11T23:02:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928280+0000", 2024-05-11T23:02:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6579999999999999e-06 2024-05-11T23:02:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967169+0000", 2024-05-11T23:02:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038888569999999997 2024-05-11T23:02:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967240+0000", 2024-05-11T23:02:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0947999999999999e-05 2024-05-11T23:02:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967301+0000", 2024-05-11T23:02:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0782000000000002e-05 2024-05-11T23:02:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24628 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 413696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928288+0000", 2024-05-11T23:02:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.710825835000001, 2024-05-11T23:02:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039162792000000002, 2024-05-11T23:02:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:25.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:25.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.702 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24628 2024-05-11T23:02:25.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.702 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:25.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:25.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928288+0000", 2024-05-11T23:02:25.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:25.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928288+0000", 2024-05-11T23:02:25.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:25.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928289+0000", 2024-05-11T23:02:25.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7399999999999998e-07 2024-05-11T23:02:25.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:25.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928291+0000", 2024-05-11T23:02:25.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.221e-06 2024-05-11T23:02:25.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:25.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928298+0000", 2024-05-11T23:02:25.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0520000000000004e-06 2024-05-11T23:02:25.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928304+0000", 2024-05-11T23:02:25.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4149999999999998e-06 2024-05-11T23:02:25.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967324+0000", 2024-05-11T23:02:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039020436999999998 2024-05-11T23:02:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967390+0000", 2024-05-11T23:02:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5821000000000006e-05 2024-05-11T23:02:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967451+0000", 2024-05-11T23:02:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1371999999999994e-05 2024-05-11T23:02:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.705 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24636 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 430080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928310+0000", 2024-05-11T23:02:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.710803970000001, 2024-05-11T23:02:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039286303000000002, 2024-05-11T23:02:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24636 2024-05-11T23:02:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.706 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928310+0000", 2024-05-11T23:02:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928310+0000", 2024-05-11T23:02:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928311+0000", 2024-05-11T23:02:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:02:25.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928313+0000", 2024-05-11T23:02:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0499999999999999e-06 2024-05-11T23:02:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928319+0000", 2024-05-11T23:02:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4049999999999997e-06 2024-05-11T23:02:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928324+0000", 2024-05-11T23:02:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:02:25.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967473+0000", 2024-05-11T23:02:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039149138999999999 2024-05-11T23:02:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967537+0000", 2024-05-11T23:02:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4109000000000006e-05 2024-05-11T23:02:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967597+0000", 2024-05-11T23:02:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9407000000000002e-05 2024-05-11T23:02:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.709 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25144 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 28672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936035+0000", 2024-05-11T23:02:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.703079064999997, 2024-05-11T23:02:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036140510000000001, 2024-05-11T23:02:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25144 2024-05-11T23:02:25.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936035+0000", 2024-05-11T23:02:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936035+0000", 2024-05-11T23:02:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936036+0000", 2024-05-11T23:02:25.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-05-11T23:02:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936038+0000", 2024-05-11T23:02:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6910000000000002e-06 2024-05-11T23:02:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936045+0000", 2024-05-11T23:02:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1670000000000004e-06 2024-05-11T23:02:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936049+0000", 2024-05-11T23:02:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9609999999999996e-06 2024-05-11T23:02:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972024+0000", 2024-05-11T23:02:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.035974221000000001 2024-05-11T23:02:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972076+0000", 2024-05-11T23:02:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1918e-05 2024-05-11T23:02:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972176+0000", 2024-05-11T23:02:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010006200000000001 2024-05-11T23:02:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25150 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 45056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936063+0000", 2024-05-11T23:02:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.703051532000003, 2024-05-11T23:02:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036298174000000002, 2024-05-11T23:02:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:25.714 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25150 2024-05-11T23:02:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936063+0000", 2024-05-11T23:02:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936063+0000", 2024-05-11T23:02:25.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936063+0000", 2024-05-11T23:02:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.39e-07 2024-05-11T23:02:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936069+0000", 2024-05-11T23:02:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2770000000000002e-06 2024-05-11T23:02:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936088+0000", 2024-05-11T23:02:25.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8841000000000001e-05 2024-05-11T23:02:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936093+0000", 2024-05-11T23:02:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2190000000000003e-06 2024-05-11T23:02:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972213+0000", 2024-05-11T23:02:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036119039999999998 2024-05-11T23:02:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972293+0000", 2024-05-11T23:02:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0437000000000003e-05 2024-05-11T23:02:25.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972361+0000", 2024-05-11T23:02:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7921000000000002e-05 2024-05-11T23:02:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25156 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936107+0000", 2024-05-11T23:02:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.703007008, 2024-05-11T23:02:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036383785000000002, 2024-05-11T23:02:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:25.718 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25156 2024-05-11T23:02:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936107+0000", 2024-05-11T23:02:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936107+0000", 2024-05-11T23:02:25.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936108+0000", 2024-05-11T23:02:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3900000000000005e-07 2024-05-11T23:02:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936110+0000", 2024-05-11T23:02:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.435e-06 2024-05-11T23:02:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936118+0000", 2024-05-11T23:02:25.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2889999999999998e-06 2024-05-11T23:02:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936124+0000", 2024-05-11T23:02:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2850000000000002e-06 2024-05-11T23:02:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972378+0000", 2024-05-11T23:02:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036254020999999997 2024-05-11T23:02:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972420+0000", 2024-05-11T23:02:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2577999999999998e-05 2024-05-11T23:02:25.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972491+0000", 2024-05-11T23:02:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0637999999999997e-05 2024-05-11T23:02:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25164 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936132+0000", 2024-05-11T23:02:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.702981764999997, 2024-05-11T23:02:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036532028000000001, 2024-05-11T23:02:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:25.722 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25164 2024-05-11T23:02:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936132+0000", 2024-05-11T23:02:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:25.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936132+0000", 2024-05-11T23:02:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936133+0000", 2024-05-11T23:02:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-05-11T23:02:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936135+0000", 2024-05-11T23:02:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0660000000000002e-06 2024-05-11T23:02:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936146+0000", 2024-05-11T23:02:25.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.098e-05 2024-05-11T23:02:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936153+0000", 2024-05-11T23:02:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8619999999999997e-06 2024-05-11T23:02:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972532+0000", 2024-05-11T23:02:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036379702999999999 2024-05-11T23:02:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972577+0000", 2024-05-11T23:02:25.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4697999999999998e-05 2024-05-11T23:02:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972664+0000", 2024-05-11T23:02:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7338999999999998e-05 2024-05-11T23:02:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25168 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 94208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936160+0000", 2024-05-11T23:02:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.702954487, 2024-05-11T23:02:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036655488999999999, 2024-05-11T23:02:25.726 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25168 2024-05-11T23:02:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936160+0000", 2024-05-11T23:02:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:25.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936160+0000", 2024-05-11T23:02:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936160+0000", 2024-05-11T23:02:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3000000000000001e-07 2024-05-11T23:02:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936162+0000", 2024-05-11T23:02:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.108e-06 2024-05-11T23:02:25.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.937848+0000", 2024-05-11T23:02:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001685284 2024-05-11T23:02:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.937858+0000", 2024-05-11T23:02:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0222e-05 2024-05-11T23:02:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972704+0000", 2024-05-11T23:02:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034845736000000002 2024-05-11T23:02:25.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972746+0000", 2024-05-11T23:02:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2077999999999999e-05 2024-05-11T23:02:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972815+0000", 2024-05-11T23:02:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9630999999999998e-05 2024-05-11T23:02:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:25.730 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:25.731 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:25.732 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:02:25.732 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:25.732+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:25.832 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:25.832+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:25.879 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:25.880 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:25.880 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:25.880 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:25.888 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:02:25.933 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:25.933+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:25.970 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7] out_osds: [3, 6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:02:25.971 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:02:25.971 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-05-11T23:02:25.971 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-05-11T23:02:25.983 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-11T23:02:26.040 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:26.040 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:26.040 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:26.040 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:26.040 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:26.049 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:02:26.060 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:26.060+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:26.064 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone33 from 0x24d4d9d (overlap 0x24d4d9d) to 0x126a6ce 2024-05-11T23:02:26.158 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:26.135+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:26.202 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:26.202 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:26.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:26.202 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:26.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67817 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.581575275999999, 2024-05-11T23:02:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044148845999999999, 2024-05-11T23:02:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67817 2024-05-11T23:02:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620227+0000", 2024-05-11T23:02:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00046792099999999999 2024-05-11T23:02:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620232+0000", 2024-05-11T23:02:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3050000000000003e-06 2024-05-11T23:02:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620243+0000", 2024-05-11T23:02:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1289e-05 2024-05-11T23:02:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620250+0000", 2024-05-11T23:02:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2130000000000002e-06 2024-05-11T23:02:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:26.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663745+0000", 2024-05-11T23:02:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043494865000000001 2024-05-11T23:02:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663815+0000", 2024-05-11T23:02:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0284999999999995e-05 2024-05-11T23:02:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663908+0000", 2024-05-11T23:02:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2967999999999995e-05 2024-05-11T23:02:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67820 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4902912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.581025769, 2024-05-11T23:02:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044096781000000002, 2024-05-11T23:02:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67820 2024-05-11T23:02:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.207 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620310+0000", 2024-05-11T23:02:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9800000000000004e-07 2024-05-11T23:02:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620312+0000", 2024-05-11T23:02:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.272e-06 2024-05-11T23:02:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620319+0000", 2024-05-11T23:02:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9310000000000002e-06 2024-05-11T23:02:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620323+0000", 2024-05-11T23:02:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:02:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664284+0000", 2024-05-11T23:02:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043960908999999999 2024-05-11T23:02:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664327+0000", 2024-05-11T23:02:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2811000000000001e-05 2024-05-11T23:02:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664406+0000", 2024-05-11T23:02:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8588e-05 2024-05-11T23:02:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.210 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67821 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4927488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.581005358999999, 2024-05-11T23:02:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044203340000000001, 2024-05-11T23:02:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67821 2024-05-11T23:02:26.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620330+0000", 2024-05-11T23:02:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:02:26.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620332+0000", 2024-05-11T23:02:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:02:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620338+0000", 2024-05-11T23:02:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6250000000000004e-06 2024-05-11T23:02:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620342+0000", 2024-05-11T23:02:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7609999999999998e-06 2024-05-11T23:02:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664433+0000", 2024-05-11T23:02:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044090455000000001 2024-05-11T23:02:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664476+0000", 2024-05-11T23:02:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2861999999999997e-05 2024-05-11T23:02:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664533+0000", 2024-05-11T23:02:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7156e-05 2024-05-11T23:02:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67822 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4976640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.580986321000001, 2024-05-11T23:02:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044409416, 2024-05-11T23:02:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:26.215 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67822 2024-05-11T23:02:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:26.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620349+0000", 2024-05-11T23:02:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.51e-07 2024-05-11T23:02:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620351+0000", 2024-05-11T23:02:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.181e-06 2024-05-11T23:02:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:26.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620358+0000", 2024-05-11T23:02:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3060000000000003e-06 2024-05-11T23:02:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620362+0000", 2024-05-11T23:02:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7949999999999998e-06 2024-05-11T23:02:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664572+0000", 2024-05-11T23:02:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044209318999999997 2024-05-11T23:02:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664654+0000", 2024-05-11T23:02:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2367999999999995e-05 2024-05-11T23:02:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664758+0000", 2024-05-11T23:02:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103896 2024-05-11T23:02:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67823 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4952064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:26.219 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.580966271000001, 2024-05-11T23:02:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044623293000000001, 2024-05-11T23:02:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67823 2024-05-11T23:02:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:26.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622228+0000", 2024-05-11T23:02:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018592280000000001 2024-05-11T23:02:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622232+0000", 2024-05-11T23:02:26.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.403e-06 2024-05-11T23:02:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622248+0000", 2024-05-11T23:02:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5449000000000001e-05 2024-05-11T23:02:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622254+0000", 2024-05-11T23:02:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5819999999999998e-06 2024-05-11T23:02:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664800+0000", 2024-05-11T23:02:26.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042546048000000003 2024-05-11T23:02:26.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:26.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664886+0000", 2024-05-11T23:02:26.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5525000000000005e-05 2024-05-11T23:02:26.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:26.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664992+0000", 2024-05-11T23:02:26.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010605800000000001 2024-05-11T23:02:26.223 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.223 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.223 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67833 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5099520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.578859798, 2024-05-11T23:02:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044076606999999997, 2024-05-11T23:02:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:26.224 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67833 2024-05-11T23:02:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8099999999999998e-07 2024-05-11T23:02:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622477+0000", 2024-05-11T23:02:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0700000000000001e-06 2024-05-11T23:02:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622483+0000", 2024-05-11T23:02:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0780000000000003e-06 2024-05-11T23:02:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:26.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622487+0000", 2024-05-11T23:02:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.352e-06 2024-05-11T23:02:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666460+0000", 2024-05-11T23:02:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043973085000000002 2024-05-11T23:02:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666498+0000", 2024-05-11T23:02:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8441e-05 2024-05-11T23:02:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:26.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666552+0000", 2024-05-11T23:02:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3199999999999999e-05 2024-05-11T23:02:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67834 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3944448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.578821519999998, 2024-05-11T23:02:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044166551999999998, 2024-05-11T23:02:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:26.228 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67834 2024-05-11T23:02:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:26.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622514+0000", 2024-05-11T23:02:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1100000000000002e-07 2024-05-11T23:02:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622515+0000", 2024-05-11T23:02:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:02:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622519+0000", 2024-05-11T23:02:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8260000000000003e-06 2024-05-11T23:02:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622522+0000", 2024-05-11T23:02:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8899999999999999e-06 2024-05-11T23:02:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666580+0000", 2024-05-11T23:02:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044058235000000001 2024-05-11T23:02:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666624+0000", 2024-05-11T23:02:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3501999999999998e-05 2024-05-11T23:02:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666680+0000", 2024-05-11T23:02:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6042999999999999e-05 2024-05-11T23:02:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67835 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.578808983999998, 2024-05-11T23:02:26.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044272019000000003, 2024-05-11T23:02:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67835 2024-05-11T23:02:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1699999999999999e-07 2024-05-11T23:02:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:26.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622527+0000", 2024-05-11T23:02:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2109999999999999e-06 2024-05-11T23:02:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622531+0000", 2024-05-11T23:02:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4910000000000001e-06 2024-05-11T23:02:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622534+0000", 2024-05-11T23:02:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7080000000000002e-06 2024-05-11T23:02:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666703+0000", 2024-05-11T23:02:26.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044169377000000003 2024-05-11T23:02:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666745+0000", 2024-05-11T23:02:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1616000000000003e-05 2024-05-11T23:02:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666798+0000", 2024-05-11T23:02:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3298999999999999e-05 2024-05-11T23:02:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67836 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.578798324000001, 2024-05-11T23:02:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044373692999999999, 2024-05-11T23:02:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67836 2024-05-11T23:02:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:26.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622537+0000", 2024-05-11T23:02:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622538+0000", 2024-05-11T23:02:26.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1510000000000001e-06 2024-05-11T23:02:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622541+0000", 2024-05-11T23:02:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:02:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622543+0000", 2024-05-11T23:02:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4870000000000001e-06 2024-05-11T23:02:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666821+0000", 2024-05-11T23:02:26.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044278038999999998 2024-05-11T23:02:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666859+0000", 2024-05-11T23:02:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7224000000000001e-05 2024-05-11T23:02:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666910+0000", 2024-05-11T23:02:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1629999999999999e-05 2024-05-11T23:02:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67837 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.578788596999999, 2024-05-11T23:02:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044477883000000003, 2024-05-11T23:02:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67837 2024-05-11T23:02:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:26.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622548+0000", 2024-05-11T23:02:26.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:02:26.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:26.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622550+0000", 2024-05-11T23:02:26.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-05-11T23:02:26.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:26.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622553+0000", 2024-05-11T23:02:26.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.43e-06 2024-05-11T23:02:26.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:26.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666932+0000", 2024-05-11T23:02:26.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044379584 2024-05-11T23:02:26.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:26.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666971+0000", 2024-05-11T23:02:26.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8427000000000003e-05 2024-05-11T23:02:26.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:26.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667024+0000", 2024-05-11T23:02:26.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3226000000000002e-05 2024-05-11T23:02:26.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.244 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.245 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.245 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67838 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:26.245 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:26.245 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.57877908, 2024-05-11T23:02:26.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044580374999999998, 2024-05-11T23:02:26.245 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:26.245 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:26.245 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:26.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:26.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:26.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:26.245 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67838 2024-05-11T23:02:26.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.245 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:26.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:26.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:26.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:26.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:26.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:26.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:26.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:02:26.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:26.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622557+0000", 2024-05-11T23:02:26.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.102e-06 2024-05-11T23:02:26.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:26.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622560+0000", 2024-05-11T23:02:26.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:02:26.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:26.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622562+0000", 2024-05-11T23:02:26.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:02:26.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:26.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667045+0000", 2024-05-11T23:02:26.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044482936000000001 2024-05-11T23:02:26.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:26.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667083+0000", 2024-05-11T23:02:26.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7443e-05 2024-05-11T23:02:26.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:26.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667136+0000", 2024-05-11T23:02:26.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3257000000000002e-05 2024-05-11T23:02:26.249 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.249 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.249 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.249 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67839 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5394432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:26.249 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:26.249 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.578769338000001, 2024-05-11T23:02:26.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04470263, 2024-05-11T23:02:26.249 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:26.249 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:26.249 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:26.249 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:26.249 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:26.249 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:26.250 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67839 2024-05-11T23:02:26.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.250 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:26.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:26.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:26.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:26.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:26.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:26.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622566+0000", 2024-05-11T23:02:26.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-05-11T23:02:26.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:26.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622572+0000", 2024-05-11T23:02:26.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5610000000000004e-06 2024-05-11T23:02:26.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:26.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622576+0000", 2024-05-11T23:02:26.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3210000000000001e-06 2024-05-11T23:02:26.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:26.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622578+0000", 2024-05-11T23:02:26.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4710000000000001e-06 2024-05-11T23:02:26.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:26.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667160+0000", 2024-05-11T23:02:26.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044582220999999998 2024-05-11T23:02:26.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:26.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667205+0000", 2024-05-11T23:02:26.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5049000000000002e-05 2024-05-11T23:02:26.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:26.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667268+0000", 2024-05-11T23:02:26.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2788999999999998e-05 2024-05-11T23:02:26.253 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.253 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.253 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.253 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67840 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5419008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:26.253 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:26.253 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.578753708000001, 2024-05-11T23:02:26.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044796006999999999, 2024-05-11T23:02:26.253 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:26.253 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:26.253 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:26.253 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:26.254 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:26.254 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:26.254 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67840 2024-05-11T23:02:26.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.254 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:26.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:26.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:26.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:26.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:26.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:26.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:26.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:02:26.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:26.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622583+0000", 2024-05-11T23:02:26.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4389999999999999e-06 2024-05-11T23:02:26.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:26.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622586+0000", 2024-05-11T23:02:26.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:02:26.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:26.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622588+0000", 2024-05-11T23:02:26.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4540000000000001e-06 2024-05-11T23:02:26.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:26.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667287+0000", 2024-05-11T23:02:26.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044698882000000002 2024-05-11T23:02:26.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:26.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667324+0000", 2024-05-11T23:02:26.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7196000000000001e-05 2024-05-11T23:02:26.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:26.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667377+0000", 2024-05-11T23:02:26.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2927999999999999e-05 2024-05-11T23:02:26.257 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.257 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.257 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.257 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67841 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5443584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:26.257 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:26.257 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.578743934999999, 2024-05-11T23:02:26.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044907337999999998, 2024-05-11T23:02:26.257 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:26.258 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:26.258 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:26.258 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:26.258 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:26.258 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:26.258 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67841 2024-05-11T23:02:26.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.258 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:26.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:26.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:26.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:26.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:26.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:26.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:26.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:26.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:26.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622592+0000", 2024-05-11T23:02:26.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-05-11T23:02:26.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:26.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622595+0000", 2024-05-11T23:02:26.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:02:26.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:26.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622598+0000", 2024-05-11T23:02:26.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5909999999999998e-06 2024-05-11T23:02:26.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:26.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667401+0000", 2024-05-11T23:02:26.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044802980999999999 2024-05-11T23:02:26.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:26.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667439+0000", 2024-05-11T23:02:26.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8840000000000001e-05 2024-05-11T23:02:26.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:26.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667498+0000", 2024-05-11T23:02:26.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8709000000000003e-05 2024-05-11T23:02:26.261 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.261 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.261 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.261 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67842 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5468160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:26.261 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:26.261 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.578734327999999, 2024-05-11T23:02:26.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045005876, 2024-05-11T23:02:26.262 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:26.262 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:26.262 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:26.262 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:26.262 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:26.262 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:26.262 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67842 2024-05-11T23:02:26.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.262 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:26.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:26.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:26.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:26.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:26.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:26.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622601+0000", 2024-05-11T23:02:26.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:26.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:26.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622602+0000", 2024-05-11T23:02:26.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0780000000000001e-06 2024-05-11T23:02:26.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:26.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622605+0000", 2024-05-11T23:02:26.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.914e-06 2024-05-11T23:02:26.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:26.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622607+0000", 2024-05-11T23:02:26.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:02:26.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:26.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667516+0000", 2024-05-11T23:02:26.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044909156999999998 2024-05-11T23:02:26.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:26.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667554+0000", 2024-05-11T23:02:26.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7920000000000003e-05 2024-05-11T23:02:26.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:26.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667606+0000", 2024-05-11T23:02:26.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2111000000000003e-05 2024-05-11T23:02:26.265 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.266 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.266 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.266 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67843 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4067328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:26.266 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:26.266 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.578724802, 2024-05-11T23:02:26.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045119014999999998, 2024-05-11T23:02:26.266 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:26.266 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:26.266 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:26.266 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:26.266 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:26.266 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:26.267 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67843 2024-05-11T23:02:26.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.267 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:26.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:26.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:26.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:26.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:26.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:26.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624228+0000", 2024-05-11T23:02:26.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001617867 2024-05-11T23:02:26.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:26.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624232+0000", 2024-05-11T23:02:26.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-05-11T23:02:26.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:26.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624240+0000", 2024-05-11T23:02:26.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4589999999999998e-06 2024-05-11T23:02:26.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:26.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624247+0000", 2024-05-11T23:02:26.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-05-11T23:02:26.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:26.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667629+0000", 2024-05-11T23:02:26.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043382012999999997 2024-05-11T23:02:26.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:26.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667674+0000", 2024-05-11T23:02:26.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5172000000000001e-05 2024-05-11T23:02:26.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:26.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667729+0000", 2024-05-11T23:02:26.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5086000000000001e-05 2024-05-11T23:02:26.270 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.270 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.270 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.270 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67849 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4190208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:26.270 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:26.270 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.576966681000002, 2024-05-11T23:02:26.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044061433999999997, 2024-05-11T23:02:26.271 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:26.271 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:26.271 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:26.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:26.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:26.271 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:26.271 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67849 2024-05-11T23:02:26.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.271 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:26.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:26.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:26.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:26.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:26.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:26.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624369+0000", 2024-05-11T23:02:26.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5900000000000002e-07 2024-05-11T23:02:26.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:26.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624371+0000", 2024-05-11T23:02:26.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0310000000000001e-06 2024-05-11T23:02:26.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:26.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624376+0000", 2024-05-11T23:02:26.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4029999999999998e-06 2024-05-11T23:02:26.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:26.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624381+0000", 2024-05-11T23:02:26.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4349999999999996e-06 2024-05-11T23:02:26.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:26.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668331+0000", 2024-05-11T23:02:26.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043949191999999998 2024-05-11T23:02:26.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:26.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668370+0000", 2024-05-11T23:02:26.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9227999999999998e-05 2024-05-11T23:02:26.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:26.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668430+0000", 2024-05-11T23:02:26.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9685999999999997e-05 2024-05-11T23:02:26.274 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.274 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.274 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.274 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67850 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:26.274 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:26.274 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.576945449, 2024-05-11T23:02:26.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044151953000000001, 2024-05-11T23:02:26.275 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:26.275 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:26.275 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:26.275 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:26.275 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:26.275 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:26.275 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67850 2024-05-11T23:02:26.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.275 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:26.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:26.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:26.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:26.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:26.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:26.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624390+0000", 2024-05-11T23:02:26.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:02:26.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:26.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624392+0000", 2024-05-11T23:02:26.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1880000000000001e-06 2024-05-11T23:02:26.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:26.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624399+0000", 2024-05-11T23:02:26.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7390000000000002e-06 2024-05-11T23:02:26.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:26.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624405+0000", 2024-05-11T23:02:26.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1059999999999996e-06 2024-05-11T23:02:26.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:26.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668450+0000", 2024-05-11T23:02:26.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044045147999999999 2024-05-11T23:02:26.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:26.277 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668487+0000", 2024-05-11T23:02:26.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7450000000000002e-05 2024-05-11T23:02:26.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:26.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668541+0000", 2024-05-11T23:02:26.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3952000000000001e-05 2024-05-11T23:02:26.278 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.278 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.278 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.278 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67851 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5541888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:26.278 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:26.278 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.576918867, 2024-05-11T23:02:26.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044264413000000002, 2024-05-11T23:02:26.278 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:26.279 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:26.279 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:26.279 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:26.279 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:26.279 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:26.279 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67851 2024-05-11T23:02:26.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.279 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:26.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:26.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:26.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:26.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:26.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:26.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:26.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-05-11T23:02:26.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:26.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624419+0000", 2024-05-11T23:02:26.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.21e-06 2024-05-11T23:02:26.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:26.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624424+0000", 2024-05-11T23:02:26.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9050000000000002e-06 2024-05-11T23:02:26.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:26.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624430+0000", 2024-05-11T23:02:26.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-05-11T23:02:26.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:26.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668566+0000", 2024-05-11T23:02:26.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044136208000000003 2024-05-11T23:02:26.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:26.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668614+0000", 2024-05-11T23:02:26.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8022000000000002e-05 2024-05-11T23:02:26.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:26.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668680+0000", 2024-05-11T23:02:26.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6181999999999998e-05 2024-05-11T23:02:26.282 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.282 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.282 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.282 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67852 3.16 3.1a4a4af6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:26.282 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:26.282 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.576898400000001, 2024-05-11T23:02:26.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044278287999999999, 2024-05-11T23:02:26.282 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:26.283 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:26.283 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:26.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:26.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:26.283 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:26.283 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67852 2024-05-11T23:02:26.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.283 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:26.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:26.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:26.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:26.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:26.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:26.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:26.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626226+0000", 2024-05-11T23:02:26.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017893 2024-05-11T23:02:26.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:26.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626229+0000", 2024-05-11T23:02:26.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0709999999999999e-06 2024-05-11T23:02:26.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:26.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626239+0000", 2024-05-11T23:02:26.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0507e-05 2024-05-11T23:02:26.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:26.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626244+0000", 2024-05-11T23:02:26.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4880000000000001e-06 2024-05-11T23:02:26.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:26.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668706+0000", 2024-05-11T23:02:26.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042462054999999999 2024-05-11T23:02:26.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:26.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:26.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:26.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668715+0000", 2024-05-11T23:02:26.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.867e-06 2024-05-11T23:02:26.286 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.286 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.286 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.286 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:26.286 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:26.286 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:26.287 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:02:26.288 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:26.287+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:26.351 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-05-11T23:02:26.360 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:02:26.388 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:26.388+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:26.393 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-11T23:02:26.392+0000 7fd16247e640 -1 mon.a@0(leader).osd e72 definitely_dead 0 2024-05-11T23:02:26.440 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:26.441 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:26.449 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:02:26.490 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:26.490+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:26.514 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:26.515 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:26.533 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:26.533 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:26.550 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:26.550 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:26.556 INFO:teuthology.orchestra.run.smithi005.stdout:3658 trunc from 0x24d4d9d to 0x4a354cf 2024-05-11T23:02:26.565 INFO:teuthology.orchestra.run.smithi005.stdout:3659 read 0x3b234bb thru 0x3b28d3c (0x5882 bytes) 2024-05-11T23:02:26.571 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:26.571 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:26.577 INFO:teuthology.orchestra.run.smithi005.stdout:3660 write 0xa60cc01 thru 0xa60da70 (0xe70 bytes) 2024-05-11T23:02:26.580 INFO:teuthology.orchestra.run.smithi005.stdout:3662 writesame 0x5a5e921 thru 0x5a62e3a data_size 0x1d(0x451a bytes) 2024-05-11T23:02:26.584 INFO:teuthology.orchestra.run.smithi005.stdout:3663 writesame 0x28d22c2 thru 0x28db1f1 data_size 0x74(0x8f30 bytes) 2024-05-11T23:02:26.586 INFO:teuthology.orchestra.run.smithi005.stdout:3665 write 0xa594af6 thru 0xa5a42a0 (0xf7ab bytes) 2024-05-11T23:02:26.587 INFO:teuthology.orchestra.run.smithi005.stdout:3667 writesame 0x5d07e5c thru 0x5d0fcfa data_size 0x5(0x7e9f bytes) 2024-05-11T23:02:26.590 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:26.589+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:26.591 INFO:teuthology.orchestra.run.smithi005.stdout:3668 write 0x21f4aa8 thru 0x22010bd (0xc616 bytes) 2024-05-11T23:02:26.592 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:26.592 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:26.600 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:26.600 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:26.600 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:26.600 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:26.600 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:26.610 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:02:26.619 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:26.619 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:26.622 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:26.622 INFO:teuthology.orchestra.run.smithi005.stdout:3670 read 0x3f20023 thru 0x3f22e39 (0x2e17 bytes) 2024-05-11T23:02:26.622 INFO:teuthology.orchestra.run.smithi005.stdout:3671 write 0x12ae6e3 thru 0x12ba23c (0xbb5a bytes) 2024-05-11T23:02:26.652 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:26.652 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:26.677 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:26.678 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:26.695 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:26.695+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:26.706 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:26.706 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:26.710 INFO:teuthology.orchestra.run.smithi005.stdout:3675 writesame 0xa75b7a1 thru 0xa769d48 data_size 0x4(0xe5a8 bytes) 2024-05-11T23:02:26.712 INFO:teuthology.orchestra.run.smithi005.stdout:3676 write 0x9552be6 thru 0x95550ba (0x24d5 bytes) 2024-05-11T23:02:26.716 INFO:teuthology.orchestra.run.smithi005.stdout:3677 trunc from 0xa769d49 to 0xbc9ac46 2024-05-11T23:02:26.721 INFO:teuthology.orchestra.run.smithi005.stdout:3678 writesame 0x955d82a thru 0x956336e data_size 0x1(0x5b45 bytes) 2024-05-11T23:02:26.733 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:26.733 INFO:teuthology.orchestra.run.smithi005.stdout:3681 punch from 0x1f5c60e to 0x1f5e4ea, (0x1edc bytes) 2024-05-11T23:02:26.733 INFO:teuthology.orchestra.run.smithi005.stdout:3683 write 0x29accdd thru 0x29baed2 (0xe1f6 bytes) 2024-05-11T23:02:26.735 INFO:teuthology.orchestra.run.smithi005.stdout:3684 write 0x6acc4d1 thru 0x6ad38bb (0x73eb bytes) 2024-05-11T23:02:26.771 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:26.772 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:26.781 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:02:26.792 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:26.792+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:26.823 INFO:teuthology.orchestra.run.smithi005.stdout:3686 punch from 0x397a98e to 0x397fe20, (0x5492 bytes) 2024-05-11T23:02:26.823 INFO:teuthology.orchestra.run.smithi005.stdout:3687 punch from 0x42311dc to 0x424015a, (0xef7e bytes) 2024-05-11T23:02:26.831 INFO:teuthology.orchestra.run.smithi005.stdout:3688 write 0xd31816d thru 0xd3195c8 (0x145c bytes) 2024-05-11T23:02:26.845 INFO:teuthology.orchestra.run.smithi005.stdout:3689 trunc from 0xd3195c9 to 0x3761a53 2024-05-11T23:02:26.893 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:26.893+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:26.952 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:26.953 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:26.961 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:02:26.991 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-05-11T23:02:26.992 INFO:teuthology.orchestra.run.smithi005.stderr:marked in osd.3. 2024-05-11T23:02:26.994 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:26.994+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:27.000 INFO:teuthology.orchestra.run.smithi005.stdout:3690 trunc from 0x3761a53 to 0xdad1f0b 2024-05-11T23:02:27.006 INFO:teuthology.orchestra.run.smithi005.stdout:3691 punch from 0x5f5afce to 0x5f678b8, (0xc8ea bytes) 2024-05-11T23:02:27.006 INFO:teuthology.orchestra.run.smithi005.stdout:3692 write 0x9a3587d thru 0x9a36d32 (0x14b6 bytes) 2024-05-11T23:02:27.009 INFO:teuthology.orchestra.run.smithi005.stdout:3693 writesame 0x75dc4d6 thru 0x75e3b3e data_size 0x1(0x7669 bytes) 2024-05-11T23:02:27.011 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-05-11T23:02:27.021 INFO:teuthology.orchestra.run.smithi005.stdout:3695 write 0x98d7ab6 thru 0x98e4481 (0xc9cc bytes) 2024-05-11T23:02:27.023 INFO:teuthology.orchestra.run.smithi005.stdout:3696 read 0x795a0b8 thru 0x795e383 (0x42cc bytes) 2024-05-11T23:02:27.094 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:27.094+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:27.107 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:27.107 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:27.117 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:02:27.196 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:27.196+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:27.262 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:27.262 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:27.271 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:02:27.296 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:27.296+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:27.389 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:27.390 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:27.398 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:02:27.399 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:27.398+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:27.499 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:27.499+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:27.548 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:27.548 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:27.557 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:02:27.600 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:27.600+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:27.701 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:27.700+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:27.706 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:27.706 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:27.715 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:02:27.802 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:27.802+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:27.824 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:27.825 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:27.833 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:02:27.917 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:27.917+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:27.985 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:27.985 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:27.999 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:02:28.004 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:28.003+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:28.104 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:28.104+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:28.149 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:28.149 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:28.158 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:02:28.205 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:28.205+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:28.268 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:28.268 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:28.277 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:02:28.307 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:28.307+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:28.408 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:28.408+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:28.427 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:28.427 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:28.436 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:02:28.509 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:28.509+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:28.586 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:28.586 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:28.595 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:02:28.610 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:28.610+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:28.690 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:02:28.705 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:28.706 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:28.711 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:28.711+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:28.715 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:02:28.812 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:28.812+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:28.820 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:28.820 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:28.838 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:28.856 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:28.856 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:28.868 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:28.868 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:28.873 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:28.873 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:28.877 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:02:28.892 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:28.892 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:28.911 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:28.912 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:28.912+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:28.929 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:28.947 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:28.989 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:28.989 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:28.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:28.989 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:28.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.989 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36828 3.b 3:d385bc8e:::rbd_data.111e5e4d8275.0000000000000020:11 [read 512000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:28.989 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:28.989 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.055304145999997, 2024-05-11T23:02:28.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051481218000000002, 2024-05-11T23:02:28.990 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:28.990 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:28.990 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:28.990 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:28.990 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:28.990 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:28.990 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36828 2024-05-11T23:02:28.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.990 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:28.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:28.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:28.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:28.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:28.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:28.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:28.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:28.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934915+0000", 2024-05-11T23:02:28.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00181814 2024-05-11T23:02:28.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:28.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934919+0000", 2024-05-11T23:02:28.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6739999999999996e-06 2024-05-11T23:02:28.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:28.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934934+0000", 2024-05-11T23:02:28.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4415e-05 2024-05-11T23:02:28.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:28.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934942+0000", 2024-05-11T23:02:28.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8619999999999992e-06 2024-05-11T23:02:28.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:28.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984433+0000", 2024-05-11T23:02:28.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049490658999999999 2024-05-11T23:02:28.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:28.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984508+0000", 2024-05-11T23:02:28.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4394999999999995e-05 2024-05-11T23:02:28.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:28.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984578+0000", 2024-05-11T23:02:28.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0073000000000005e-05 2024-05-11T23:02:28.993 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:28.993 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:28.993 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:28.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.993 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36950 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 520192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:28.994 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:28.994 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.050998380000003, 2024-05-11T23:02:28.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051450856000000003, 2024-05-11T23:02:28.994 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:28.994 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:28.994 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:28.994 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:28.994 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:28.994 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:28.994 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36950 2024-05-11T23:02:28.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.994 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:28.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:28.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:28.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:28.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:28.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:28.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:28.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:28.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938914+0000", 2024-05-11T23:02:28.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015121480000000001 2024-05-11T23:02:28.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:28.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938918+0000", 2024-05-11T23:02:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4149999999999999e-06 2024-05-11T23:02:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938925+0000", 2024-05-11T23:02:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0779999999999998e-06 2024-05-11T23:02:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938929+0000", 2024-05-11T23:02:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2930000000000002e-06 2024-05-11T23:02:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988675+0000", 2024-05-11T23:02:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049746394999999999 2024-05-11T23:02:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988756+0000", 2024-05-11T23:02:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0499000000000003e-05 2024-05-11T23:02:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:28.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988853+0000", 2024-05-11T23:02:28.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7027999999999993e-05 2024-05-11T23:02:28.998 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:28.998 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:28.998 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:28.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.998 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36957 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 778240~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:28.998 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:28.998 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.049376590999998, 2024-05-11T23:02:28.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051328273000000001, 2024-05-11T23:02:28.998 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:28.998 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:28.998 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:28.998 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:28.999 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:28.999 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:28.999 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36957 2024-05-11T23:02:28.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.999 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:28.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:28.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:28.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:28.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:28.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:28.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:28.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:28.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:28.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:29.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:29.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2099999999999998e-07 2024-05-11T23:02:29.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:29.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939026+0000", 2024-05-11T23:02:29.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5069999999999999e-06 2024-05-11T23:02:29.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:29.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939030+0000", 2024-05-11T23:02:29.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7919999999999999e-06 2024-05-11T23:02:29.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939032+0000", 2024-05-11T23:02:29.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9129999999999999e-06 2024-05-11T23:02:29.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:29.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990178+0000", 2024-05-11T23:02:29.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051145373000000001 2024-05-11T23:02:29.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:29.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990255+0000", 2024-05-11T23:02:29.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7195999999999997e-05 2024-05-11T23:02:29.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:29.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990352+0000", 2024-05-11T23:02:29.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7170999999999995e-05 2024-05-11T23:02:29.002 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.002 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:29.002 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.002 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36958 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 815104~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.002 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:29.002 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.049364271999998, 2024-05-11T23:02:29.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051531625999999997, 2024-05-11T23:02:29.002 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:29.002 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:29.003 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:29.003 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:29.003 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:29.003 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.003 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36958 2024-05-11T23:02:29.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.003 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:29.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:29.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:29.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:29.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:29.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:29.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939037+0000", 2024-05-11T23:02:29.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0400000000000002e-07 2024-05-11T23:02:29.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:29.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939038+0000", 2024-05-11T23:02:29.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.164e-06 2024-05-11T23:02:29.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:29.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939041+0000", 2024-05-11T23:02:29.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1109999999999999e-06 2024-05-11T23:02:29.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939043+0000", 2024-05-11T23:02:29.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.554e-06 2024-05-11T23:02:29.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:29.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990382+0000", 2024-05-11T23:02:29.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051338953999999999 2024-05-11T23:02:29.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:29.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990465+0000", 2024-05-11T23:02:29.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2461999999999998e-05 2024-05-11T23:02:29.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:29.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990568+0000", 2024-05-11T23:02:29.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000103077 2024-05-11T23:02:29.006 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.006 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:29.006 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.006 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36959 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 851968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.007 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:29.007 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.049352376000002, 2024-05-11T23:02:29.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051758936999999998, 2024-05-11T23:02:29.007 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:29.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:29.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:29.007 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:29.007 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:29.007 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.007 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36959 2024-05-11T23:02:29.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.009 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:29.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:29.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:29.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:29.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:29.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:29.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:29.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.16e-07 2024-05-11T23:02:29.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:29.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939050+0000", 2024-05-11T23:02:29.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.176e-06 2024-05-11T23:02:29.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:29.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939053+0000", 2024-05-11T23:02:29.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1140000000000002e-06 2024-05-11T23:02:29.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939055+0000", 2024-05-11T23:02:29.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.621e-06 2024-05-11T23:02:29.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:29.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990610+0000", 2024-05-11T23:02:29.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051554244999999999 2024-05-11T23:02:29.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:29.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990700+0000", 2024-05-11T23:02:29.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0284999999999994e-05 2024-05-11T23:02:29.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:29.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990807+0000", 2024-05-11T23:02:29.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010728 2024-05-11T23:02:29.012 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.012 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:29.012 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.013 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36960 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 888832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.013 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:29.013 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.049342140999997, 2024-05-11T23:02:29.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051886831000000001, 2024-05-11T23:02:29.013 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:29.013 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:29.013 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:29.013 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:29.013 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:29.013 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.013 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36960 2024-05-11T23:02:29.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.013 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:29.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:29.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:29.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:29.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:29.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:29.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940913+0000", 2024-05-11T23:02:29.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001855104 2024-05-11T23:02:29.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:29.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940916+0000", 2024-05-11T23:02:29.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0000000000000001e-06 2024-05-11T23:02:29.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:29.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940924+0000", 2024-05-11T23:02:29.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0999999999999998e-06 2024-05-11T23:02:29.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940927+0000", 2024-05-11T23:02:29.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7160000000000001e-06 2024-05-11T23:02:29.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:29.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990826+0000", 2024-05-11T23:02:29.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049898832999999997 2024-05-11T23:02:29.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:29.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990872+0000", 2024-05-11T23:02:29.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5717000000000003e-05 2024-05-11T23:02:29.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:29.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990945+0000", 2024-05-11T23:02:29.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3361000000000005e-05 2024-05-11T23:02:29.016 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.016 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:29.016 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.016 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36986 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 528384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.017 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:29.017 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.047335922999999, 2024-05-11T23:02:29.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051341053999999997, 2024-05-11T23:02:29.017 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:29.017 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:29.017 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:29.017 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:29.017 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:29.017 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.017 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36986 2024-05-11T23:02:29.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.017 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:29.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:29.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:29.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:29.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:29.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:29.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:29.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:02:29.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:29.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941066+0000", 2024-05-11T23:02:29.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1629999999999999e-06 2024-05-11T23:02:29.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:29.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941069+0000", 2024-05-11T23:02:29.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.118e-06 2024-05-11T23:02:29.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941072+0000", 2024-05-11T23:02:29.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5720000000000001e-06 2024-05-11T23:02:29.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:29.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992292+0000", 2024-05-11T23:02:29.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051220620000000001 2024-05-11T23:02:29.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:29.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992344+0000", 2024-05-11T23:02:29.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1737000000000003e-05 2024-05-11T23:02:29.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:29.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992406+0000", 2024-05-11T23:02:29.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1623000000000003e-05 2024-05-11T23:02:29.020 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.020 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:29.020 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.021 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36987 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 565248~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.021 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:29.021 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.047325802000003, 2024-05-11T23:02:29.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051568725000000003, 2024-05-11T23:02:29.021 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:29.021 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:29.021 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:29.021 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:29.021 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:29.021 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.021 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36987 2024-05-11T23:02:29.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.021 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:29.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:29.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:29.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:29.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:29.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:29.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:29.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.91e-07 2024-05-11T23:02:29.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:29.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941076+0000", 2024-05-11T23:02:29.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1379999999999999e-06 2024-05-11T23:02:29.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:29.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941079+0000", 2024-05-11T23:02:29.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9739999999999998e-06 2024-05-11T23:02:29.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941082+0000", 2024-05-11T23:02:29.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-05-11T23:02:29.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:29.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992446+0000", 2024-05-11T23:02:29.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051364058999999997 2024-05-11T23:02:29.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:29.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992536+0000", 2024-05-11T23:02:29.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0085000000000002e-05 2024-05-11T23:02:29.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:29.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992643+0000", 2024-05-11T23:02:29.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107377 2024-05-11T23:02:29.024 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.024 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:29.024 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.025 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36988 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 602112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.025 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:29.025 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.047315433000001, 2024-05-11T23:02:29.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051700007999999999, 2024-05-11T23:02:29.025 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:29.025 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:29.025 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:29.025 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:29.025 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:29.025 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.025 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36988 2024-05-11T23:02:29.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.025 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:29.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:29.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:29.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:29.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:29.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:29.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:29.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.05e-07 2024-05-11T23:02:29.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:29.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941086+0000", 2024-05-11T23:02:29.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.125e-06 2024-05-11T23:02:29.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:29.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941089+0000", 2024-05-11T23:02:29.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0599999999999999e-06 2024-05-11T23:02:29.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941092+0000", 2024-05-11T23:02:29.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6529999999999998e-06 2024-05-11T23:02:29.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:29.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992667+0000", 2024-05-11T23:02:29.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051574587999999998 2024-05-11T23:02:29.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:29.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992723+0000", 2024-05-11T23:02:29.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6682999999999999e-05 2024-05-11T23:02:29.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:29.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992785+0000", 2024-05-11T23:02:29.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1693999999999996e-05 2024-05-11T23:02:29.028 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.028 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:29.028 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.029 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36989 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 638976~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.029 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:29.029 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.047305244, 2024-05-11T23:02:29.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05192811, 2024-05-11T23:02:29.029 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:29.029 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:29.029 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:29.029 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:29.029 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:29.029 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.029 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36989 2024-05-11T23:02:29.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.029 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:29.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:29.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:29.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:29.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:29.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:29.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942916+0000", 2024-05-11T23:02:29.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001820835 2024-05-11T23:02:29.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:29.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942920+0000", 2024-05-11T23:02:29.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5310000000000002e-06 2024-05-11T23:02:29.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:29.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942931+0000", 2024-05-11T23:02:29.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1839e-05 2024-05-11T23:02:29.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942940+0000", 2024-05-11T23:02:29.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2309999999999996e-06 2024-05-11T23:02:29.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:29.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992820+0000", 2024-05-11T23:02:29.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049880751000000001 2024-05-11T23:02:29.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:29.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992918+0000", 2024-05-11T23:02:29.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7051000000000003e-05 2024-05-11T23:02:29.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:29.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.993023+0000", 2024-05-11T23:02:29.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010587200000000001 2024-05-11T23:02:29.032 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.032 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:29.033 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.033 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36997 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 933888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.033 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:29.033 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.045271409000001, 2024-05-11T23:02:29.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051390602000000001, 2024-05-11T23:02:29.033 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:29.033 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:29.033 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:29.033 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:29.033 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:29.033 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.033 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36997 2024-05-11T23:02:29.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.033 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:29.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:29.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:29.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:29.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:29.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:29.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943130+0000", 2024-05-11T23:02:29.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0200000000000003e-07 2024-05-11T23:02:29.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:29.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943132+0000", 2024-05-11T23:02:29.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.305e-06 2024-05-11T23:02:29.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:29.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943138+0000", 2024-05-11T23:02:29.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4219999999999997e-06 2024-05-11T23:02:29.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943143+0000", 2024-05-11T23:02:29.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0610000000000003e-06 2024-05-11T23:02:29.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:29.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994323+0000", 2024-05-11T23:02:29.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051179592000000003 2024-05-11T23:02:29.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:29.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994412+0000", 2024-05-11T23:02:29.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8709999999999996e-05 2024-05-11T23:02:29.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:29.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994520+0000", 2024-05-11T23:02:29.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010811000000000001 2024-05-11T23:02:29.036 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.036 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:29.036 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.037 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36998 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 970752~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.037 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:29.037 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.045250529999997, 2024-05-11T23:02:29.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051517669000000002, 2024-05-11T23:02:29.037 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:29.037 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:29.037 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:29.037 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:29.037 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:29.037 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.037 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36998 2024-05-11T23:02:29.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.037 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:29.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:29.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:29.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:29.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:29.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:29.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:29.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:02:29.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:29.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943153+0000", 2024-05-11T23:02:29.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.306e-06 2024-05-11T23:02:29.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:29.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943160+0000", 2024-05-11T23:02:29.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.452e-06 2024-05-11T23:02:29.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943165+0000", 2024-05-11T23:02:29.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.152e-06 2024-05-11T23:02:29.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:29.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994561+0000", 2024-05-11T23:02:29.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051395765000000003 2024-05-11T23:02:29.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:29.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994607+0000", 2024-05-11T23:02:29.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6415000000000002e-05 2024-05-11T23:02:29.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:29.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994668+0000", 2024-05-11T23:02:29.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0179000000000001e-05 2024-05-11T23:02:29.040 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.040 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36999 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 1007616~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.045205649000003, 2024-05-11T23:02:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051696919000000001, 2024-05-11T23:02:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36999 2024-05-11T23:02:29.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.042 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:29.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:29.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:29.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:29.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:29.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:29.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:29.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:02:29.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:29.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943197+0000", 2024-05-11T23:02:29.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1280000000000002e-06 2024-05-11T23:02:29.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:29.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943202+0000", 2024-05-11T23:02:29.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8910000000000003e-06 2024-05-11T23:02:29.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943206+0000", 2024-05-11T23:02:29.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3550000000000001e-06 2024-05-11T23:02:29.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:29.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994703+0000", 2024-05-11T23:02:29.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051497045999999998 2024-05-11T23:02:29.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:29.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994789+0000", 2024-05-11T23:02:29.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6458000000000005e-05 2024-05-11T23:02:29.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:29.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994892+0000", 2024-05-11T23:02:29.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102705 2024-05-11T23:02:29.045 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.045 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:29.045 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.045 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37000 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 1044480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.045 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:29.045 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.045189704000002, 2024-05-11T23:02:29.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051842371999999998, 2024-05-11T23:02:29.045 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:29.045 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:29.045 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:29.045 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:29.045 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:29.046 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.046 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37000 2024-05-11T23:02:29.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.046 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:29.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:29.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:29.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:29.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:29.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:29.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944916+0000", 2024-05-11T23:02:29.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001705097 2024-05-11T23:02:29.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:29.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944921+0000", 2024-05-11T23:02:29.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5809999999999998e-06 2024-05-11T23:02:29.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:29.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944930+0000", 2024-05-11T23:02:29.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5529999999999998e-06 2024-05-11T23:02:29.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944938+0000", 2024-05-11T23:02:29.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2519999999999999e-06 2024-05-11T23:02:29.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:29.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994911+0000", 2024-05-11T23:02:29.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049973113999999999 2024-05-11T23:02:29.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:29.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994982+0000", 2024-05-11T23:02:29.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0544999999999996e-05 2024-05-11T23:02:29.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:29.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.995053+0000", 2024-05-11T23:02:29.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1229999999999994e-05 2024-05-11T23:02:29.049 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.049 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:29.049 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.049 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37041 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 618496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.049 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:29.049 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.043345725000002, 2024-05-11T23:02:29.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051352768999999999, 2024-05-11T23:02:29.049 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:29.049 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:29.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:29.049 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:29.050 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:29.050 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.050 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37041 2024-05-11T23:02:29.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.050 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:29.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:29.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:29.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:29.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:29.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:29.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:29.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:02:29.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:29.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945056+0000", 2024-05-11T23:02:29.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:02:29.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:29.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945059+0000", 2024-05-11T23:02:29.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.965e-06 2024-05-11T23:02:29.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945062+0000", 2024-05-11T23:02:29.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:02:29.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:29.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996213+0000", 2024-05-11T23:02:29.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051151532999999999 2024-05-11T23:02:29.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:29.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996301+0000", 2024-05-11T23:02:29.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8143e-05 2024-05-11T23:02:29.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:29.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996408+0000", 2024-05-11T23:02:29.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010623700000000001 2024-05-11T23:02:29.053 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.053 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:29.053 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.053 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37042 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 655360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.053 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:29.053 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.043335740000003, 2024-05-11T23:02:29.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051462075000000003, 2024-05-11T23:02:29.053 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:29.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:29.054 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:29.054 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:29.054 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:29.054 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.054 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37042 2024-05-11T23:02:29.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.054 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:29.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:29.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:29.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:29.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:29.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:29.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:29.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0599999999999999e-07 2024-05-11T23:02:29.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:29.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945066+0000", 2024-05-11T23:02:29.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-05-11T23:02:29.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:29.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945069+0000", 2024-05-11T23:02:29.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0079999999999998e-06 2024-05-11T23:02:29.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945072+0000", 2024-05-11T23:02:29.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6740000000000001e-06 2024-05-11T23:02:29.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:29.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996422+0000", 2024-05-11T23:02:29.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051349999 2024-05-11T23:02:29.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:29.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996467+0000", 2024-05-11T23:02:29.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.528e-05 2024-05-11T23:02:29.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:29.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996527+0000", 2024-05-11T23:02:29.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9759999999999997e-05 2024-05-11T23:02:29.057 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.057 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:29.057 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.057 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37043 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 692224~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.058 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:29.058 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.043323667000003, 2024-05-11T23:02:29.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051679535999999998, 2024-05-11T23:02:29.058 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:29.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:29.058 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:29.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:29.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:29.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37043 2024-05-11T23:02:29.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.058 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:29.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:29.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:29.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:29.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:29.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:29.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:29.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8999999999999997e-07 2024-05-11T23:02:29.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:29.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945079+0000", 2024-05-11T23:02:29.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.001e-06 2024-05-11T23:02:29.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:29.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945087+0000", 2024-05-11T23:02:29.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1749999999999999e-06 2024-05-11T23:02:29.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945093+0000", 2024-05-11T23:02:29.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9639999999999997e-06 2024-05-11T23:02:29.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:29.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996563+0000", 2024-05-11T23:02:29.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051469050000000002 2024-05-11T23:02:29.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:29.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996651+0000", 2024-05-11T23:02:29.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8356000000000006e-05 2024-05-11T23:02:29.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:29.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996756+0000", 2024-05-11T23:02:29.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0001055 2024-05-11T23:02:29.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.061 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:29.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.062 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37044 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 729088~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.062 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:29.062 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.043289755000004, 2024-05-11T23:02:29.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051763445999999998, 2024-05-11T23:02:29.062 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:29.062 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:29.062 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:29.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:29.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:29.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.062 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37044 2024-05-11T23:02:29.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.062 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:29.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:29.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:29.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:29.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:29.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:29.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946915+0000", 2024-05-11T23:02:29.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018042760000000001 2024-05-11T23:02:29.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:29.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946918+0000", 2024-05-11T23:02:29.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.36e-06 2024-05-11T23:02:29.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:29.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946926+0000", 2024-05-11T23:02:29.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3130000000000001e-06 2024-05-11T23:02:29.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946930+0000", 2024-05-11T23:02:29.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5249999999999999e-06 2024-05-11T23:02:29.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:29.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996772+0000", 2024-05-11T23:02:29.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049841868999999997 2024-05-11T23:02:29.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:29.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996817+0000", 2024-05-11T23:02:29.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5351e-05 2024-05-11T23:02:29.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:29.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996874+0000", 2024-05-11T23:02:29.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6752000000000001e-05 2024-05-11T23:02:29.065 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.065 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:29.065 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.066 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37052 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 1024000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.066 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:29.066 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.041382904000002, 2024-05-11T23:02:29.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051326140999999999, 2024-05-11T23:02:29.066 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:29.066 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:29.066 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:29.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:29.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:29.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.066 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37052 2024-05-11T23:02:29.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.067 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:29.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:29.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:29.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:29.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:29.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:29.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:29.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:29.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:29.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947019+0000", 2024-05-11T23:02:29.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.218e-06 2024-05-11T23:02:29.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:29.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947022+0000", 2024-05-11T23:02:29.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1889999999999999e-06 2024-05-11T23:02:29.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947025+0000", 2024-05-11T23:02:29.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.464e-06 2024-05-11T23:02:29.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:29.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998237+0000", 2024-05-11T23:02:29.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051212494999999997 2024-05-11T23:02:29.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:29.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998284+0000", 2024-05-11T23:02:29.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7246000000000001e-05 2024-05-11T23:02:29.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:29.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998344+0000", 2024-05-11T23:02:29.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9320000000000001e-05 2024-05-11T23:02:29.070 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.070 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:29.070 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.070 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37071 3.b 3:d385bc8e:::rbd_data.111e5e4d8275.0000000000000020:11 [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.070 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:29.070 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.041372695, 2024-05-11T23:02:29.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051528796000000002, 2024-05-11T23:02:29.070 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:29.070 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:29.070 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:29.070 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:29.070 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:29.071 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.071 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37071 2024-05-11T23:02:29.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.071 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:29.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:29.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:29.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:29.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:29.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:29.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:29.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948913+0000", 2024-05-11T23:02:29.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018854970000000001 2024-05-11T23:02:29.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:29.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948917+0000", 2024-05-11T23:02:29.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8089999999999999e-06 2024-05-11T23:02:29.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:29.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948926+0000", 2024-05-11T23:02:29.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8389999999999996e-06 2024-05-11T23:02:29.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948930+0000", 2024-05-11T23:02:29.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4519999999999999e-06 2024-05-11T23:02:29.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:29.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998379+0000", 2024-05-11T23:02:29.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049449011000000001 2024-05-11T23:02:29.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:29.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998462+0000", 2024-05-11T23:02:29.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2739999999999997e-05 2024-05-11T23:02:29.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:29.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:29.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:29.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998557+0000", 2024-05-11T23:02:29.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4448000000000001e-05 2024-05-11T23:02:29.074 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.074 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:29.074 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.074 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:29.074 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:29.074 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:29.074 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-11T23:02:29.076 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:02:29.076 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:29.075+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:29.176 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:29.176+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:29.193 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:29.193 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:29.194 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:29.203 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:02:29.272 INFO:teuthology.orchestra.run.smithi005.stdout:3701 writesame 0xe51b1ce thru 0xe5205d0 data_size 0x3(0x5403 bytes) 2024-05-11T23:02:29.277 INFO:teuthology.orchestra.run.smithi005.stdout:3703 writesame 0xe61b692 thru 0xe625280 data_size 0x13(0x9bef bytes) 2024-05-11T23:02:29.277 INFO:teuthology.orchestra.run.smithi005.stdout:3704 writesame 0x5c0a5e3 thru 0x5c10352 data_size 0x1a(0x5d70 bytes) 2024-05-11T23:02:29.277 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:29.277+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:29.281 INFO:teuthology.orchestra.run.smithi005.stdout:3705 writesame 0xb4b5b4a thru 0xb4c52e7 data_size 0xf(0xf79e bytes) 2024-05-11T23:02:29.285 INFO:teuthology.orchestra.run.smithi005.stdout:3706 write 0x1ee3cf0 thru 0x1ef15fd (0xd90e bytes) 2024-05-11T23:02:29.287 INFO:teuthology.orchestra.run.smithi005.stdout:3707 punch from 0xb090893 to 0xb0962a4, (0x5a11 bytes) 2024-05-11T23:02:29.287 INFO:teuthology.orchestra.run.smithi005.stdout:3708 writesame 0x3e12bcc thru 0x3e1efc6 data_size 0x1(0xc3fb bytes) 2024-05-11T23:02:29.308 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:29.308 INFO:teuthology.orchestra.run.smithi005.stdout:3710 write 0x588c0f5 thru 0x588e099 (0x1fa5 bytes) 2024-05-11T23:02:29.309 INFO:teuthology.orchestra.run.smithi005.stdout:3711 write 0x67517af thru 0x6761427 (0xfc79 bytes) 2024-05-11T23:02:29.311 INFO:teuthology.orchestra.run.smithi005.stdout:3712 read 0x65c6bc6 thru 0x65c872a (0x1b65 bytes) 2024-05-11T23:02:29.312 INFO:teuthology.orchestra.run.smithi005.stdout:3713 read 0x3e0116 thru 0x3e5ac9 (0x59b4 bytes) 2024-05-11T23:02:29.315 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:29.315 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:29.315 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:29.318 INFO:teuthology.orchestra.run.smithi005.stdout:3714 trunc from 0xe625281 to 0xa1aa12c 2024-05-11T23:02:29.325 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:02:29.334 INFO:teuthology.orchestra.run.smithi005.stdout:3715 write 0xc7bdbc7 thru 0xc7c7037 (0x9471 bytes) 2024-05-11T23:02:29.344 INFO:teuthology.orchestra.run.smithi005.stdout:3716 write 0xcaf2b2f thru 0xcaf6583 (0x3a55 bytes) 2024-05-11T23:02:29.347 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:29.347 INFO:teuthology.orchestra.run.smithi005.stdout:3718 writesame 0x6cdca44 thru 0x6cea41d data_size 0x6e(0xd9da bytes) 2024-05-11T23:02:29.348 INFO:teuthology.orchestra.run.smithi005.stdout:3719 read 0xb6547c3 thru 0xb65b941 (0x717f bytes) 2024-05-11T23:02:29.349 INFO:teuthology.orchestra.run.smithi005.stdout:3721 punch from 0x4a6672b to 0x4a6c8b8, (0x618d bytes) 2024-05-11T23:02:29.349 INFO:teuthology.orchestra.run.smithi005.stdout:3722 writesame 0x87071f5 thru 0x8713ea1 data_size 0x1(0xccad bytes) 2024-05-11T23:02:29.378 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:29.378+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:29.385 INFO:teuthology.orchestra.run.smithi005.stdout:3723 trunc from 0xcaf6584 to 0x37a6e54 2024-05-11T23:02:29.412 INFO:teuthology.orchestra.run.smithi005.stdout:3724 write 0x5df0196 thru 0x5df0a18 (0x883 bytes) 2024-05-11T23:02:29.415 INFO:teuthology.orchestra.run.smithi005.stdout:3725 write 0x3a51b91 thru 0x3a5401d (0x248d bytes) 2024-05-11T23:02:29.418 INFO:teuthology.orchestra.run.smithi005.stdout:3727 read 0x515cec5 thru 0x5164176 (0x72b2 bytes) 2024-05-11T23:02:29.427 INFO:teuthology.orchestra.run.smithi005.stdout:3728 write 0x881e7ad thru 0x8826430 (0x7c84 bytes) 2024-05-11T23:02:29.429 INFO:teuthology.orchestra.run.smithi005.stdout:3729 write 0x1f21ea5 thru 0x1f26ed1 (0x502d bytes) 2024-05-11T23:02:29.430 INFO:teuthology.orchestra.run.smithi005.stdout:3731 write 0x4b4d4e5 thru 0x4b4eee8 (0x1a04 bytes) 2024-05-11T23:02:29.440 INFO:teuthology.orchestra.run.smithi005.stdout:3733 write 0x8eadb2a thru 0x8ebd338 (0xf80f bytes) 2024-05-11T23:02:29.442 INFO:teuthology.orchestra.run.smithi005.stdout:3734 write 0x213060d thru 0x21335fb (0x2fef bytes) 2024-05-11T23:02:29.445 INFO:teuthology.orchestra.run.smithi005.stdout:3738 writesame 0x20a5e55 thru 0x20b25ee data_size 0x1d(0xc79a bytes) 2024-05-11T23:02:29.446 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:29.447 INFO:teuthology.orchestra.run.smithi005.stdout:3740 read 0x32dc0fc thru 0x32e4a3c (0x8941 bytes) 2024-05-11T23:02:29.447 INFO:teuthology.orchestra.run.smithi005.stdout:3741 write 0x4bf790e thru 0x4bfc0ba (0x47ad bytes) 2024-05-11T23:02:29.449 INFO:teuthology.orchestra.run.smithi005.stdout:3742 read 0x55d3403 thru 0x55dd931 (0xa52f bytes) 2024-05-11T23:02:29.449 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:29.449 INFO:teuthology.orchestra.run.smithi005.stdout:3744 writesame 0x2a6d48f thru 0x2a79570 data_size 0x7(0xc0e2 bytes) 2024-05-11T23:02:29.453 INFO:teuthology.orchestra.run.smithi005.stdout:3745 read 0x26c12f2 thru 0x26c527f (0x3f8e bytes) 2024-05-11T23:02:29.453 INFO:teuthology.orchestra.run.smithi005.stdout:3747 writesame 0x1678add thru 0x167dd49 data_size 0x1(0x526d bytes) 2024-05-11T23:02:29.465 INFO:teuthology.orchestra.run.smithi005.stdout:3748 punch from 0xd48a0 to 0xdc0ef, (0x784f bytes) 2024-05-11T23:02:29.466 INFO:teuthology.orchestra.run.smithi005.stdout:3749 write 0x138b255 thru 0x1399fdf (0xed8b bytes) 2024-05-11T23:02:29.466 INFO:teuthology.orchestra.run.smithi005.stdout:3751 punch from 0x19663e8 to 0x1966c94, (0x8ac bytes) 2024-05-11T23:02:29.473 INFO:teuthology.orchestra.run.smithi005.stdout:3752 write 0xad151b1 thru 0xad18ce6 (0x3b36 bytes) 2024-05-11T23:02:29.479 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:29.479+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:29.485 INFO:teuthology.orchestra.run.smithi005.stdout:3753 writesame 0xe56c612 thru 0xe57c3cf data_size 0x1(0xfdbe bytes) 2024-05-11T23:02:29.490 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:29.490 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:29.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:29.490 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:29.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.490 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24548 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 200704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.490 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.927949+0000", 2024-05-11T23:02:29.490 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.560638206, 2024-05-11T23:02:29.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037300396999999999, 2024-05-11T23:02:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24548 2024-05-11T23:02:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927949+0000", 2024-05-11T23:02:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927949+0000", 2024-05-11T23:02:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927950+0000", 2024-05-11T23:02:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9400000000000005e-07 2024-05-11T23:02:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927952+0000", 2024-05-11T23:02:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2330000000000001e-06 2024-05-11T23:02:29.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927959+0000", 2024-05-11T23:02:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8820000000000003e-06 2024-05-11T23:02:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927964+0000", 2024-05-11T23:02:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.899e-06 2024-05-11T23:02:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.959280+0000", 2024-05-11T23:02:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.031315904999999998 2024-05-11T23:02:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.959348+0000", 2024-05-11T23:02:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8214999999999997e-05 2024-05-11T23:02:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965250+0000", 2024-05-11T23:02:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.005901669 2024-05-11T23:02:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.494 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24555 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.927974+0000", 2024-05-11T23:02:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.560612939000002, 2024-05-11T23:02:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.03739746, 2024-05-11T23:02:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24555 2024-05-11T23:02:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927974+0000", 2024-05-11T23:02:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927974+0000", 2024-05-11T23:02:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927975+0000", 2024-05-11T23:02:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6899999999999998e-07 2024-05-11T23:02:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927977+0000", 2024-05-11T23:02:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.074e-06 2024-05-11T23:02:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927984+0000", 2024-05-11T23:02:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7309999999999996e-06 2024-05-11T23:02:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927989+0000", 2024-05-11T23:02:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.079e-06 2024-05-11T23:02:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965279+0000", 2024-05-11T23:02:29.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037290271999999999 2024-05-11T23:02:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965324+0000", 2024-05-11T23:02:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4638999999999997e-05 2024-05-11T23:02:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965372+0000", 2024-05-11T23:02:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8195999999999997e-05 2024-05-11T23:02:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24562 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 233472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.927995+0000", 2024-05-11T23:02:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.560592004, 2024-05-11T23:02:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037703408000000001, 2024-05-11T23:02:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24562 2024-05-11T23:02:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927995+0000", 2024-05-11T23:02:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927995+0000", 2024-05-11T23:02:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927996+0000", 2024-05-11T23:02:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9499999999999998e-07 2024-05-11T23:02:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.927998+0000", 2024-05-11T23:02:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0889999999999998e-06 2024-05-11T23:02:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928004+0000", 2024-05-11T23:02:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4740000000000002e-06 2024-05-11T23:02:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928009+0000", 2024-05-11T23:02:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:02:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965549+0000", 2024-05-11T23:02:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037539826999999998 2024-05-11T23:02:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965616+0000", 2024-05-11T23:02:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7387999999999999e-05 2024-05-11T23:02:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965699+0000", 2024-05-11T23:02:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2347000000000003e-05 2024-05-11T23:02:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24568 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 249856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928016+0000", 2024-05-11T23:02:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.560571818, 2024-05-11T23:02:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037850338999999997, 2024-05-11T23:02:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24568 2024-05-11T23:02:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928016+0000", 2024-05-11T23:02:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928016+0000", 2024-05-11T23:02:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928016+0000", 2024-05-11T23:02:29.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-05-11T23:02:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928025+0000", 2024-05-11T23:02:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.363e-06 2024-05-11T23:02:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928032+0000", 2024-05-11T23:02:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.114e-06 2024-05-11T23:02:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928038+0000", 2024-05-11T23:02:29.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1250000000000001e-06 2024-05-11T23:02:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965720+0000", 2024-05-11T23:02:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037682488 2024-05-11T23:02:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965785+0000", 2024-05-11T23:02:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5180999999999998e-05 2024-05-11T23:02:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965866+0000", 2024-05-11T23:02:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0687999999999997e-05 2024-05-11T23:02:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24574 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 266240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928045+0000", 2024-05-11T23:02:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.560542841, 2024-05-11T23:02:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037990532, 2024-05-11T23:02:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.507 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24574 2024-05-11T23:02:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928045+0000", 2024-05-11T23:02:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928045+0000", 2024-05-11T23:02:29.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928045+0000", 2024-05-11T23:02:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:02:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928047+0000", 2024-05-11T23:02:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9649999999999998e-06 2024-05-11T23:02:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928053+0000", 2024-05-11T23:02:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1550000000000004e-06 2024-05-11T23:02:29.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928058+0000", 2024-05-11T23:02:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9060000000000001e-06 2024-05-11T23:02:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965890+0000", 2024-05-11T23:02:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037831754000000002 2024-05-11T23:02:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.965957+0000", 2024-05-11T23:02:29.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6798999999999995e-05 2024-05-11T23:02:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966035+0000", 2024-05-11T23:02:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8532e-05 2024-05-11T23:02:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24581 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928064+0000", 2024-05-11T23:02:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.560523025000002, 2024-05-11T23:02:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038134135, 2024-05-11T23:02:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.511 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24581 2024-05-11T23:02:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928064+0000", 2024-05-11T23:02:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928064+0000", 2024-05-11T23:02:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928065+0000", 2024-05-11T23:02:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3099999999999998e-07 2024-05-11T23:02:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928067+0000", 2024-05-11T23:02:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9549999999999999e-06 2024-05-11T23:02:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928073+0000", 2024-05-11T23:02:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1929999999999998e-06 2024-05-11T23:02:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928078+0000", 2024-05-11T23:02:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9289999999999997e-06 2024-05-11T23:02:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966058+0000", 2024-05-11T23:02:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.037980452999999997 2024-05-11T23:02:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966123+0000", 2024-05-11T23:02:29.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4832000000000005e-05 2024-05-11T23:02:29.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966198+0000", 2024-05-11T23:02:29.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5341999999999998e-05 2024-05-11T23:02:29.515 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.515 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.515 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.515 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24585 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 299008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.515 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928095+0000", 2024-05-11T23:02:29.515 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.560492504999999, 2024-05-11T23:02:29.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038282688000000002, 2024-05-11T23:02:29.516 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.516 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.516 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.516 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24585 2024-05-11T23:02:29.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.516 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928095+0000", 2024-05-11T23:02:29.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928095+0000", 2024-05-11T23:02:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928095+0000", 2024-05-11T23:02:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1900000000000003e-07 2024-05-11T23:02:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928098+0000", 2024-05-11T23:02:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1880000000000001e-06 2024-05-11T23:02:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928105+0000", 2024-05-11T23:02:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3690000000000001e-06 2024-05-11T23:02:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928120+0000", 2024-05-11T23:02:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4762e-05 2024-05-11T23:02:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966233+0000", 2024-05-11T23:02:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038113526000000002 2024-05-11T23:02:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966300+0000", 2024-05-11T23:02:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6637999999999995e-05 2024-05-11T23:02:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966378+0000", 2024-05-11T23:02:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7686000000000001e-05 2024-05-11T23:02:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24592 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 315392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928128+0000", 2024-05-11T23:02:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.560459199999997, 2024-05-11T23:02:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038417382, 2024-05-11T23:02:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24592 2024-05-11T23:02:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928128+0000", 2024-05-11T23:02:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928128+0000", 2024-05-11T23:02:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928129+0000", 2024-05-11T23:02:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.44e-07 2024-05-11T23:02:29.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928131+0000", 2024-05-11T23:02:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2910000000000002e-06 2024-05-11T23:02:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928138+0000", 2024-05-11T23:02:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9430000000000003e-06 2024-05-11T23:02:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928143+0000", 2024-05-11T23:02:29.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4739999999999996e-06 2024-05-11T23:02:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966400+0000", 2024-05-11T23:02:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038257001999999998 2024-05-11T23:02:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966465+0000", 2024-05-11T23:02:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4468999999999995e-05 2024-05-11T23:02:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966546+0000", 2024-05-11T23:02:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0659000000000001e-05 2024-05-11T23:02:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24598 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 331776~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928150+0000", 2024-05-11T23:02:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.560436891000002, 2024-05-11T23:02:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038544469999999997, 2024-05-11T23:02:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.524 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24598 2024-05-11T23:02:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928150+0000", 2024-05-11T23:02:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928150+0000", 2024-05-11T23:02:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928151+0000", 2024-05-11T23:02:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999999e-07 2024-05-11T23:02:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928153+0000", 2024-05-11T23:02:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.21e-06 2024-05-11T23:02:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928160+0000", 2024-05-11T23:02:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5200000000000003e-06 2024-05-11T23:02:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928165+0000", 2024-05-11T23:02:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9269999999999996e-06 2024-05-11T23:02:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966569+0000", 2024-05-11T23:02:29.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038404595999999999 2024-05-11T23:02:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966634+0000", 2024-05-11T23:02:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5221000000000005e-05 2024-05-11T23:02:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966695+0000", 2024-05-11T23:02:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0547999999999997e-05 2024-05-11T23:02:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.528 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24604 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 348160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928171+0000", 2024-05-11T23:02:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.560416529000001, 2024-05-11T23:02:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038676227000000001, 2024-05-11T23:02:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24604 2024-05-11T23:02:29.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928171+0000", 2024-05-11T23:02:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928171+0000", 2024-05-11T23:02:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928171+0000", 2024-05-11T23:02:29.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-05-11T23:02:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928173+0000", 2024-05-11T23:02:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0619999999999999e-06 2024-05-11T23:02:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928180+0000", 2024-05-11T23:02:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4269999999999996e-06 2024-05-11T23:02:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928185+0000", 2024-05-11T23:02:29.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.04e-06 2024-05-11T23:02:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966718+0000", 2024-05-11T23:02:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038532759 2024-05-11T23:02:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966784+0000", 2024-05-11T23:02:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6865000000000004e-05 2024-05-11T23:02:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966847+0000", 2024-05-11T23:02:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2694000000000007e-05 2024-05-11T23:02:29.532 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24609 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 364544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928191+0000", 2024-05-11T23:02:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.560396461000003, 2024-05-11T23:02:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038806908000000001, 2024-05-11T23:02:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.533 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24609 2024-05-11T23:02:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928191+0000", 2024-05-11T23:02:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928191+0000", 2024-05-11T23:02:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928191+0000", 2024-05-11T23:02:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6300000000000001e-07 2024-05-11T23:02:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928198+0000", 2024-05-11T23:02:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2180000000000002e-06 2024-05-11T23:02:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928205+0000", 2024-05-11T23:02:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5899999999999996e-06 2024-05-11T23:02:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928210+0000", 2024-05-11T23:02:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.118e-06 2024-05-11T23:02:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966869+0000", 2024-05-11T23:02:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038659106999999998 2024-05-11T23:02:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966933+0000", 2024-05-11T23:02:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4158999999999993e-05 2024-05-11T23:02:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.966998+0000", 2024-05-11T23:02:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4352999999999999e-05 2024-05-11T23:02:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24616 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 380928~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.537 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928217+0000", 2024-05-11T23:02:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.560370317999997, 2024-05-11T23:02:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038928536999999999, 2024-05-11T23:02:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24616 2024-05-11T23:02:29.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928217+0000", 2024-05-11T23:02:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928217+0000", 2024-05-11T23:02:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928217+0000", 2024-05-11T23:02:29.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.89e-07 2024-05-11T23:02:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928219+0000", 2024-05-11T23:02:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0609999999999998e-06 2024-05-11T23:02:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928233+0000", 2024-05-11T23:02:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3715e-05 2024-05-11T23:02:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928238+0000", 2024-05-11T23:02:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1889999999999998e-06 2024-05-11T23:02:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967021+0000", 2024-05-11T23:02:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038782284 2024-05-11T23:02:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967085+0000", 2024-05-11T23:02:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4319000000000005e-05 2024-05-11T23:02:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967146+0000", 2024-05-11T23:02:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0579999999999999e-05 2024-05-11T23:02:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24622 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 397312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928262+0000", 2024-05-11T23:02:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.560325013000003, 2024-05-11T23:02:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039038392999999998, 2024-05-11T23:02:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24622 2024-05-11T23:02:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.543 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928262+0000", 2024-05-11T23:02:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928262+0000", 2024-05-11T23:02:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928263+0000", 2024-05-11T23:02:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0299999999999999e-07 2024-05-11T23:02:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928266+0000", 2024-05-11T23:02:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6690000000000002e-06 2024-05-11T23:02:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928275+0000", 2024-05-11T23:02:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1630000000000002e-06 2024-05-11T23:02:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928280+0000", 2024-05-11T23:02:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6579999999999999e-06 2024-05-11T23:02:29.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967169+0000", 2024-05-11T23:02:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.038888569999999997 2024-05-11T23:02:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967240+0000", 2024-05-11T23:02:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0947999999999999e-05 2024-05-11T23:02:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967301+0000", 2024-05-11T23:02:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0782000000000002e-05 2024-05-11T23:02:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24628 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 413696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928288+0000", 2024-05-11T23:02:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.560298965000001, 2024-05-11T23:02:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039162792000000002, 2024-05-11T23:02:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.547 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24628 2024-05-11T23:02:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928288+0000", 2024-05-11T23:02:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928288+0000", 2024-05-11T23:02:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928289+0000", 2024-05-11T23:02:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7399999999999998e-07 2024-05-11T23:02:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928291+0000", 2024-05-11T23:02:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.221e-06 2024-05-11T23:02:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928298+0000", 2024-05-11T23:02:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0520000000000004e-06 2024-05-11T23:02:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928304+0000", 2024-05-11T23:02:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4149999999999998e-06 2024-05-11T23:02:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967324+0000", 2024-05-11T23:02:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039020436999999998 2024-05-11T23:02:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967390+0000", 2024-05-11T23:02:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5821000000000006e-05 2024-05-11T23:02:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967451+0000", 2024-05-11T23:02:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1371999999999994e-05 2024-05-11T23:02:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:24636 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 430080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.928310+0000", 2024-05-11T23:02:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.5602771, 2024-05-11T23:02:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039286303000000002, 2024-05-11T23:02:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 24636 2024-05-11T23:02:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928310+0000", 2024-05-11T23:02:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928310+0000", 2024-05-11T23:02:29.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928311+0000", 2024-05-11T23:02:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:02:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928313+0000", 2024-05-11T23:02:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0499999999999999e-06 2024-05-11T23:02:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928319+0000", 2024-05-11T23:02:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4049999999999997e-06 2024-05-11T23:02:29.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.928324+0000", 2024-05-11T23:02:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:02:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967473+0000", 2024-05-11T23:02:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039149138999999999 2024-05-11T23:02:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967537+0000", 2024-05-11T23:02:29.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4109000000000006e-05 2024-05-11T23:02:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.967597+0000", 2024-05-11T23:02:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9407000000000002e-05 2024-05-11T23:02:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25144 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 28672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936035+0000", 2024-05-11T23:02:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.552552194999997, 2024-05-11T23:02:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036140510000000001, 2024-05-11T23:02:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.555 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25144 2024-05-11T23:02:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936035+0000", 2024-05-11T23:02:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936035+0000", 2024-05-11T23:02:29.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936036+0000", 2024-05-11T23:02:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8999999999999997e-07 2024-05-11T23:02:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936038+0000", 2024-05-11T23:02:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6910000000000002e-06 2024-05-11T23:02:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936045+0000", 2024-05-11T23:02:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1670000000000004e-06 2024-05-11T23:02:29.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936049+0000", 2024-05-11T23:02:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9609999999999996e-06 2024-05-11T23:02:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972024+0000", 2024-05-11T23:02:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.035974221000000001 2024-05-11T23:02:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972076+0000", 2024-05-11T23:02:29.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1918e-05 2024-05-11T23:02:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972176+0000", 2024-05-11T23:02:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010006200000000001 2024-05-11T23:02:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25150 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 45056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936063+0000", 2024-05-11T23:02:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.552524662000003, 2024-05-11T23:02:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036298174000000002, 2024-05-11T23:02:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.559 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25150 2024-05-11T23:02:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936063+0000", 2024-05-11T23:02:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936063+0000", 2024-05-11T23:02:29.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936063+0000", 2024-05-11T23:02:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.39e-07 2024-05-11T23:02:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936069+0000", 2024-05-11T23:02:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2770000000000002e-06 2024-05-11T23:02:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936088+0000", 2024-05-11T23:02:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8841000000000001e-05 2024-05-11T23:02:29.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936093+0000", 2024-05-11T23:02:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2190000000000003e-06 2024-05-11T23:02:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972213+0000", 2024-05-11T23:02:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036119039999999998 2024-05-11T23:02:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972293+0000", 2024-05-11T23:02:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0437000000000003e-05 2024-05-11T23:02:29.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972361+0000", 2024-05-11T23:02:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7921000000000002e-05 2024-05-11T23:02:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25156 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 61440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936107+0000", 2024-05-11T23:02:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.552480138, 2024-05-11T23:02:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036383785000000002, 2024-05-11T23:02:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.563 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25156 2024-05-11T23:02:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936107+0000", 2024-05-11T23:02:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936107+0000", 2024-05-11T23:02:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936108+0000", 2024-05-11T23:02:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3900000000000005e-07 2024-05-11T23:02:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936110+0000", 2024-05-11T23:02:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.435e-06 2024-05-11T23:02:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936118+0000", 2024-05-11T23:02:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2889999999999998e-06 2024-05-11T23:02:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936124+0000", 2024-05-11T23:02:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2850000000000002e-06 2024-05-11T23:02:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972378+0000", 2024-05-11T23:02:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036254020999999997 2024-05-11T23:02:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972420+0000", 2024-05-11T23:02:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2577999999999998e-05 2024-05-11T23:02:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972491+0000", 2024-05-11T23:02:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0637999999999997e-05 2024-05-11T23:02:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25164 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936132+0000", 2024-05-11T23:02:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.552454894999997, 2024-05-11T23:02:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036532028000000001, 2024-05-11T23:02:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.567 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25164 2024-05-11T23:02:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936132+0000", 2024-05-11T23:02:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936132+0000", 2024-05-11T23:02:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936133+0000", 2024-05-11T23:02:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8000000000000001e-07 2024-05-11T23:02:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936135+0000", 2024-05-11T23:02:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0660000000000002e-06 2024-05-11T23:02:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936146+0000", 2024-05-11T23:02:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.098e-05 2024-05-11T23:02:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936153+0000", 2024-05-11T23:02:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8619999999999997e-06 2024-05-11T23:02:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972532+0000", 2024-05-11T23:02:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036379702999999999 2024-05-11T23:02:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972577+0000", 2024-05-11T23:02:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4697999999999998e-05 2024-05-11T23:02:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972664+0000", 2024-05-11T23:02:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7338999999999998e-05 2024-05-11T23:02:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:25168 3.9 3:91531ee2:::rbd_data.111e70525772.000000000000000c:e [read 94208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:51.936160+0000", 2024-05-11T23:02:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.552427616999999, 2024-05-11T23:02:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.036655488999999999, 2024-05-11T23:02:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 25168 2024-05-11T23:02:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936160+0000", 2024-05-11T23:02:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936160+0000", 2024-05-11T23:02:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936160+0000", 2024-05-11T23:02:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3000000000000001e-07 2024-05-11T23:02:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.936162+0000", 2024-05-11T23:02:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.108e-06 2024-05-11T23:02:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.937848+0000", 2024-05-11T23:02:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001685284 2024-05-11T23:02:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.937858+0000", 2024-05-11T23:02:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0222e-05 2024-05-11T23:02:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972704+0000", 2024-05-11T23:02:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.034845736000000002 2024-05-11T23:02:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972746+0000", 2024-05-11T23:02:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2077999999999999e-05 2024-05-11T23:02:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:51.972815+0000", 2024-05-11T23:02:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9630999999999998e-05 2024-05-11T23:02:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.575 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.575 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:29.577 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:02:29.578 INFO:teuthology.orchestra.run.smithi005.stdout:3754 writesame 0xeb212ca thru 0xeb21c4d data_size 0x1d(0x984 bytes) 2024-05-11T23:02:29.578 INFO:teuthology.orchestra.run.smithi005.stdout:3755 read 0x5e25dc7 thru 0x5e32e54 (0xd08e bytes) 2024-05-11T23:02:29.578 INFO:teuthology.orchestra.run.smithi005.stdout:3757 write 0x71b3c62 thru 0x71b9b8d (0x5f2c bytes) 2024-05-11T23:02:29.578 INFO:teuthology.orchestra.run.smithi005.stdout:3758 read 0x99737d4 thru 0x9982248 (0xea75 bytes) 2024-05-11T23:02:29.579 INFO:teuthology.orchestra.run.smithi005.stdout:3759 read 0xb46c0f5 thru 0xb4792ce (0xd1da bytes) 2024-05-11T23:02:29.579 INFO:teuthology.orchestra.run.smithi005.stdout:3760 trunc from 0xeb21c4e to 0x42fb587 2024-05-11T23:02:29.579 INFO:teuthology.orchestra.run.smithi005.stdout:3761 trunc from 0x42fb587 to 0x6439ddb 2024-05-11T23:02:29.579 INFO:teuthology.orchestra.run.smithi005.stdout:3762 write 0x1d25815 thru 0x1d2e52c (0x8d18 bytes) 2024-05-11T23:02:29.579 INFO:teuthology.orchestra.run.smithi005.stdout:3763 write 0xcd81915 thru 0xcd88aed (0x71d9 bytes) 2024-05-11T23:02:29.579 INFO:teuthology.orchestra.run.smithi005.stdout:3764 punch from 0x24d9a77 to 0x24dfdf1, (0x637a bytes) 2024-05-11T23:02:29.579 INFO:teuthology.orchestra.run.smithi005.stdout:3766 write 0xdf42fd thru 0xdfbfba (0x7cbe bytes) 2024-05-11T23:02:29.579 INFO:teuthology.orchestra.run.smithi005.stdout:3767 read 0x81849d2 thru 0x8185c5a (0x1289 bytes) 2024-05-11T23:02:29.579 INFO:teuthology.orchestra.run.smithi005.stdout:3768 write 0x2c79db0 thru 0x2c7b66b (0x18bc bytes) 2024-05-11T23:02:29.579 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:29.579 INFO:teuthology.orchestra.run.smithi005.stdout:3771 read 0x8f3bea6 thru 0x8f416f2 (0x584d bytes) 2024-05-11T23:02:29.580 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:29.580 INFO:teuthology.orchestra.run.smithi005.stdout:3773 writesame 0xc76d6f thru 0xc82d34 data_size 0x1(0xbfc6 bytes) 2024-05-11T23:02:29.580 INFO:teuthology.orchestra.run.smithi005.stdout:3775 trunc from 0xcd88aee to 0x87df299 2024-05-11T23:02:29.581 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:29.581+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:29.591 INFO:teuthology.orchestra.run.smithi005.stdout:3776 trunc from 0x87df299 to 0xd8d2ecb 2024-05-11T23:02:29.597 INFO:teuthology.orchestra.run.smithi005.stdout:3777 writesame 0xc0bf6e4 thru 0xc0c3782 data_size 0x1(0x409f bytes) 2024-05-11T23:02:29.606 INFO:teuthology.orchestra.run.smithi005.stdout:3778 read 0x17acc38 thru 0x17b30d6 (0x649f bytes) 2024-05-11T23:02:29.624 INFO:teuthology.orchestra.run.smithi005.stdout:3783 trunc from 0xd8d2ecb to 0x127699 2024-05-11T23:02:29.647 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:29.648 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:29.648 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:29.648 INFO:teuthology.orchestra.run.smithi005.stdout:3787 punch from 0x51aa8 to 0x56a98, (0x4ff0 bytes) 2024-05-11T23:02:29.648 INFO:teuthology.orchestra.run.smithi005.stdout:3788 read 0xda57c thru 0xe7b04 (0xd589 bytes) 2024-05-11T23:02:29.667 INFO:teuthology.orchestra.run.smithi005.stdout:3789 write 0xb256e3a thru 0xb25d32d (0x64f4 bytes) 2024-05-11T23:02:29.669 INFO:teuthology.orchestra.run.smithi005.stdout:3790 write 0x700f248 thru 0x7010725 (0x14de bytes) 2024-05-11T23:02:29.672 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:29.672 INFO:teuthology.orchestra.run.smithi005.stdout:3794 punch from 0x5c2f9b2 to 0x5c30c1c, (0x126a bytes) 2024-05-11T23:02:29.672 INFO:teuthology.orchestra.run.smithi005.stdout:3795 punch from 0x5245f56 to 0x5246e85, (0xf2f bytes) 2024-05-11T23:02:29.672 INFO:teuthology.orchestra.run.smithi005.stdout:3796 write 0x245c509 thru 0x245db9e (0x1696 bytes) 2024-05-11T23:02:29.675 INFO:teuthology.orchestra.run.smithi005.stdout:3797 write 0x10166d2 thru 0x1022d82 (0xc6b1 bytes) 2024-05-11T23:02:29.676 INFO:teuthology.orchestra.run.smithi005.stdout:3798 write 0x3816085 thru 0x3817415 (0x1391 bytes) 2024-05-11T23:02:29.679 INFO:teuthology.orchestra.run.smithi005.stdout:3799 write 0x84fb174 thru 0x8500e8c (0x5d19 bytes) 2024-05-11T23:02:29.681 INFO:teuthology.orchestra.run.smithi005.stdout:3800 read 0xb2a509 thru 0xb317ae (0x72a6 bytes) 2024-05-11T23:02:29.681 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:29.681+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:29.690 INFO:teuthology.orchestra.run.smithi005.stdout:3801 write 0xe3e9659 thru 0xe3e9864 (0x20c bytes) 2024-05-11T23:02:29.693 INFO:teuthology.orchestra.run.smithi005.stdout:3803 writesame 0x875d2dd thru 0x87686cc data_size 0x8(0xb3f0 bytes) 2024-05-11T23:02:29.695 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:29.695 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:29.695 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:29.695 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:29.696 INFO:teuthology.orchestra.run.smithi005.stdout:3806 write 0x38e0cf2 thru 0x38e577f (0x4a8e bytes) 2024-05-11T23:02:29.697 INFO:teuthology.orchestra.run.smithi005.stdout:3807 writesame 0xa49ed8b thru 0xa4ab262 data_size 0x4(0xc4d8 bytes) 2024-05-11T23:02:29.705 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:02:29.709 INFO:teuthology.orchestra.run.smithi005.stdout:3809 trunc from 0xe3e9865 to 0xcf8c492 2024-05-11T23:02:29.732 INFO:teuthology.orchestra.run.smithi005.stdout:3810 trunc from 0xcf8c492 to 0x195b43e 2024-05-11T23:02:29.753 INFO:teuthology.orchestra.run.smithi005.stdout:3811 write 0x2a2532f thru 0x2a3368c (0xe35e bytes) 2024-05-11T23:02:29.775 INFO:teuthology.orchestra.run.smithi005.stdout:3812 writesame 0xd455b80 thru 0xd46198e data_size 0x25(0xbe0f bytes) 2024-05-11T23:02:29.775 INFO:teuthology.orchestra.run.smithi005.stdout:3813 write 0x2f1048 thru 0x2f7db9 (0x6d72 bytes) 2024-05-11T23:02:29.782 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:29.782+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:29.782 INFO:teuthology.orchestra.run.smithi005.stdout:3814 write 0xe42bca0 thru 0xe43471c (0x8a7d bytes) 2024-05-11T23:02:29.786 INFO:teuthology.orchestra.run.smithi005.stdout:3816 read 0xad23b9c thru 0xad2a2e6 (0x674b bytes) 2024-05-11T23:02:29.786 INFO:teuthology.orchestra.run.smithi005.stdout:3817 punch from 0x617f6db to 0x6186417, (0x6d3c bytes) 2024-05-11T23:02:29.786 INFO:teuthology.orchestra.run.smithi005.stdout:3818 read 0xbd31078 thru 0xbd3b851 (0xa7da bytes) 2024-05-11T23:02:29.786 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:29.792 INFO:teuthology.orchestra.run.smithi005.stdout:3820 write 0xe934fd8 thru 0xe941e30 (0xce59 bytes) 2024-05-11T23:02:29.795 INFO:teuthology.orchestra.run.smithi005.stdout:3823 punch from 0xc71ff3b to 0xc72f8a1, (0xf966 bytes) 2024-05-11T23:02:29.795 INFO:teuthology.orchestra.run.smithi005.stdout:3824 writesame 0x7754cf4 thru 0x7754ed8 data_size 0x1(0x1e5 bytes) 2024-05-11T23:02:29.796 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:29.797 INFO:teuthology.orchestra.run.smithi005.stdout:3829 write 0x751f116 thru 0x752de3f (0xed2a bytes) 2024-05-11T23:02:29.798 INFO:teuthology.orchestra.run.smithi005.stdout:3831 clone 35 order 19 su 262144 sc 5 2024-05-11T23:02:29.815 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:29.815 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:29.815 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:29.815 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:29.815 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:29.824 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:02:29.908 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:29.908+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:29.972 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:29.972 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:29.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:29.972 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:29.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.972 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67817 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:29.972 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:29.972 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.351378212, 2024-05-11T23:02:29.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044148845999999999, 2024-05-11T23:02:29.972 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67817 2024-05-11T23:02:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620227+0000", 2024-05-11T23:02:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00046792099999999999 2024-05-11T23:02:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620232+0000", 2024-05-11T23:02:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3050000000000003e-06 2024-05-11T23:02:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620243+0000", 2024-05-11T23:02:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1289e-05 2024-05-11T23:02:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620250+0000", 2024-05-11T23:02:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2130000000000002e-06 2024-05-11T23:02:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663745+0000", 2024-05-11T23:02:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043494865000000001 2024-05-11T23:02:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663815+0000", 2024-05-11T23:02:29.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0284999999999995e-05 2024-05-11T23:02:29.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663908+0000", 2024-05-11T23:02:29.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2967999999999995e-05 2024-05-11T23:02:29.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.976 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.976 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67820 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4902912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:29.976 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:29.976 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.350828705000001, 2024-05-11T23:02:29.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044096781000000002, 2024-05-11T23:02:29.977 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.977 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.977 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.977 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.977 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.977 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.977 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67820 2024-05-11T23:02:29.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.977 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:29.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:29.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620310+0000", 2024-05-11T23:02:29.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9800000000000004e-07 2024-05-11T23:02:29.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620312+0000", 2024-05-11T23:02:29.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.272e-06 2024-05-11T23:02:29.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620319+0000", 2024-05-11T23:02:29.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9310000000000002e-06 2024-05-11T23:02:29.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620323+0000", 2024-05-11T23:02:29.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:02:29.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664284+0000", 2024-05-11T23:02:29.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043960908999999999 2024-05-11T23:02:29.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664327+0000", 2024-05-11T23:02:29.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2811000000000001e-05 2024-05-11T23:02:29.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664406+0000", 2024-05-11T23:02:29.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8588e-05 2024-05-11T23:02:29.980 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.980 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.980 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.980 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67821 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4927488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:29.980 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:29.980 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.350808295, 2024-05-11T23:02:29.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044203340000000001, 2024-05-11T23:02:29.981 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.981 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.981 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.981 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.981 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.981 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.981 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67821 2024-05-11T23:02:29.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.981 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:29.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:29.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620330+0000", 2024-05-11T23:02:29.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:02:29.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620332+0000", 2024-05-11T23:02:29.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:02:29.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620338+0000", 2024-05-11T23:02:29.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6250000000000004e-06 2024-05-11T23:02:29.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620342+0000", 2024-05-11T23:02:29.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7609999999999998e-06 2024-05-11T23:02:29.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664433+0000", 2024-05-11T23:02:29.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044090455000000001 2024-05-11T23:02:29.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664476+0000", 2024-05-11T23:02:29.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2861999999999997e-05 2024-05-11T23:02:29.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664533+0000", 2024-05-11T23:02:29.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7156e-05 2024-05-11T23:02:29.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.984 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.984 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67822 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4976640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:29.984 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:29.984 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.350789256999999, 2024-05-11T23:02:29.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044409416, 2024-05-11T23:02:29.985 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.985 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.985 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.985 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67822 2024-05-11T23:02:29.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.985 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:29.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:29.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620349+0000", 2024-05-11T23:02:29.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.51e-07 2024-05-11T23:02:29.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620351+0000", 2024-05-11T23:02:29.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.181e-06 2024-05-11T23:02:29.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620358+0000", 2024-05-11T23:02:29.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3060000000000003e-06 2024-05-11T23:02:29.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620362+0000", 2024-05-11T23:02:29.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7949999999999998e-06 2024-05-11T23:02:29.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664572+0000", 2024-05-11T23:02:29.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044209318999999997 2024-05-11T23:02:29.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664654+0000", 2024-05-11T23:02:29.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2367999999999995e-05 2024-05-11T23:02:29.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664758+0000", 2024-05-11T23:02:29.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103896 2024-05-11T23:02:29.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.988 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.989 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67823 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4952064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:29.989 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:29.989 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.350769206999999, 2024-05-11T23:02:29.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044623293000000001, 2024-05-11T23:02:29.989 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.989 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.989 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.989 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67823 2024-05-11T23:02:29.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.989 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:29.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:29.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622228+0000", 2024-05-11T23:02:29.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018592280000000001 2024-05-11T23:02:29.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622232+0000", 2024-05-11T23:02:29.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.403e-06 2024-05-11T23:02:29.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622248+0000", 2024-05-11T23:02:29.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5449000000000001e-05 2024-05-11T23:02:29.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622254+0000", 2024-05-11T23:02:29.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5819999999999998e-06 2024-05-11T23:02:29.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664800+0000", 2024-05-11T23:02:29.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042546048000000003 2024-05-11T23:02:29.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664886+0000", 2024-05-11T23:02:29.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5525000000000005e-05 2024-05-11T23:02:29.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664992+0000", 2024-05-11T23:02:29.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010605800000000001 2024-05-11T23:02:29.992 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.992 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.993 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67833 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5099520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:29.993 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:29.993 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.348662734000001, 2024-05-11T23:02:29.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044076606999999997, 2024-05-11T23:02:29.993 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.993 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.993 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.993 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.993 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.993 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.993 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67833 2024-05-11T23:02:29.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.994 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:29.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:29.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:29.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8099999999999998e-07 2024-05-11T23:02:29.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622477+0000", 2024-05-11T23:02:29.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0700000000000001e-06 2024-05-11T23:02:29.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622483+0000", 2024-05-11T23:02:29.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0780000000000003e-06 2024-05-11T23:02:29.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:29.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622487+0000", 2024-05-11T23:02:29.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.352e-06 2024-05-11T23:02:29.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:29.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666460+0000", 2024-05-11T23:02:29.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043973085000000002 2024-05-11T23:02:29.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:29.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666498+0000", 2024-05-11T23:02:29.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8441e-05 2024-05-11T23:02:29.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:29.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666552+0000", 2024-05-11T23:02:29.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3199999999999999e-05 2024-05-11T23:02:29.997 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.997 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:29.997 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:29.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.997 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67834 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3944448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:29.997 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:29.997 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.348624456, 2024-05-11T23:02:29.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044166551999999998, 2024-05-11T23:02:29.997 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:29.997 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:29.997 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:29.997 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:29.998 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:29.998 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:29.998 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67834 2024-05-11T23:02:29.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.998 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:29.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:29.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:29.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:29.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:29.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:29.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:29.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622514+0000", 2024-05-11T23:02:29.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1100000000000002e-07 2024-05-11T23:02:29.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:29.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622515+0000", 2024-05-11T23:02:29.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:02:29.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:29.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:29.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:29.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622519+0000", 2024-05-11T23:02:29.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8260000000000003e-06 2024-05-11T23:02:29.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622522+0000", 2024-05-11T23:02:30.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8899999999999999e-06 2024-05-11T23:02:30.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666580+0000", 2024-05-11T23:02:30.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044058235000000001 2024-05-11T23:02:30.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666624+0000", 2024-05-11T23:02:30.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3501999999999998e-05 2024-05-11T23:02:30.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666680+0000", 2024-05-11T23:02:30.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6042999999999999e-05 2024-05-11T23:02:30.001 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.001 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.001 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.001 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67835 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:30.001 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:30.001 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.34861192, 2024-05-11T23:02:30.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044272019000000003, 2024-05-11T23:02:30.001 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.001 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.002 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.002 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.002 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.002 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.002 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67835 2024-05-11T23:02:30.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.002 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:30.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:30.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:30.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1699999999999999e-07 2024-05-11T23:02:30.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622527+0000", 2024-05-11T23:02:30.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2109999999999999e-06 2024-05-11T23:02:30.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622531+0000", 2024-05-11T23:02:30.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4910000000000001e-06 2024-05-11T23:02:30.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622534+0000", 2024-05-11T23:02:30.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7080000000000002e-06 2024-05-11T23:02:30.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666703+0000", 2024-05-11T23:02:30.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044169377000000003 2024-05-11T23:02:30.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666745+0000", 2024-05-11T23:02:30.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1616000000000003e-05 2024-05-11T23:02:30.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666798+0000", 2024-05-11T23:02:30.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3298999999999999e-05 2024-05-11T23:02:30.005 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.005 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.005 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.005 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67836 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:30.005 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:30.005 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.348601259999999, 2024-05-11T23:02:30.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044373692999999999, 2024-05-11T23:02:30.005 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.005 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.006 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.006 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.006 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.006 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.006 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67836 2024-05-11T23:02:30.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.006 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:30.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:30.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622537+0000", 2024-05-11T23:02:30.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:30.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622538+0000", 2024-05-11T23:02:30.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1510000000000001e-06 2024-05-11T23:02:30.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622541+0000", 2024-05-11T23:02:30.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:02:30.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622543+0000", 2024-05-11T23:02:30.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4870000000000001e-06 2024-05-11T23:02:30.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666821+0000", 2024-05-11T23:02:30.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044278038999999998 2024-05-11T23:02:30.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666859+0000", 2024-05-11T23:02:30.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7224000000000001e-05 2024-05-11T23:02:30.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666910+0000", 2024-05-11T23:02:30.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1629999999999999e-05 2024-05-11T23:02:30.009 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.009 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.009 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.009 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67837 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:30.009 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:30.009 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.348591533, 2024-05-11T23:02:30.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044477883000000003, 2024-05-11T23:02:30.009 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.010 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.010 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.010 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.010 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.010 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.010 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67837 2024-05-11T23:02:30.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.010 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:30.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:30.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:30.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:30.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622548+0000", 2024-05-11T23:02:30.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:02:30.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622550+0000", 2024-05-11T23:02:30.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-05-11T23:02:30.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622553+0000", 2024-05-11T23:02:30.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.43e-06 2024-05-11T23:02:30.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666932+0000", 2024-05-11T23:02:30.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044379584 2024-05-11T23:02:30.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666971+0000", 2024-05-11T23:02:30.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8427000000000003e-05 2024-05-11T23:02:30.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667024+0000", 2024-05-11T23:02:30.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3226000000000002e-05 2024-05-11T23:02:30.013 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.013 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.013 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.013 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67838 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:30.014 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:30.014 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.348582016000002, 2024-05-11T23:02:30.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044580374999999998, 2024-05-11T23:02:30.014 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.014 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.014 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.014 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.014 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.014 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.014 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67838 2024-05-11T23:02:30.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.014 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:30.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:30.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:30.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:02:30.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622557+0000", 2024-05-11T23:02:30.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.102e-06 2024-05-11T23:02:30.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622560+0000", 2024-05-11T23:02:30.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:02:30.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622562+0000", 2024-05-11T23:02:30.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:02:30.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667045+0000", 2024-05-11T23:02:30.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044482936000000001 2024-05-11T23:02:30.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667083+0000", 2024-05-11T23:02:30.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7443e-05 2024-05-11T23:02:30.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667136+0000", 2024-05-11T23:02:30.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3257000000000002e-05 2024-05-11T23:02:30.017 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.017 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.017 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.018 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67839 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5394432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:30.018 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:30.018 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.348572273999999, 2024-05-11T23:02:30.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04470263, 2024-05-11T23:02:30.018 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.018 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.018 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.018 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.018 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.018 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.018 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67839 2024-05-11T23:02:30.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.018 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:30.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:30.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622566+0000", 2024-05-11T23:02:30.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-05-11T23:02:30.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622572+0000", 2024-05-11T23:02:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5610000000000004e-06 2024-05-11T23:02:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622576+0000", 2024-05-11T23:02:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3210000000000001e-06 2024-05-11T23:02:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622578+0000", 2024-05-11T23:02:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4710000000000001e-06 2024-05-11T23:02:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667160+0000", 2024-05-11T23:02:30.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044582220999999998 2024-05-11T23:02:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667205+0000", 2024-05-11T23:02:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5049000000000002e-05 2024-05-11T23:02:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667268+0000", 2024-05-11T23:02:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2788999999999998e-05 2024-05-11T23:02:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.021 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67840 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5419008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.348556643999999, 2024-05-11T23:02:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044796006999999999, 2024-05-11T23:02:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.022 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67840 2024-05-11T23:02:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:30.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:02:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622583+0000", 2024-05-11T23:02:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4389999999999999e-06 2024-05-11T23:02:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622586+0000", 2024-05-11T23:02:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:02:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622588+0000", 2024-05-11T23:02:30.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4540000000000001e-06 2024-05-11T23:02:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667287+0000", 2024-05-11T23:02:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044698882000000002 2024-05-11T23:02:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667324+0000", 2024-05-11T23:02:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7196000000000001e-05 2024-05-11T23:02:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667377+0000", 2024-05-11T23:02:30.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2927999999999999e-05 2024-05-11T23:02:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67841 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5443584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.348546871, 2024-05-11T23:02:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044907337999999998, 2024-05-11T23:02:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.026 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67841 2024-05-11T23:02:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622592+0000", 2024-05-11T23:02:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-05-11T23:02:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622595+0000", 2024-05-11T23:02:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:02:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622598+0000", 2024-05-11T23:02:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5909999999999998e-06 2024-05-11T23:02:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667401+0000", 2024-05-11T23:02:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044802980999999999 2024-05-11T23:02:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667439+0000", 2024-05-11T23:02:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8840000000000001e-05 2024-05-11T23:02:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667498+0000", 2024-05-11T23:02:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8709000000000003e-05 2024-05-11T23:02:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67842 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5468160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.348537264000001, 2024-05-11T23:02:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045005876, 2024-05-11T23:02:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67842 2024-05-11T23:02:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622601+0000", 2024-05-11T23:02:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622602+0000", 2024-05-11T23:02:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0780000000000001e-06 2024-05-11T23:02:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622605+0000", 2024-05-11T23:02:30.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.914e-06 2024-05-11T23:02:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622607+0000", 2024-05-11T23:02:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:02:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667516+0000", 2024-05-11T23:02:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044909156999999998 2024-05-11T23:02:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667554+0000", 2024-05-11T23:02:30.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7920000000000003e-05 2024-05-11T23:02:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667606+0000", 2024-05-11T23:02:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2111000000000003e-05 2024-05-11T23:02:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67843 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4067328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.348527738000001, 2024-05-11T23:02:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045119014999999998, 2024-05-11T23:02:30.034 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67843 2024-05-11T23:02:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624228+0000", 2024-05-11T23:02:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001617867 2024-05-11T23:02:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624232+0000", 2024-05-11T23:02:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-05-11T23:02:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624240+0000", 2024-05-11T23:02:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4589999999999998e-06 2024-05-11T23:02:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624247+0000", 2024-05-11T23:02:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-05-11T23:02:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667629+0000", 2024-05-11T23:02:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043382012999999997 2024-05-11T23:02:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667674+0000", 2024-05-11T23:02:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5172000000000001e-05 2024-05-11T23:02:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667729+0000", 2024-05-11T23:02:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5086000000000001e-05 2024-05-11T23:02:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67849 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4190208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:30.038 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.346769617, 2024-05-11T23:02:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044061433999999997, 2024-05-11T23:02:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67849 2024-05-11T23:02:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624369+0000", 2024-05-11T23:02:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5900000000000002e-07 2024-05-11T23:02:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624371+0000", 2024-05-11T23:02:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0310000000000001e-06 2024-05-11T23:02:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624376+0000", 2024-05-11T23:02:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4029999999999998e-06 2024-05-11T23:02:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624381+0000", 2024-05-11T23:02:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4349999999999996e-06 2024-05-11T23:02:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668331+0000", 2024-05-11T23:02:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043949191999999998 2024-05-11T23:02:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668370+0000", 2024-05-11T23:02:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9227999999999998e-05 2024-05-11T23:02:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668430+0000", 2024-05-11T23:02:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9685999999999997e-05 2024-05-11T23:02:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67850 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.346748385000001, 2024-05-11T23:02:30.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044151953000000001, 2024-05-11T23:02:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67850 2024-05-11T23:02:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624390+0000", 2024-05-11T23:02:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:02:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624392+0000", 2024-05-11T23:02:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1880000000000001e-06 2024-05-11T23:02:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624399+0000", 2024-05-11T23:02:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7390000000000002e-06 2024-05-11T23:02:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624405+0000", 2024-05-11T23:02:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1059999999999996e-06 2024-05-11T23:02:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668450+0000", 2024-05-11T23:02:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044045147999999999 2024-05-11T23:02:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668487+0000", 2024-05-11T23:02:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7450000000000002e-05 2024-05-11T23:02:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668541+0000", 2024-05-11T23:02:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3952000000000001e-05 2024-05-11T23:02:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67851 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5541888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:30.046 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.346721803000001, 2024-05-11T23:02:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044264413000000002, 2024-05-11T23:02:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67851 2024-05-11T23:02:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-05-11T23:02:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624419+0000", 2024-05-11T23:02:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.21e-06 2024-05-11T23:02:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624424+0000", 2024-05-11T23:02:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9050000000000002e-06 2024-05-11T23:02:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624430+0000", 2024-05-11T23:02:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-05-11T23:02:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668566+0000", 2024-05-11T23:02:30.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044136208000000003 2024-05-11T23:02:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668614+0000", 2024-05-11T23:02:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8022000000000002e-05 2024-05-11T23:02:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668680+0000", 2024-05-11T23:02:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6181999999999998e-05 2024-05-11T23:02:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67852 3.16 3.1a4a4af6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.346701335999999, 2024-05-11T23:02:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044278287999999999, 2024-05-11T23:02:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67852 2024-05-11T23:02:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626226+0000", 2024-05-11T23:02:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017893 2024-05-11T23:02:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626229+0000", 2024-05-11T23:02:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0709999999999999e-06 2024-05-11T23:02:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626239+0000", 2024-05-11T23:02:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0507e-05 2024-05-11T23:02:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626244+0000", 2024-05-11T23:02:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4880000000000001e-06 2024-05-11T23:02:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668706+0000", 2024-05-11T23:02:30.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042462054999999999 2024-05-11T23:02:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668715+0000", 2024-05-11T23:02:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.867e-06 2024-05-11T23:02:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.054 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.054 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:30.056 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-05-11T23:02:30.056 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:02:30.057 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:30.055+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:30.156 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:30.156+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:30.202 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:30.202 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:30.202 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:30.202 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:30.210 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:02:30.257 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:30.257+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:30.357 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:30.357 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:30.357 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:30.357 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:30.357 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:30.358 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:30.358+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:30.366 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:02:30.459 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:30.459+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:30.515 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:30.515 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.860949929999997, 2024-05-11T23:02:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:02:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.516 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:02:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:02:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:02:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:02:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:02:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:02:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:02:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:02:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:02:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:02:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:02:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:02:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:02:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:02:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.857568741000001, 2024-05-11T23:02:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:02:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.520 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:02:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:02:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:02:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:02:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:02:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:02:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:02:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:02:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:02:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:02:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:02:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:02:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:02:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:02:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:30.524 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.857550033999999, 2024-05-11T23:02:30.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:02:30.525 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.525 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.525 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.525 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.525 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.525 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.525 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:02:30.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.525 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:30.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:30.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:02:30.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:02:30.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:02:30.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:02:30.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:02:30.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:02:30.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:02:30.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:02:30.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:02:30.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:02:30.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:02:30.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:02:30.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:02:30.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:02:30.528 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.528 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.528 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.529 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:30.529 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:30.529 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.857531891000001, 2024-05-11T23:02:30.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:02:30.529 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.529 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.529 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.529 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.529 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.529 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.529 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:02:30.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.529 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:30.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:30.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:30.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:02:30.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:02:30.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:02:30.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:02:30.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:02:30.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:02:30.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:02:30.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:02:30.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:02:30.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:02:30.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:02:30.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:02:30.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:02:30.533 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.533 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.533 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.533 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:30.533 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:30.533 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.857504403999997, 2024-05-11T23:02:30.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:02:30.533 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.533 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.533 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.533 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.533 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.533 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.534 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:02:30.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.534 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:30.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:30.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:02:30.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:02:30.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:02:30.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:02:30.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:02:30.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:02:30.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:02:30.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:02:30.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:02:30.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:02:30.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:02:30.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:02:30.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:02:30.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:02:30.537 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.537 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.537 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.537 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:30.537 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:30.537 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.857483876000003, 2024-05-11T23:02:30.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:02:30.537 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.537 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.537 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.537 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.538 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.538 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.538 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:02:30.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.538 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:30.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:30.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:30.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:02:30.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:02:30.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:02:30.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:02:30.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:02:30.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:02:30.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:02:30.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:02:30.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:02:30.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:02:30.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:02:30.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:02:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:02:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.857464100999998, 2024-05-11T23:02:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:02:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.541 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:02:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:02:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:02:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:02:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:02:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:02:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:02:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:02:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:02:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:02:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:02:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:02:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:02:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:02:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:02:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.857445294999998, 2024-05-11T23:02:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:02:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:02:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:02:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:02:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:02:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:02:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:02:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:02:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:02:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:02:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:02:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:02:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:02:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:02:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:02:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:02:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.857426371999999, 2024-05-11T23:02:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:02:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:02:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:02:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:02:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:02:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:02:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:02:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:02:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:02:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:02:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:02:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:02:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:02:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:02:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:02:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.857407709999997, 2024-05-11T23:02:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:02:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:02:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:02:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:02:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:02:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:02:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:02:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:02:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:02:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:02:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:02:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:02:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:02:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:02:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:02:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.857062548000002, 2024-05-11T23:02:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:02:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:02:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:02:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:02:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:02:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:02:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:02:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:02:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:02:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:02:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:02:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:02:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:02:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:02:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:02:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:02:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.857036997999998, 2024-05-11T23:02:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:02:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:02:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:02:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:02:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:02:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:02:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:02:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:02:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:02:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:02:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:02:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:02:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:02:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:02:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:02:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.856954739999999, 2024-05-11T23:02:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:02:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:02:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:02:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:02:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:02:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:02:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:02:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:02:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:02:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:02:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:02:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:02:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:02:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:02:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:02:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.856932293, 2024-05-11T23:02:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:02:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:02:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:02:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:02:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:02:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:02:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:02:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:02:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:02:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:02:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:02:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:02:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:02:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:02:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:02:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:02:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.856912835000003, 2024-05-11T23:02:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:02:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:02:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:02:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:02:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:02:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:02:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:02:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:02:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:02:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:02:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:02:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:02:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:02:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:02:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:02:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.856892774999999, 2024-05-11T23:02:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:02:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:02:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:02:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:02:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:02:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:02:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:02:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:02:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:02:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:02:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:02:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:02:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:02:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:02:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:02:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.856873505000003, 2024-05-11T23:02:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:02:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:02:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:02:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:02:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:02:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:02:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:02:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:02:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:02:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:02:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:02:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:02:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:02:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:02:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:02:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.856327438999998, 2024-05-11T23:02:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:02:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:02:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:02:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:02:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:02:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:02:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:02:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:02:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:02:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:02:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:02:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:02:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:02:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:02:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:02:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:02:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.856295654999997, 2024-05-11T23:02:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:02:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:02:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:02:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:02:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:02:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:02:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:02:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:02:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:02:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:02:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:02:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:02:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:02:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:02:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:02:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:02:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.419050191, 2024-05-11T23:02:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:02:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:02:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:02:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:02:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:02:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:02:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:02:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:02:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:02:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:02:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:02:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:02:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:02:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:02:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:02:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:02:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:02:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:02:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:02:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:02:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:02:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:02:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:02:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:02:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:02:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.601 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.601 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.601 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.601 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:30.602 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:02:30.603 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:30.602+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:30.703 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:30.703+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:30.749 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:30.749 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:30.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.749 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:105234 3.15 3.a7a11fb5 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-11T23:02:30.749 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:02:30.749 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.74199865200000004, 2024-05-11T23:02:30.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.74201980199999995, 2024-05-11T23:02:30.749 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:30.749 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:30.749 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for peered", 2024-05-11T23:02:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 105234 2024-05-11T23:02:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:02:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:02:30.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:30.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:30.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007623+0000", 2024-05-11T23:02:30.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2240000000000001e-06 2024-05-11T23:02:30.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:30.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007627+0000", 2024-05-11T23:02:30.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:02:30.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:30.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007636+0000", 2024-05-11T23:02:30.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2409999999999995e-06 2024-05-11T23:02:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007641+0000", 2024-05-11T23:02:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6810000000000003e-06 2024-05-11T23:02:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007676+0000", 2024-05-11T23:02:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5061999999999997e-05 2024-05-11T23:02:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:02:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007682+0000", 2024-05-11T23:02:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0399999999999998e-06 2024-05-11T23:02:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.752 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:30.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.753 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:30.753 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-11T23:02:30.753 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 1 2024-05-11T23:02:30.753 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:30.757 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:02:30.804 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:30.804+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:30.902 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:30.902 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:30.902 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:30.902 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:30.902 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:30.904 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:30.904+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:30.910 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:02:30.959 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:02:31.005 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:31.005+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:31.020 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:02:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.855669685999999, 2024-05-11T23:02:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:02:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:31.020 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:02:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:02:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:02:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:02:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:02:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:02:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:02:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:02:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:02:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:02:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:02:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:02:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:02:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:02:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:02:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:02:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:02:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:02:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:02:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:02:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:02:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:02:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:02:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:02:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:02:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:02:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:02:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:31.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:02:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:02:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:02:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:02:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.026 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78429 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:02:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.054400343999999, 2024-05-11T23:02:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044563497000000001, 2024-05-11T23:02:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78429 2024-05-11T23:02:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:31.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:02:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:02:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964872+0000", 2024-05-11T23:02:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035070300000000001 2024-05-11T23:02:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964876+0000", 2024-05-11T23:02:31.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5009999999999998e-06 2024-05-11T23:02:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964902+0000", 2024-05-11T23:02:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5715999999999999e-05 2024-05-11T23:02:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964910+0000", 2024-05-11T23:02:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3930000000000002e-06 2024-05-11T23:02:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008898+0000", 2024-05-11T23:02:31.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043988772000000002 2024-05-11T23:02:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008981+0000", 2024-05-11T23:02:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3140999999999996e-05 2024-05-11T23:02:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009085+0000", 2024-05-11T23:02:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010327099999999999 2024-05-11T23:02:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78431 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:02:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.053940036, 2024-05-11T23:02:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044542033000000002, 2024-05-11T23:02:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78431 2024-05-11T23:02:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:02:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:02:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965038+0000", 2024-05-11T23:02:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6993000000000001e-05 2024-05-11T23:02:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965043+0000", 2024-05-11T23:02:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7040000000000002e-06 2024-05-11T23:02:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965053+0000", 2024-05-11T23:02:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8539999999999992e-06 2024-05-11T23:02:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965061+0000", 2024-05-11T23:02:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9810000000000003e-06 2024-05-11T23:02:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009350+0000", 2024-05-11T23:02:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044288739000000001 2024-05-11T23:02:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009429+0000", 2024-05-11T23:02:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9354e-05 2024-05-11T23:02:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009524+0000", 2024-05-11T23:02:31.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4407999999999995e-05 2024-05-11T23:02:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78432 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:02:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.053849187999999, 2024-05-11T23:02:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044635421000000002, 2024-05-11T23:02:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:31.035 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78432 2024-05-11T23:02:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:02:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:31.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:02:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965343+0000", 2024-05-11T23:02:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027056500000000001 2024-05-11T23:02:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965347+0000", 2024-05-11T23:02:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3540000000000002e-06 2024-05-11T23:02:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965356+0000", 2024-05-11T23:02:31.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2140000000000002e-06 2024-05-11T23:02:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965363+0000", 2024-05-11T23:02:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4150000000000004e-06 2024-05-11T23:02:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009551+0000", 2024-05-11T23:02:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044187917 2024-05-11T23:02:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009626+0000", 2024-05-11T23:02:31.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5024000000000005e-05 2024-05-11T23:02:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009708+0000", 2024-05-11T23:02:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1932000000000001e-05 2024-05-11T23:02:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78433 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:02:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.053548300999999, 2024-05-11T23:02:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044564864000000003, 2024-05-11T23:02:31.039 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78433 2024-05-11T23:02:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:02:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:31.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:02:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966225+0000", 2024-05-11T23:02:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085220700000000001 2024-05-11T23:02:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966230+0000", 2024-05-11T23:02:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0690000000000003e-06 2024-05-11T23:02:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966239+0000", 2024-05-11T23:02:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-05-11T23:02:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966245+0000", 2024-05-11T23:02:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5520000000000001e-06 2024-05-11T23:02:31.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009743+0000", 2024-05-11T23:02:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043498311999999997 2024-05-11T23:02:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009848+0000", 2024-05-11T23:02:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104885 2024-05-11T23:02:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009938+0000", 2024-05-11T23:02:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9812e-05 2024-05-11T23:02:31.043 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78570 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.966350+0000", 2024-05-11T23:02:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.052571741, 2024-05-11T23:02:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044456832000000002, 2024-05-11T23:02:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78570 2024-05-11T23:02:31.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966350+0000", 2024-05-11T23:02:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966350+0000", 2024-05-11T23:02:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968227+0000", 2024-05-11T23:02:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018775969999999999 2024-05-11T23:02:31.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968232+0000", 2024-05-11T23:02:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7190000000000001e-06 2024-05-11T23:02:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968243+0000", 2024-05-11T23:02:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1105e-05 2024-05-11T23:02:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968251+0000", 2024-05-11T23:02:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2470000000000008e-06 2024-05-11T23:02:31.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.010665+0000", 2024-05-11T23:02:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042413803 2024-05-11T23:02:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.010748+0000", 2024-05-11T23:02:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2676999999999995e-05 2024-05-11T23:02:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.010807+0000", 2024-05-11T23:02:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8684000000000003e-05 2024-05-11T23:02:31.047 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78580 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 999424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:02:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.050435618, 2024-05-11T23:02:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044493076999999999, 2024-05-11T23:02:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78580 2024-05-11T23:02:31.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:02:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:02:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:02:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:02:31.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968489+0000", 2024-05-11T23:02:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.52e-06 2024-05-11T23:02:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968496+0000", 2024-05-11T23:02:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9419999999999998e-06 2024-05-11T23:02:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968501+0000", 2024-05-11T23:02:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8720000000000001e-06 2024-05-11T23:02:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012690+0000", 2024-05-11T23:02:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044189655000000001 2024-05-11T23:02:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012848+0000", 2024-05-11T23:02:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015758100000000001 2024-05-11T23:02:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012979+0000", 2024-05-11T23:02:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000131066 2024-05-11T23:02:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.051 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78581 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:02:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.050414613999999, 2024-05-11T23:02:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044616243, 2024-05-11T23:02:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78581 2024-05-11T23:02:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.052 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:02:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:02:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970227+0000", 2024-05-11T23:02:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017200049999999999 2024-05-11T23:02:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970232+0000", 2024-05-11T23:02:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000003e-06 2024-05-11T23:02:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970242+0000", 2024-05-11T23:02:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0192e-05 2024-05-11T23:02:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970249+0000", 2024-05-11T23:02:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.888e-06 2024-05-11T23:02:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013008+0000", 2024-05-11T23:02:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042759495000000002 2024-05-11T23:02:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013063+0000", 2024-05-11T23:02:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5071999999999997e-05 2024-05-11T23:02:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013123+0000", 2024-05-11T23:02:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9930999999999999e-05 2024-05-11T23:02:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78588 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 876544~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970382+0000", 2024-05-11T23:02:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.048539435, 2024-05-11T23:02:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044394415999999999, 2024-05-11T23:02:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78588 2024-05-11T23:02:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970382+0000", 2024-05-11T23:02:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970382+0000", 2024-05-11T23:02:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970382+0000", 2024-05-11T23:02:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8599999999999999e-07 2024-05-11T23:02:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970385+0000", 2024-05-11T23:02:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1220000000000002e-06 2024-05-11T23:02:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970390+0000", 2024-05-11T23:02:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5350000000000004e-06 2024-05-11T23:02:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970395+0000", 2024-05-11T23:02:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8709999999999996e-06 2024-05-11T23:02:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014537+0000", 2024-05-11T23:02:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044141996000000003 2024-05-11T23:02:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:31.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014659+0000", 2024-05-11T23:02:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000121698 2024-05-11T23:02:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014777+0000", 2024-05-11T23:02:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011780799999999999 2024-05-11T23:02:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78589 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:02:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.048495369999999, 2024-05-11T23:02:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044587697000000003, 2024-05-11T23:02:31.060 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78589 2024-05-11T23:02:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:02:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:02:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970427+0000", 2024-05-11T23:02:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:02:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970429+0000", 2024-05-11T23:02:31.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3800000000000001e-06 2024-05-11T23:02:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970435+0000", 2024-05-11T23:02:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2530000000000001e-06 2024-05-11T23:02:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970441+0000", 2024-05-11T23:02:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-05-11T23:02:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014822+0000", 2024-05-11T23:02:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04438168 2024-05-11T23:02:31.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014911+0000", 2024-05-11T23:02:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8882999999999996e-05 2024-05-11T23:02:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015014+0000", 2024-05-11T23:02:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010263399999999999 2024-05-11T23:02:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78590 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:31.064 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:02:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.048474571, 2024-05-11T23:02:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044880120000000003, 2024-05-11T23:02:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78590 2024-05-11T23:02:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:02:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:02:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970918+0000", 2024-05-11T23:02:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000470834 2024-05-11T23:02:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970925+0000", 2024-05-11T23:02:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1690000000000003e-06 2024-05-11T23:02:31.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970934+0000", 2024-05-11T23:02:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9099999999999994e-06 2024-05-11T23:02:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970941+0000", 2024-05-11T23:02:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7410000000000003e-06 2024-05-11T23:02:31.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015049+0000", 2024-05-11T23:02:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044108392000000003 2024-05-11T23:02:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015133+0000", 2024-05-11T23:02:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3568999999999999e-05 2024-05-11T23:02:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015327+0000", 2024-05-11T23:02:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019450500000000001 2024-05-11T23:02:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78591 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:02:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.047971155000001, 2024-05-11T23:02:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044618497, 2024-05-11T23:02:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:31.069 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78591 2024-05-11T23:02:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:02:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:02:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970951+0000", 2024-05-11T23:02:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:02:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970953+0000", 2024-05-11T23:02:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:02:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970956+0000", 2024-05-11T23:02:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-05-11T23:02:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970959+0000", 2024-05-11T23:02:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9409999999999999e-06 2024-05-11T23:02:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015367+0000", 2024-05-11T23:02:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044407621000000001 2024-05-11T23:02:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015453+0000", 2024-05-11T23:02:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6685000000000001e-05 2024-05-11T23:02:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015569+0000", 2024-05-11T23:02:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115396 2024-05-11T23:02:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78592 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:02:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.047958347, 2024-05-11T23:02:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044831546999999999, 2024-05-11T23:02:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:31.073 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78592 2024-05-11T23:02:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:02:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:02:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972226+0000", 2024-05-11T23:02:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00126246 2024-05-11T23:02:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972230+0000", 2024-05-11T23:02:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1860000000000002e-06 2024-05-11T23:02:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972237+0000", 2024-05-11T23:02:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:02:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972241+0000", 2024-05-11T23:02:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.284e-06 2024-05-11T23:02:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015611+0000", 2024-05-11T23:02:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043369461999999998 2024-05-11T23:02:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015695+0000", 2024-05-11T23:02:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4458999999999999e-05 2024-05-11T23:02:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015795+0000", 2024-05-11T23:02:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9475999999999994e-05 2024-05-11T23:02:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78593 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:02:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.046671428, 2024-05-11T23:02:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046045214000000001, 2024-05-11T23:02:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:31.077 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78593 2024-05-11T23:02:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:02:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:02:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:02:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7599999999999998e-07 2024-05-11T23:02:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972252+0000", 2024-05-11T23:02:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7349999999999999e-06 2024-05-11T23:02:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972256+0000", 2024-05-11T23:02:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5659999999999998e-06 2024-05-11T23:02:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972259+0000", 2024-05-11T23:02:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8040000000000002e-06 2024-05-11T23:02:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018088+0000", 2024-05-11T23:02:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045829316000000002 2024-05-11T23:02:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018179+0000", 2024-05-11T23:02:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1299999999999997e-05 2024-05-11T23:02:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018295+0000", 2024-05-11T23:02:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116117 2024-05-11T23:02:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78594 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:02:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.046659068, 2024-05-11T23:02:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046245204999999998, 2024-05-11T23:02:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:31.081 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78594 2024-05-11T23:02:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:02:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:02:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972263+0000", 2024-05-11T23:02:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:02:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972264+0000", 2024-05-11T23:02:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.206e-06 2024-05-11T23:02:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972267+0000", 2024-05-11T23:02:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0690000000000002e-06 2024-05-11T23:02:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972270+0000", 2024-05-11T23:02:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5610000000000001e-06 2024-05-11T23:02:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018330+0000", 2024-05-11T23:02:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04606085 2024-05-11T23:02:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018414+0000", 2024-05-11T23:02:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3125000000000001e-05 2024-05-11T23:02:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:31.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018508+0000", 2024-05-11T23:02:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4153000000000005e-05 2024-05-11T23:02:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78595 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:02:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.046646203, 2024-05-11T23:02:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046396698, 2024-05-11T23:02:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:31.085 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78595 2024-05-11T23:02:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:02:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:02:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972276+0000", 2024-05-11T23:02:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5600000000000002e-07 2024-05-11T23:02:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972277+0000", 2024-05-11T23:02:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.576e-06 2024-05-11T23:02:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972280+0000", 2024-05-11T23:02:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-05-11T23:02:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972283+0000", 2024-05-11T23:02:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:02:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018538+0000", 2024-05-11T23:02:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046255255000000002 2024-05-11T23:02:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018605+0000", 2024-05-11T23:02:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6847000000000006e-05 2024-05-11T23:02:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018672+0000", 2024-05-11T23:02:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7045000000000006e-05 2024-05-11T23:02:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78596 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:02:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.046635353999999, 2024-05-11T23:02:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046586806000000001, 2024-05-11T23:02:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:31.089 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78596 2024-05-11T23:02:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:02:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:02:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:02:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:02:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972288+0000", 2024-05-11T23:02:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2550000000000001e-06 2024-05-11T23:02:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972291+0000", 2024-05-11T23:02:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9490000000000001e-06 2024-05-11T23:02:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972293+0000", 2024-05-11T23:02:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5069999999999999e-06 2024-05-11T23:02:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018707+0000", 2024-05-11T23:02:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046413604999999997 2024-05-11T23:02:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018782+0000", 2024-05-11T23:02:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5556000000000006e-05 2024-05-11T23:02:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018873+0000", 2024-05-11T23:02:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0703000000000002e-05 2024-05-11T23:02:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78597 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:02:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.046625581000001, 2024-05-11T23:02:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046825787000000001, 2024-05-11T23:02:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:31.093 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78597 2024-05-11T23:02:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:02:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:02:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:02:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-05-11T23:02:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972297+0000", 2024-05-11T23:02:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3069999999999999e-06 2024-05-11T23:02:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972300+0000", 2024-05-11T23:02:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:02:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972303+0000", 2024-05-11T23:02:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5509999999999998e-06 2024-05-11T23:02:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018911+0000", 2024-05-11T23:02:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046608194999999998 2024-05-11T23:02:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018986+0000", 2024-05-11T23:02:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5360999999999999e-05 2024-05-11T23:02:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:31.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019122+0000", 2024-05-11T23:02:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013526 2024-05-11T23:02:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78598 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:02:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 15.046615681, 2024-05-11T23:02:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046998572000000002, 2024-05-11T23:02:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:31.097 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78598 2024-05-11T23:02:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:02:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:02:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:02:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:02:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972307+0000", 2024-05-11T23:02:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1799999999999999e-06 2024-05-11T23:02:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972310+0000", 2024-05-11T23:02:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:02:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972313+0000", 2024-05-11T23:02:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4760000000000001e-06 2024-05-11T23:02:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019149+0000", 2024-05-11T23:02:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046836056000000001 2024-05-11T23:02:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019203+0000", 2024-05-11T23:02:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-05 2024-05-11T23:02:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019304+0000", 2024-05-11T23:02:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101537 2024-05-11T23:02:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:105234 3.15 3:adf885e5:::rbd_object_map.111e913a615f.0000000000000025:head [writefull 0~39] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-11T23:02:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:02:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 1.0112999709999999, 2024-05-11T23:02:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77797599100000003, 2024-05-11T23:02:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:31.101 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 105234 2024-05-11T23:02:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:02:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:02:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:31.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007623+0000", 2024-05-11T23:02:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2240000000000001e-06 2024-05-11T23:02:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007627+0000", 2024-05-11T23:02:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:02:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007636+0000", 2024-05-11T23:02:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2409999999999995e-06 2024-05-11T23:02:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007641+0000", 2024-05-11T23:02:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6810000000000003e-06 2024-05-11T23:02:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007676+0000", 2024-05-11T23:02:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5061999999999997e-05 2024-05-11T23:02:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:02:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007682+0000", 2024-05-11T23:02:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0399999999999998e-06 2024-05-11T23:02:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783880+0000", 2024-05-11T23:02:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77619752900000005 2024-05-11T23:02:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783995+0000", 2024-05-11T23:02:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011550800000000001 2024-05-11T23:02:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:02:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784082+0000", 2024-05-11T23:02:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6347999999999994e-05 2024-05-11T23:02:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:31.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784682+0000", 2024-05-11T23:02:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000600162 2024-05-11T23:02:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785539+0000", 2024-05-11T23:02:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085699099999999998 2024-05-11T23:02:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785572+0000", 2024-05-11T23:02:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3182999999999997e-05 2024-05-11T23:02:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:31.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785598+0000", 2024-05-11T23:02:31.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5522000000000001e-05 2024-05-11T23:02:31.107 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.107 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.107 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.107 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:31.107 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:31.107 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:31.108 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:31.108 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:31.109 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:31.108+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:31.109 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:02:31.110 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:31.128 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:31.145 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:31.164 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:31.164 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:31.182 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:31.182 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:31.200 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:31.200 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:31.209 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:31.210+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:31.217 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:31.218 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:31.222 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:31.222 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:31.231 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:02:31.310 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:31.310+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:31.381 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:31.381 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:31.390 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:02:31.433 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:31.433+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:31.500 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:31.500 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:31.509 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:02:31.513 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:31.513+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:31.614 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:31.614+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:31.659 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:31.660 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:31.671 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:02:31.716 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:31.715+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:31.786 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:31.787 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:31.798 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:02:31.815 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone34 from 0xe941e31 (overlap 0x127699) to 0xcec9e 2024-05-11T23:02:31.816 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:31.816+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:31.910 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-05-11T23:02:31.922 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:31.922+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:31.928 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:31.929 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:31.944 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:02:32.012 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 3] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:02:32.012 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:02:32.013 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T23:02:32.017 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:32.017+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:32.058 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-11T23:02:32.083 INFO:teuthology.orchestra.run.smithi005.stdout:3832 clone 36 order 20 su 8192 sc 5 2024-05-11T23:02:32.119 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:32.119+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:32.126 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:32.126 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:32.136 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:02:32.220 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:32.220+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:32.287 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:32.288 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:32.297 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:02:32.338 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:32.338+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:02:32.348 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-11T23:02:32.359 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-05-11T23:02:31.776181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1277397632,"num_objects":1730,"num_object_clones":464,"num_object_copies":3460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1730,"num_whiteouts":766,"num_read":132337,"num_read_kb":696099,"num_write":12351,"num_write_kb":383841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1524,"num_bytes_recovered":136059230,"num_keys_recovered":226,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10184,"ondisk_log_size":10184,"up":82,"acting":79,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":308072,"kb_used_data":71748,"kb_used_omap":10,"kb_used_meta":236213,"kb_avail":660294808,"statfs":{"total":676457349120,"available":676141883392,"internally_reserved":0,"allocated":73469952,"data_stored":802070258,"data_compressed":42803887,"data_compressed_allocated":46632960,"data_compressed_original":781242041,"omap_allocated":11158,"internal_metadata":241882218},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[13],"upper_bound":2},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":102389757,"num_objects":134,"num_object_clones":38,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-18,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":70,"num_read":6077,"num_read_kb":71836,"num_write":664,"num_write_kb":54974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":617,"num_bytes_recovered":59424681,"num_keys_recovered":82,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.786904"},"pg_stats":[{"pgid":"3.1f","version":"72'246","reported_seq":3163,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-11T23:02:30.002322+0000","last_change":"2024-05-11T23:01:29.462665+0000","last_active":"2024-05-11T23:02:30.002322+0000","last_peered":"2024-05-11T23:02:30.002322+0000","last_clean":"2024-05-11T23:02:30.002322+0000","last_became_active":"2024-05-11T23:01:27.459623+0000","last_became_peered":"2024-05-11T23:01:27.459623+0000","last_unstale":"2024-05-11T23:02:30.002322+0000","last_undegraded":"2024-05-11T23:02:30.002322+0000","last_fullsized":"2024-05-11T23:02:30.002322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:41:53.138078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096814299999999998,"stat_sum":{"num_bytes":20256460,"num_objects":55,"num_object_clones":15,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":28,"num_read":3767,"num_read_kb":11030,"num_write":243,"num_write_kb":16776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"72'366","reported_seq":3381,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-11T23:02:30.784663+0000","last_change":"2024-05-11T23:02:30.783716+0000","last_active":"2024-05-11T23:02:30.784663+0000","last_peered":"2024-05-11T23:02:30.784663+0000","last_clean":"2024-05-11T23:02:30.784663+0000","last_became_active":"2024-05-11T23:02:30.782432+0000","last_became_peered":"2024-05-11T23:02:30.782432+0000","last_unstale":"2024-05-11T23:02:30.784663+0000","last_undegraded":"2024-05-11T23:02:30.784663+0000","last_fullsized":"2024-05-11T23:02:30.784663+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:21:52.661180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062673000000000004,"stat_sum":{"num_bytes":55453569,"num_objects":60,"num_object_clones":18,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":28,"num_read":3719,"num_read_kb":40172,"num_write":409,"num_write_kb":18162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":37039436,"num_keys_recovered":72,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"72'190","reported_seq":1728,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T23:02:27.992026+0000","last_change":"2024-05-11T23:01:29.474888+0000","last_active":"2024-05-11T23:02:27.992026+0000","last_peered":"2024-05-11T23:02:27.992026+0000","last_clean":"2024-05-11T23:02:27.992026+0000","last_became_active":"2024-05-11T23:01:27.458054+0000","last_became_peered":"2024-05-11T23:01:27.458054+0000","last_unstale":"2024-05-11T23:02:27.992026+0000","last_undegraded":"2024-05-11T23:02:27.992026+0000","last_fullsized":"2024-05-11T23:02:27.992026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:16:25.547925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001000658,"stat_sum":{"num_bytes":25006204,"num_objects":39,"num_object_clones":9,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":23,"num_read":1490,"num_read_kb":11915,"num_write":182,"num_write_kb":8333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"72'311","reported_seq":5786,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T23:02:27.992015+0000","last_change":"2024-05-11T23:01:29.474685+0000","last_active":"2024-05-11T23:02:27.992015+0000","last_peered":"2024-05-11T23:02:27.992015+0000","last_clean":"2024-05-11T23:02:27.992015+0000","last_became_active":"2024-05-11T23:01:27.463341+0000","last_became_peered":"2024-05-11T23:01:27.463341+0000","last_unstale":"2024-05-11T23:02:27.992015+0000","last_undegraded":"2024-05-11T23:02:27.992015+0000","last_fullsized":"2024-05-11T23:02:27.992015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:25:12.933966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091161300000000003,"stat_sum":{"num_bytes":40040479,"num_objects":63,"num_object_clones":14,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":31,"num_read":6251,"num_read_kb":30247,"num_write":266,"num_write_kb":5951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"73'237","reported_seq":4434,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T23:02:27.994821+0000","last_change":"2024-05-11T23:01:39.800279+0000","last_active":"2024-05-11T23:02:27.994821+0000","last_peered":"2024-05-11T23:02:27.994821+0000","last_clean":"2024-05-11T23:02:27.994821+0000","last_became_active":"2024-05-11T23:01:39.787165+0000","last_became_peered":"2024-05-11T23:01:39.787165+0000","last_unstale":"2024-05-11T23:02:27.994821+0000","last_undegraded":"2024-05-11T23:02:27.994821+0000","last_fullsized":"2024-05-11T23:02:27.994821+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:12:26.301816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065062200000000001,"stat_sum":{"num_bytes":85493261,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":24,"num_read":4316,"num_read_kb":37312,"num_write":318,"num_write_kb":33989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"72'137","reported_seq":6515,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T23:02:27.994289+0000","last_change":"2024-05-11T23:01:29.466255+0000","last_active":"2024-05-11T23:02:27.994289+0000","last_peered":"2024-05-11T23:02:27.994289+0000","last_clean":"2024-05-11T23:02:27.994289+0000","last_became_active":"2024-05-11T23:01:27.460541+0000","last_became_peered":"2024-05-11T23:01:27.460541+0000","last_unstale":"2024-05-11T23:02:27.994289+0000","last_undegraded":"2024-05-11T23:02:27.994289+0000","last_fullsized":"2024-05-11T23:02:27.994289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:52:38.161402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087678099999999998,"stat_sum":{"num_bytes":64034755,"num_objects":46,"num_object_clones":8,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":16,"num_read":6280,"num_read_kb":26311,"num_write":208,"num_write_kb":17624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"72'649","reported_seq":4568,"reported_epoch":76,"state":"peering","last_fresh":"2024-05-11T23:02:30.282532+0000","last_change":"2024-05-11T23:02:30.780807+0000","last_active":"2024-05-11T23:02:30.780807+0000","last_peered":"2024-05-11T23:02:30.282532+0000","last_clean":"2024-05-11T23:02:26.927074+0000","last_became_active":"2024-05-11T23:02:29.266610+0000","last_became_peered":"2024-05-11T23:02:29.266610+0000","last_unstale":"2024-05-11T23:02:30.282532+0000","last_undegraded":"2024-05-11T23:02:30.282532+0000","last_fullsized":"2024-05-11T23:02:28.996188+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":649,"log_dups_size":0,"ondisk_log_size":649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000433275,"stat_sum":{"num_bytes":65710759,"num_objects":58,"num_object_clones":14,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":23,"num_read":6079,"num_read_kb":33611,"num_write":1026,"num_write_kb":12426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":13589947,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"75'207","reported_seq":1715,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-05-11T23:02:30.354446+0000","last_change":"2024-05-11T23:02:31.160949+0000","last_active":"2024-05-11T23:02:31.160949+0000","last_peered":"2024-05-11T23:02:30.354446+0000","last_clean":"2024-05-11T23:02:26.978971+0000","last_became_active":"2024-05-11T23:02:29.265179+0000","last_became_peered":"2024-05-11T23:02:29.265179+0000","last_unstale":"2024-05-11T23:02:30.354446+0000","last_undegraded":"2024-05-11T23:02:30.354446+0000","last_fullsized":"2024-05-11T23:02:28.992624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079167699999999996,"stat_sum":{"num_bytes":17637540,"num_objects":44,"num_object_clones":12,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":21,"num_read":1561,"num_read_kb":3220,"num_write":191,"num_write_kb":12074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":4822628,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"75'213","reported_seq":2411,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-11T23:02:30.002515+0000","last_change":"2024-05-11T23:01:29.462988+0000","last_active":"2024-05-11T23:02:30.002515+0000","last_peered":"2024-05-11T23:02:30.002515+0000","last_clean":"2024-05-11T23:02:30.002515+0000","last_became_active":"2024-05-11T23:01:27.460088+0000","last_became_peered":"2024-05-11T23:01:27.460088+0000","last_unstale":"2024-05-11T23:02:30.002515+0000","last_undegraded":"2024-05-11T23:02:30.002515+0000","last_fullsized":"2024-05-11T23:02:30.002515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:19:19.181829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045912,"stat_sum":{"num_bytes":8665959,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":2561,"num_read_kb":11541,"num_write":271,"num_write_kb":2610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"72'207","reported_seq":3257,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T23:02:27.992093+0000","last_change":"2024-05-11T23:01:29.474960+0000","last_active":"2024-05-11T23:02:27.992093+0000","last_peered":"2024-05-11T23:02:27.992093+0000","last_clean":"2024-05-11T23:02:27.992093+0000","last_became_active":"2024-05-11T23:01:27.463623+0000","last_became_peered":"2024-05-11T23:01:27.463623+0000","last_unstale":"2024-05-11T23:02:27.992093+0000","last_undegraded":"2024-05-11T23:02:27.992093+0000","last_fullsized":"2024-05-11T23:02:27.992093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:49:20.891115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001047748,"stat_sum":{"num_bytes":38994882,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":27,"num_read":3193,"num_read_kb":34540,"num_write":215,"num_write_kb":15328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"72'221","reported_seq":4489,"reported_epoch":73,"state":"remapped+peering","last_fresh":"2024-05-11T23:02:26.993634+0000","last_change":"2024-05-11T23:02:27.994734+0000","last_active":"2024-05-11T23:02:26.993634+0000","last_peered":"2024-05-11T23:02:26.918860+0000","last_clean":"2024-05-11T23:02:26.918860+0000","last_became_active":"2024-05-11T23:01:58.868753+0000","last_became_peered":"2024-05-11T23:01:58.868753+0000","last_unstale":"2024-05-11T23:02:26.993634+0000","last_undegraded":"2024-05-11T23:02:26.993634+0000","last_fullsized":"2024-05-11T23:02:26.993634+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039810900000000001,"stat_sum":{"num_bytes":50322988,"num_objects":54,"num_object_clones":17,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":26,"num_read":3207,"num_read_kb":13366,"num_write":272,"num_write_kb":10989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":2214117,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3,4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"72'259","reported_seq":2592,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T23:02:27.994007+0000","last_change":"2024-05-11T23:01:29.466206+0000","last_active":"2024-05-11T23:02:27.994007+0000","last_peered":"2024-05-11T23:02:27.994007+0000","last_clean":"2024-05-11T23:02:27.994007+0000","last_became_active":"2024-05-11T23:01:27.465952+0000","last_became_peered":"2024-05-11T23:01:27.465952+0000","last_unstale":"2024-05-11T23:02:27.994007+0000","last_undegraded":"2024-05-11T23:02:27.994007+0000","last_fullsized":"2024-05-11T23:02:27.994007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:50:07.285482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066166100000000002,"stat_sum":{"num_bytes":25334660,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":23,"num_read":3173,"num_read_kb":9100,"num_write":245,"num_write_kb":4372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":57,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T23:02:27.993957+0000","last_change":"2024-05-11T23:02:04.882401+0000","last_active":"2024-05-11T23:02:27.993957+0000","last_peered":"2024-05-11T23:02:27.993957+0000","last_clean":"2024-05-11T23:02:27.993957+0000","last_became_active":"2024-05-11T23:02:04.882078+0000","last_became_peered":"2024-05-11T23:02:04.882078+0000","last_unstale":"2024-05-11T23:02:27.993957+0000","last_undegraded":"2024-05-11T23:02:27.993957+0000","last_fullsized":"2024-05-11T23:02:27.993957+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:44:09.968438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"72'544","reported_seq":3242,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-11T23:02:30.002161+0000","last_change":"2024-05-11T23:01:29.462499+0000","last_active":"2024-05-11T23:02:30.002161+0000","last_peered":"2024-05-11T23:02:30.002161+0000","last_clean":"2024-05-11T23:02:30.002161+0000","last_became_active":"2024-05-11T23:01:27.458177+0000","last_became_peered":"2024-05-11T23:01:27.458177+0000","last_unstale":"2024-05-11T23:02:30.002161+0000","last_undegraded":"2024-05-11T23:02:30.002161+0000","last_fullsized":"2024-05-11T23:02:30.002161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:23:50.053429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099453400000000004,"stat_sum":{"num_bytes":25806994,"num_objects":49,"num_object_clones":7,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":24,"num_read":5563,"num_read_kb":16867,"num_write":796,"num_write_kb":8580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":135,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-11T23:02:30.781153+0000","last_change":"2024-05-11T23:02:04.881007+0000","last_active":"2024-05-11T23:02:30.781153+0000","last_peered":"2024-05-11T23:02:30.781153+0000","last_clean":"2024-05-11T23:02:30.781153+0000","last_became_active":"2024-05-11T23:02:04.880724+0000","last_became_peered":"2024-05-11T23:02:04.880724+0000","last_unstale":"2024-05-11T23:02:30.781153+0000","last_undegraded":"2024-05-11T23:02:30.781153+0000","last_fullsized":"2024-05-11T23:02:30.781153+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:43:37.396847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"75'557","reported_seq":4417,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-05-11T23:02:30.166572+0000","last_change":"2024-05-11T23:02:31.161026+0000","last_active":"2024-05-11T23:02:31.161026+0000","last_peered":"2024-05-11T23:02:30.166572+0000","last_clean":"2024-05-11T23:02:26.111205+0000","last_became_active":"2024-05-11T23:02:29.264862+0000","last_became_peered":"2024-05-11T23:02:29.264862+0000","last_unstale":"2024-05-11T23:02:30.166572+0000","last_undegraded":"2024-05-11T23:02:30.166572+0000","last_fullsized":"2024-05-11T23:02:30.166572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058786099999999996,"stat_sum":{"num_bytes":29090289,"num_objects":50,"num_object_clones":12,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":6196,"num_read_kb":16637,"num_write":787,"num_write_kb":4639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":4415701,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":124,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T23:02:27.994292+0000","last_change":"2024-05-11T23:01:19.216700+0000","last_active":"2024-05-11T23:02:27.994292+0000","last_peered":"2024-05-11T23:02:27.994292+0000","last_clean":"2024-05-11T23:02:27.994292+0000","last_became_active":"2024-05-11T23:01:19.216413+0000","last_became_peered":"2024-05-11T23:01:19.216413+0000","last_unstale":"2024-05-11T23:02:27.994292+0000","last_undegraded":"2024-05-11T23:02:27.994292+0000","last_fullsized":"2024-05-11T23:02:27.994292+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:38.639409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'172","reported_seq":3319,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-11T23:02:30.781385+0000","last_change":"2024-05-11T23:01:29.468176+0000","last_active":"2024-05-11T23:02:30.781385+0000","last_peered":"2024-05-11T23:02:30.781385+0000","last_clean":"2024-05-11T23:02:30.781385+0000","last_became_active":"2024-05-11T23:01:27.460866+0000","last_became_peered":"2024-05-11T23:01:27.460866+0000","last_unstale":"2024-05-11T23:02:30.781385+0000","last_undegraded":"2024-05-11T23:02:30.781385+0000","last_fullsized":"2024-05-11T23:02:30.781385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:07:14.226380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050028300000000002,"stat_sum":{"num_bytes":64000655,"num_objects":47,"num_object_clones":17,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":20,"num_read":3152,"num_read_kb":40589,"num_write":216,"num_write_kb":31433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":130,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-11T23:02:30.781492+0000","last_change":"2024-05-11T23:01:19.220232+0000","last_active":"2024-05-11T23:02:30.781492+0000","last_peered":"2024-05-11T23:02:30.781492+0000","last_clean":"2024-05-11T23:02:30.781492+0000","last_became_active":"2024-05-11T23:01:19.219917+0000","last_became_peered":"2024-05-11T23:01:19.219917+0000","last_unstale":"2024-05-11T23:02:30.781492+0000","last_undegraded":"2024-05-11T23:02:30.781492+0000","last_fullsized":"2024-05-11T23:02:30.781492+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:44:02.868252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"72'175","reported_seq":3196,"reported_epoch":73,"state":"remapped+peering","last_fresh":"2024-05-11T23:02:26.987309+0000","last_change":"2024-05-11T23:02:27.993149+0000","last_active":"2024-05-11T23:02:26.987309+0000","last_peered":"2024-05-11T23:02:26.176608+0000","last_clean":"2024-05-11T23:02:26.176608+0000","last_became_active":"2024-05-11T23:01:58.869645+0000","last_became_peered":"2024-05-11T23:01:58.869645+0000","last_unstale":"2024-05-11T23:02:26.987309+0000","last_undegraded":"2024-05-11T23:02:26.987309+0000","last_fullsized":"2024-05-11T23:02:26.987309+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012391380000000001,"stat_sum":{"num_bytes":45794799,"num_objects":60,"num_object_clones":19,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":28,"num_read":2881,"num_read_kb":20686,"num_write":150,"num_write_kb":9000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":5716609,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"72'369","reported_seq":3067,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-11T23:02:30.007348+0000","last_change":"2024-05-11T23:02:15.933489+0000","last_active":"2024-05-11T23:02:30.007348+0000","last_peered":"2024-05-11T23:02:30.007348+0000","last_clean":"2024-05-11T23:02:30.007348+0000","last_became_active":"2024-05-11T23:02:15.932604+0000","last_became_peered":"2024-05-11T23:02:15.932604+0000","last_unstale":"2024-05-11T23:02:30.007348+0000","last_undegraded":"2024-05-11T23:02:30.007348+0000","last_fullsized":"2024-05-11T23:02:30.007348+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:00:50.763439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038468999999999998,"stat_sum":{"num_bytes":61899402,"num_objects":60,"num_object_clones":14,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":28,"num_read":3996,"num_read_kb":23365,"num_write":432,"num_write_kb":11056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":3164613,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":124,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T23:02:27.992443+0000","last_change":"2024-05-11T23:01:19.221118+0000","last_active":"2024-05-11T23:02:27.992443+0000","last_peered":"2024-05-11T23:02:27.992443+0000","last_clean":"2024-05-11T23:02:27.992443+0000","last_became_active":"2024-05-11T23:01:19.220855+0000","last_became_peered":"2024-05-11T23:01:19.220855+0000","last_unstale":"2024-05-11T23:02:27.992443+0000","last_undegraded":"2024-05-11T23:02:27.992443+0000","last_fullsized":"2024-05-11T23:02:27.992443+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:01:56.076963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"77'347","reported_seq":3601,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-11T23:02:30.816118+0000","last_change":"2024-05-11T23:01:29.467921+0000","last_active":"2024-05-11T23:02:30.816118+0000","last_peered":"2024-05-11T23:02:30.816118+0000","last_clean":"2024-05-11T23:02:30.816118+0000","last_became_active":"2024-05-11T23:01:27.459593+0000","last_became_peered":"2024-05-11T23:01:27.459593+0000","last_unstale":"2024-05-11T23:02:30.816118+0000","last_undegraded":"2024-05-11T23:02:30.816118+0000","last_fullsized":"2024-05-11T23:02:30.816118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:48:32.160822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047314299999999998,"stat_sum":{"num_bytes":15254749,"num_objects":54,"num_object_clones":15,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":26,"num_read":4800,"num_read_kb":12439,"num_write":417,"num_write_kb":2275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":138,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-11T23:02:30.007320+0000","last_change":"2024-05-11T23:02:27.995538+0000","last_active":"2024-05-11T23:02:30.007320+0000","last_peered":"2024-05-11T23:02:30.007320+0000","last_clean":"2024-05-11T23:02:30.007320+0000","last_became_active":"2024-05-11T23:02:27.995242+0000","last_became_peered":"2024-05-11T23:02:27.995242+0000","last_unstale":"2024-05-11T23:02:30.007320+0000","last_undegraded":"2024-05-11T23:02:30.007320+0000","last_fullsized":"2024-05-11T23:02:30.007320+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T09:00:41.429605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"72'209","reported_seq":2023,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-11T23:02:30.001950+0000","last_change":"2024-05-11T23:01:29.462388+0000","last_active":"2024-05-11T23:02:30.001950+0000","last_peered":"2024-05-11T23:02:30.001950+0000","last_clean":"2024-05-11T23:02:30.001950+0000","last_became_active":"2024-05-11T23:01:27.460113+0000","last_became_peered":"2024-05-11T23:01:27.460113+0000","last_unstale":"2024-05-11T23:02:30.001950+0000","last_undegraded":"2024-05-11T23:02:30.001950+0000","last_fullsized":"2024-05-11T23:02:30.001950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:58:43.773147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062926200000000001,"stat_sum":{"num_bytes":17605041,"num_objects":51,"num_object_clones":14,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":24,"num_read":2416,"num_read_kb":12414,"num_write":184,"num_write_kb":4253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":124,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T23:02:27.993157+0000","last_change":"2024-05-11T23:01:19.219755+0000","last_active":"2024-05-11T23:02:27.993157+0000","last_peered":"2024-05-11T23:02:27.993157+0000","last_clean":"2024-05-11T23:02:27.993157+0000","last_became_active":"2024-05-11T23:01:19.219471+0000","last_became_peered":"2024-05-11T23:01:19.219471+0000","last_unstale":"2024-05-11T23:02:27.993157+0000","last_undegraded":"2024-05-11T23:02:27.993157+0000","last_fullsized":"2024-05-11T23:02:27.993157+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:39:05.284213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"72'304","reported_seq":1925,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-11T23:02:30.784798+0000","last_change":"2024-05-11T23:02:30.783741+0000","last_active":"2024-05-11T23:02:30.784798+0000","last_peered":"2024-05-11T23:02:30.784798+0000","last_clean":"2024-05-11T23:02:30.784798+0000","last_became_active":"2024-05-11T23:02:30.782707+0000","last_became_peered":"2024-05-11T23:02:30.782707+0000","last_unstale":"2024-05-11T23:02:30.784798+0000","last_undegraded":"2024-05-11T23:02:30.784798+0000","last_fullsized":"2024-05-11T23:02:30.784798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:04:17.726232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000407852,"stat_sum":{"num_bytes":33025735,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":27,"num_read":1848,"num_read_kb":9353,"num_write":353,"num_write_kb":4644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":10719266,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":124,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T23:02:27.992003+0000","last_change":"2024-05-11T23:01:19.219128+0000","last_active":"2024-05-11T23:02:27.992003+0000","last_peered":"2024-05-11T23:02:27.992003+0000","last_clean":"2024-05-11T23:02:27.992003+0000","last_became_active":"2024-05-11T23:01:19.218846+0000","last_became_peered":"2024-05-11T23:01:19.218846+0000","last_unstale":"2024-05-11T23:02:27.992003+0000","last_undegraded":"2024-05-11T23:02:27.992003+0000","last_fullsized":"2024-05-11T23:02:27.992003+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:41:09.111000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":276,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T23:02:27.993584+0000","last_change":"2024-05-11T23:01:18.164452+0000","last_active":"2024-05-11T23:02:27.993584+0000","last_peered":"2024-05-11T23:02:27.993584+0000","last_clean":"2024-05-11T23:02:27.993584+0000","last_became_active":"2024-05-11T23:01:18.163431+0000","last_became_peered":"2024-05-11T23:01:18.163431+0000","last_unstale":"2024-05-11T23:02:27.993584+0000","last_undegraded":"2024-05-11T23:02:27.993584+0000","last_fullsized":"2024-05-11T23:02:27.993584+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_clean_scrub_stamp":"2024-05-11T23:01:16.147564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:40:59.974639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"72'603","reported_seq":2461,"reported_epoch":77,"state":"peering","last_fresh":"2024-05-11T23:02:30.453203+0000","last_change":"2024-05-11T23:02:31.160922+0000","last_active":"2024-05-11T23:02:31.160922+0000","last_peered":"2024-05-11T23:02:30.453203+0000","last_clean":"2024-05-11T23:02:26.594203+0000","last_became_active":"2024-05-11T23:02:29.266533+0000","last_became_peered":"2024-05-11T23:02:29.266533+0000","last_unstale":"2024-05-11T23:02:30.453203+0000","last_undegraded":"2024-05-11T23:02:30.453203+0000","last_fullsized":"2024-05-11T23:02:28.993307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061400699999999996,"stat_sum":{"num_bytes":35711848,"num_objects":57,"num_object_clones":12,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":28,"num_read":3757,"num_read_kb":11997,"num_write":862,"num_write_kb":4887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":4220183,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"73'215","reported_seq":5681,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T23:02:27.993787+0000","last_change":"2024-05-11T23:01:29.465998+0000","last_active":"2024-05-11T23:02:27.993787+0000","last_peered":"2024-05-11T23:02:27.993787+0000","last_clean":"2024-05-11T23:02:27.993787+0000","last_became_active":"2024-05-11T23:01:27.461824+0000","last_became_peered":"2024-05-11T23:01:27.461824+0000","last_unstale":"2024-05-11T23:02:27.993787+0000","last_undegraded":"2024-05-11T23:02:27.993787+0000","last_fullsized":"2024-05-11T23:02:27.993787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:07:25.859972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055969500000000003,"stat_sum":{"num_bytes":71858646,"num_objects":65,"num_object_clones":19,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":23,"num_read":5569,"num_read_kb":39396,"num_write":206,"num_write_kb":25887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"72'264","reported_seq":2901,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T23:02:27.994079+0000","last_change":"2024-05-11T23:02:07.889202+0000","last_active":"2024-05-11T23:02:27.994079+0000","last_peered":"2024-05-11T23:02:27.994079+0000","last_clean":"2024-05-11T23:02:27.994079+0000","last_became_active":"2024-05-11T23:02:07.888795+0000","last_became_peered":"2024-05-11T23:02:07.888795+0000","last_unstale":"2024-05-11T23:02:27.994079+0000","last_undegraded":"2024-05-11T23:02:27.994079+0000","last_fullsized":"2024-05-11T23:02:27.994079+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:38:40.058749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030522600000000002,"stat_sum":{"num_bytes":29099032,"num_objects":61,"num_object_clones":17,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":24,"num_read":2818,"num_read_kb":14880,"num_write":262,"num_write_kb":5762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":3317854,"num_keys_recovered":15,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"71'255","reported_seq":6834,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-11T23:02:30.784630+0000","last_change":"2024-05-11T23:02:30.783606+0000","last_active":"2024-05-11T23:02:30.784630+0000","last_peered":"2024-05-11T23:02:30.784630+0000","last_clean":"2024-05-11T23:02:30.784630+0000","last_became_active":"2024-05-11T23:02:30.779706+0000","last_became_peered":"2024-05-11T23:02:30.779706+0000","last_unstale":"2024-05-11T23:02:30.784630+0000","last_undegraded":"2024-05-11T23:02:30.784630+0000","last_fullsized":"2024-05-11T23:02:30.784630+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:51:04.851278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042056999999999999,"stat_sum":{"num_bytes":73036906,"num_objects":65,"num_object_clones":23,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":19,"num_read":6130,"num_read_kb":40275,"num_write":305,"num_write_kb":40797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":30909049,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"72'271","reported_seq":2790,"reported_epoch":73,"state":"remapped+peering","last_fresh":"2024-05-11T23:02:26.987326+0000","last_change":"2024-05-11T23:02:27.993134+0000","last_active":"2024-05-11T23:02:26.987326+0000","last_peered":"2024-05-11T23:02:26.882592+0000","last_clean":"2024-05-11T23:02:26.882592+0000","last_became_active":"2024-05-11T23:02:07.890269+0000","last_became_peered":"2024-05-11T23:02:07.890269+0000","last_unstale":"2024-05-11T23:02:26.987326+0000","last_undegraded":"2024-05-11T23:02:26.987326+0000","last_fullsized":"2024-05-11T23:02:26.987326+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084388499999999995,"stat_sum":{"num_bytes":14001193,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":17,"num_read":3552,"num_read_kb":19708,"num_write":318,"num_write_kb":9238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":3150248,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"72'264","reported_seq":5466,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T23:02:27.993681+0000","last_change":"2024-05-11T23:01:29.864324+0000","last_active":"2024-05-11T23:02:27.993681+0000","last_peered":"2024-05-11T23:02:27.993681+0000","last_clean":"2024-05-11T23:02:27.993681+0000","last_became_active":"2024-05-11T23:01:27.465570+0000","last_became_peered":"2024-05-11T23:01:27.465570+0000","last_unstale":"2024-05-11T23:02:27.993681+0000","last_undegraded":"2024-05-11T23:02:27.993681+0000","last_fullsized":"2024-05-11T23:02:27.993681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:31:52.989645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062217299999999995,"stat_sum":{"num_bytes":29829189,"num_objects":55,"num_object_clones":17,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":25,"num_read":5699,"num_read_kb":25080,"num_write":249,"num_write_kb":7001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"72'603","reported_seq":5585,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T23:02:27.993553+0000","last_change":"2024-05-11T23:01:29.864587+0000","last_active":"2024-05-11T23:02:27.993553+0000","last_peered":"2024-05-11T23:02:27.993553+0000","last_clean":"2024-05-11T23:02:27.993553+0000","last_became_active":"2024-05-11T23:01:27.466572+0000","last_became_peered":"2024-05-11T23:01:27.466572+0000","last_unstale":"2024-05-11T23:02:27.993553+0000","last_undegraded":"2024-05-11T23:02:27.993553+0000","last_fullsized":"2024-05-11T23:02:27.993553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:05:09.619183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087175200000000005,"stat_sum":{"num_bytes":50292214,"num_objects":82,"num_object_clones":24,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":35,"num_read":8465,"num_read_kb":34123,"num_write":771,"num_write_kb":15131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"72'220","reported_seq":3658,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T23:02:27.993920+0000","last_change":"2024-05-11T23:01:29.864577+0000","last_active":"2024-05-11T23:02:27.993920+0000","last_peered":"2024-05-11T23:02:27.993920+0000","last_clean":"2024-05-11T23:02:27.993920+0000","last_became_active":"2024-05-11T23:01:27.466778+0000","last_became_peered":"2024-05-11T23:01:27.466778+0000","last_unstale":"2024-05-11T23:02:27.993920+0000","last_undegraded":"2024-05-11T23:02:27.993920+0000","last_fullsized":"2024-05-11T23:02:27.993920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:02:56.919219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080844100000000002,"stat_sum":{"num_bytes":29245947,"num_objects":64,"num_object_clones":20,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":34,"num_read":3519,"num_read_kb":26228,"num_write":214,"num_write_kb":10513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"65'279","reported_seq":2929,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-11T23:02:30.001887+0000","last_change":"2024-05-11T23:02:07.889732+0000","last_active":"2024-05-11T23:02:30.001887+0000","last_peered":"2024-05-11T23:02:30.001887+0000","last_clean":"2024-05-11T23:02:30.001887+0000","last_became_active":"2024-05-11T23:02:07.888575+0000","last_became_peered":"2024-05-11T23:02:07.888575+0000","last_unstale":"2024-05-11T23:02:30.001887+0000","last_undegraded":"2024-05-11T23:02:30.001887+0000","last_fullsized":"2024-05-11T23:02:30.001887+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:38:23.640354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033171200000000001,"stat_sum":{"num_bytes":35280528,"num_objects":53,"num_object_clones":14,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":20,"num_read":4126,"num_read_kb":19524,"num_write":330,"num_write_kb":9215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":4284961,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"73'381","reported_seq":3406,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T23:02:27.994116+0000","last_change":"2024-05-11T23:02:24.979353+0000","last_active":"2024-05-11T23:02:27.994116+0000","last_peered":"2024-05-11T23:02:27.994116+0000","last_clean":"2024-05-11T23:02:27.994116+0000","last_became_active":"2024-05-11T23:02:24.978284+0000","last_became_peered":"2024-05-11T23:02:24.978284+0000","last_unstale":"2024-05-11T23:02:27.994116+0000","last_undegraded":"2024-05-11T23:02:27.994116+0000","last_fullsized":"2024-05-11T23:02:27.994116+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:38:32.500162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044654299999999998,"stat_sum":{"num_bytes":34524875,"num_objects":52,"num_object_clones":17,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":17,"num_read":4632,"num_read_kb":17594,"num_write":484,"num_write_kb":5197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":8486426,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"75'631","reported_seq":3562,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-11T23:02:30.001938+0000","last_change":"2024-05-11T23:01:29.462751+0000","last_active":"2024-05-11T23:02:30.001938+0000","last_peered":"2024-05-11T23:02:30.001938+0000","last_clean":"2024-05-11T23:02:30.001938+0000","last_became_active":"2024-05-11T23:01:27.459137+0000","last_became_peered":"2024-05-11T23:01:27.459137+0000","last_unstale":"2024-05-11T23:02:30.001938+0000","last_undegraded":"2024-05-11T23:02:30.001938+0000","last_fullsized":"2024-05-11T23:02:30.001938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:06:54.682906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010326510000000001,"stat_sum":{"num_bytes":70495727,"num_objects":62,"num_object_clones":15,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":23,"num_read":5196,"num_read_kb":20876,"num_write":872,"num_write_kb":10120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"72'161","reported_seq":2271,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-05-11T23:02:27.993483+0000","last_change":"2024-05-11T23:01:29.864290+0000","last_active":"2024-05-11T23:02:27.993483+0000","last_peered":"2024-05-11T23:02:27.993483+0000","last_clean":"2024-05-11T23:02:27.993483+0000","last_became_active":"2024-05-11T23:01:27.465953+0000","last_became_peered":"2024-05-11T23:01:27.465953+0000","last_unstale":"2024-05-11T23:02:27.993483+0000","last_undegraded":"2024-05-11T23:02:27.993483+0000","last_fullsized":"2024-05-11T23:02:27.993483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:14:02.950660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065432399999999996,"stat_sum":{"num_bytes":14001979,"num_objects":38,"num_object_clones":11,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":19,"num_read":2309,"num_read_kb":11603,"num_write":138,"num_write_kb":3163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1276807264,"num_objects":1728,"num_object_clones":464,"num_object_copies":3456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1728,"num_whiteouts":766,"num_read":132221,"num_read_kb":695999,"num_write":12192,"num_write_kb":381425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1524,"num_bytes_recovered":136059230,"num_keys_recovered":226,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":68366336,"data_stored":804897626,"data_compressed":43104241,"data_compressed_allocated":46940160,"data_compressed_original":786314833,"omap_allocated":0,"internal_metadata":0},"log_size":10067,"ondisk_log_size":10067,"up":64,"acting":61,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65020,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56972,"kb_used_data":11900,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94314868,"statfs":{"total":96636764160,"available":96578424832,"internally_reserved":0,"allocated":12185600,"data_stored":124716248,"data_compressed":7135290,"data_compressed_allocated":7557120,"data_compressed_original":120985728,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42984,"kb_used_data":7644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328856,"statfs":{"total":96636764160,"available":96592748544,"internally_reserved":0,"allocated":7827456,"data_stored":80610085,"data_compressed":4581870,"data_compressed_allocated":4853760,"data_compressed_original":78406719,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43588,"kb_used_data":16568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94328252,"statfs":{"total":96636764160,"available":96592130048,"internally_reserved":0,"allocated":16965632,"data_stored":191538351,"data_compressed":10021488,"data_compressed_allocated":11034624,"data_compressed_original":186575964,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[13],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":0,"seq":64424509456,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36876,"kb_used_data":9860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334964,"statfs":{"total":96636764160,"available":96599003136,"internally_reserved":0,"allocated":10096640,"data_stored":110574993,"data_compressed":5735067,"data_compressed_allocated":6230016,"data_compressed_original":107592328,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40740,"kb_used_data":9560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331100,"statfs":{"total":96636764160,"available":96595046400,"internally_reserved":0,"allocated":9789440,"data_stored":95928223,"data_compressed":5038314,"data_compressed_allocated":6225920,"data_compressed_original":93278321,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47644,"kb_used_data":8136,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94324196,"statfs":{"total":96636764160,"available":96587976704,"internally_reserved":0,"allocated":8331264,"data_stored":102048268,"data_compressed":5231237,"data_compressed_allocated":5713920,"data_compressed_original":100150232,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39268,"kb_used_data":8080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332572,"statfs":{"total":96636764160,"available":96596553728,"internally_reserved":0,"allocated":8273920,"data_stored":96654090,"data_compressed":5060621,"data_compressed_allocated":5017600,"data_compressed_original":94252749,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7323648,"data_stored":95655142,"data_compressed":5028111,"data_compressed_allocated":4972544,"data_compressed_original":93679309,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7426048,"data_stored":78185801,"data_compressed":4040509,"data_compressed_allocated":5079040,"data_compressed_original":76214385,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6926336,"data_stored":93257494,"data_compressed":4828821,"data_compressed_allocated":5181440,"data_compressed_original":91761624,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9330688,"data_stored":110848858,"data_compressed":5776054,"data_compressed_allocated":6283264,"data_compressed_original":108235400,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16117760,"data_stored":191148655,"data_compressed":10021488,"data_compressed_allocated":11034624,"data_compressed_original":186575964,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6967296,"data_stored":80214023,"data_compressed":4581870,"data_compressed_allocated":4853760,"data_compressed_original":78406719,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3014656,"data_stored":31857835,"data_compressed":1724608,"data_compressed_allocated":2023424,"data_compressed_original":31029144,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":11259904,"data_stored":123729818,"data_compressed":7102780,"data_compressed_allocated":7512064,"data_compressed_original":120412288,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T23:02:32.422 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:32.422+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:32.448 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:32.448 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:32.457 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:02:32.523 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:32.523+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:32.607 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:32.607 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:32.616 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:02:32.659 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:32.659+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:32.725 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:32.725 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:32.726 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:32.726+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:32.733 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:02:32.826 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:32.826+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:32.883 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:32.884 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:32.893 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:02:32.938 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:32.938+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:33.008 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:33.008 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-05-11T23:02:33.019 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:02:33.060 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:33.060+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:33.106 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone35 from 0xe941e31 (overlap 0xe941e31) to 0x1753638 2024-05-11T23:02:33.129 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:33.129+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:33.205 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:33.206 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:33.219 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:02:33.230 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:02:33.263 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:33.263+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:33.330 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:33.331+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:33.356 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:33.356 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:33.365 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:02:33.390 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:33.390 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:33.409 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:33.409 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:33.427 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:33.428 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:33.432 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:33.432+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:33.443 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-05-11T23:02:33.446 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:33.447 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:33.447 INFO:teuthology.orchestra.run.smithi005.stdout:3833 write 0x2bada49 thru 0x2bb6fe7 (0x959f bytes) 2024-05-11T23:02:33.466 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:33.467 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:33.485 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:33.485 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:33.504 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:33.504 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:33.522 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:33.522 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:33.527 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:33.527 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:33.527 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:33.527 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:33.532 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:33.532+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:33.539 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:02:33.545 INFO:teuthology.orchestra.run.smithi005.stdout:3834 read 0xb1597d3 thru 0xb15e803 (0x5031 bytes) 2024-05-11T23:02:33.545 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:33.545 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:33.546 INFO:teuthology.orchestra.run.smithi005.stdout:3838 writesame 0x77fc28b thru 0x7801b5e data_size 0x1e(0x58d4 bytes) 2024-05-11T23:02:33.591 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:33.592 INFO:teuthology.orchestra.run.smithi005.stdout:3842 trunc from 0xe941e31 to 0xdbc2361 2024-05-11T23:02:33.605 INFO:teuthology.orchestra.run.smithi005.stdout:3843 read 0xcee7099 thru 0xcef4882 (0xd7ea bytes) 2024-05-11T23:02:33.605 INFO:teuthology.orchestra.run.smithi005.stdout:3846 clone 37 order 20 su 16384 sc 2 2024-05-11T23:02:33.633 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:33.633+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:33.650 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:33.650 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:33.651 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:33.651 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:33.651 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:33.659 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:02:33.734 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:33.734+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:33.767 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:33.767 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:33.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:33.767 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:33.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.767 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67817 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:33.767 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:33.767 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.146360956999999, 2024-05-11T23:02:33.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044148845999999999, 2024-05-11T23:02:33.767 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:33.768 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:33.768 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:33.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:33.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:33.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:33.768 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67817 2024-05-11T23:02:33.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.768 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:33.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:33.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:33.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:33.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:33.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:33.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620227+0000", 2024-05-11T23:02:33.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00046792099999999999 2024-05-11T23:02:33.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:33.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620232+0000", 2024-05-11T23:02:33.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3050000000000003e-06 2024-05-11T23:02:33.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:33.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620243+0000", 2024-05-11T23:02:33.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1289e-05 2024-05-11T23:02:33.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:33.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620250+0000", 2024-05-11T23:02:33.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2130000000000002e-06 2024-05-11T23:02:33.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:33.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663745+0000", 2024-05-11T23:02:33.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043494865000000001 2024-05-11T23:02:33.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:33.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663815+0000", 2024-05-11T23:02:33.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0284999999999995e-05 2024-05-11T23:02:33.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:33.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663908+0000", 2024-05-11T23:02:33.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2967999999999995e-05 2024-05-11T23:02:33.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.771 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.771 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67820 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4902912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:33.772 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:33.772 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.14581145, 2024-05-11T23:02:33.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044096781000000002, 2024-05-11T23:02:33.772 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:33.772 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:33.772 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:33.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:33.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:33.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:33.772 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67820 2024-05-11T23:02:33.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.772 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:33.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:33.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:33.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:33.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:33.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:33.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620310+0000", 2024-05-11T23:02:33.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9800000000000004e-07 2024-05-11T23:02:33.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:33.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620312+0000", 2024-05-11T23:02:33.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.272e-06 2024-05-11T23:02:33.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:33.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620319+0000", 2024-05-11T23:02:33.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9310000000000002e-06 2024-05-11T23:02:33.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:33.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620323+0000", 2024-05-11T23:02:33.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:02:33.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:33.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664284+0000", 2024-05-11T23:02:33.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043960908999999999 2024-05-11T23:02:33.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:33.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664327+0000", 2024-05-11T23:02:33.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2811000000000001e-05 2024-05-11T23:02:33.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:33.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664406+0000", 2024-05-11T23:02:33.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8588e-05 2024-05-11T23:02:33.775 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.775 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.775 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.775 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67821 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4927488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:33.776 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:33.776 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.145791039999999, 2024-05-11T23:02:33.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044203340000000001, 2024-05-11T23:02:33.776 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:33.776 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:33.776 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:33.776 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:33.776 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:33.776 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:33.776 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67821 2024-05-11T23:02:33.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.776 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:33.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:33.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:33.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:33.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:33.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:33.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620330+0000", 2024-05-11T23:02:33.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:02:33.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:33.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620332+0000", 2024-05-11T23:02:33.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:02:33.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:33.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620338+0000", 2024-05-11T23:02:33.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6250000000000004e-06 2024-05-11T23:02:33.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:33.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620342+0000", 2024-05-11T23:02:33.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7609999999999998e-06 2024-05-11T23:02:33.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:33.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664433+0000", 2024-05-11T23:02:33.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044090455000000001 2024-05-11T23:02:33.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:33.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664476+0000", 2024-05-11T23:02:33.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2861999999999997e-05 2024-05-11T23:02:33.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:33.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664533+0000", 2024-05-11T23:02:33.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7156e-05 2024-05-11T23:02:33.779 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.779 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.779 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.779 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67822 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4976640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:33.779 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:33.779 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.145772002000001, 2024-05-11T23:02:33.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044409416, 2024-05-11T23:02:33.780 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:33.780 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:33.780 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:33.780 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:33.780 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:33.780 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:33.780 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67822 2024-05-11T23:02:33.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.780 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:33.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:33.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:33.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:33.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:33.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:33.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620349+0000", 2024-05-11T23:02:33.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.51e-07 2024-05-11T23:02:33.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:33.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620351+0000", 2024-05-11T23:02:33.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.181e-06 2024-05-11T23:02:33.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:33.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620358+0000", 2024-05-11T23:02:33.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3060000000000003e-06 2024-05-11T23:02:33.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:33.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620362+0000", 2024-05-11T23:02:33.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7949999999999998e-06 2024-05-11T23:02:33.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:33.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664572+0000", 2024-05-11T23:02:33.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044209318999999997 2024-05-11T23:02:33.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:33.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664654+0000", 2024-05-11T23:02:33.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2367999999999995e-05 2024-05-11T23:02:33.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:33.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664758+0000", 2024-05-11T23:02:33.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103896 2024-05-11T23:02:33.783 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.783 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.783 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.783 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67823 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4952064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:33.783 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:33.783 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.145751952000001, 2024-05-11T23:02:33.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044623293000000001, 2024-05-11T23:02:33.784 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:33.784 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:33.784 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:33.784 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:33.784 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:33.784 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:33.784 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67823 2024-05-11T23:02:33.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.784 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:33.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:33.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:33.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:33.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:33.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:33.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622228+0000", 2024-05-11T23:02:33.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018592280000000001 2024-05-11T23:02:33.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:33.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622232+0000", 2024-05-11T23:02:33.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.403e-06 2024-05-11T23:02:33.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:33.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622248+0000", 2024-05-11T23:02:33.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5449000000000001e-05 2024-05-11T23:02:33.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:33.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622254+0000", 2024-05-11T23:02:33.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5819999999999998e-06 2024-05-11T23:02:33.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:33.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664800+0000", 2024-05-11T23:02:33.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042546048000000003 2024-05-11T23:02:33.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:33.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664886+0000", 2024-05-11T23:02:33.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5525000000000005e-05 2024-05-11T23:02:33.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:33.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664992+0000", 2024-05-11T23:02:33.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010605800000000001 2024-05-11T23:02:33.787 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.787 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.787 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.787 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67833 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5099520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:33.787 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:33.787 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.143645479, 2024-05-11T23:02:33.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044076606999999997, 2024-05-11T23:02:33.788 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:33.788 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:33.788 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:33.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:33.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:33.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:33.788 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67833 2024-05-11T23:02:33.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.788 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:33.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:33.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:33.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:33.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:33.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:33.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:33.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8099999999999998e-07 2024-05-11T23:02:33.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:33.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622477+0000", 2024-05-11T23:02:33.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0700000000000001e-06 2024-05-11T23:02:33.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:33.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622483+0000", 2024-05-11T23:02:33.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0780000000000003e-06 2024-05-11T23:02:33.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:33.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622487+0000", 2024-05-11T23:02:33.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.352e-06 2024-05-11T23:02:33.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:33.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666460+0000", 2024-05-11T23:02:33.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043973085000000002 2024-05-11T23:02:33.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:33.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666498+0000", 2024-05-11T23:02:33.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8441e-05 2024-05-11T23:02:33.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:33.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666552+0000", 2024-05-11T23:02:33.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3199999999999999e-05 2024-05-11T23:02:33.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.791 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.791 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.791 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67834 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3944448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:33.791 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:33.792 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.143607200999998, 2024-05-11T23:02:33.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044166551999999998, 2024-05-11T23:02:33.792 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:33.792 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:33.792 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:33.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:33.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:33.792 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:33.792 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67834 2024-05-11T23:02:33.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.792 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:33.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:33.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:33.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:33.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:33.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:33.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622514+0000", 2024-05-11T23:02:33.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1100000000000002e-07 2024-05-11T23:02:33.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:33.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622515+0000", 2024-05-11T23:02:33.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:02:33.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622519+0000", 2024-05-11T23:02:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8260000000000003e-06 2024-05-11T23:02:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622522+0000", 2024-05-11T23:02:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8899999999999999e-06 2024-05-11T23:02:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666580+0000", 2024-05-11T23:02:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044058235000000001 2024-05-11T23:02:33.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666624+0000", 2024-05-11T23:02:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3501999999999998e-05 2024-05-11T23:02:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666680+0000", 2024-05-11T23:02:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6042999999999999e-05 2024-05-11T23:02:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67835 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:33.795 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.143594664999998, 2024-05-11T23:02:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044272019000000003, 2024-05-11T23:02:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67835 2024-05-11T23:02:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1699999999999999e-07 2024-05-11T23:02:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:33.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622527+0000", 2024-05-11T23:02:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2109999999999999e-06 2024-05-11T23:02:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622531+0000", 2024-05-11T23:02:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4910000000000001e-06 2024-05-11T23:02:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622534+0000", 2024-05-11T23:02:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7080000000000002e-06 2024-05-11T23:02:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666703+0000", 2024-05-11T23:02:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044169377000000003 2024-05-11T23:02:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666745+0000", 2024-05-11T23:02:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1616000000000003e-05 2024-05-11T23:02:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666798+0000", 2024-05-11T23:02:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3298999999999999e-05 2024-05-11T23:02:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67836 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.143584005000001, 2024-05-11T23:02:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044373692999999999, 2024-05-11T23:02:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67836 2024-05-11T23:02:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.800 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622537+0000", 2024-05-11T23:02:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:33.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622538+0000", 2024-05-11T23:02:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1510000000000001e-06 2024-05-11T23:02:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622541+0000", 2024-05-11T23:02:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:02:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622543+0000", 2024-05-11T23:02:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4870000000000001e-06 2024-05-11T23:02:33.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666821+0000", 2024-05-11T23:02:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044278038999999998 2024-05-11T23:02:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666859+0000", 2024-05-11T23:02:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7224000000000001e-05 2024-05-11T23:02:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666910+0000", 2024-05-11T23:02:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1629999999999999e-05 2024-05-11T23:02:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.803 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67837 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.143574277999999, 2024-05-11T23:02:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044477883000000003, 2024-05-11T23:02:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:33.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67837 2024-05-11T23:02:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622548+0000", 2024-05-11T23:02:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:02:33.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622550+0000", 2024-05-11T23:02:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-05-11T23:02:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622553+0000", 2024-05-11T23:02:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.43e-06 2024-05-11T23:02:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666932+0000", 2024-05-11T23:02:33.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044379584 2024-05-11T23:02:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666971+0000", 2024-05-11T23:02:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8427000000000003e-05 2024-05-11T23:02:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667024+0000", 2024-05-11T23:02:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3226000000000002e-05 2024-05-11T23:02:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67838 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.143564761, 2024-05-11T23:02:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044580374999999998, 2024-05-11T23:02:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67838 2024-05-11T23:02:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:02:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622557+0000", 2024-05-11T23:02:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.102e-06 2024-05-11T23:02:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622560+0000", 2024-05-11T23:02:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:02:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622562+0000", 2024-05-11T23:02:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:02:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:33.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667045+0000", 2024-05-11T23:02:33.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044482936000000001 2024-05-11T23:02:33.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:33.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667083+0000", 2024-05-11T23:02:33.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7443e-05 2024-05-11T23:02:33.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:33.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667136+0000", 2024-05-11T23:02:33.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3257000000000002e-05 2024-05-11T23:02:33.812 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.812 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.812 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.813 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67839 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5394432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:33.813 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:33.813 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.143555019000001, 2024-05-11T23:02:33.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04470263, 2024-05-11T23:02:33.813 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:33.813 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:33.813 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:33.813 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:33.813 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:33.813 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:33.813 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67839 2024-05-11T23:02:33.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.814 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:33.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:33.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:33.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:33.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:33.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:33.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622566+0000", 2024-05-11T23:02:33.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-05-11T23:02:33.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:33.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622572+0000", 2024-05-11T23:02:33.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5610000000000004e-06 2024-05-11T23:02:33.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:33.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622576+0000", 2024-05-11T23:02:33.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3210000000000001e-06 2024-05-11T23:02:33.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:33.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622578+0000", 2024-05-11T23:02:33.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4710000000000001e-06 2024-05-11T23:02:33.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:33.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667160+0000", 2024-05-11T23:02:33.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044582220999999998 2024-05-11T23:02:33.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:33.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667205+0000", 2024-05-11T23:02:33.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5049000000000002e-05 2024-05-11T23:02:33.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:33.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667268+0000", 2024-05-11T23:02:33.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2788999999999998e-05 2024-05-11T23:02:33.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.816 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.817 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.817 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67840 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5419008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:33.817 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:33.817 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.143539389000001, 2024-05-11T23:02:33.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044796006999999999, 2024-05-11T23:02:33.817 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:33.817 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:33.817 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:33.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:33.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:33.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:33.817 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67840 2024-05-11T23:02:33.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.818 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:33.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:33.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:33.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:33.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:33.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:33.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:33.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:02:33.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:33.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622583+0000", 2024-05-11T23:02:33.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4389999999999999e-06 2024-05-11T23:02:33.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:33.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622586+0000", 2024-05-11T23:02:33.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:02:33.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:33.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622588+0000", 2024-05-11T23:02:33.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4540000000000001e-06 2024-05-11T23:02:33.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:33.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667287+0000", 2024-05-11T23:02:33.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044698882000000002 2024-05-11T23:02:33.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:33.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667324+0000", 2024-05-11T23:02:33.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7196000000000001e-05 2024-05-11T23:02:33.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:33.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667377+0000", 2024-05-11T23:02:33.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2927999999999999e-05 2024-05-11T23:02:33.821 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.821 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.821 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.821 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67841 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5443584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:33.821 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:33.821 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.143529615999999, 2024-05-11T23:02:33.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044907337999999998, 2024-05-11T23:02:33.821 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:33.821 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:33.821 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:33.821 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:33.822 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:33.822 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:33.822 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67841 2024-05-11T23:02:33.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.822 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:33.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:33.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:33.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:33.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:33.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:33.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:33.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:33.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:33.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622592+0000", 2024-05-11T23:02:33.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-05-11T23:02:33.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:33.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622595+0000", 2024-05-11T23:02:33.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:02:33.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:33.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622598+0000", 2024-05-11T23:02:33.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5909999999999998e-06 2024-05-11T23:02:33.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:33.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667401+0000", 2024-05-11T23:02:33.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044802980999999999 2024-05-11T23:02:33.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:33.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667439+0000", 2024-05-11T23:02:33.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8840000000000001e-05 2024-05-11T23:02:33.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:33.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667498+0000", 2024-05-11T23:02:33.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8709000000000003e-05 2024-05-11T23:02:33.825 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.825 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.825 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.825 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67842 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5468160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:33.825 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:33.825 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.143520009, 2024-05-11T23:02:33.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045005876, 2024-05-11T23:02:33.825 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:33.825 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:33.825 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:33.826 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:33.826 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:33.826 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:33.826 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67842 2024-05-11T23:02:33.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.826 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:33.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:33.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:33.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:33.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:33.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:33.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622601+0000", 2024-05-11T23:02:33.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:33.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:33.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622602+0000", 2024-05-11T23:02:33.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0780000000000001e-06 2024-05-11T23:02:33.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:33.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622605+0000", 2024-05-11T23:02:33.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.914e-06 2024-05-11T23:02:33.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:33.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622607+0000", 2024-05-11T23:02:33.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:02:33.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:33.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667516+0000", 2024-05-11T23:02:33.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044909156999999998 2024-05-11T23:02:33.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:33.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667554+0000", 2024-05-11T23:02:33.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7920000000000003e-05 2024-05-11T23:02:33.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:33.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667606+0000", 2024-05-11T23:02:33.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2111000000000003e-05 2024-05-11T23:02:33.829 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.829 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.829 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.829 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67843 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4067328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:33.829 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:33.829 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.143510483, 2024-05-11T23:02:33.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045119014999999998, 2024-05-11T23:02:33.829 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:33.829 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:33.830 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:33.830 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:33.830 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:33.830 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:33.830 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67843 2024-05-11T23:02:33.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.830 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:33.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:33.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:33.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:33.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:33.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:33.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624228+0000", 2024-05-11T23:02:33.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001617867 2024-05-11T23:02:33.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:33.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624232+0000", 2024-05-11T23:02:33.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-05-11T23:02:33.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:33.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624240+0000", 2024-05-11T23:02:33.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4589999999999998e-06 2024-05-11T23:02:33.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:33.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624247+0000", 2024-05-11T23:02:33.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-05-11T23:02:33.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:33.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667629+0000", 2024-05-11T23:02:33.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043382012999999997 2024-05-11T23:02:33.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:33.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667674+0000", 2024-05-11T23:02:33.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5172000000000001e-05 2024-05-11T23:02:33.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:33.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667729+0000", 2024-05-11T23:02:33.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5086000000000001e-05 2024-05-11T23:02:33.833 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.833 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.833 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.833 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67849 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4190208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:33.833 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:33.833 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.141752361999998, 2024-05-11T23:02:33.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044061433999999997, 2024-05-11T23:02:33.833 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:33.834 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:33.834 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:33.834 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:33.834 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:33.834 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:33.834 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67849 2024-05-11T23:02:33.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.834 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:33.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:33.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:33.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:33.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:33.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:33.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624369+0000", 2024-05-11T23:02:33.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5900000000000002e-07 2024-05-11T23:02:33.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:33.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624371+0000", 2024-05-11T23:02:33.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0310000000000001e-06 2024-05-11T23:02:33.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:33.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624376+0000", 2024-05-11T23:02:33.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4029999999999998e-06 2024-05-11T23:02:33.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:33.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624381+0000", 2024-05-11T23:02:33.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4349999999999996e-06 2024-05-11T23:02:33.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:33.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668331+0000", 2024-05-11T23:02:33.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043949191999999998 2024-05-11T23:02:33.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:33.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668370+0000", 2024-05-11T23:02:33.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9227999999999998e-05 2024-05-11T23:02:33.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:33.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668430+0000", 2024-05-11T23:02:33.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9685999999999997e-05 2024-05-11T23:02:33.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.837 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.837 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67850 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:33.837 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:33.837 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.14173113, 2024-05-11T23:02:33.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044151953000000001, 2024-05-11T23:02:33.838 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:33.838 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:33.838 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:33.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:33.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:33.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:33.838 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67850 2024-05-11T23:02:33.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.838 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:33.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:33.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:33.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:33.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:33.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:33.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624390+0000", 2024-05-11T23:02:33.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:02:33.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:33.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624392+0000", 2024-05-11T23:02:33.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1880000000000001e-06 2024-05-11T23:02:33.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:33.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624399+0000", 2024-05-11T23:02:33.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7390000000000002e-06 2024-05-11T23:02:33.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:33.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624405+0000", 2024-05-11T23:02:33.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1059999999999996e-06 2024-05-11T23:02:33.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:33.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668450+0000", 2024-05-11T23:02:33.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044045147999999999 2024-05-11T23:02:33.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:33.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668487+0000", 2024-05-11T23:02:33.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7450000000000002e-05 2024-05-11T23:02:33.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:33.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668541+0000", 2024-05-11T23:02:33.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3952000000000001e-05 2024-05-11T23:02:33.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.841 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.841 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67851 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5541888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:33.841 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:33.842 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.141704548, 2024-05-11T23:02:33.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044264413000000002, 2024-05-11T23:02:33.842 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:33.842 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:33.842 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:33.842 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:33.842 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:33.842 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:33.842 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67851 2024-05-11T23:02:33.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.842 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:33.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:33.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:33.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:33.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:33.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:33.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:33.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-05-11T23:02:33.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:33.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624419+0000", 2024-05-11T23:02:33.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.21e-06 2024-05-11T23:02:33.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:33.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624424+0000", 2024-05-11T23:02:33.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9050000000000002e-06 2024-05-11T23:02:33.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:33.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624430+0000", 2024-05-11T23:02:33.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-05-11T23:02:33.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:33.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668566+0000", 2024-05-11T23:02:33.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044136208000000003 2024-05-11T23:02:33.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:33.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668614+0000", 2024-05-11T23:02:33.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8022000000000002e-05 2024-05-11T23:02:33.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:33.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668680+0000", 2024-05-11T23:02:33.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6181999999999998e-05 2024-05-11T23:02:33.845 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.846 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.846 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.846 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67852 3.16 3.1a4a4af6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:33.846 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:33.846 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.141684081000001, 2024-05-11T23:02:33.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044278287999999999, 2024-05-11T23:02:33.846 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:33.846 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:33.846 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:33.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:33.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:33.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:33.847 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67852 2024-05-11T23:02:33.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.847 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:33.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:33.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:33.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:33.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:33.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:33.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:33.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626226+0000", 2024-05-11T23:02:33.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017893 2024-05-11T23:02:33.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:33.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626229+0000", 2024-05-11T23:02:33.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0709999999999999e-06 2024-05-11T23:02:33.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:33.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626239+0000", 2024-05-11T23:02:33.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0507e-05 2024-05-11T23:02:33.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:33.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626244+0000", 2024-05-11T23:02:33.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4880000000000001e-06 2024-05-11T23:02:33.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:33.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668706+0000", 2024-05-11T23:02:33.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042462054999999999 2024-05-11T23:02:33.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:33.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:33.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:33.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668715+0000", 2024-05-11T23:02:33.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.867e-06 2024-05-11T23:02:33.849 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.850 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.850 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.850 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:33.850 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:33.850 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:33.851 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:02:33.852 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:33.851+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:33.952 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:33.952+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:33.957 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:33.957 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:33.958 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:33.958 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:33.966 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:02:34.053 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:34.053+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:34.118 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:34.118 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:34.118 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:34.118 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:34.118 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:34.127 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:02:34.129 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone36 from 0xdbc2361 (overlap 0xdbc2361) to 0x2bf3a46 2024-05-11T23:02:34.171 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:34.171+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:34.256 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:34.256+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:34.277 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:34.277 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:34.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:34.277 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:34.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.278 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:34.278 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:34.278 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.621763356999999, 2024-05-11T23:02:34.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:02:34.278 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.278 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.278 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.279 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.279 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.279 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.279 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:02:34.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.279 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:34.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:34.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:02:34.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:02:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:02:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:02:34.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:02:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:02:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:02:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:02:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:02:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:02:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:02:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:02:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:02:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.618382167999997, 2024-05-11T23:02:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:02:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.288 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:02:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.289 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:02:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:02:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:02:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:02:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:02:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:02:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:02:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:02:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:02:34.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:02:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:02:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:02:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:02:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.618363461000001, 2024-05-11T23:02:34.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:02:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.296 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:02:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:02:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:02:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:02:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:02:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:02:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:02:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:02:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:02:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:02:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:02:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:02:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:02:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:02:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:02:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.618345318000003, 2024-05-11T23:02:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:02:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.300 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:02:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:34.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:02:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:02:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:02:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:02:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:02:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:02:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:02:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:02:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:02:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:02:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:02:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:02:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:02:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.618317830999999, 2024-05-11T23:02:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:02:34.304 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:02:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:34.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:02:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:02:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:02:34.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:02:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:02:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:02:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:02:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:02:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:02:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:02:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:02:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:02:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:02:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:02:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.308 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.618297302999999, 2024-05-11T23:02:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:02:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:02:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.309 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:34.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:02:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:02:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:02:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:02:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:02:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:02:34.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:02:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:02:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:02:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:02:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:02:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:02:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:02:34.312 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.618277528, 2024-05-11T23:02:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:02:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.313 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:02:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:02:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:02:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:02:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:02:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:02:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:02:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:02:34.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:02:34.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:02:34.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:02:34.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:02:34.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:02:34.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:02:34.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:02:34.317 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.317 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.317 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.317 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:34.317 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:34.317 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.618258722, 2024-05-11T23:02:34.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:02:34.317 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.317 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.317 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.317 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.317 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.317 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.317 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:02:34.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.318 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:34.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:34.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:02:34.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:02:34.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:02:34.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:02:34.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:02:34.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:02:34.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:02:34.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:02:34.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:02:34.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:02:34.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:02:34.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:02:34.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:02:34.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:02:34.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.324 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.325 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:34.325 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:34.325 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.618239799000001, 2024-05-11T23:02:34.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:02:34.325 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.326 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.326 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.326 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:02:34.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.327 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:34.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:34.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:34.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:02:34.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:02:34.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:02:34.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:02:34.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:02:34.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:02:34.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:02:34.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:02:34.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:02:34.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:02:34.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:02:34.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:02:34.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:02:34.332 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.332 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.332 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.332 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:34.333 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:34.333 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.618221136999999, 2024-05-11T23:02:34.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:02:34.333 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.333 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.333 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.333 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.333 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.333 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.333 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:02:34.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.333 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:34.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:34.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:34.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:02:34.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:02:34.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:02:34.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:02:34.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:02:34.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:02:34.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:02:34.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:02:34.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:02:34.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:02:34.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:02:34.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:02:34.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:02:34.336 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.336 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.336 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.336 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:34.337 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:34.337 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.617875974999997, 2024-05-11T23:02:34.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:02:34.337 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.337 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.337 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.337 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:02:34.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.337 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:34.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:34.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:02:34.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:02:34.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:02:34.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:02:34.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:02:34.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:02:34.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:02:34.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:02:34.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:02:34.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:02:34.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:02:34.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:02:34.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:02:34.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:02:34.340 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.340 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.340 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.341 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:34.341 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:34.341 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.617850425, 2024-05-11T23:02:34.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:02:34.341 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.341 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.341 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.341 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.341 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.341 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.341 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:02:34.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.341 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:34.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:34.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:34.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:02:34.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:02:34.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:02:34.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:02:34.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:02:34.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:02:34.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:02:34.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:02:34.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:02:34.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:02:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:02:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:02:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:02:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.344 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.617768167000001, 2024-05-11T23:02:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:02:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:02:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:02:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:02:34.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:02:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:02:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:02:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:02:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:02:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:02:34.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:02:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:02:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:02:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:02:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:02:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.348 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.617745720000002, 2024-05-11T23:02:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:02:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:02:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:34.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:02:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:02:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:02:34.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:02:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:02:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:02:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:02:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:02:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:02:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:02:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:02:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:02:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:02:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:02:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.617726261999998, 2024-05-11T23:02:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:02:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:02:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:02:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:02:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:02:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:02:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:02:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:02:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:02:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:02:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:02:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:02:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:02:34.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:02:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:02:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.617706202000001, 2024-05-11T23:02:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:02:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.357 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:02:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:02:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:02:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:02:34.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:02:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:02:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:02:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:02:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:02:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:02:34.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:02:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:02:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:02:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:02:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:34.361 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.617686931999998, 2024-05-11T23:02:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:02:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:02:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:34.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:34.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:34.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:02:34.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:02:34.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:02:34.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:02:34.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:02:34.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:02:34.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:02:34.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:02:34.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:02:34.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:02:34.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:02:34.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:02:34.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:02:34.365 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.365 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.365 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.366 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:34.366 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:34.366 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.617140866, 2024-05-11T23:02:34.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:02:34.366 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.366 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.366 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.366 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.366 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.366 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.366 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:02:34.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.366 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:34.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:34.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:02:34.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:02:34.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:02:34.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:02:34.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:02:34.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:02:34.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:02:34.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:02:34.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:02:34.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:02:34.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:02:34.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:02:34.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:02:34.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:02:34.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.369 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.370 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:34.370 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:34.370 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.617109081999999, 2024-05-11T23:02:34.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:02:34.370 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.370 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.370 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.370 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:02:34.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.370 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:34.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:34.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:34.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:02:34.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:02:34.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:02:34.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:02:34.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:02:34.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:02:34.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:02:34.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:02:34.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:02:34.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:02:34.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:02:34.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:02:34.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:02:34.373 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.373 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.373 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.373 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:02:34.374 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:34.374 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 32.179863617999999, 2024-05-11T23:02:34.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:02:34.374 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.374 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.374 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:34.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.374 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:02:34.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.374 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:34.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:34.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:02:34.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:02:34.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:02:34.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:02:34.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:02:34.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:02:34.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:02:34.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:02:34.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:02:34.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:02:34.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:02:34.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:02:34.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:02:34.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:02:34.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:02:34.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:34.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:02:34.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:02:34.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:34.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:02:34.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:02:34.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:34.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:02:34.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:02:34.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:02:34.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:02:34.379 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.379 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.379 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.379 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.379 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.379 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:34.380 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:02:34.381 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:34.380+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:34.482 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:34.481+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:34.573 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:34.573 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:34.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.573 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113761 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [call rbd.sparse_copyup in=999496b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:02:34.574 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:02:34.574 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.043462636999999998, 2024-05-11T23:02:34.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043492784, 2024-05-11T23:02:34.574 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.574 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.574 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for sub ops", 2024-05-11T23:02:34.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.574 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113761 2024-05-11T23:02:34.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.574 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:02:34.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:02:34.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530309+0000", 2024-05-11T23:02:34.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:02:34.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531242+0000", 2024-05-11T23:02:34.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093316899999999997 2024-05-11T23:02:34.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531249+0000", 2024-05-11T23:02:34.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8140000000000003e-06 2024-05-11T23:02:34.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531253+0000", 2024-05-11T23:02:34.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4290000000000003e-06 2024-05-11T23:02:34.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570325+0000", 2024-05-11T23:02:34.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039071379000000003 2024-05-11T23:02:34.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570382+0000", 2024-05-11T23:02:34.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6988999999999999e-05 2024-05-11T23:02:34.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:02:34.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570555+0000", 2024-05-11T23:02:34.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017379799999999999 2024-05-11T23:02:34.577 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.577 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.577 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113762 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [delete] snapc 27=[27] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:02:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:02:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.042508685999999997, 2024-05-11T23:02:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042621780999999997, 2024-05-11T23:02:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for sub ops", 2024-05-11T23:02:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113762 2024-05-11T23:02:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.578 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:02:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:02:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531263+0000", 2024-05-11T23:02:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.85e-07 2024-05-11T23:02:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531265+0000", 2024-05-11T23:02:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.827e-06 2024-05-11T23:02:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531268+0000", 2024-05-11T23:02:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3780000000000001e-06 2024-05-11T23:02:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531271+0000", 2024-05-11T23:02:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8509999999999999e-06 2024-05-11T23:02:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572298+0000", 2024-05-11T23:02:34.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041026990999999999 2024-05-11T23:02:34.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572345+0000", 2024-05-11T23:02:34.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6878000000000001e-05 2024-05-11T23:02:34.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:02:34.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572443+0000", 2024-05-11T23:02:34.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8184999999999996e-05 2024-05-11T23:02:34.581 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.581 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.581 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.581 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.581 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-11T23:02:34.582 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 2 2024-05-11T23:02:34.582 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:34.583 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:34.582+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:34.584 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:02:34.684 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:34.683+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:34.725 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-05-11T23:02:34.735 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:34.735 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:34.735 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:34.735 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:34.735 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:34.744 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:02:34.770 INFO:teuthology.orchestra.run.smithi005.stdout:3847 trunc from 0xdbc2361 to 0x4780719 2024-05-11T23:02:34.783 INFO:teuthology.orchestra.run.smithi005.stdout:3849 clone 38 order 18 su 1024 sc 15 2024-05-11T23:02:34.784 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:34.784+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:34.886 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:34.886+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:34.889 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.889 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:02:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 77.725342916000002, 2024-05-11T23:02:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:02:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:02:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:02:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:02:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:02:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:02:34.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:02:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:02:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:02:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:02:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:02:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:02:34.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:02:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:02:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:02:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:02:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:02:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:02:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:02:34.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:02:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:02:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:02:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:02:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:02:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:02:34.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:02:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:02:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:02:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:02:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:02:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:02:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:02:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78429 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:02:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.924073574000001, 2024-05-11T23:02:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044563497000000001, 2024-05-11T23:02:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78429 2024-05-11T23:02:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:02:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:02:34.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964872+0000", 2024-05-11T23:02:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035070300000000001 2024-05-11T23:02:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964876+0000", 2024-05-11T23:02:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5009999999999998e-06 2024-05-11T23:02:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964902+0000", 2024-05-11T23:02:34.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5715999999999999e-05 2024-05-11T23:02:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964910+0000", 2024-05-11T23:02:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3930000000000002e-06 2024-05-11T23:02:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008898+0000", 2024-05-11T23:02:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043988772000000002 2024-05-11T23:02:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008981+0000", 2024-05-11T23:02:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3140999999999996e-05 2024-05-11T23:02:34.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009085+0000", 2024-05-11T23:02:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010327099999999999 2024-05-11T23:02:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78431 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:02:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.923613266, 2024-05-11T23:02:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044542033000000002, 2024-05-11T23:02:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.900 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78431 2024-05-11T23:02:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:02:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:02:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965038+0000", 2024-05-11T23:02:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6993000000000001e-05 2024-05-11T23:02:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965043+0000", 2024-05-11T23:02:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7040000000000002e-06 2024-05-11T23:02:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965053+0000", 2024-05-11T23:02:34.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8539999999999992e-06 2024-05-11T23:02:34.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965061+0000", 2024-05-11T23:02:34.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9810000000000003e-06 2024-05-11T23:02:34.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009350+0000", 2024-05-11T23:02:34.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044288739000000001 2024-05-11T23:02:34.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009429+0000", 2024-05-11T23:02:34.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9354e-05 2024-05-11T23:02:34.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009524+0000", 2024-05-11T23:02:34.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4407999999999995e-05 2024-05-11T23:02:34.904 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.904 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.904 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.904 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78432 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:34.904 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:02:34.904 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.923522418000001, 2024-05-11T23:02:34.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044635421000000002, 2024-05-11T23:02:34.905 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.905 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.905 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.905 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.905 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.905 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.905 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78432 2024-05-11T23:02:34.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.905 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:02:34.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:02:34.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965343+0000", 2024-05-11T23:02:34.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027056500000000001 2024-05-11T23:02:34.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965347+0000", 2024-05-11T23:02:34.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3540000000000002e-06 2024-05-11T23:02:34.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965356+0000", 2024-05-11T23:02:34.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2140000000000002e-06 2024-05-11T23:02:34.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965363+0000", 2024-05-11T23:02:34.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4150000000000004e-06 2024-05-11T23:02:34.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009551+0000", 2024-05-11T23:02:34.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044187917 2024-05-11T23:02:34.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009626+0000", 2024-05-11T23:02:34.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5024000000000005e-05 2024-05-11T23:02:34.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009708+0000", 2024-05-11T23:02:34.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1932000000000001e-05 2024-05-11T23:02:34.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.908 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.908 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78433 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:34.908 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:02:34.908 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.923221530999999, 2024-05-11T23:02:34.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044564864000000003, 2024-05-11T23:02:34.909 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.909 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.909 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.909 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78433 2024-05-11T23:02:34.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.909 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:02:34.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:02:34.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966225+0000", 2024-05-11T23:02:34.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085220700000000001 2024-05-11T23:02:34.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966230+0000", 2024-05-11T23:02:34.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0690000000000003e-06 2024-05-11T23:02:34.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966239+0000", 2024-05-11T23:02:34.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-05-11T23:02:34.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966245+0000", 2024-05-11T23:02:34.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5520000000000001e-06 2024-05-11T23:02:34.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009743+0000", 2024-05-11T23:02:34.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043498311999999997 2024-05-11T23:02:34.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009848+0000", 2024-05-11T23:02:34.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104885 2024-05-11T23:02:34.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009938+0000", 2024-05-11T23:02:34.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9812e-05 2024-05-11T23:02:34.912 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.912 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.912 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.912 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78580 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 999424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:34.912 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:02:34.912 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.920108848000002, 2024-05-11T23:02:34.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044493076999999999, 2024-05-11T23:02:34.913 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.913 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.913 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.913 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.913 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.913 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.913 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78580 2024-05-11T23:02:34.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.913 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:02:34.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:02:34.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:02:34.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:02:34.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968489+0000", 2024-05-11T23:02:34.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.52e-06 2024-05-11T23:02:34.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968496+0000", 2024-05-11T23:02:34.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9419999999999998e-06 2024-05-11T23:02:34.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968501+0000", 2024-05-11T23:02:34.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8720000000000001e-06 2024-05-11T23:02:34.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012690+0000", 2024-05-11T23:02:34.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044189655000000001 2024-05-11T23:02:34.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012848+0000", 2024-05-11T23:02:34.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015758100000000001 2024-05-11T23:02:34.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012979+0000", 2024-05-11T23:02:34.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000131066 2024-05-11T23:02:34.916 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.916 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.916 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.916 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78581 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:34.917 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:02:34.917 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.920087844000001, 2024-05-11T23:02:34.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044616243, 2024-05-11T23:02:34.917 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.917 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.917 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.917 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.917 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.917 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.917 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78581 2024-05-11T23:02:34.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.917 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:02:34.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:02:34.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970227+0000", 2024-05-11T23:02:34.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017200049999999999 2024-05-11T23:02:34.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970232+0000", 2024-05-11T23:02:34.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000003e-06 2024-05-11T23:02:34.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970242+0000", 2024-05-11T23:02:34.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0192e-05 2024-05-11T23:02:34.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970249+0000", 2024-05-11T23:02:34.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.888e-06 2024-05-11T23:02:34.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013008+0000", 2024-05-11T23:02:34.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042759495000000002 2024-05-11T23:02:34.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013063+0000", 2024-05-11T23:02:34.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5071999999999997e-05 2024-05-11T23:02:34.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013123+0000", 2024-05-11T23:02:34.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9930999999999999e-05 2024-05-11T23:02:34.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.920 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.920 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78589 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:34.921 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:02:34.921 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.918168600000001, 2024-05-11T23:02:34.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044587697000000003, 2024-05-11T23:02:34.921 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.921 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.921 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.921 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78589 2024-05-11T23:02:34.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.921 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:02:34.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:02:34.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970427+0000", 2024-05-11T23:02:34.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:02:34.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970429+0000", 2024-05-11T23:02:34.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3800000000000001e-06 2024-05-11T23:02:34.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970435+0000", 2024-05-11T23:02:34.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2530000000000001e-06 2024-05-11T23:02:34.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970441+0000", 2024-05-11T23:02:34.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-05-11T23:02:34.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014822+0000", 2024-05-11T23:02:34.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04438168 2024-05-11T23:02:34.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014911+0000", 2024-05-11T23:02:34.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8882999999999996e-05 2024-05-11T23:02:34.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015014+0000", 2024-05-11T23:02:34.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010263399999999999 2024-05-11T23:02:34.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.924 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.924 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.925 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78590 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:34.925 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:02:34.925 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.918147801, 2024-05-11T23:02:34.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044880120000000003, 2024-05-11T23:02:34.925 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.925 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.925 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.925 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.925 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78590 2024-05-11T23:02:34.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.925 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:02:34.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:02:34.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970918+0000", 2024-05-11T23:02:34.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000470834 2024-05-11T23:02:34.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970925+0000", 2024-05-11T23:02:34.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1690000000000003e-06 2024-05-11T23:02:34.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970934+0000", 2024-05-11T23:02:34.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9099999999999994e-06 2024-05-11T23:02:34.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970941+0000", 2024-05-11T23:02:34.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7410000000000003e-06 2024-05-11T23:02:34.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015049+0000", 2024-05-11T23:02:34.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044108392000000003 2024-05-11T23:02:34.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015133+0000", 2024-05-11T23:02:34.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3568999999999999e-05 2024-05-11T23:02:34.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015327+0000", 2024-05-11T23:02:34.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019450500000000001 2024-05-11T23:02:34.928 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.928 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.928 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.929 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78591 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:34.929 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:02:34.929 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.917644384999999, 2024-05-11T23:02:34.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044618497, 2024-05-11T23:02:34.929 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.929 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.929 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.929 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.929 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.929 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.929 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78591 2024-05-11T23:02:34.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.929 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:02:34.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:02:34.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970951+0000", 2024-05-11T23:02:34.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:02:34.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970953+0000", 2024-05-11T23:02:34.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:02:34.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970956+0000", 2024-05-11T23:02:34.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-05-11T23:02:34.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970959+0000", 2024-05-11T23:02:34.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9409999999999999e-06 2024-05-11T23:02:34.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015367+0000", 2024-05-11T23:02:34.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044407621000000001 2024-05-11T23:02:34.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015453+0000", 2024-05-11T23:02:34.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6685000000000001e-05 2024-05-11T23:02:34.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015569+0000", 2024-05-11T23:02:34.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115396 2024-05-11T23:02:34.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.932 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.933 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78592 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:34.933 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:02:34.933 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.917631577000002, 2024-05-11T23:02:34.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044831546999999999, 2024-05-11T23:02:34.933 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.933 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.933 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.933 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.933 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78592 2024-05-11T23:02:34.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.933 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:02:34.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:02:34.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972226+0000", 2024-05-11T23:02:34.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00126246 2024-05-11T23:02:34.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972230+0000", 2024-05-11T23:02:34.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1860000000000002e-06 2024-05-11T23:02:34.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972237+0000", 2024-05-11T23:02:34.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:02:34.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972241+0000", 2024-05-11T23:02:34.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.284e-06 2024-05-11T23:02:34.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015611+0000", 2024-05-11T23:02:34.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043369461999999998 2024-05-11T23:02:34.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015695+0000", 2024-05-11T23:02:34.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4458999999999999e-05 2024-05-11T23:02:34.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015795+0000", 2024-05-11T23:02:34.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9475999999999994e-05 2024-05-11T23:02:34.936 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.936 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.936 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.937 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78593 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:34.937 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:02:34.937 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.916344658, 2024-05-11T23:02:34.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046045214000000001, 2024-05-11T23:02:34.937 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.937 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.937 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.937 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.937 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.937 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.937 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78593 2024-05-11T23:02:34.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.937 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:02:34.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:02:34.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:02:34.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7599999999999998e-07 2024-05-11T23:02:34.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972252+0000", 2024-05-11T23:02:34.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7349999999999999e-06 2024-05-11T23:02:34.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972256+0000", 2024-05-11T23:02:34.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5659999999999998e-06 2024-05-11T23:02:34.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972259+0000", 2024-05-11T23:02:34.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8040000000000002e-06 2024-05-11T23:02:34.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018088+0000", 2024-05-11T23:02:34.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045829316000000002 2024-05-11T23:02:34.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018179+0000", 2024-05-11T23:02:34.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1299999999999997e-05 2024-05-11T23:02:34.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018295+0000", 2024-05-11T23:02:34.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116117 2024-05-11T23:02:34.940 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.940 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.940 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.941 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78594 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:34.941 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:02:34.941 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.916332298, 2024-05-11T23:02:34.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046245204999999998, 2024-05-11T23:02:34.941 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.941 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.941 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.941 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.941 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.941 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.941 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78594 2024-05-11T23:02:34.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.941 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:02:34.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:02:34.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972263+0000", 2024-05-11T23:02:34.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:02:34.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972264+0000", 2024-05-11T23:02:34.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.206e-06 2024-05-11T23:02:34.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972267+0000", 2024-05-11T23:02:34.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0690000000000002e-06 2024-05-11T23:02:34.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972270+0000", 2024-05-11T23:02:34.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5610000000000001e-06 2024-05-11T23:02:34.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018330+0000", 2024-05-11T23:02:34.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04606085 2024-05-11T23:02:34.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018414+0000", 2024-05-11T23:02:34.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3125000000000001e-05 2024-05-11T23:02:34.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018508+0000", 2024-05-11T23:02:34.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4153000000000005e-05 2024-05-11T23:02:34.945 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.945 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.945 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.945 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78595 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:34.945 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:02:34.945 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.916319432999998, 2024-05-11T23:02:34.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046396698, 2024-05-11T23:02:34.945 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.945 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.945 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.945 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.946 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78595 2024-05-11T23:02:34.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.946 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:02:34.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:02:34.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972276+0000", 2024-05-11T23:02:34.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5600000000000002e-07 2024-05-11T23:02:34.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972277+0000", 2024-05-11T23:02:34.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.576e-06 2024-05-11T23:02:34.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972280+0000", 2024-05-11T23:02:34.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-05-11T23:02:34.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972283+0000", 2024-05-11T23:02:34.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:02:34.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018538+0000", 2024-05-11T23:02:34.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046255255000000002 2024-05-11T23:02:34.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018605+0000", 2024-05-11T23:02:34.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6847000000000006e-05 2024-05-11T23:02:34.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018672+0000", 2024-05-11T23:02:34.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7045000000000006e-05 2024-05-11T23:02:34.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.949 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.949 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78596 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:34.949 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:02:34.949 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.916308583999999, 2024-05-11T23:02:34.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046586806000000001, 2024-05-11T23:02:34.949 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.949 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.949 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.950 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78596 2024-05-11T23:02:34.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.950 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:02:34.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:02:34.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:02:34.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:02:34.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972288+0000", 2024-05-11T23:02:34.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2550000000000001e-06 2024-05-11T23:02:34.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972291+0000", 2024-05-11T23:02:34.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9490000000000001e-06 2024-05-11T23:02:34.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972293+0000", 2024-05-11T23:02:34.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5069999999999999e-06 2024-05-11T23:02:34.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018707+0000", 2024-05-11T23:02:34.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046413604999999997 2024-05-11T23:02:34.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018782+0000", 2024-05-11T23:02:34.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5556000000000006e-05 2024-05-11T23:02:34.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018873+0000", 2024-05-11T23:02:34.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0703000000000002e-05 2024-05-11T23:02:34.953 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.953 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.953 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.953 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78597 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:34.953 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:02:34.953 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.916298811000001, 2024-05-11T23:02:34.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046825787000000001, 2024-05-11T23:02:34.953 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.953 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.953 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.954 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.954 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78597 2024-05-11T23:02:34.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.954 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:02:34.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:02:34.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:02:34.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-05-11T23:02:34.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972297+0000", 2024-05-11T23:02:34.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3069999999999999e-06 2024-05-11T23:02:34.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972300+0000", 2024-05-11T23:02:34.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:02:34.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972303+0000", 2024-05-11T23:02:34.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5509999999999998e-06 2024-05-11T23:02:34.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018911+0000", 2024-05-11T23:02:34.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046608194999999998 2024-05-11T23:02:34.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018986+0000", 2024-05-11T23:02:34.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5360999999999999e-05 2024-05-11T23:02:34.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019122+0000", 2024-05-11T23:02:34.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013526 2024-05-11T23:02:34.957 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.957 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.957 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.957 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78598 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:34.957 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:02:34.957 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.916288910999999, 2024-05-11T23:02:34.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046998572000000002, 2024-05-11T23:02:34.957 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.957 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.957 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:34.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.958 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.958 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.958 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78598 2024-05-11T23:02:34.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.958 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:02:34.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:02:34.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:02:34.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:02:34.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972307+0000", 2024-05-11T23:02:34.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1799999999999999e-06 2024-05-11T23:02:34.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972310+0000", 2024-05-11T23:02:34.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:02:34.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972313+0000", 2024-05-11T23:02:34.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4760000000000001e-06 2024-05-11T23:02:34.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019149+0000", 2024-05-11T23:02:34.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046836056000000001 2024-05-11T23:02:34.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019203+0000", 2024-05-11T23:02:34.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-05 2024-05-11T23:02:34.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019304+0000", 2024-05-11T23:02:34.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101537 2024-05-11T23:02:34.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.961 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.961 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:105234 3.15 3:adf885e5:::rbd_object_map.111e913a615f.0000000000000025:head [writefull 0~39] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-11T23:02:34.961 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:02:34.961 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.8809732009999998, 2024-05-11T23:02:34.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77797599100000003, 2024-05-11T23:02:34.961 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.961 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.962 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:34.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.962 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 105234 2024-05-11T23:02:34.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.962 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:02:34.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:02:34.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007623+0000", 2024-05-11T23:02:34.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2240000000000001e-06 2024-05-11T23:02:34.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007627+0000", 2024-05-11T23:02:34.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:02:34.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007636+0000", 2024-05-11T23:02:34.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2409999999999995e-06 2024-05-11T23:02:34.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007641+0000", 2024-05-11T23:02:34.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6810000000000003e-06 2024-05-11T23:02:34.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007676+0000", 2024-05-11T23:02:34.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5061999999999997e-05 2024-05-11T23:02:34.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:02:34.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007682+0000", 2024-05-11T23:02:34.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0399999999999998e-06 2024-05-11T23:02:34.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783880+0000", 2024-05-11T23:02:34.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77619752900000005 2024-05-11T23:02:34.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783995+0000", 2024-05-11T23:02:34.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011550800000000001 2024-05-11T23:02:34.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:02:34.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784082+0000", 2024-05-11T23:02:34.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6347999999999994e-05 2024-05-11T23:02:34.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:34.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784682+0000", 2024-05-11T23:02:34.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000600162 2024-05-11T23:02:34.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:34.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785539+0000", 2024-05-11T23:02:34.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085699099999999998 2024-05-11T23:02:34.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:34.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785572+0000", 2024-05-11T23:02:34.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3182999999999997e-05 2024-05-11T23:02:34.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785598+0000", 2024-05-11T23:02:34.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5522000000000001e-05 2024-05-11T23:02:34.967 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.967 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.967 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.967 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113761 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [call rbd.sparse_copyup in=999496b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:02:34.967 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:02:34.967 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.35828628200000001, 2024-05-11T23:02:34.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048572964000000003, 2024-05-11T23:02:34.967 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.967 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.967 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:34.967 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.968 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.968 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.968 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113761 2024-05-11T23:02:34.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.968 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:02:34.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:02:34.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530309+0000", 2024-05-11T23:02:34.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:02:34.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531242+0000", 2024-05-11T23:02:34.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093316899999999997 2024-05-11T23:02:34.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531249+0000", 2024-05-11T23:02:34.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8140000000000003e-06 2024-05-11T23:02:34.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531253+0000", 2024-05-11T23:02:34.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4290000000000003e-06 2024-05-11T23:02:34.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570325+0000", 2024-05-11T23:02:34.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039071379000000003 2024-05-11T23:02:34.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570382+0000", 2024-05-11T23:02:34.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6988999999999999e-05 2024-05-11T23:02:34.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:02:34.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570555+0000", 2024-05-11T23:02:34.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017379799999999999 2024-05-11T23:02:34.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:34.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.575953+0000", 2024-05-11T23:02:34.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053975539999999997 2024-05-11T23:02:34.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:34.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578778+0000", 2024-05-11T23:02:34.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002825616 2024-05-11T23:02:34.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:34.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578837+0000", 2024-05-11T23:02:34.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9005000000000002e-05 2024-05-11T23:02:34.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578881+0000", 2024-05-11T23:02:34.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3970000000000001e-05 2024-05-11T23:02:34.972 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.972 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.972 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.972 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113762 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [delete] snapc 27=[27] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:02:34.972 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:02:34.972 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.357332331, 2024-05-11T23:02:34.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047916260000000002, 2024-05-11T23:02:34.973 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:34.973 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:34.973 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:34.973 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:34.973 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:34.973 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:34.973 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113762 2024-05-11T23:02:34.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.973 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:34.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:34.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:02:34.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:34.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:02:34.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:34.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:34.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531263+0000", 2024-05-11T23:02:34.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.85e-07 2024-05-11T23:02:34.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:34.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531265+0000", 2024-05-11T23:02:34.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.827e-06 2024-05-11T23:02:34.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:34.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531268+0000", 2024-05-11T23:02:34.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3780000000000001e-06 2024-05-11T23:02:34.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:34.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531271+0000", 2024-05-11T23:02:34.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8509999999999999e-06 2024-05-11T23:02:34.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:34.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572298+0000", 2024-05-11T23:02:34.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041026990999999999 2024-05-11T23:02:34.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:34.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572345+0000", 2024-05-11T23:02:34.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6878000000000001e-05 2024-05-11T23:02:34.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:02:34.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572443+0000", 2024-05-11T23:02:34.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8184999999999996e-05 2024-05-11T23:02:34.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:34.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578585+0000", 2024-05-11T23:02:34.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006141919 2024-05-11T23:02:34.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:34.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579126+0000", 2024-05-11T23:02:34.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00054063799999999997 2024-05-11T23:02:34.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:34.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579154+0000", 2024-05-11T23:02:34.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8042999999999999e-05 2024-05-11T23:02:34.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:34.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:34.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:34.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579179+0000", 2024-05-11T23:02:34.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4765000000000001e-05 2024-05-11T23:02:34.977 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.977 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.977 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.977 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:34.977 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:34.977 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:34.979 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:02:35.023 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:35.023+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:35.033 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-11T23:02:35.088 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:35.087+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:35.105 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:35.105 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:35.105 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:02:35.105 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:35.115 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:02:35.138 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone37 from 0x4780719 (overlap 0x4780719) to 0x0 2024-05-11T23:02:35.189 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:35.189+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:35.261 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:35.261 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:35.261 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:35.261 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:35.261 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:35.273 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:02:35.290 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:35.290+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:35.391 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:35.391+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:35.456 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:35.457 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:35.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:35.457 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:35.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.457 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71308 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:35.457 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.146883+0000", 2024-05-11T23:02:35.457 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.308981049, 2024-05-11T23:02:35.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030851522999999999, 2024-05-11T23:02:35.457 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:35.457 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:35.457 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:35.457 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:35.458 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:35.458 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:35.458 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71308 2024-05-11T23:02:35.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.458 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:35.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:35.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146883+0000", 2024-05-11T23:02:35.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:35.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146883+0000", 2024-05-11T23:02:35.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:35.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146884+0000", 2024-05-11T23:02:35.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3399999999999999e-07 2024-05-11T23:02:35.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:35.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146893+0000", 2024-05-11T23:02:35.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2159999999999995e-06 2024-05-11T23:02:35.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:35.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146899+0000", 2024-05-11T23:02:35.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.302e-06 2024-05-11T23:02:35.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:35.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146905+0000", 2024-05-11T23:02:35.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9120000000000003e-06 2024-05-11T23:02:35.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:35.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177730+0000", 2024-05-11T23:02:35.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030824449 2024-05-11T23:02:35.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:35.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177735+0000", 2024-05-11T23:02:35.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1100000000000002e-06 2024-05-11T23:02:35.461 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.461 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.461 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.461 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71309 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:35.461 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.146921+0000", 2024-05-11T23:02:35.461 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.308942786999999, 2024-05-11T23:02:35.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030854467999999999, 2024-05-11T23:02:35.461 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:35.461 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:35.461 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:35.461 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:35.462 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:35.462 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:35.462 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71309 2024-05-11T23:02:35.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.462 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:35.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:35.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146921+0000", 2024-05-11T23:02:35.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:35.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146921+0000", 2024-05-11T23:02:35.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:35.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146922+0000", 2024-05-11T23:02:35.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1799999999999995e-07 2024-05-11T23:02:35.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:35.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146924+0000", 2024-05-11T23:02:35.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5009999999999999e-06 2024-05-11T23:02:35.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:35.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146938+0000", 2024-05-11T23:02:35.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3286e-05 2024-05-11T23:02:35.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:35.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146944+0000", 2024-05-11T23:02:35.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7839999999999998e-06 2024-05-11T23:02:35.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:35.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177766+0000", 2024-05-11T23:02:35.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030821988000000002 2024-05-11T23:02:35.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:35.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177776+0000", 2024-05-11T23:02:35.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3910000000000004e-06 2024-05-11T23:02:35.465 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.465 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.465 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.465 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71310 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:35.465 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.146954+0000", 2024-05-11T23:02:35.465 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.308910449999999, 2024-05-11T23:02:35.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030837251999999999, 2024-05-11T23:02:35.466 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:35.466 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:35.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:35.466 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:35.466 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:35.466 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:35.466 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71310 2024-05-11T23:02:35.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.466 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:35.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:35.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146954+0000", 2024-05-11T23:02:35.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:35.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146954+0000", 2024-05-11T23:02:35.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:35.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146954+0000", 2024-05-11T23:02:35.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:02:35.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:35.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146957+0000", 2024-05-11T23:02:35.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.463e-06 2024-05-11T23:02:35.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:35.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146963+0000", 2024-05-11T23:02:35.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6460000000000004e-06 2024-05-11T23:02:35.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:35.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146968+0000", 2024-05-11T23:02:35.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0250000000000002e-06 2024-05-11T23:02:35.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:35.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177787+0000", 2024-05-11T23:02:35.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030818680000000001 2024-05-11T23:02:35.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:35.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177791+0000", 2024-05-11T23:02:35.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0090000000000001e-06 2024-05-11T23:02:35.469 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.469 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.469 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.470 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71313 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:35.470 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147016+0000", 2024-05-11T23:02:35.470 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.308847768, 2024-05-11T23:02:35.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030841770000000001, 2024-05-11T23:02:35.470 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:35.470 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:35.470 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:35.470 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:35.471 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:35.471 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:35.471 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71313 2024-05-11T23:02:35.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.471 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:35.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:35.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147016+0000", 2024-05-11T23:02:35.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:35.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147016+0000", 2024-05-11T23:02:35.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:35.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147017+0000", 2024-05-11T23:02:35.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8599999999999999e-07 2024-05-11T23:02:35.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:35.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147019+0000", 2024-05-11T23:02:35.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0329999999999998e-06 2024-05-11T23:02:35.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:35.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147025+0000", 2024-05-11T23:02:35.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0299999999999999e-06 2024-05-11T23:02:35.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:35.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147029+0000", 2024-05-11T23:02:35.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5979999999999999e-06 2024-05-11T23:02:35.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:35.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177848+0000", 2024-05-11T23:02:35.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030818662 2024-05-11T23:02:35.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:35.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177858+0000", 2024-05-11T23:02:35.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0061000000000001e-05 2024-05-11T23:02:35.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.474 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.475 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71327 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:35.475 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147299+0000", 2024-05-11T23:02:35.475 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.308565080000001, 2024-05-11T23:02:35.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030857026999999999, 2024-05-11T23:02:35.475 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:35.475 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:35.475 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:35.475 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:35.475 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:35.476 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:35.476 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71327 2024-05-11T23:02:35.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.476 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:35.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:35.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147299+0000", 2024-05-11T23:02:35.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:35.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147299+0000", 2024-05-11T23:02:35.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:35.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147299+0000", 2024-05-11T23:02:35.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1500000000000001e-07 2024-05-11T23:02:35.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:35.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147303+0000", 2024-05-11T23:02:35.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1459999999999998e-06 2024-05-11T23:02:35.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:35.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147312+0000", 2024-05-11T23:02:35.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3299999999999999e-06 2024-05-11T23:02:35.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:35.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147321+0000", 2024-05-11T23:02:35.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7420000000000003e-06 2024-05-11T23:02:35.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:35.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178145+0000", 2024-05-11T23:02:35.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030824852999999999 2024-05-11T23:02:35.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:35.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178156+0000", 2024-05-11T23:02:35.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0740999999999999e-05 2024-05-11T23:02:35.479 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.479 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.479 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.480 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71328 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:35.480 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147326+0000", 2024-05-11T23:02:35.480 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.308538597999998, 2024-05-11T23:02:35.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030846630999999999, 2024-05-11T23:02:35.480 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:35.480 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:35.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:35.480 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:35.480 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:35.480 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:35.480 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71328 2024-05-11T23:02:35.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.480 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:35.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:35.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147326+0000", 2024-05-11T23:02:35.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:35.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147326+0000", 2024-05-11T23:02:35.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:35.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147326+0000", 2024-05-11T23:02:35.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5400000000000002e-07 2024-05-11T23:02:35.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:35.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147327+0000", 2024-05-11T23:02:35.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2500000000000001e-06 2024-05-11T23:02:35.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:35.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147331+0000", 2024-05-11T23:02:35.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5499999999999999e-06 2024-05-11T23:02:35.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:35.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147333+0000", 2024-05-11T23:02:35.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5450000000000002e-06 2024-05-11T23:02:35.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:35.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178168+0000", 2024-05-11T23:02:35.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030834934000000001 2024-05-11T23:02:35.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:35.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178172+0000", 2024-05-11T23:02:35.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0980000000000004e-06 2024-05-11T23:02:35.483 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.483 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.483 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.484 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71329 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:35.484 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147336+0000", 2024-05-11T23:02:35.484 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.308527743999999, 2024-05-11T23:02:35.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030865317999999999, 2024-05-11T23:02:35.484 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:35.484 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:35.484 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:35.484 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:35.484 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:35.484 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:35.484 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71329 2024-05-11T23:02:35.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.484 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:35.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:35.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147336+0000", 2024-05-11T23:02:35.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:35.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147336+0000", 2024-05-11T23:02:35.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:35.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147337+0000", 2024-05-11T23:02:35.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:35.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147338+0000", 2024-05-11T23:02:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5090000000000001e-06 2024-05-11T23:02:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147344+0000", 2024-05-11T23:02:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.66e-06 2024-05-11T23:02:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147348+0000", 2024-05-11T23:02:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2980000000000002e-06 2024-05-11T23:02:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178192+0000", 2024-05-11T23:02:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030843425000000001 2024-05-11T23:02:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178202+0000", 2024-05-11T23:02:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0217000000000001e-05 2024-05-11T23:02:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71330 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147355+0000", 2024-05-11T23:02:35.487 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.30850959, 2024-05-11T23:02:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030862031000000002, 2024-05-11T23:02:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71330 2024-05-11T23:02:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147355+0000", 2024-05-11T23:02:35.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147355+0000", 2024-05-11T23:02:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147355+0000", 2024-05-11T23:02:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.39e-07 2024-05-11T23:02:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147365+0000", 2024-05-11T23:02:35.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7140000000000005e-06 2024-05-11T23:02:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147370+0000", 2024-05-11T23:02:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2979999999999999e-06 2024-05-11T23:02:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147373+0000", 2024-05-11T23:02:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.41e-06 2024-05-11T23:02:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:35.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178213+0000", 2024-05-11T23:02:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030839325000000001 2024-05-11T23:02:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178217+0000", 2024-05-11T23:02:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9450000000000003e-06 2024-05-11T23:02:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71331 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147391+0000", 2024-05-11T23:02:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.308472669, 2024-05-11T23:02:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030863854999999999, 2024-05-11T23:02:35.491 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71331 2024-05-11T23:02:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147391+0000", 2024-05-11T23:02:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:35.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147391+0000", 2024-05-11T23:02:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147392+0000", 2024-05-11T23:02:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.68e-07 2024-05-11T23:02:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147395+0000", 2024-05-11T23:02:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.503e-06 2024-05-11T23:02:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:35.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147401+0000", 2024-05-11T23:02:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7889999999999997e-06 2024-05-11T23:02:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147406+0000", 2024-05-11T23:02:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.831e-06 2024-05-11T23:02:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178245+0000", 2024-05-11T23:02:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030838884 2024-05-11T23:02:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:35.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178255+0000", 2024-05-11T23:02:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.028e-05 2024-05-11T23:02:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71332 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147414+0000", 2024-05-11T23:02:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.308450617999998, 2024-05-11T23:02:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030874374, 2024-05-11T23:02:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:35.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71332 2024-05-11T23:02:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147414+0000", 2024-05-11T23:02:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147414+0000", 2024-05-11T23:02:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:35.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147414+0000", 2024-05-11T23:02:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6899999999999998e-07 2024-05-11T23:02:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147416+0000", 2024-05-11T23:02:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:02:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147428+0000", 2024-05-11T23:02:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1428e-05 2024-05-11T23:02:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:35.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147436+0000", 2024-05-11T23:02:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9689999999999994e-06 2024-05-11T23:02:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178283+0000", 2024-05-11T23:02:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030847764 2024-05-11T23:02:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178288+0000", 2024-05-11T23:02:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6430000000000003e-06 2024-05-11T23:02:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71333 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147444+0000", 2024-05-11T23:02:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.308420637000001, 2024-05-11T23:02:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.03086906, 2024-05-11T23:02:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71333 2024-05-11T23:02:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:35.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147444+0000", 2024-05-11T23:02:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147444+0000", 2024-05-11T23:02:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147444+0000", 2024-05-11T23:02:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:02:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147446+0000", 2024-05-11T23:02:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3590000000000002e-06 2024-05-11T23:02:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147453+0000", 2024-05-11T23:02:35.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7580000000000003e-06 2024-05-11T23:02:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147460+0000", 2024-05-11T23:02:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5010000000000001e-06 2024-05-11T23:02:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178308+0000", 2024-05-11T23:02:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030848179 2024-05-11T23:02:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178313+0000", 2024-05-11T23:02:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8219999999999997e-06 2024-05-11T23:02:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71334 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147466+0000", 2024-05-11T23:02:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.308397967000001, 2024-05-11T23:02:35.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030868219999999998, 2024-05-11T23:02:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71334 2024-05-11T23:02:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147466+0000", 2024-05-11T23:02:35.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147466+0000", 2024-05-11T23:02:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147467+0000", 2024-05-11T23:02:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9099999999999999e-07 2024-05-11T23:02:35.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147469+0000", 2024-05-11T23:02:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9769999999999999e-06 2024-05-11T23:02:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147475+0000", 2024-05-11T23:02:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1369999999999999e-06 2024-05-11T23:02:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147479+0000", 2024-05-11T23:02:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6639999999999997e-06 2024-05-11T23:02:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178330+0000", 2024-05-11T23:02:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030850776 2024-05-11T23:02:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178334+0000", 2024-05-11T23:02:35.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2749999999999997e-06 2024-05-11T23:02:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71335 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147489+0000", 2024-05-11T23:02:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.30837541, 2024-05-11T23:02:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030866874999999998, 2024-05-11T23:02:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:35.508 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71335 2024-05-11T23:02:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147489+0000", 2024-05-11T23:02:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:35.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147489+0000", 2024-05-11T23:02:35.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:35.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147489+0000", 2024-05-11T23:02:35.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7899999999999999e-07 2024-05-11T23:02:35.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:35.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147491+0000", 2024-05-11T23:02:35.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0600000000000002e-06 2024-05-11T23:02:35.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:35.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147498+0000", 2024-05-11T23:02:35.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3339999999999998e-06 2024-05-11T23:02:35.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:35.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147502+0000", 2024-05-11T23:02:35.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8629999999999999e-06 2024-05-11T23:02:35.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:35.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178350+0000", 2024-05-11T23:02:35.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030847795000000001 2024-05-11T23:02:35.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:35.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178356+0000", 2024-05-11T23:02:35.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4439999999999999e-06 2024-05-11T23:02:35.513 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.513 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.513 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.513 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71336 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:35.513 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147508+0000", 2024-05-11T23:02:35.513 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.308355669000001, 2024-05-11T23:02:35.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030867196999999999, 2024-05-11T23:02:35.514 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:35.514 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:35.514 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:35.514 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:35.514 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:35.514 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:35.514 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71336 2024-05-11T23:02:35.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.514 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:35.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:35.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147508+0000", 2024-05-11T23:02:35.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147508+0000", 2024-05-11T23:02:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147509+0000", 2024-05-11T23:02:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:02:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147511+0000", 2024-05-11T23:02:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.88e-06 2024-05-11T23:02:35.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147517+0000", 2024-05-11T23:02:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7849999999999997e-06 2024-05-11T23:02:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147521+0000", 2024-05-11T23:02:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4240000000000003e-06 2024-05-11T23:02:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178371+0000", 2024-05-11T23:02:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030850452 2024-05-11T23:02:35.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178376+0000", 2024-05-11T23:02:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2560000000000004e-06 2024-05-11T23:02:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71337 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147527+0000", 2024-05-11T23:02:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.308337072, 2024-05-11T23:02:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030869829000000001, 2024-05-11T23:02:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71337 2024-05-11T23:02:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147527+0000", 2024-05-11T23:02:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147527+0000", 2024-05-11T23:02:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147527+0000", 2024-05-11T23:02:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3099999999999999e-07 2024-05-11T23:02:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147529+0000", 2024-05-11T23:02:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8870000000000001e-06 2024-05-11T23:02:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147536+0000", 2024-05-11T23:02:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2910000000000004e-06 2024-05-11T23:02:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147540+0000", 2024-05-11T23:02:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8189999999999999e-06 2024-05-11T23:02:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178393+0000", 2024-05-11T23:02:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030852118000000001 2024-05-11T23:02:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178397+0000", 2024-05-11T23:02:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3830000000000002e-06 2024-05-11T23:02:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71338 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147546+0000", 2024-05-11T23:02:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.308317812999999, 2024-05-11T23:02:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030872143000000001, 2024-05-11T23:02:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71338 2024-05-11T23:02:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147546+0000", 2024-05-11T23:02:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147546+0000", 2024-05-11T23:02:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147547+0000", 2024-05-11T23:02:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7500000000000001e-07 2024-05-11T23:02:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147549+0000", 2024-05-11T23:02:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0040000000000002e-06 2024-05-11T23:02:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147555+0000", 2024-05-11T23:02:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1959999999999996e-06 2024-05-11T23:02:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147559+0000", 2024-05-11T23:02:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3329999999999998e-06 2024-05-11T23:02:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178414+0000", 2024-05-11T23:02:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030854837999999999 2024-05-11T23:02:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178418+0000", 2024-05-11T23:02:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3970000000000004e-06 2024-05-11T23:02:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71339 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147570+0000", 2024-05-11T23:02:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.308293814999999, 2024-05-11T23:02:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030868237, 2024-05-11T23:02:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71339 2024-05-11T23:02:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147570+0000", 2024-05-11T23:02:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147570+0000", 2024-05-11T23:02:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147571+0000", 2024-05-11T23:02:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2399999999999999e-07 2024-05-11T23:02:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147573+0000", 2024-05-11T23:02:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.43e-06 2024-05-11T23:02:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147589+0000", 2024-05-11T23:02:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5977999999999998e-05 2024-05-11T23:02:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147595+0000", 2024-05-11T23:02:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.463e-06 2024-05-11T23:02:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178434+0000", 2024-05-11T23:02:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030839582000000001 2024-05-11T23:02:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178439+0000", 2024-05-11T23:02:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3599999999999998e-06 2024-05-11T23:02:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71340 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147614+0000", 2024-05-11T23:02:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.308249721999999, 2024-05-11T23:02:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030844255000000001, 2024-05-11T23:02:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71340 2024-05-11T23:02:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147614+0000", 2024-05-11T23:02:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147614+0000", 2024-05-11T23:02:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147615+0000", 2024-05-11T23:02:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4200000000000001e-07 2024-05-11T23:02:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147617+0000", 2024-05-11T23:02:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5150000000000001e-06 2024-05-11T23:02:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147626+0000", 2024-05-11T23:02:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1950000000000003e-06 2024-05-11T23:02:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147631+0000", 2024-05-11T23:02:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3369999999999999e-06 2024-05-11T23:02:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178454+0000", 2024-05-11T23:02:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030823481999999999 2024-05-11T23:02:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178459+0000", 2024-05-11T23:02:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.284e-06 2024-05-11T23:02:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71341 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147639+0000", 2024-05-11T23:02:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.308225639, 2024-05-11T23:02:35.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030842306999999999, 2024-05-11T23:02:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71341 2024-05-11T23:02:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147639+0000", 2024-05-11T23:02:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147639+0000", 2024-05-11T23:02:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147639+0000", 2024-05-11T23:02:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6600000000000002e-07 2024-05-11T23:02:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147646+0000", 2024-05-11T23:02:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8870000000000003e-06 2024-05-11T23:02:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147650+0000", 2024-05-11T23:02:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4510000000000002e-06 2024-05-11T23:02:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147653+0000", 2024-05-11T23:02:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8700000000000001e-06 2024-05-11T23:02:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178476+0000", 2024-05-11T23:02:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030823350999999999 2024-05-11T23:02:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178481+0000", 2024-05-11T23:02:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3819999999999997e-06 2024-05-11T23:02:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:81250 3.b 3:d7828050:::rbd_data.111e7ffeb370.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:16.371363+0000", 2024-05-11T23:02:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.084501283000002, 2024-05-11T23:02:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.037265637999999997, 2024-05-11T23:02:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:35.536 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 81250 2024-05-11T23:02:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.371363+0000", 2024-05-11T23:02:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.371363+0000", 2024-05-11T23:02:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.371365+0000", 2024-05-11T23:02:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1330000000000002e-06 2024-05-11T23:02:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.372770+0000", 2024-05-11T23:02:35.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0014046110000000001 2024-05-11T23:02:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.372790+0000", 2024-05-11T23:02:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0530000000000002e-05 2024-05-11T23:02:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.372799+0000", 2024-05-11T23:02:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3000000000000007e-06 2024-05-11T23:02:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.372883+0000", 2024-05-11T23:02:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3709999999999996e-05 2024-05-11T23:02:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.373076+0000", 2024-05-11T23:02:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019249499999999999 2024-05-11T23:02:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-05-11T23:02:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.380091+0000", 2024-05-11T23:02:35.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.007015533 2024-05-11T23:02:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.408542+0000", 2024-05-11T23:02:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028451026000000001 2024-05-11T23:02:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:02:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.408555+0000", 2024-05-11T23:02:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3151e-05 2024-05-11T23:02:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.408592+0000", 2024-05-11T23:02:35.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6463e-05 2024-05-11T23:02:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.408629+0000", 2024-05-11T23:02:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6686000000000001e-05 2024-05-11T23:02:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:35.542 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:35.543 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:35.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:02:35.548 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:02:35.549 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:35.547+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:35.584 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-05-11T23:02:35.648 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:35.648+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:35.701 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:35.702 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:35.702 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:02:35.702 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:35.711 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:02:35.739 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:35.740 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:35.749 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:35.749+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:35.761 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:35.761 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:35.780 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:35.780 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:35.803 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:35.803 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:35.822 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:35.840 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:35.841 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:35.841 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:35.841 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:35.849 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:35.849+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:35.851 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:02:35.855 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:35.859 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:35.859 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:35.866 INFO:teuthology.orchestra.run.smithi005.stdout:3852 writesame 0x698b2c6 thru 0x698c8e3 data_size 0x13(0x161e bytes) 2024-05-11T23:02:35.866 INFO:teuthology.orchestra.run.smithi005.stdout:3853 clone 39 order 18 su 1024 sc 14 2024-05-11T23:02:35.878 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:35.878 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:35.950 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:35.951+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:35.967 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:35.967 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:35.975 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:02:36.026 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-05-11T23:02:36.051 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:36.051+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:36.092 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:36.092 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:36.101 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:02:36.140 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone38 from 0x698c8e4 (overlap 0x4780719) to 0x1c99c70 2024-05-11T23:02:36.152 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:36.152+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:36.235 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:36.236 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:36.251 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:02:36.292 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:36.292+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:36.353 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:36.353+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:36.454 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:36.454+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:36.471 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:36.471 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:36.484 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:02:36.568 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:36.568+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:36.656 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:36.656+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:36.690 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:36.690 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:36.699 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:02:36.758 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:36.758+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:36.849 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:36.849 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:36.858 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:02:36.859 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:36.859+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:36.960 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:36.960+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:36.979 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:36.980 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:36.989 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:02:37.061 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:37.061+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:37.136 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:37.136 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:37.145 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:02:37.162 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:37.162+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:37.263 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:37.263+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:37.266 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:37.266 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:37.276 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:02:37.363 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 3] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:02:37.363 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:02:37.363 INFO:tasks.thrashosds.thrasher:Adding osd 6 2024-05-11T23:02:37.363 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 6 2024-05-11T23:02:37.364 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:37.363+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:37.409 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:37.409 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:37.419 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:02:37.464 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:37.464+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:37.532 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:37.532 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:37.541 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:02:37.566 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-05-11T23:02:37.578 INFO:teuthology.orchestra.run.smithi005.stdout:3854 writesame 0x8ea4a3a thru 0x8ea52d5 data_size 0x4(0x89c bytes) 2024-05-11T23:02:37.578 INFO:teuthology.orchestra.run.smithi005.stdout:3855 punch from 0x750f7dc to 0x751f5fe, (0xfe22 bytes) 2024-05-11T23:02:37.578 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:37.579 INFO:teuthology.orchestra.run.smithi005.stdout:3863 punch from 0x480429a to 0x4807fd8, (0x3d3e bytes) 2024-05-11T23:02:37.579 INFO:teuthology.orchestra.run.smithi005.stdout:3864 read 0x2782aca thru 0x278ed38 (0xc26f bytes) 2024-05-11T23:02:37.579 INFO:teuthology.orchestra.run.smithi005.stdout:3866 write 0x44eaf50 thru 0x44ec61e (0x16cf bytes) 2024-05-11T23:02:37.582 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:37.582+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:37.601 INFO:teuthology.orchestra.run.smithi005.stdout:3867 write 0x78aedce thru 0x78b1a16 (0x2c49 bytes) 2024-05-11T23:02:37.616 INFO:teuthology.orchestra.run.smithi005.stdout:3868 write 0x1452f9 thru 0x148a59 (0x3761 bytes) 2024-05-11T23:02:37.666 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:37.666+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:37.724 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-11T23:02:37.724+0000 7fd16247e640 -1 mon.a@0(leader).osd e82 definitely_dead 0 2024-05-11T23:02:37.731 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:37.731 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:37.744 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:02:37.768 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:37.767+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:37.868 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:37.868+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:37.891 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:02:37.930 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:37.930 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:37.941 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:02:37.969 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:37.968+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:38.028 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:38.028 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:38.028 INFO:teuthology.orchestra.run.smithi005.stdout:3869 trunc from 0x8ea52d6 to 0xe37ea6 2024-05-11T23:02:38.038 INFO:teuthology.orchestra.run.smithi005.stderr:marked in osd.6. 2024-05-11T23:02:38.043 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-11T23:02:38.053 INFO:tasks.thrashosds.thrasher:Added osd 6 2024-05-11T23:02:38.054 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:38.054 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:38.060 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:38.060 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:38.070 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:02:38.070 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:38.069+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:38.071 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:38.071 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:38.087 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:38.088 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:38.105 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:38.106 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:38.124 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:38.124 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:38.142 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:38.142 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:38.161 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:38.161 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:38.170 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:38.170+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:38.215 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:38.216 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:38.216 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:38.216 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:38.216 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:38.225 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:02:38.271 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:38.271+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:38.331 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:38.331 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:38.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:38.331 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:38.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.331 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:38.332 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:38.332 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.676706864000003, 2024-05-11T23:02:38.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:02:38.332 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.332 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.332 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.332 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:02:38.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.332 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:38.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:38.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:38.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:02:38.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:02:38.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:02:38.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:02:38.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:02:38.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:02:38.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:02:38.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:02:38.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:02:38.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:02:38.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:02:38.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:02:38.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:02:38.336 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.336 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.336 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.336 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:38.336 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:38.336 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.673325675000001, 2024-05-11T23:02:38.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:02:38.336 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.336 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.336 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.336 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.336 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.336 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.337 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:02:38.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.337 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:38.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:38.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:38.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:02:38.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:02:38.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:02:38.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:02:38.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:02:38.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:02:38.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:02:38.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:02:38.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:02:38.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:02:38.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:02:38.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:02:38.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:02:38.340 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.340 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.340 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.340 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:38.340 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:38.340 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.673306967999999, 2024-05-11T23:02:38.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:02:38.340 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.340 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.341 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.341 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.341 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.341 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.341 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:02:38.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.341 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:38.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:38.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:02:38.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:02:38.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:02:38.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:02:38.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:02:38.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:02:38.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:02:38.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:02:38.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:02:38.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:02:38.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:02:38.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:02:38.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:02:38.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:02:38.344 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.344 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.344 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.344 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:38.344 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:38.344 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.673288825, 2024-05-11T23:02:38.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:02:38.344 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.345 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.345 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.345 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:02:38.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.345 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:38.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:38.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:38.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:02:38.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:02:38.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:02:38.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:02:38.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:02:38.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:02:38.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:02:38.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:02:38.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:02:38.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:02:38.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:02:38.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:02:38.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:02:38.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.348 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.348 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:38.348 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:38.348 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.673261338000003, 2024-05-11T23:02:38.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:02:38.349 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.349 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.349 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.349 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:02:38.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.349 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:38.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:38.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:02:38.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:02:38.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:02:38.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:02:38.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:02:38.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:02:38.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:02:38.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:02:38.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:02:38.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:02:38.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:02:38.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:02:38.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:02:38.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:02:38.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.352 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.353 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:38.353 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:38.353 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.673240810000003, 2024-05-11T23:02:38.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:02:38.353 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.353 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.353 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.353 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:02:38.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.353 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:38.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:38.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:38.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:02:38.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:02:38.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:02:38.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:02:38.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:02:38.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:02:38.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:02:38.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:02:38.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:02:38.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:02:38.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:02:38.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:02:38.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:02:38.356 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.356 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.356 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.357 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:38.357 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:38.357 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.673221034999997, 2024-05-11T23:02:38.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:02:38.357 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.357 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.357 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.357 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:02:38.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.357 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:38.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:38.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:02:38.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:02:38.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:02:38.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:02:38.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:02:38.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:02:38.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:02:38.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:02:38.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:02:38.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:02:38.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:02:38.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:02:38.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:02:38.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:02:38.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.360 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.361 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:38.361 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:38.361 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.673202228999997, 2024-05-11T23:02:38.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:02:38.361 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.361 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.361 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.361 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:02:38.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.362 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:38.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:38.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:02:38.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:02:38.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:02:38.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:02:38.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:02:38.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:02:38.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:02:38.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:02:38.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:02:38.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:02:38.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:02:38.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:02:38.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:02:38.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:02:38.365 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.365 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.365 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.365 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:38.365 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:38.365 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.673183305999999, 2024-05-11T23:02:38.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:02:38.365 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.365 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.365 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.365 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.366 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.366 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.366 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:02:38.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.366 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:38.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:38.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:38.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:02:38.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:02:38.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:02:38.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:02:38.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:02:38.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:02:38.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:02:38.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:02:38.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:02:38.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:02:38.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:02:38.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:02:38.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:02:38.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.369 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.369 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:38.369 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:38.369 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.673164644000003, 2024-05-11T23:02:38.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:02:38.369 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.369 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.369 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.370 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:02:38.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.370 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:38.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:38.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:38.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:02:38.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:02:38.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:02:38.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:02:38.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:02:38.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:02:38.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:02:38.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:02:38.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:02:38.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:02:38.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:02:38.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:02:38.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:02:38.373 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.373 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.373 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.373 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:38.373 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:38.373 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.672819482000001, 2024-05-11T23:02:38.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:02:38.373 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.373 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.374 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.374 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:02:38.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.374 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:38.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:38.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:02:38.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:02:38.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:02:38.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:02:38.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:02:38.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:02:38.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:02:38.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:02:38.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:02:38.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:02:38.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:02:38.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:02:38.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:02:38.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:02:38.377 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.377 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.377 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.377 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:38.377 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:38.377 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.672793931999998, 2024-05-11T23:02:38.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:02:38.377 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.378 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.378 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.378 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.378 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:02:38.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.378 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:38.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:38.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:38.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:02:38.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:02:38.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:02:38.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:02:38.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:02:38.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:02:38.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:02:38.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:02:38.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:02:38.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:02:38.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:02:38.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:02:38.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:02:38.381 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.381 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.381 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.381 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:38.381 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:38.381 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.672711673999999, 2024-05-11T23:02:38.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:02:38.382 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.382 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.382 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.382 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:02:38.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.382 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:38.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:38.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:38.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:02:38.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:02:38.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:02:38.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:02:38.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:02:38.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:02:38.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:02:38.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:02:38.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:02:38.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:02:38.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:02:38.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:02:38.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:02:38.385 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.385 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.385 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.386 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:38.386 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:38.386 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.672689226999999, 2024-05-11T23:02:38.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:02:38.386 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.386 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.386 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.386 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:02:38.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.386 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:38.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:38.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:02:38.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:02:38.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:02:38.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:02:38.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:02:38.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:02:38.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:02:38.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:02:38.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:02:38.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:02:38.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:02:38.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:02:38.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:02:38.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:02:38.389 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.389 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.389 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.390 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:38.390 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:38.390 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.672669769000002, 2024-05-11T23:02:38.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:02:38.390 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.390 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.390 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.390 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.390 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.390 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.390 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:02:38.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.391 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:38.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:38.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:38.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:02:38.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:02:38.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:02:38.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:02:38.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:02:38.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:02:38.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:02:38.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:02:38.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:02:38.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:02:38.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:02:38.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:02:38.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:02:38.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.394 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.394 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:38.394 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:38.394 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.672649708999998, 2024-05-11T23:02:38.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:02:38.394 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.394 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.394 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.394 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.394 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.395 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:02:38.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.395 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:38.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:38.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:38.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:02:38.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:02:38.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:02:38.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:02:38.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:02:38.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:02:38.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:02:38.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:02:38.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:02:38.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:02:38.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:02:38.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:02:38.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:02:38.398 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.398 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.398 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.398 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:38.398 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:38.398 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.672630439000002, 2024-05-11T23:02:38.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:02:38.399 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.399 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.399 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.399 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.399 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.399 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.399 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:02:38.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.399 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:38.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:38.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:38.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:02:38.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:02:38.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:02:38.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:02:38.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:02:38.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:02:38.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:02:38.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:02:38.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:02:38.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:02:38.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:02:38.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:02:38.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:02:38.402 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.402 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.403 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:38.403 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:38.403 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.672084372999997, 2024-05-11T23:02:38.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:02:38.403 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.403 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.403 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.403 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.403 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.403 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.403 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:02:38.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.404 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:38.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:38.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:02:38.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:02:38.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:02:38.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:02:38.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:02:38.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:02:38.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:02:38.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:02:38.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:02:38.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:02:38.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:02:38.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:02:38.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:02:38.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:02:38.406 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.407 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.407 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.407 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:38.407 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:38.407 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 45.672052589000003, 2024-05-11T23:02:38.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:02:38.407 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.407 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.407 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.407 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.407 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.407 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.407 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:02:38.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.408 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:38.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:38.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:38.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:02:38.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:02:38.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:02:38.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:02:38.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:02:38.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:02:38.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:02:38.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:02:38.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:02:38.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:02:38.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:02:38.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:02:38.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:02:38.411 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.411 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.411 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.411 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:02:38.411 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:38.411 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.234807125000003, 2024-05-11T23:02:38.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:02:38.412 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.412 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.412 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:38.412 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.412 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.412 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.412 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:02:38.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.412 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:38.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:38.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:02:38.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:02:38.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:02:38.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:02:38.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:02:38.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:02:38.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:02:38.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:02:38.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:02:38.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:02:38.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:02:38.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:02:38.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:02:38.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:02:38.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:02:38.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:38.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:02:38.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:02:38.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:38.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:02:38.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:02:38.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:38.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:02:38.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:02:38.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:02:38.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:02:38.416 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.416 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.416 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.417 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.417 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.417 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:38.418 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:02:38.419 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:38.418+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:38.519 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:38.519+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:38.525 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:38.525 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:38.525 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:38.525 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:38.535 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:02:38.619 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:38.619+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:38.680 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:38.680 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:38.680 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:38.680 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:38.680 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:38.688 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:02:38.729 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:38.729+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:38.822 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:38.822+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:38.835 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:38.835 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:38.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:38.835 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:38.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.835 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:02:38.835 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:38.836 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.671223835999996, 2024-05-11T23:02:38.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:02:38.836 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.836 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.836 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:38.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:02:38.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:02:38.836 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:02:38.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.836 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:38.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:38.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:02:38.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:02:38.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:02:38.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:02:38.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:02:38.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:02:38.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:02:38.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:02:38.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:02:38.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:02:38.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:02:38.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:02:38.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:02:38.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:02:38.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:02:38.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:02:38.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:02:38.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:02:38.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:02:38.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:02:38.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:38.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:02:38.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:02:38.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:38.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:02:38.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:02:38.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:38.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:02:38.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:02:38.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:02:38.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:02:38.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.841 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.842 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78429 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:38.842 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:02:38.842 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.869954494000002, 2024-05-11T23:02:38.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044563497000000001, 2024-05-11T23:02:38.842 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.842 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.842 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.842 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.842 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.842 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.842 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78429 2024-05-11T23:02:38.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.843 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:02:38.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:02:38.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964872+0000", 2024-05-11T23:02:38.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035070300000000001 2024-05-11T23:02:38.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964876+0000", 2024-05-11T23:02:38.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5009999999999998e-06 2024-05-11T23:02:38.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964902+0000", 2024-05-11T23:02:38.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5715999999999999e-05 2024-05-11T23:02:38.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964910+0000", 2024-05-11T23:02:38.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3930000000000002e-06 2024-05-11T23:02:38.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008898+0000", 2024-05-11T23:02:38.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043988772000000002 2024-05-11T23:02:38.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008981+0000", 2024-05-11T23:02:38.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3140999999999996e-05 2024-05-11T23:02:38.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009085+0000", 2024-05-11T23:02:38.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010327099999999999 2024-05-11T23:02:38.845 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.846 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.846 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.846 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78431 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:38.846 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:02:38.846 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.869494186000001, 2024-05-11T23:02:38.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044542033000000002, 2024-05-11T23:02:38.846 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.846 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.846 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.846 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78431 2024-05-11T23:02:38.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.847 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:02:38.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:02:38.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965038+0000", 2024-05-11T23:02:38.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6993000000000001e-05 2024-05-11T23:02:38.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965043+0000", 2024-05-11T23:02:38.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7040000000000002e-06 2024-05-11T23:02:38.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965053+0000", 2024-05-11T23:02:38.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8539999999999992e-06 2024-05-11T23:02:38.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965061+0000", 2024-05-11T23:02:38.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9810000000000003e-06 2024-05-11T23:02:38.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009350+0000", 2024-05-11T23:02:38.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044288739000000001 2024-05-11T23:02:38.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009429+0000", 2024-05-11T23:02:38.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9354e-05 2024-05-11T23:02:38.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009524+0000", 2024-05-11T23:02:38.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4407999999999995e-05 2024-05-11T23:02:38.850 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.850 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.850 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.850 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78432 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:38.850 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:02:38.850 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.869403338000001, 2024-05-11T23:02:38.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044635421000000002, 2024-05-11T23:02:38.850 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.850 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.850 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.850 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.851 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78432 2024-05-11T23:02:38.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.851 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:02:38.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:02:38.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965343+0000", 2024-05-11T23:02:38.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027056500000000001 2024-05-11T23:02:38.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965347+0000", 2024-05-11T23:02:38.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3540000000000002e-06 2024-05-11T23:02:38.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965356+0000", 2024-05-11T23:02:38.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2140000000000002e-06 2024-05-11T23:02:38.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965363+0000", 2024-05-11T23:02:38.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4150000000000004e-06 2024-05-11T23:02:38.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009551+0000", 2024-05-11T23:02:38.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044187917 2024-05-11T23:02:38.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009626+0000", 2024-05-11T23:02:38.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5024000000000005e-05 2024-05-11T23:02:38.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009708+0000", 2024-05-11T23:02:38.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1932000000000001e-05 2024-05-11T23:02:38.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.854 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.854 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78433 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:38.854 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:02:38.854 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.869102451, 2024-05-11T23:02:38.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044564864000000003, 2024-05-11T23:02:38.854 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.854 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.855 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.855 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78433 2024-05-11T23:02:38.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.855 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:02:38.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:02:38.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966225+0000", 2024-05-11T23:02:38.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085220700000000001 2024-05-11T23:02:38.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966230+0000", 2024-05-11T23:02:38.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0690000000000003e-06 2024-05-11T23:02:38.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966239+0000", 2024-05-11T23:02:38.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-05-11T23:02:38.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966245+0000", 2024-05-11T23:02:38.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5520000000000001e-06 2024-05-11T23:02:38.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009743+0000", 2024-05-11T23:02:38.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043498311999999997 2024-05-11T23:02:38.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009848+0000", 2024-05-11T23:02:38.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104885 2024-05-11T23:02:38.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009938+0000", 2024-05-11T23:02:38.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9812e-05 2024-05-11T23:02:38.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.858 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.858 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78580 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 999424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:38.858 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:02:38.858 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.865989767999999, 2024-05-11T23:02:38.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044493076999999999, 2024-05-11T23:02:38.859 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.859 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.859 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.859 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78580 2024-05-11T23:02:38.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.859 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:02:38.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:02:38.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:02:38.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:02:38.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968489+0000", 2024-05-11T23:02:38.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.52e-06 2024-05-11T23:02:38.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968496+0000", 2024-05-11T23:02:38.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9419999999999998e-06 2024-05-11T23:02:38.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968501+0000", 2024-05-11T23:02:38.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8720000000000001e-06 2024-05-11T23:02:38.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012690+0000", 2024-05-11T23:02:38.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044189655000000001 2024-05-11T23:02:38.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012848+0000", 2024-05-11T23:02:38.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015758100000000001 2024-05-11T23:02:38.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012979+0000", 2024-05-11T23:02:38.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000131066 2024-05-11T23:02:38.862 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.862 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.862 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.862 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78581 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:38.862 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:02:38.862 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.865968764000002, 2024-05-11T23:02:38.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044616243, 2024-05-11T23:02:38.863 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.863 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.863 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.863 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.863 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.863 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.863 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78581 2024-05-11T23:02:38.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.863 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:02:38.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:02:38.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970227+0000", 2024-05-11T23:02:38.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017200049999999999 2024-05-11T23:02:38.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970232+0000", 2024-05-11T23:02:38.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000003e-06 2024-05-11T23:02:38.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970242+0000", 2024-05-11T23:02:38.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0192e-05 2024-05-11T23:02:38.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970249+0000", 2024-05-11T23:02:38.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.888e-06 2024-05-11T23:02:38.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013008+0000", 2024-05-11T23:02:38.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042759495000000002 2024-05-11T23:02:38.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013063+0000", 2024-05-11T23:02:38.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5071999999999997e-05 2024-05-11T23:02:38.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013123+0000", 2024-05-11T23:02:38.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9930999999999999e-05 2024-05-11T23:02:38.866 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.866 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.867 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.867 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78589 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:38.867 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:02:38.867 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.864049520000002, 2024-05-11T23:02:38.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044587697000000003, 2024-05-11T23:02:38.867 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.867 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.867 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.867 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.867 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.867 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.867 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78589 2024-05-11T23:02:38.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.867 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:02:38.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:02:38.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970427+0000", 2024-05-11T23:02:38.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:02:38.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970429+0000", 2024-05-11T23:02:38.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3800000000000001e-06 2024-05-11T23:02:38.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970435+0000", 2024-05-11T23:02:38.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2530000000000001e-06 2024-05-11T23:02:38.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970441+0000", 2024-05-11T23:02:38.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-05-11T23:02:38.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014822+0000", 2024-05-11T23:02:38.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04438168 2024-05-11T23:02:38.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014911+0000", 2024-05-11T23:02:38.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8882999999999996e-05 2024-05-11T23:02:38.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015014+0000", 2024-05-11T23:02:38.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010263399999999999 2024-05-11T23:02:38.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.871 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.871 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78590 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:38.871 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:02:38.871 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.864028721, 2024-05-11T23:02:38.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044880120000000003, 2024-05-11T23:02:38.871 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.871 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.871 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.872 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78590 2024-05-11T23:02:38.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.872 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:02:38.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:02:38.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970918+0000", 2024-05-11T23:02:38.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000470834 2024-05-11T23:02:38.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970925+0000", 2024-05-11T23:02:38.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1690000000000003e-06 2024-05-11T23:02:38.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970934+0000", 2024-05-11T23:02:38.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9099999999999994e-06 2024-05-11T23:02:38.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970941+0000", 2024-05-11T23:02:38.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7410000000000003e-06 2024-05-11T23:02:38.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015049+0000", 2024-05-11T23:02:38.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044108392000000003 2024-05-11T23:02:38.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015133+0000", 2024-05-11T23:02:38.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3568999999999999e-05 2024-05-11T23:02:38.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015327+0000", 2024-05-11T23:02:38.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019450500000000001 2024-05-11T23:02:38.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.875 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.875 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78591 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:38.875 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:02:38.875 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.863525305, 2024-05-11T23:02:38.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044618497, 2024-05-11T23:02:38.875 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.875 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.875 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.876 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78591 2024-05-11T23:02:38.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.876 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:02:38.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:02:38.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970951+0000", 2024-05-11T23:02:38.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:02:38.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970953+0000", 2024-05-11T23:02:38.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:02:38.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970956+0000", 2024-05-11T23:02:38.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-05-11T23:02:38.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970959+0000", 2024-05-11T23:02:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9409999999999999e-06 2024-05-11T23:02:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015367+0000", 2024-05-11T23:02:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044407621000000001 2024-05-11T23:02:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015453+0000", 2024-05-11T23:02:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6685000000000001e-05 2024-05-11T23:02:38.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015569+0000", 2024-05-11T23:02:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115396 2024-05-11T23:02:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78592 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:02:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.863512496999999, 2024-05-11T23:02:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044831546999999999, 2024-05-11T23:02:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.879 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78592 2024-05-11T23:02:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:02:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:02:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972226+0000", 2024-05-11T23:02:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00126246 2024-05-11T23:02:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972230+0000", 2024-05-11T23:02:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1860000000000002e-06 2024-05-11T23:02:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972237+0000", 2024-05-11T23:02:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:02:38.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972241+0000", 2024-05-11T23:02:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.284e-06 2024-05-11T23:02:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015611+0000", 2024-05-11T23:02:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043369461999999998 2024-05-11T23:02:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015695+0000", 2024-05-11T23:02:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4458999999999999e-05 2024-05-11T23:02:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015795+0000", 2024-05-11T23:02:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9475999999999994e-05 2024-05-11T23:02:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78593 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:02:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.862225578, 2024-05-11T23:02:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046045214000000001, 2024-05-11T23:02:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.883 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78593 2024-05-11T23:02:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:02:38.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:02:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:02:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7599999999999998e-07 2024-05-11T23:02:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972252+0000", 2024-05-11T23:02:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7349999999999999e-06 2024-05-11T23:02:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972256+0000", 2024-05-11T23:02:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5659999999999998e-06 2024-05-11T23:02:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972259+0000", 2024-05-11T23:02:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8040000000000002e-06 2024-05-11T23:02:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018088+0000", 2024-05-11T23:02:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045829316000000002 2024-05-11T23:02:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018179+0000", 2024-05-11T23:02:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1299999999999997e-05 2024-05-11T23:02:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018295+0000", 2024-05-11T23:02:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116117 2024-05-11T23:02:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.887 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78594 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:02:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.862213218000001, 2024-05-11T23:02:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046245204999999998, 2024-05-11T23:02:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78594 2024-05-11T23:02:38.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:02:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:02:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972263+0000", 2024-05-11T23:02:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:02:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972264+0000", 2024-05-11T23:02:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.206e-06 2024-05-11T23:02:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972267+0000", 2024-05-11T23:02:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0690000000000002e-06 2024-05-11T23:02:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972270+0000", 2024-05-11T23:02:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5610000000000001e-06 2024-05-11T23:02:38.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018330+0000", 2024-05-11T23:02:38.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04606085 2024-05-11T23:02:38.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018414+0000", 2024-05-11T23:02:38.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3125000000000001e-05 2024-05-11T23:02:38.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018508+0000", 2024-05-11T23:02:38.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4153000000000005e-05 2024-05-11T23:02:38.891 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.891 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.892 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78595 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:38.892 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:02:38.892 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.862200352999999, 2024-05-11T23:02:38.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046396698, 2024-05-11T23:02:38.892 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.892 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.892 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.892 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78595 2024-05-11T23:02:38.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.893 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:02:38.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:02:38.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972276+0000", 2024-05-11T23:02:38.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5600000000000002e-07 2024-05-11T23:02:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972277+0000", 2024-05-11T23:02:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.576e-06 2024-05-11T23:02:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972280+0000", 2024-05-11T23:02:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-05-11T23:02:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972283+0000", 2024-05-11T23:02:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:02:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018538+0000", 2024-05-11T23:02:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046255255000000002 2024-05-11T23:02:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018605+0000", 2024-05-11T23:02:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6847000000000006e-05 2024-05-11T23:02:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018672+0000", 2024-05-11T23:02:38.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7045000000000006e-05 2024-05-11T23:02:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78596 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:02:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.862189504, 2024-05-11T23:02:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046586806000000001, 2024-05-11T23:02:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78596 2024-05-11T23:02:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:02:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:02:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:02:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:02:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972288+0000", 2024-05-11T23:02:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2550000000000001e-06 2024-05-11T23:02:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972291+0000", 2024-05-11T23:02:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9490000000000001e-06 2024-05-11T23:02:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972293+0000", 2024-05-11T23:02:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5069999999999999e-06 2024-05-11T23:02:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018707+0000", 2024-05-11T23:02:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046413604999999997 2024-05-11T23:02:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018782+0000", 2024-05-11T23:02:38.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5556000000000006e-05 2024-05-11T23:02:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018873+0000", 2024-05-11T23:02:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0703000000000002e-05 2024-05-11T23:02:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78597 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:02:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.862179731000001, 2024-05-11T23:02:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046825787000000001, 2024-05-11T23:02:38.900 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78597 2024-05-11T23:02:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:02:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:02:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:02:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-05-11T23:02:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972297+0000", 2024-05-11T23:02:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3069999999999999e-06 2024-05-11T23:02:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972300+0000", 2024-05-11T23:02:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:02:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972303+0000", 2024-05-11T23:02:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5509999999999998e-06 2024-05-11T23:02:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018911+0000", 2024-05-11T23:02:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046608194999999998 2024-05-11T23:02:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018986+0000", 2024-05-11T23:02:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5360999999999999e-05 2024-05-11T23:02:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019122+0000", 2024-05-11T23:02:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013526 2024-05-11T23:02:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78598 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:02:38.904 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.862169830999999, 2024-05-11T23:02:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046998572000000002, 2024-05-11T23:02:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78598 2024-05-11T23:02:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:02:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:02:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:02:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:02:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972307+0000", 2024-05-11T23:02:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1799999999999999e-06 2024-05-11T23:02:38.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972310+0000", 2024-05-11T23:02:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:02:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972313+0000", 2024-05-11T23:02:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4760000000000001e-06 2024-05-11T23:02:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019149+0000", 2024-05-11T23:02:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046836056000000001 2024-05-11T23:02:38.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019203+0000", 2024-05-11T23:02:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-05 2024-05-11T23:02:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019304+0000", 2024-05-11T23:02:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101537 2024-05-11T23:02:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:105234 3.15 3:adf885e5:::rbd_object_map.111e913a615f.0000000000000025:head [writefull 0~39] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-11T23:02:38.908 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:02:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 8.8268541210000002, 2024-05-11T23:02:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77797599100000003, 2024-05-11T23:02:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 105234 2024-05-11T23:02:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:02:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:02:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007623+0000", 2024-05-11T23:02:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2240000000000001e-06 2024-05-11T23:02:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007627+0000", 2024-05-11T23:02:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:02:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007636+0000", 2024-05-11T23:02:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2409999999999995e-06 2024-05-11T23:02:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007641+0000", 2024-05-11T23:02:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6810000000000003e-06 2024-05-11T23:02:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007676+0000", 2024-05-11T23:02:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5061999999999997e-05 2024-05-11T23:02:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:02:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007682+0000", 2024-05-11T23:02:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0399999999999998e-06 2024-05-11T23:02:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783880+0000", 2024-05-11T23:02:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77619752900000005 2024-05-11T23:02:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783995+0000", 2024-05-11T23:02:38.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011550800000000001 2024-05-11T23:02:38.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:02:38.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784082+0000", 2024-05-11T23:02:38.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6347999999999994e-05 2024-05-11T23:02:38.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:38.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784682+0000", 2024-05-11T23:02:38.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000600162 2024-05-11T23:02:38.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:38.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785539+0000", 2024-05-11T23:02:38.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085699099999999998 2024-05-11T23:02:38.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:38.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785572+0000", 2024-05-11T23:02:38.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3182999999999997e-05 2024-05-11T23:02:38.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785598+0000", 2024-05-11T23:02:38.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5522000000000001e-05 2024-05-11T23:02:38.914 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.914 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.915 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113761 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [call rbd.sparse_copyup in=999496b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:02:38.915 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:02:38.915 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.3041672020000004, 2024-05-11T23:02:38.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048572964000000003, 2024-05-11T23:02:38.915 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.915 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.915 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:38.915 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.915 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.915 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.915 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113761 2024-05-11T23:02:38.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.916 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:02:38.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:02:38.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530309+0000", 2024-05-11T23:02:38.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:02:38.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531242+0000", 2024-05-11T23:02:38.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093316899999999997 2024-05-11T23:02:38.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531249+0000", 2024-05-11T23:02:38.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8140000000000003e-06 2024-05-11T23:02:38.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531253+0000", 2024-05-11T23:02:38.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4290000000000003e-06 2024-05-11T23:02:38.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570325+0000", 2024-05-11T23:02:38.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039071379000000003 2024-05-11T23:02:38.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570382+0000", 2024-05-11T23:02:38.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6988999999999999e-05 2024-05-11T23:02:38.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:02:38.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570555+0000", 2024-05-11T23:02:38.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017379799999999999 2024-05-11T23:02:38.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:38.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.575953+0000", 2024-05-11T23:02:38.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053975539999999997 2024-05-11T23:02:38.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:38.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578778+0000", 2024-05-11T23:02:38.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002825616 2024-05-11T23:02:38.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:38.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578837+0000", 2024-05-11T23:02:38.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9005000000000002e-05 2024-05-11T23:02:38.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578881+0000", 2024-05-11T23:02:38.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3970000000000001e-05 2024-05-11T23:02:38.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.920 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.920 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.920 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113762 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [delete] snapc 27=[27] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:02:38.920 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:02:38.920 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.3032132509999999, 2024-05-11T23:02:38.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047916260000000002, 2024-05-11T23:02:38.921 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:38.921 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:38.921 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:38.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:38.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:38.921 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:38.921 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113762 2024-05-11T23:02:38.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.921 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:38.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:38.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:02:38.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:38.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:02:38.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:38.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:38.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531263+0000", 2024-05-11T23:02:38.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.85e-07 2024-05-11T23:02:38.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:38.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531265+0000", 2024-05-11T23:02:38.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.827e-06 2024-05-11T23:02:38.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:38.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531268+0000", 2024-05-11T23:02:38.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3780000000000001e-06 2024-05-11T23:02:38.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:38.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531271+0000", 2024-05-11T23:02:38.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8509999999999999e-06 2024-05-11T23:02:38.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:38.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572298+0000", 2024-05-11T23:02:38.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041026990999999999 2024-05-11T23:02:38.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:38.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572345+0000", 2024-05-11T23:02:38.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6878000000000001e-05 2024-05-11T23:02:38.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:02:38.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572443+0000", 2024-05-11T23:02:38.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8184999999999996e-05 2024-05-11T23:02:38.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:38.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578585+0000", 2024-05-11T23:02:38.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006141919 2024-05-11T23:02:38.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:38.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579126+0000", 2024-05-11T23:02:38.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00054063799999999997 2024-05-11T23:02:38.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:38.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579154+0000", 2024-05-11T23:02:38.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8042999999999999e-05 2024-05-11T23:02:38.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:38.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:38.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:38.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579179+0000", 2024-05-11T23:02:38.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4765000000000001e-05 2024-05-11T23:02:38.926 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.926 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.926 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.926 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:38.926 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:38.926 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:38.927 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:38.927+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:38.928 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:02:39.029 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:39.028+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-05-11T23:02:39.058 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:39.058 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:39.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.058 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:133806 3.10 3.af113e70 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e84)", 2024-05-11T23:02:39.058 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:39.044307+0000", 2024-05-11T23:02:39.058 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.014643814999999999, 2024-05-11T23:02:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014671202, 2024-05-11T23:02:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "waiting for peered", 2024-05-11T23:02:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 133806 2024-05-11T23:02:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:39.044307+0000", 2024-05-11T23:02:39.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:39.044307+0000", 2024-05-11T23:02:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:39.044308+0000", 2024-05-11T23:02:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.088e-06 2024-05-11T23:02:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:39.044311+0000", 2024-05-11T23:02:39.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.999e-06 2024-05-11T23:02:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:39.044320+0000", 2024-05-11T23:02:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8389999999999996e-06 2024-05-11T23:02:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:39.044328+0000", 2024-05-11T23:02:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8320000000000006e-06 2024-05-11T23:02:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:39.044366+0000", 2024-05-11T23:02:39.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8849e-05 2024-05-11T23:02:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:02:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:39.044373+0000", 2024-05-11T23:02:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6869999999999997e-06 2024-05-11T23:02:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:133808 3.10 3.86dfb50 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e84)", 2024-05-11T23:02:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:39.044339+0000", 2024-05-11T23:02:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.014611476, 2024-05-11T23:02:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014684318999999999, 2024-05-11T23:02:39.062 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "waiting for peered", 2024-05-11T23:02:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 133808 2024-05-11T23:02:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:39.044339+0000", 2024-05-11T23:02:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:39.044339+0000", 2024-05-11T23:02:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:39.044340+0000", 2024-05-11T23:02:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.51e-07 2024-05-11T23:02:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:39.044342+0000", 2024-05-11T23:02:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7949999999999999e-06 2024-05-11T23:02:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:39.044350+0000", 2024-05-11T23:02:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4540000000000001e-06 2024-05-11T23:02:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:39.044356+0000", 2024-05-11T23:02:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8780000000000003e-06 2024-05-11T23:02:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:39.044390+0000", 2024-05-11T23:02:39.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4270000000000002e-05 2024-05-11T23:02:39.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:02:39.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:39.044393+0000", 2024-05-11T23:02:39.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2490000000000001e-06 2024-05-11T23:02:39.066 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.066 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.066 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.066 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.066 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:02:39.066 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 2 2024-05-11T23:02:39.066 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:39.068 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:02:39.129 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:39.129+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:39.178 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:39.178 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:39.179 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:39.179 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:39.187 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:02:39.231 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:39.231+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:39.298 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71308 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.146883+0000", 2024-05-11T23:02:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.151153785000002, 2024-05-11T23:02:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030851522999999999, 2024-05-11T23:02:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.299 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71308 2024-05-11T23:02:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.300 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146883+0000", 2024-05-11T23:02:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146883+0000", 2024-05-11T23:02:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146884+0000", 2024-05-11T23:02:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3399999999999999e-07 2024-05-11T23:02:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146893+0000", 2024-05-11T23:02:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2159999999999995e-06 2024-05-11T23:02:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146899+0000", 2024-05-11T23:02:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.302e-06 2024-05-11T23:02:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146905+0000", 2024-05-11T23:02:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9120000000000003e-06 2024-05-11T23:02:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177730+0000", 2024-05-11T23:02:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030824449 2024-05-11T23:02:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177735+0000", 2024-05-11T23:02:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1100000000000002e-06 2024-05-11T23:02:39.303 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71309 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.146921+0000", 2024-05-11T23:02:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.151115523000001, 2024-05-11T23:02:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030854467999999999, 2024-05-11T23:02:39.304 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71309 2024-05-11T23:02:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146921+0000", 2024-05-11T23:02:39.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146921+0000", 2024-05-11T23:02:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146922+0000", 2024-05-11T23:02:39.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1799999999999995e-07 2024-05-11T23:02:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146924+0000", 2024-05-11T23:02:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5009999999999999e-06 2024-05-11T23:02:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.307 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146938+0000", 2024-05-11T23:02:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3286e-05 2024-05-11T23:02:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146944+0000", 2024-05-11T23:02:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7839999999999998e-06 2024-05-11T23:02:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.308 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177766+0000", 2024-05-11T23:02:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030821988000000002 2024-05-11T23:02:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177776+0000", 2024-05-11T23:02:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3910000000000004e-06 2024-05-11T23:02:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71310 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.146954+0000", 2024-05-11T23:02:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.151083186000001, 2024-05-11T23:02:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030837251999999999, 2024-05-11T23:02:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71310 2024-05-11T23:02:39.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146954+0000", 2024-05-11T23:02:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146954+0000", 2024-05-11T23:02:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146954+0000", 2024-05-11T23:02:39.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:02:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146957+0000", 2024-05-11T23:02:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.463e-06 2024-05-11T23:02:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146963+0000", 2024-05-11T23:02:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6460000000000004e-06 2024-05-11T23:02:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146968+0000", 2024-05-11T23:02:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0250000000000002e-06 2024-05-11T23:02:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177787+0000", 2024-05-11T23:02:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030818680000000001 2024-05-11T23:02:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177791+0000", 2024-05-11T23:02:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0090000000000001e-06 2024-05-11T23:02:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71313 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147016+0000", 2024-05-11T23:02:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.151020504000002, 2024-05-11T23:02:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030841770000000001, 2024-05-11T23:02:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.314 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71313 2024-05-11T23:02:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147016+0000", 2024-05-11T23:02:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147016+0000", 2024-05-11T23:02:39.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147017+0000", 2024-05-11T23:02:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8599999999999999e-07 2024-05-11T23:02:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147019+0000", 2024-05-11T23:02:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0329999999999998e-06 2024-05-11T23:02:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147025+0000", 2024-05-11T23:02:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0299999999999999e-06 2024-05-11T23:02:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147029+0000", 2024-05-11T23:02:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5979999999999999e-06 2024-05-11T23:02:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177848+0000", 2024-05-11T23:02:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030818662 2024-05-11T23:02:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177858+0000", 2024-05-11T23:02:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0061000000000001e-05 2024-05-11T23:02:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71327 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147299+0000", 2024-05-11T23:02:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.150737815999999, 2024-05-11T23:02:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030857026999999999, 2024-05-11T23:02:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.318 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71327 2024-05-11T23:02:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147299+0000", 2024-05-11T23:02:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147299+0000", 2024-05-11T23:02:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147299+0000", 2024-05-11T23:02:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1500000000000001e-07 2024-05-11T23:02:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147303+0000", 2024-05-11T23:02:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1459999999999998e-06 2024-05-11T23:02:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147312+0000", 2024-05-11T23:02:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3299999999999999e-06 2024-05-11T23:02:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147321+0000", 2024-05-11T23:02:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7420000000000003e-06 2024-05-11T23:02:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178145+0000", 2024-05-11T23:02:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030824852999999999 2024-05-11T23:02:39.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178156+0000", 2024-05-11T23:02:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0740999999999999e-05 2024-05-11T23:02:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71328 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147326+0000", 2024-05-11T23:02:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.150711334, 2024-05-11T23:02:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030846630999999999, 2024-05-11T23:02:39.322 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71328 2024-05-11T23:02:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147326+0000", 2024-05-11T23:02:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147326+0000", 2024-05-11T23:02:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147326+0000", 2024-05-11T23:02:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5400000000000002e-07 2024-05-11T23:02:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147327+0000", 2024-05-11T23:02:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2500000000000001e-06 2024-05-11T23:02:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147331+0000", 2024-05-11T23:02:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5499999999999999e-06 2024-05-11T23:02:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147333+0000", 2024-05-11T23:02:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5450000000000002e-06 2024-05-11T23:02:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178168+0000", 2024-05-11T23:02:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030834934000000001 2024-05-11T23:02:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178172+0000", 2024-05-11T23:02:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0980000000000004e-06 2024-05-11T23:02:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71329 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147336+0000", 2024-05-11T23:02:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.150700480000001, 2024-05-11T23:02:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030865317999999999, 2024-05-11T23:02:39.326 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71329 2024-05-11T23:02:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147336+0000", 2024-05-11T23:02:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147336+0000", 2024-05-11T23:02:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147337+0000", 2024-05-11T23:02:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147338+0000", 2024-05-11T23:02:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5090000000000001e-06 2024-05-11T23:02:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147344+0000", 2024-05-11T23:02:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.66e-06 2024-05-11T23:02:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147348+0000", 2024-05-11T23:02:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2980000000000002e-06 2024-05-11T23:02:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178192+0000", 2024-05-11T23:02:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030843425000000001 2024-05-11T23:02:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178202+0000", 2024-05-11T23:02:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0217000000000001e-05 2024-05-11T23:02:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71330 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147355+0000", 2024-05-11T23:02:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.150682325999998, 2024-05-11T23:02:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030862031000000002, 2024-05-11T23:02:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.330 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71330 2024-05-11T23:02:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147355+0000", 2024-05-11T23:02:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147355+0000", 2024-05-11T23:02:39.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147355+0000", 2024-05-11T23:02:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.39e-07 2024-05-11T23:02:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147365+0000", 2024-05-11T23:02:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7140000000000005e-06 2024-05-11T23:02:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147370+0000", 2024-05-11T23:02:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2979999999999999e-06 2024-05-11T23:02:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147373+0000", 2024-05-11T23:02:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.41e-06 2024-05-11T23:02:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178213+0000", 2024-05-11T23:02:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030839325000000001 2024-05-11T23:02:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178217+0000", 2024-05-11T23:02:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9450000000000003e-06 2024-05-11T23:02:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71331 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147391+0000", 2024-05-11T23:02:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.150645404999999, 2024-05-11T23:02:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030863854999999999, 2024-05-11T23:02:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.334 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71331 2024-05-11T23:02:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147391+0000", 2024-05-11T23:02:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147391+0000", 2024-05-11T23:02:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147392+0000", 2024-05-11T23:02:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.68e-07 2024-05-11T23:02:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147395+0000", 2024-05-11T23:02:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.503e-06 2024-05-11T23:02:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147401+0000", 2024-05-11T23:02:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7889999999999997e-06 2024-05-11T23:02:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147406+0000", 2024-05-11T23:02:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.831e-06 2024-05-11T23:02:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178245+0000", 2024-05-11T23:02:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030838884 2024-05-11T23:02:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178255+0000", 2024-05-11T23:02:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.028e-05 2024-05-11T23:02:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71332 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147414+0000", 2024-05-11T23:02:39.338 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.150623354, 2024-05-11T23:02:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030874374, 2024-05-11T23:02:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71332 2024-05-11T23:02:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147414+0000", 2024-05-11T23:02:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147414+0000", 2024-05-11T23:02:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147414+0000", 2024-05-11T23:02:39.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6899999999999998e-07 2024-05-11T23:02:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147416+0000", 2024-05-11T23:02:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:02:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147428+0000", 2024-05-11T23:02:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1428e-05 2024-05-11T23:02:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147436+0000", 2024-05-11T23:02:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9689999999999994e-06 2024-05-11T23:02:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178283+0000", 2024-05-11T23:02:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030847764 2024-05-11T23:02:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178288+0000", 2024-05-11T23:02:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6430000000000003e-06 2024-05-11T23:02:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71333 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147444+0000", 2024-05-11T23:02:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.150593373, 2024-05-11T23:02:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.03086906, 2024-05-11T23:02:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.343 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71333 2024-05-11T23:02:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147444+0000", 2024-05-11T23:02:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147444+0000", 2024-05-11T23:02:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147444+0000", 2024-05-11T23:02:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:02:39.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147446+0000", 2024-05-11T23:02:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3590000000000002e-06 2024-05-11T23:02:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147453+0000", 2024-05-11T23:02:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7580000000000003e-06 2024-05-11T23:02:39.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147460+0000", 2024-05-11T23:02:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5010000000000001e-06 2024-05-11T23:02:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178308+0000", 2024-05-11T23:02:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030848179 2024-05-11T23:02:39.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178313+0000", 2024-05-11T23:02:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8219999999999997e-06 2024-05-11T23:02:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.348 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71334 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147466+0000", 2024-05-11T23:02:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.150570703, 2024-05-11T23:02:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030868219999999998, 2024-05-11T23:02:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71334 2024-05-11T23:02:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147466+0000", 2024-05-11T23:02:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147466+0000", 2024-05-11T23:02:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147467+0000", 2024-05-11T23:02:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9099999999999999e-07 2024-05-11T23:02:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147469+0000", 2024-05-11T23:02:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9769999999999999e-06 2024-05-11T23:02:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147475+0000", 2024-05-11T23:02:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1369999999999999e-06 2024-05-11T23:02:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147479+0000", 2024-05-11T23:02:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6639999999999997e-06 2024-05-11T23:02:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178330+0000", 2024-05-11T23:02:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030850776 2024-05-11T23:02:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178334+0000", 2024-05-11T23:02:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2749999999999997e-06 2024-05-11T23:02:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71335 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147489+0000", 2024-05-11T23:02:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.150548145999998, 2024-05-11T23:02:39.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030866874999999998, 2024-05-11T23:02:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71335 2024-05-11T23:02:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147489+0000", 2024-05-11T23:02:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147489+0000", 2024-05-11T23:02:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147489+0000", 2024-05-11T23:02:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7899999999999999e-07 2024-05-11T23:02:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147491+0000", 2024-05-11T23:02:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0600000000000002e-06 2024-05-11T23:02:39.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147498+0000", 2024-05-11T23:02:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3339999999999998e-06 2024-05-11T23:02:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147502+0000", 2024-05-11T23:02:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8629999999999999e-06 2024-05-11T23:02:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178350+0000", 2024-05-11T23:02:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030847795000000001 2024-05-11T23:02:39.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178356+0000", 2024-05-11T23:02:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4439999999999999e-06 2024-05-11T23:02:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71336 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147508+0000", 2024-05-11T23:02:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.150528404999999, 2024-05-11T23:02:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030867196999999999, 2024-05-11T23:02:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71336 2024-05-11T23:02:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147508+0000", 2024-05-11T23:02:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147508+0000", 2024-05-11T23:02:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147509+0000", 2024-05-11T23:02:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:02:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147511+0000", 2024-05-11T23:02:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.88e-06 2024-05-11T23:02:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147517+0000", 2024-05-11T23:02:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7849999999999997e-06 2024-05-11T23:02:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147521+0000", 2024-05-11T23:02:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4240000000000003e-06 2024-05-11T23:02:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178371+0000", 2024-05-11T23:02:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030850452 2024-05-11T23:02:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178376+0000", 2024-05-11T23:02:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2560000000000004e-06 2024-05-11T23:02:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71337 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147527+0000", 2024-05-11T23:02:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.150509807999999, 2024-05-11T23:02:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030869829000000001, 2024-05-11T23:02:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71337 2024-05-11T23:02:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147527+0000", 2024-05-11T23:02:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147527+0000", 2024-05-11T23:02:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147527+0000", 2024-05-11T23:02:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3099999999999999e-07 2024-05-11T23:02:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147529+0000", 2024-05-11T23:02:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8870000000000001e-06 2024-05-11T23:02:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147536+0000", 2024-05-11T23:02:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2910000000000004e-06 2024-05-11T23:02:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147540+0000", 2024-05-11T23:02:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8189999999999999e-06 2024-05-11T23:02:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178393+0000", 2024-05-11T23:02:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030852118000000001 2024-05-11T23:02:39.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178397+0000", 2024-05-11T23:02:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3830000000000002e-06 2024-05-11T23:02:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71338 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147546+0000", 2024-05-11T23:02:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.150490549000001, 2024-05-11T23:02:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030872143000000001, 2024-05-11T23:02:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.365 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71338 2024-05-11T23:02:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147546+0000", 2024-05-11T23:02:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147546+0000", 2024-05-11T23:02:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147547+0000", 2024-05-11T23:02:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7500000000000001e-07 2024-05-11T23:02:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147549+0000", 2024-05-11T23:02:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0040000000000002e-06 2024-05-11T23:02:39.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147555+0000", 2024-05-11T23:02:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1959999999999996e-06 2024-05-11T23:02:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147559+0000", 2024-05-11T23:02:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3329999999999998e-06 2024-05-11T23:02:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178414+0000", 2024-05-11T23:02:39.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030854837999999999 2024-05-11T23:02:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178418+0000", 2024-05-11T23:02:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3970000000000004e-06 2024-05-11T23:02:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71339 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147570+0000", 2024-05-11T23:02:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.150466551000001, 2024-05-11T23:02:39.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030868237, 2024-05-11T23:02:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71339 2024-05-11T23:02:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147570+0000", 2024-05-11T23:02:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147570+0000", 2024-05-11T23:02:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147571+0000", 2024-05-11T23:02:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2399999999999999e-07 2024-05-11T23:02:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147573+0000", 2024-05-11T23:02:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.43e-06 2024-05-11T23:02:39.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147589+0000", 2024-05-11T23:02:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5977999999999998e-05 2024-05-11T23:02:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147595+0000", 2024-05-11T23:02:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.463e-06 2024-05-11T23:02:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178434+0000", 2024-05-11T23:02:39.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030839582000000001 2024-05-11T23:02:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178439+0000", 2024-05-11T23:02:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3599999999999998e-06 2024-05-11T23:02:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71340 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147614+0000", 2024-05-11T23:02:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.150422458000001, 2024-05-11T23:02:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030844255000000001, 2024-05-11T23:02:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71340 2024-05-11T23:02:39.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147614+0000", 2024-05-11T23:02:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147614+0000", 2024-05-11T23:02:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147615+0000", 2024-05-11T23:02:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4200000000000001e-07 2024-05-11T23:02:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147617+0000", 2024-05-11T23:02:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5150000000000001e-06 2024-05-11T23:02:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147626+0000", 2024-05-11T23:02:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1950000000000003e-06 2024-05-11T23:02:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147631+0000", 2024-05-11T23:02:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3369999999999999e-06 2024-05-11T23:02:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178454+0000", 2024-05-11T23:02:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030823481999999999 2024-05-11T23:02:39.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178459+0000", 2024-05-11T23:02:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.284e-06 2024-05-11T23:02:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71341 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147639+0000", 2024-05-11T23:02:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.150398375000002, 2024-05-11T23:02:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030842306999999999, 2024-05-11T23:02:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71341 2024-05-11T23:02:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147639+0000", 2024-05-11T23:02:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147639+0000", 2024-05-11T23:02:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147639+0000", 2024-05-11T23:02:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6600000000000002e-07 2024-05-11T23:02:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147646+0000", 2024-05-11T23:02:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8870000000000003e-06 2024-05-11T23:02:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147650+0000", 2024-05-11T23:02:39.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4510000000000002e-06 2024-05-11T23:02:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147653+0000", 2024-05-11T23:02:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8700000000000001e-06 2024-05-11T23:02:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178476+0000", 2024-05-11T23:02:39.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030823350999999999 2024-05-11T23:02:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178481+0000", 2024-05-11T23:02:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3819999999999997e-06 2024-05-11T23:02:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:81250 3.b 3:d7828050:::rbd_data.111e7ffeb370.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:16.371363+0000", 2024-05-11T23:02:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.926674019, 2024-05-11T23:02:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.037265637999999997, 2024-05-11T23:02:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.384 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 81250 2024-05-11T23:02:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.371363+0000", 2024-05-11T23:02:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.371363+0000", 2024-05-11T23:02:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.371365+0000", 2024-05-11T23:02:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1330000000000002e-06 2024-05-11T23:02:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.372770+0000", 2024-05-11T23:02:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0014046110000000001 2024-05-11T23:02:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.372790+0000", 2024-05-11T23:02:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0530000000000002e-05 2024-05-11T23:02:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.372799+0000", 2024-05-11T23:02:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3000000000000007e-06 2024-05-11T23:02:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.372883+0000", 2024-05-11T23:02:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3709999999999996e-05 2024-05-11T23:02:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.373076+0000", 2024-05-11T23:02:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019249499999999999 2024-05-11T23:02:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-05-11T23:02:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.380091+0000", 2024-05-11T23:02:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.007015533 2024-05-11T23:02:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.408542+0000", 2024-05-11T23:02:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028451026000000001 2024-05-11T23:02:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:02:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.408555+0000", 2024-05-11T23:02:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3151e-05 2024-05-11T23:02:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.408592+0000", 2024-05-11T23:02:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6463e-05 2024-05-11T23:02:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.408629+0000", 2024-05-11T23:02:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6686000000000001e-05 2024-05-11T23:02:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.390 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.390 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:39.392 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:02:39.392 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:39.391+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:39.492 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:39.493+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:39.542 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:39.543 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:02:39.543 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:39.551 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:02:39.595 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:39.595+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:39.661 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:39.661 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:39.661 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:39.669 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:02:39.711 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:39.711+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:39.796 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:39.796+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:39.821 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28445 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.739665711000001, 2024-05-11T23:02:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031719429, 2024-05-11T23:02:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.822 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28445 2024-05-11T23:02:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-05-11T23:02:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081234+0000", 2024-05-11T23:02:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.578e-06 2024-05-11T23:02:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081237+0000", 2024-05-11T23:02:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3340000000000002e-06 2024-05-11T23:02:39.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081240+0000", 2024-05-11T23:02:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7429999999999998e-06 2024-05-11T23:02:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112828+0000", 2024-05-11T23:02:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031587941000000001 2024-05-11T23:02:39.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112878+0000", 2024-05-11T23:02:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0535999999999999e-05 2024-05-11T23:02:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112951+0000", 2024-05-11T23:02:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3072999999999996e-05 2024-05-11T23:02:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28448 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 507904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:02:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.739633210000001, 2024-05-11T23:02:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031916923, 2024-05-11T23:02:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.827 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28448 2024-05-11T23:02:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:02:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:02:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081265+0000", 2024-05-11T23:02:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2899999999999999e-07 2024-05-11T23:02:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081267+0000", 2024-05-11T23:02:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6749999999999998e-06 2024-05-11T23:02:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081273+0000", 2024-05-11T23:02:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4049999999999999e-06 2024-05-11T23:02:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081278+0000", 2024-05-11T23:02:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0139999999999998e-06 2024-05-11T23:02:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112996+0000", 2024-05-11T23:02:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031718283 2024-05-11T23:02:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113080+0000", 2024-05-11T23:02:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4334999999999998e-05 2024-05-11T23:02:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113181+0000", 2024-05-11T23:02:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000100882 2024-05-11T23:02:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.831 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28451 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 540672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.739604708000002, 2024-05-11T23:02:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032054412999999997, 2024-05-11T23:02:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:39.832 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28451 2024-05-11T23:02:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2499999999999995e-07 2024-05-11T23:02:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081300+0000", 2024-05-11T23:02:39.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0929999999999999e-06 2024-05-11T23:02:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081306+0000", 2024-05-11T23:02:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6290000000000004e-06 2024-05-11T23:02:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081311+0000", 2024-05-11T23:02:39.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9289999999999997e-06 2024-05-11T23:02:39.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113208+0000", 2024-05-11T23:02:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031897162 2024-05-11T23:02:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113277+0000", 2024-05-11T23:02:39.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8811000000000002e-05 2024-05-11T23:02:39.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113347+0000", 2024-05-11T23:02:39.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0264000000000004e-05 2024-05-11T23:02:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28454 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 573440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.739530086999999, 2024-05-11T23:02:39.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032210564999999997, 2024-05-11T23:02:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28454 2024-05-11T23:02:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.99e-07 2024-05-11T23:02:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081371+0000", 2024-05-11T23:02:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1200000000000002e-06 2024-05-11T23:02:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081378+0000", 2024-05-11T23:02:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6089999999999997e-06 2024-05-11T23:02:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081384+0000", 2024-05-11T23:02:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0859999999999998e-06 2024-05-11T23:02:39.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113388+0000", 2024-05-11T23:02:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032004248999999999 2024-05-11T23:02:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113470+0000", 2024-05-11T23:02:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1871000000000003e-05 2024-05-11T23:02:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113578+0000", 2024-05-11T23:02:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000108031 2024-05-11T23:02:39.843 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28457 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 606208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.739499862999999, 2024-05-11T23:02:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032410130000000002, 2024-05-11T23:02:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.844 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28457 2024-05-11T23:02:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.32e-07 2024-05-11T23:02:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081400+0000", 2024-05-11T23:02:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9120000000000001e-06 2024-05-11T23:02:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081404+0000", 2024-05-11T23:02:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7900000000000001e-06 2024-05-11T23:02:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081407+0000", 2024-05-11T23:02:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8839999999999998e-06 2024-05-11T23:02:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113621+0000", 2024-05-11T23:02:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032213841 2024-05-11T23:02:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113702+0000", 2024-05-11T23:02:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1574000000000002e-05 2024-05-11T23:02:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113808+0000", 2024-05-11T23:02:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000105697 2024-05-11T23:02:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28460 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 638976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.739487032, 2024-05-11T23:02:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032616804999999999, 2024-05-11T23:02:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.848 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28460 2024-05-11T23:02:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5900000000000003e-07 2024-05-11T23:02:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081412+0000", 2024-05-11T23:02:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1990000000000001e-06 2024-05-11T23:02:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081415+0000", 2024-05-11T23:02:39.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.971e-06 2024-05-11T23:02:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081418+0000", 2024-05-11T23:02:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-05-11T23:02:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113853+0000", 2024-05-11T23:02:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032435155 2024-05-11T23:02:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113936+0000", 2024-05-11T23:02:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3441000000000003e-05 2024-05-11T23:02:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114027+0000", 2024-05-11T23:02:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1108999999999998e-05 2024-05-11T23:02:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28463 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 671744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.739470109999999, 2024-05-11T23:02:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032814958999999998, 2024-05-11T23:02:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28463 2024-05-11T23:02:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4999999999999998e-07 2024-05-11T23:02:39.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081430+0000", 2024-05-11T23:02:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6109999999999999e-06 2024-05-11T23:02:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081433+0000", 2024-05-11T23:02:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5590000000000001e-06 2024-05-11T23:02:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081437+0000", 2024-05-11T23:02:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3950000000000001e-06 2024-05-11T23:02:39.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114061+0000", 2024-05-11T23:02:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032624092 2024-05-11T23:02:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114142+0000", 2024-05-11T23:02:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0893e-05 2024-05-11T23:02:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114242+0000", 2024-05-11T23:02:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010095899999999999 2024-05-11T23:02:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28466 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 704512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.739457576, 2024-05-11T23:02:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032933658999999997, 2024-05-11T23:02:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.857 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28466 2024-05-11T23:02:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4700000000000002e-07 2024-05-11T23:02:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081442+0000", 2024-05-11T23:02:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.296e-06 2024-05-11T23:02:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081445+0000", 2024-05-11T23:02:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9009999999999998e-06 2024-05-11T23:02:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081447+0000", 2024-05-11T23:02:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4200000000000001e-06 2024-05-11T23:02:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114280+0000", 2024-05-11T23:02:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032832667000000003 2024-05-11T23:02:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114325+0000", 2024-05-11T23:02:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5018000000000002e-05 2024-05-11T23:02:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114374+0000", 2024-05-11T23:02:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.901e-05 2024-05-11T23:02:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28468 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 737280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:39.861 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.739437571000003, 2024-05-11T23:02:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033118776000000003, 2024-05-11T23:02:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28468 2024-05-11T23:02:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:39.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3799999999999998e-07 2024-05-11T23:02:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081462+0000", 2024-05-11T23:02:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4899999999999999e-06 2024-05-11T23:02:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081465+0000", 2024-05-11T23:02:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4079999999999998e-06 2024-05-11T23:02:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081468+0000", 2024-05-11T23:02:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7e-06 2024-05-11T23:02:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114415+0000", 2024-05-11T23:02:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032946766000000002 2024-05-11T23:02:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114497+0000", 2024-05-11T23:02:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1854000000000006e-05 2024-05-11T23:02:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114579+0000", 2024-05-11T23:02:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2120000000000007e-05 2024-05-11T23:02:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28471 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 770048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.739422863999998, 2024-05-11T23:02:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033234970000000003, 2024-05-11T23:02:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.866 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28471 2024-05-11T23:02:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:39.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-05-11T23:02:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081476+0000", 2024-05-11T23:02:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3120000000000001e-06 2024-05-11T23:02:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081480+0000", 2024-05-11T23:02:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1470000000000001e-06 2024-05-11T23:02:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081482+0000", 2024-05-11T23:02:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7480000000000002e-06 2024-05-11T23:02:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114616+0000", 2024-05-11T23:02:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033133322 2024-05-11T23:02:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114662+0000", 2024-05-11T23:02:39.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6020999999999999e-05 2024-05-11T23:02:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114710+0000", 2024-05-11T23:02:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8059999999999997e-05 2024-05-11T23:02:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.871 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28474 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.739406789999997, 2024-05-11T23:02:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033336431, 2024-05-11T23:02:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28474 2024-05-11T23:02:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.872 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:39.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.15e-07 2024-05-11T23:02:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081493+0000", 2024-05-11T23:02:39.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4950000000000001e-06 2024-05-11T23:02:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081496+0000", 2024-05-11T23:02:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.309e-06 2024-05-11T23:02:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081499+0000", 2024-05-11T23:02:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4570000000000001e-06 2024-05-11T23:02:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114740+0000", 2024-05-11T23:02:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033240619999999999 2024-05-11T23:02:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114781+0000", 2024-05-11T23:02:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1106999999999998e-05 2024-05-11T23:02:39.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114827+0000", 2024-05-11T23:02:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6028e-05 2024-05-11T23:02:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28477 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 835584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.739389776000003, 2024-05-11T23:02:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033527225000000001, 2024-05-11T23:02:39.877 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28477 2024-05-11T23:02:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1300000000000001e-07 2024-05-11T23:02:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081510+0000", 2024-05-11T23:02:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4610000000000001e-06 2024-05-11T23:02:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081513+0000", 2024-05-11T23:02:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2559999999999998e-06 2024-05-11T23:02:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081516+0000", 2024-05-11T23:02:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7099999999999999e-06 2024-05-11T23:02:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114869+0000", 2024-05-11T23:02:39.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033353718999999997 2024-05-11T23:02:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114954+0000", 2024-05-11T23:02:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4969000000000006e-05 2024-05-11T23:02:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115035+0000", 2024-05-11T23:02:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0697000000000003e-05 2024-05-11T23:02:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.881 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28480 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 868352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.739372805999999, 2024-05-11T23:02:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033695939000000001, 2024-05-11T23:02:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28480 2024-05-11T23:02:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.46e-07 2024-05-11T23:02:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081526+0000", 2024-05-11T23:02:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.347e-06 2024-05-11T23:02:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081530+0000", 2024-05-11T23:02:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.348e-06 2024-05-11T23:02:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081533+0000", 2024-05-11T23:02:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4369999999999999e-06 2024-05-11T23:02:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115078+0000", 2024-05-11T23:02:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033545063 2024-05-11T23:02:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115164+0000", 2024-05-11T23:02:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5662999999999996e-05 2024-05-11T23:02:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115221+0000", 2024-05-11T23:02:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6734999999999998e-05 2024-05-11T23:02:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.885 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28483 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.739355633000002, 2024-05-11T23:02:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033803466999999997, 2024-05-11T23:02:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28483 2024-05-11T23:02:39.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:39.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4999999999999998e-07 2024-05-11T23:02:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081544+0000", 2024-05-11T23:02:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:02:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081547+0000", 2024-05-11T23:02:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3139999999999999e-06 2024-05-11T23:02:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081550+0000", 2024-05-11T23:02:39.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7779999999999999e-06 2024-05-11T23:02:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115253+0000", 2024-05-11T23:02:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033703516000000003 2024-05-11T23:02:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115302+0000", 2024-05-11T23:02:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8869999999999998e-05 2024-05-11T23:02:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115345+0000", 2024-05-11T23:02:39.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3340999999999997e-05 2024-05-11T23:02:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28487 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 933888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.739344432000003, 2024-05-11T23:02:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033903918999999998, 2024-05-11T23:02:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.890 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28487 2024-05-11T23:02:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9999999999999999e-07 2024-05-11T23:02:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081559+0000", 2024-05-11T23:02:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-05-11T23:02:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081563+0000", 2024-05-11T23:02:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2370000000000003e-06 2024-05-11T23:02:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081566+0000", 2024-05-11T23:02:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6199999999999999e-06 2024-05-11T23:02:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115375+0000", 2024-05-11T23:02:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033808887000000003 2024-05-11T23:02:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115413+0000", 2024-05-11T23:02:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8309000000000002e-05 2024-05-11T23:02:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115457+0000", 2024-05-11T23:02:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3809e-05 2024-05-11T23:02:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28490 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 966656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.739327908, 2024-05-11T23:02:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034114903000000002, 2024-05-11T23:02:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:39.894 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28490 2024-05-11T23:02:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4400000000000001e-07 2024-05-11T23:02:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081571+0000", 2024-05-11T23:02:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2130000000000001e-06 2024-05-11T23:02:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081574+0000", 2024-05-11T23:02:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8270000000000002e-06 2024-05-11T23:02:39.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081577+0000", 2024-05-11T23:02:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-05-11T23:02:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115507+0000", 2024-05-11T23:02:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033929925999999999 2024-05-11T23:02:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115599+0000", 2024-05-11T23:02:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1571999999999997e-05 2024-05-11T23:02:39.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115685+0000", 2024-05-11T23:02:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6071999999999999e-05 2024-05-11T23:02:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28493 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 999424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.739315804999997, 2024-05-11T23:02:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034232944000000001, 2024-05-11T23:02:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28493 2024-05-11T23:02:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1300000000000001e-07 2024-05-11T23:02:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081584+0000", 2024-05-11T23:02:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9479999999999998e-06 2024-05-11T23:02:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081590+0000", 2024-05-11T23:02:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.728e-06 2024-05-11T23:02:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081595+0000", 2024-05-11T23:02:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8579999999999999e-06 2024-05-11T23:02:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115723+0000", 2024-05-11T23:02:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034128473999999999 2024-05-11T23:02:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115768+0000", 2024-05-11T23:02:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5108999999999998e-05 2024-05-11T23:02:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115815+0000", 2024-05-11T23:02:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6514000000000002e-05 2024-05-11T23:02:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.902 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28496 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:02:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.739296346000003, 2024-05-11T23:02:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034427754999999997, 2024-05-11T23:02:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.903 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28496 2024-05-11T23:02:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:02:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:02:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081602+0000", 2024-05-11T23:02:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:02:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081604+0000", 2024-05-11T23:02:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3999999999999999e-06 2024-05-11T23:02:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081610+0000", 2024-05-11T23:02:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.224e-06 2024-05-11T23:02:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081615+0000", 2024-05-11T23:02:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8269999999999997e-06 2024-05-11T23:02:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115859+0000", 2024-05-11T23:02:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034243498999999997 2024-05-11T23:02:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115946+0000", 2024-05-11T23:02:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7122999999999999e-05 2024-05-11T23:02:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116029+0000", 2024-05-11T23:02:39.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3332999999999996e-05 2024-05-11T23:02:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28498 3.0 3:05d5189e:::rbd_object_map.111e5e4d8275.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:02:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.737777694000002, 2024-05-11T23:02:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.035035108000000002, 2024-05-11T23:02:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.908 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28498 2024-05-11T23:02:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:02:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:02:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083121+0000", 2024-05-11T23:02:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5700000000000001e-07 2024-05-11T23:02:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083126+0000", 2024-05-11T23:02:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8860000000000003e-06 2024-05-11T23:02:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083135+0000", 2024-05-11T23:02:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8369999999999992e-06 2024-05-11T23:02:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083142+0000", 2024-05-11T23:02:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4779999999999996e-06 2024-05-11T23:02:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116072+0000", 2024-05-11T23:02:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032929991999999998 2024-05-11T23:02:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116158+0000", 2024-05-11T23:02:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6077999999999998e-05 2024-05-11T23:02:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-05-11T23:02:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116513+0000", 2024-05-11T23:02:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00035508300000000002 2024-05-11T23:02:39.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118025+0000", 2024-05-11T23:02:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015118919999999999 2024-05-11T23:02:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:02:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118039+0000", 2024-05-11T23:02:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4188999999999999e-05 2024-05-11T23:02:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:39.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118094+0000", 2024-05-11T23:02:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4929000000000002e-05 2024-05-11T23:02:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118155+0000", 2024-05-11T23:02:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0887000000000002e-05 2024-05-11T23:02:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:64049 3.8 3:11d98727:::rbd_object_map.111e8974a681:head [read 0~38] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-05-11T23:02:39.915 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:02:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 39.041016384000002, 2024-05-11T23:02:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99884604399999999, 2024-05-11T23:02:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 64049 2024-05-11T23:02:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:02:39.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:02:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779883+0000", 2024-05-11T23:02:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.57e-06 2024-05-11T23:02:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:39.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779887+0000", 2024-05-11T23:02:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5709999999999999e-06 2024-05-11T23:02:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779898+0000", 2024-05-11T23:02:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.042e-05 2024-05-11T23:02:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779905+0000", 2024-05-11T23:02:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0110000000000003e-06 2024-05-11T23:02:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779962+0000", 2024-05-11T23:02:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7395999999999997e-05 2024-05-11T23:02:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:02:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779969+0000", 2024-05-11T23:02:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2459999999999997e-06 2024-05-11T23:02:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778379+0000", 2024-05-11T23:02:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99840986300000001 2024-05-11T23:02:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778564+0000", 2024-05-11T23:02:39.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018420900000000001 2024-05-11T23:02:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778727+0000", 2024-05-11T23:02:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016375800000000001 2024-05-11T23:02:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:39.920 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:39.920 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:39.922 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:02:39.922 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:39.922+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:40.022 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:40.022+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:40.077 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:40.077 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:40.078 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:02:40.078 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:40.086 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:02:40.123 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:40.122+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:40.172 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:02:40.197 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:40.197 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:40.197 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:40.207 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:02:40.251 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:40.250+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:40.302 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:40.302 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:40.320 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:40.320 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:40.324 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:40.324+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:40.337 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:40.338 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:40.354 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:40.355 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:40.361 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:40.361 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:40.370 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:02:40.373 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:40.393 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:40.393 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:40.411 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:40.411 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:40.426 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:40.425+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:40.430 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:40.430 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:40.482 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:40.482 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:40.491 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:02:40.527 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:40.527+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:40.600 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:40.600 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:40.609 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:02:40.628 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:40.628+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:40.719 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:40.719 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:40.728 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:02:40.729 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:40.729+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:40.830 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:40.830+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:40.839 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:40.839 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:40.849 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:02:40.931 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:40.930+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:40.990 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:41.013 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:41.014 INFO:teuthology.orchestra.run.smithi005.stdout:3870 read 0xa159bd thru 0xa234f2 (0xdb36 bytes) 2024-05-11T23:02:41.014 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:02:41.022 INFO:teuthology.orchestra.run.smithi005.stdout:3871 write 0x57172e0 thru 0x5725722 (0xe443 bytes) 2024-05-11T23:02:41.032 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:41.032+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:41.037 INFO:teuthology.orchestra.run.smithi005.stdout:3873 write 0x4e5da6 thru 0x4f480d (0xea68 bytes) 2024-05-11T23:02:41.068 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-11T23:02:41.133 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:41.133+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:41.220 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:41.220 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:41.232 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:02:41.273 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:41.273+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:41.335 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:41.335+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:41.418 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:41.418 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:41.430 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:02:41.473 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:41.470+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:41.537 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:41.537+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:41.583 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:41.583 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:41.591 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:02:41.639 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:41.639+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:41.696 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:41.696 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:41.704 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:02:41.740 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:41.740+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:41.811 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:41.811 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:41.820 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:02:41.841 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:41.841+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:41.925 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:41.925 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:41.934 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:02:41.942 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:41.942+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:42.039 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:42.039 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:42.042 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:42.042+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:42.048 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:02:42.053 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-05-11T23:02:42.071 INFO:teuthology.orchestra.run.smithi005.stdout:3875 trunc from 0x5725723 to 0xac35b50 2024-05-11T23:02:42.079 INFO:teuthology.orchestra.run.smithi005.stdout:3876 read 0x665a7be thru 0x666092c (0x616f bytes) 2024-05-11T23:02:42.079 INFO:teuthology.orchestra.run.smithi005.stdout:3877 writesame 0xf611b7 thru 0xf680de data_size 0x1(0x6f28 bytes) 2024-05-11T23:02:42.143 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:42.143+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:42.157 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:42.157 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:42.166 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:02:42.244 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:42.244+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:42.311 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:42.311 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:42.320 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:02:42.345 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:42.345+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:42.425 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:42.426 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:42.435 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:02:42.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:02:42.445 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:42.445+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:42.546 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:42.547+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:42.558 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:42.558 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:42.568 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:02:42.614 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:42.614 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:42.632 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:42.632 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:42.647 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:42.647+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:42.649 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:42.650 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:42.667 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:42.667 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:42.681 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:42.681 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:42.681 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:42.681 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:42.681 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:42.685 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:42.685 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:42.690 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:02:42.704 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:42.704 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:42.721 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:42.721 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:42.739 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:42.739 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:42.748 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:42.749+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:42.819 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:42.820 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:42.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:42.820 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:42.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.820 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71308 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:42.820 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.146883+0000", 2024-05-11T23:02:42.820 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.672141099000001, 2024-05-11T23:02:42.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030851522999999999, 2024-05-11T23:02:42.820 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:42.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:42.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:42.820 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:42.820 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:42.821 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:42.821 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71308 2024-05-11T23:02:42.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.821 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:42.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:42.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146883+0000", 2024-05-11T23:02:42.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:42.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146883+0000", 2024-05-11T23:02:42.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:42.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146884+0000", 2024-05-11T23:02:42.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3399999999999999e-07 2024-05-11T23:02:42.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:42.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146893+0000", 2024-05-11T23:02:42.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2159999999999995e-06 2024-05-11T23:02:42.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:42.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146899+0000", 2024-05-11T23:02:42.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.302e-06 2024-05-11T23:02:42.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:42.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146905+0000", 2024-05-11T23:02:42.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9120000000000003e-06 2024-05-11T23:02:42.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:42.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177730+0000", 2024-05-11T23:02:42.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030824449 2024-05-11T23:02:42.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:42.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177735+0000", 2024-05-11T23:02:42.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1100000000000002e-06 2024-05-11T23:02:42.823 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.823 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.824 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.824 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71309 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:42.824 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.146921+0000", 2024-05-11T23:02:42.824 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.672102837000001, 2024-05-11T23:02:42.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030854467999999999, 2024-05-11T23:02:42.824 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:42.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:42.824 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:42.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:42.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:42.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:42.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71309 2024-05-11T23:02:42.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.825 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:42.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:42.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146921+0000", 2024-05-11T23:02:42.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:42.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146921+0000", 2024-05-11T23:02:42.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:42.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146922+0000", 2024-05-11T23:02:42.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1799999999999995e-07 2024-05-11T23:02:42.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:42.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146924+0000", 2024-05-11T23:02:42.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5009999999999999e-06 2024-05-11T23:02:42.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:42.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146938+0000", 2024-05-11T23:02:42.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3286e-05 2024-05-11T23:02:42.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:42.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146944+0000", 2024-05-11T23:02:42.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7839999999999998e-06 2024-05-11T23:02:42.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:42.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177766+0000", 2024-05-11T23:02:42.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030821988000000002 2024-05-11T23:02:42.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:42.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177776+0000", 2024-05-11T23:02:42.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3910000000000004e-06 2024-05-11T23:02:42.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.827 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.827 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71310 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:42.827 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.146954+0000", 2024-05-11T23:02:42.828 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.6720705, 2024-05-11T23:02:42.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030837251999999999, 2024-05-11T23:02:42.828 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:42.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:42.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:42.828 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:42.828 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:42.828 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:42.828 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71310 2024-05-11T23:02:42.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.828 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:42.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:42.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146954+0000", 2024-05-11T23:02:42.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:42.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146954+0000", 2024-05-11T23:02:42.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:42.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146954+0000", 2024-05-11T23:02:42.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:02:42.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:42.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146957+0000", 2024-05-11T23:02:42.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.463e-06 2024-05-11T23:02:42.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:42.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146963+0000", 2024-05-11T23:02:42.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6460000000000004e-06 2024-05-11T23:02:42.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:42.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.146968+0000", 2024-05-11T23:02:42.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0250000000000002e-06 2024-05-11T23:02:42.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:42.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177787+0000", 2024-05-11T23:02:42.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030818680000000001 2024-05-11T23:02:42.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:42.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177791+0000", 2024-05-11T23:02:42.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0090000000000001e-06 2024-05-11T23:02:42.831 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.831 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.831 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.831 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71313 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:42.831 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147016+0000", 2024-05-11T23:02:42.831 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.672007818000001, 2024-05-11T23:02:42.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030841770000000001, 2024-05-11T23:02:42.831 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:42.831 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:42.831 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:42.831 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:42.832 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:42.832 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:42.832 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71313 2024-05-11T23:02:42.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.832 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:42.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:42.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147016+0000", 2024-05-11T23:02:42.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:42.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147016+0000", 2024-05-11T23:02:42.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:42.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147017+0000", 2024-05-11T23:02:42.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8599999999999999e-07 2024-05-11T23:02:42.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:42.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147019+0000", 2024-05-11T23:02:42.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0329999999999998e-06 2024-05-11T23:02:42.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:42.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147025+0000", 2024-05-11T23:02:42.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0299999999999999e-06 2024-05-11T23:02:42.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:42.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147029+0000", 2024-05-11T23:02:42.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5979999999999999e-06 2024-05-11T23:02:42.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:42.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177848+0000", 2024-05-11T23:02:42.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030818662 2024-05-11T23:02:42.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:42.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.177858+0000", 2024-05-11T23:02:42.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0061000000000001e-05 2024-05-11T23:02:42.834 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.835 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.835 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.835 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71327 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:42.835 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147299+0000", 2024-05-11T23:02:42.835 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.671725129999999, 2024-05-11T23:02:42.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030857026999999999, 2024-05-11T23:02:42.835 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:42.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:42.835 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:42.835 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:42.835 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:42.835 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:42.835 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71327 2024-05-11T23:02:42.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.836 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:42.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:42.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147299+0000", 2024-05-11T23:02:42.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:42.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147299+0000", 2024-05-11T23:02:42.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:42.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147299+0000", 2024-05-11T23:02:42.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1500000000000001e-07 2024-05-11T23:02:42.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:42.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147303+0000", 2024-05-11T23:02:42.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1459999999999998e-06 2024-05-11T23:02:42.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:42.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147312+0000", 2024-05-11T23:02:42.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3299999999999999e-06 2024-05-11T23:02:42.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:42.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147321+0000", 2024-05-11T23:02:42.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7420000000000003e-06 2024-05-11T23:02:42.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:42.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178145+0000", 2024-05-11T23:02:42.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030824852999999999 2024-05-11T23:02:42.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:42.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178156+0000", 2024-05-11T23:02:42.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0740999999999999e-05 2024-05-11T23:02:42.838 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.838 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.838 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.838 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71328 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:42.838 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147326+0000", 2024-05-11T23:02:42.839 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.671698648, 2024-05-11T23:02:42.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030846630999999999, 2024-05-11T23:02:42.839 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:42.839 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:42.839 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:42.839 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:42.839 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:42.839 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:42.839 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71328 2024-05-11T23:02:42.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.839 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:42.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:42.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147326+0000", 2024-05-11T23:02:42.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:42.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147326+0000", 2024-05-11T23:02:42.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:42.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147326+0000", 2024-05-11T23:02:42.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5400000000000002e-07 2024-05-11T23:02:42.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:42.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147327+0000", 2024-05-11T23:02:42.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2500000000000001e-06 2024-05-11T23:02:42.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:42.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147331+0000", 2024-05-11T23:02:42.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5499999999999999e-06 2024-05-11T23:02:42.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:42.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147333+0000", 2024-05-11T23:02:42.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5450000000000002e-06 2024-05-11T23:02:42.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:42.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178168+0000", 2024-05-11T23:02:42.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030834934000000001 2024-05-11T23:02:42.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:42.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178172+0000", 2024-05-11T23:02:42.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0980000000000004e-06 2024-05-11T23:02:42.842 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.842 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.842 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.842 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71329 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:42.842 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147336+0000", 2024-05-11T23:02:42.842 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.671687794, 2024-05-11T23:02:42.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030865317999999999, 2024-05-11T23:02:42.842 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:42.842 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:42.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:42.843 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:42.843 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:42.843 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:42.843 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71329 2024-05-11T23:02:42.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.843 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:42.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:42.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147336+0000", 2024-05-11T23:02:42.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:42.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147336+0000", 2024-05-11T23:02:42.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:42.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147337+0000", 2024-05-11T23:02:42.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:42.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:42.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147338+0000", 2024-05-11T23:02:42.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5090000000000001e-06 2024-05-11T23:02:42.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:42.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147344+0000", 2024-05-11T23:02:42.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.66e-06 2024-05-11T23:02:42.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:42.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147348+0000", 2024-05-11T23:02:42.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2980000000000002e-06 2024-05-11T23:02:42.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:42.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178192+0000", 2024-05-11T23:02:42.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030843425000000001 2024-05-11T23:02:42.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:42.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178202+0000", 2024-05-11T23:02:42.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0217000000000001e-05 2024-05-11T23:02:42.845 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.846 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.846 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.846 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71330 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:42.846 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147355+0000", 2024-05-11T23:02:42.846 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.671669640000001, 2024-05-11T23:02:42.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030862031000000002, 2024-05-11T23:02:42.846 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:42.846 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:42.846 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:42.846 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:42.846 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:42.846 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:42.847 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71330 2024-05-11T23:02:42.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.847 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:42.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:42.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147355+0000", 2024-05-11T23:02:42.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:42.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147355+0000", 2024-05-11T23:02:42.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:42.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147355+0000", 2024-05-11T23:02:42.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.39e-07 2024-05-11T23:02:42.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:42.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147365+0000", 2024-05-11T23:02:42.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7140000000000005e-06 2024-05-11T23:02:42.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:42.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147370+0000", 2024-05-11T23:02:42.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2979999999999999e-06 2024-05-11T23:02:42.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:42.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147373+0000", 2024-05-11T23:02:42.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.41e-06 2024-05-11T23:02:42.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:42.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178213+0000", 2024-05-11T23:02:42.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030839325000000001 2024-05-11T23:02:42.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:42.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178217+0000", 2024-05-11T23:02:42.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9450000000000003e-06 2024-05-11T23:02:42.849 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.849 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.850 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71331 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:42.850 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147391+0000", 2024-05-11T23:02:42.850 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.671632719000002, 2024-05-11T23:02:42.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030863854999999999, 2024-05-11T23:02:42.850 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:42.850 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:42.850 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:42.850 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:42.850 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:42.850 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:42.850 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71331 2024-05-11T23:02:42.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.851 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:42.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:42.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147391+0000", 2024-05-11T23:02:42.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:42.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147391+0000", 2024-05-11T23:02:42.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:42.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147392+0000", 2024-05-11T23:02:42.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.68e-07 2024-05-11T23:02:42.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:42.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147395+0000", 2024-05-11T23:02:42.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.503e-06 2024-05-11T23:02:42.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:42.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147401+0000", 2024-05-11T23:02:42.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7889999999999997e-06 2024-05-11T23:02:42.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:42.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147406+0000", 2024-05-11T23:02:42.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.831e-06 2024-05-11T23:02:42.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:42.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178245+0000", 2024-05-11T23:02:42.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030838884 2024-05-11T23:02:42.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:42.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178255+0000", 2024-05-11T23:02:42.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.028e-05 2024-05-11T23:02:42.853 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.853 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.853 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.854 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71332 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:42.854 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147414+0000", 2024-05-11T23:02:42.854 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.671610668, 2024-05-11T23:02:42.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030874374, 2024-05-11T23:02:42.854 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:42.854 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:42.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:42.854 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:42.854 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:42.854 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:42.854 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71332 2024-05-11T23:02:42.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.854 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:42.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:42.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147414+0000", 2024-05-11T23:02:42.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:42.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147414+0000", 2024-05-11T23:02:42.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:42.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147414+0000", 2024-05-11T23:02:42.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6899999999999998e-07 2024-05-11T23:02:42.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:42.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147416+0000", 2024-05-11T23:02:42.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:02:42.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:42.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147428+0000", 2024-05-11T23:02:42.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1428e-05 2024-05-11T23:02:42.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:42.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147436+0000", 2024-05-11T23:02:42.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9689999999999994e-06 2024-05-11T23:02:42.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:42.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178283+0000", 2024-05-11T23:02:42.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030847764 2024-05-11T23:02:42.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:42.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178288+0000", 2024-05-11T23:02:42.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6430000000000003e-06 2024-05-11T23:02:42.857 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.857 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.857 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.857 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71333 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:42.857 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147444+0000", 2024-05-11T23:02:42.857 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.671580686999999, 2024-05-11T23:02:42.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.03086906, 2024-05-11T23:02:42.858 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:42.858 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:42.858 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:42.858 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:42.858 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:42.858 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:42.858 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71333 2024-05-11T23:02:42.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.858 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:42.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:42.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147444+0000", 2024-05-11T23:02:42.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:42.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147444+0000", 2024-05-11T23:02:42.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:42.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147444+0000", 2024-05-11T23:02:42.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:02:42.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:42.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147446+0000", 2024-05-11T23:02:42.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3590000000000002e-06 2024-05-11T23:02:42.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:42.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147453+0000", 2024-05-11T23:02:42.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7580000000000003e-06 2024-05-11T23:02:42.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:42.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147460+0000", 2024-05-11T23:02:42.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5010000000000001e-06 2024-05-11T23:02:42.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:42.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178308+0000", 2024-05-11T23:02:42.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030848179 2024-05-11T23:02:42.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:42.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178313+0000", 2024-05-11T23:02:42.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8219999999999997e-06 2024-05-11T23:02:42.861 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.861 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.861 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.861 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71334 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:42.861 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147466+0000", 2024-05-11T23:02:42.861 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.671558016999999, 2024-05-11T23:02:42.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030868219999999998, 2024-05-11T23:02:42.861 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:42.861 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:42.862 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:42.862 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:42.862 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:42.862 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:42.862 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71334 2024-05-11T23:02:42.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.862 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:42.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:42.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147466+0000", 2024-05-11T23:02:42.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:42.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147466+0000", 2024-05-11T23:02:42.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:42.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147467+0000", 2024-05-11T23:02:42.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9099999999999999e-07 2024-05-11T23:02:42.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:42.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147469+0000", 2024-05-11T23:02:42.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9769999999999999e-06 2024-05-11T23:02:42.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:42.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147475+0000", 2024-05-11T23:02:42.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1369999999999999e-06 2024-05-11T23:02:42.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:42.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147479+0000", 2024-05-11T23:02:42.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6639999999999997e-06 2024-05-11T23:02:42.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:42.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178330+0000", 2024-05-11T23:02:42.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030850776 2024-05-11T23:02:42.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:42.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178334+0000", 2024-05-11T23:02:42.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2749999999999997e-06 2024-05-11T23:02:42.865 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.865 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.865 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.865 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71335 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:42.865 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147489+0000", 2024-05-11T23:02:42.865 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.671535460000001, 2024-05-11T23:02:42.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030866874999999998, 2024-05-11T23:02:42.865 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:42.865 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:42.865 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:42.865 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:42.865 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:42.865 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:42.866 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71335 2024-05-11T23:02:42.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.866 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:42.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:42.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147489+0000", 2024-05-11T23:02:42.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:42.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147489+0000", 2024-05-11T23:02:42.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:42.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147489+0000", 2024-05-11T23:02:42.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7899999999999999e-07 2024-05-11T23:02:42.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:42.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147491+0000", 2024-05-11T23:02:42.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0600000000000002e-06 2024-05-11T23:02:42.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:42.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147498+0000", 2024-05-11T23:02:42.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3339999999999998e-06 2024-05-11T23:02:42.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:42.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147502+0000", 2024-05-11T23:02:42.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8629999999999999e-06 2024-05-11T23:02:42.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:42.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178350+0000", 2024-05-11T23:02:42.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030847795000000001 2024-05-11T23:02:42.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:42.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178356+0000", 2024-05-11T23:02:42.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4439999999999999e-06 2024-05-11T23:02:42.868 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.868 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.868 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.869 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71336 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:42.869 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147508+0000", 2024-05-11T23:02:42.869 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.671515718999999, 2024-05-11T23:02:42.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030867196999999999, 2024-05-11T23:02:42.869 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:42.869 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:42.869 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:42.869 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:42.869 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:42.869 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:42.869 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71336 2024-05-11T23:02:42.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.869 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:42.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:42.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147508+0000", 2024-05-11T23:02:42.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:42.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147508+0000", 2024-05-11T23:02:42.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:42.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147509+0000", 2024-05-11T23:02:42.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:02:42.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:42.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147511+0000", 2024-05-11T23:02:42.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.88e-06 2024-05-11T23:02:42.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:42.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147517+0000", 2024-05-11T23:02:42.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7849999999999997e-06 2024-05-11T23:02:42.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:42.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147521+0000", 2024-05-11T23:02:42.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4240000000000003e-06 2024-05-11T23:02:42.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:42.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178371+0000", 2024-05-11T23:02:42.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030850452 2024-05-11T23:02:42.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:42.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178376+0000", 2024-05-11T23:02:42.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2560000000000004e-06 2024-05-11T23:02:42.872 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.872 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.872 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.872 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71337 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:42.872 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147527+0000", 2024-05-11T23:02:42.873 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.671497122000002, 2024-05-11T23:02:42.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030869829000000001, 2024-05-11T23:02:42.873 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:42.873 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:42.873 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:42.873 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:42.873 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:42.873 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:42.873 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71337 2024-05-11T23:02:42.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.873 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:42.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:42.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147527+0000", 2024-05-11T23:02:42.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:42.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147527+0000", 2024-05-11T23:02:42.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:42.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147527+0000", 2024-05-11T23:02:42.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3099999999999999e-07 2024-05-11T23:02:42.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:42.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147529+0000", 2024-05-11T23:02:42.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8870000000000001e-06 2024-05-11T23:02:42.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:42.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147536+0000", 2024-05-11T23:02:42.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2910000000000004e-06 2024-05-11T23:02:42.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:42.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147540+0000", 2024-05-11T23:02:42.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8189999999999999e-06 2024-05-11T23:02:42.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:42.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178393+0000", 2024-05-11T23:02:42.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030852118000000001 2024-05-11T23:02:42.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:42.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178397+0000", 2024-05-11T23:02:42.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3830000000000002e-06 2024-05-11T23:02:42.876 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.876 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.876 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.876 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71338 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:42.876 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147546+0000", 2024-05-11T23:02:42.876 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.671477863, 2024-05-11T23:02:42.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030872143000000001, 2024-05-11T23:02:42.876 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:42.876 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:42.876 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:42.877 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:42.877 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:42.877 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:42.877 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71338 2024-05-11T23:02:42.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.877 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:42.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:42.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147546+0000", 2024-05-11T23:02:42.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:42.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147546+0000", 2024-05-11T23:02:42.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:42.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147547+0000", 2024-05-11T23:02:42.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7500000000000001e-07 2024-05-11T23:02:42.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:42.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147549+0000", 2024-05-11T23:02:42.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0040000000000002e-06 2024-05-11T23:02:42.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:42.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147555+0000", 2024-05-11T23:02:42.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1959999999999996e-06 2024-05-11T23:02:42.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:42.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147559+0000", 2024-05-11T23:02:42.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3329999999999998e-06 2024-05-11T23:02:42.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:42.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178414+0000", 2024-05-11T23:02:42.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030854837999999999 2024-05-11T23:02:42.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:42.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178418+0000", 2024-05-11T23:02:42.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3970000000000004e-06 2024-05-11T23:02:42.880 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.880 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.880 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.880 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71339 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:42.880 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147570+0000", 2024-05-11T23:02:42.880 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.671453865, 2024-05-11T23:02:42.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030868237, 2024-05-11T23:02:42.880 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:42.880 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:42.880 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:42.880 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:42.880 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:42.880 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:42.880 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71339 2024-05-11T23:02:42.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.881 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:42.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:42.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147570+0000", 2024-05-11T23:02:42.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:42.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147570+0000", 2024-05-11T23:02:42.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:42.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147571+0000", 2024-05-11T23:02:42.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2399999999999999e-07 2024-05-11T23:02:42.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:42.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147573+0000", 2024-05-11T23:02:42.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.43e-06 2024-05-11T23:02:42.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:42.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147589+0000", 2024-05-11T23:02:42.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5977999999999998e-05 2024-05-11T23:02:42.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:42.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147595+0000", 2024-05-11T23:02:42.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.463e-06 2024-05-11T23:02:42.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:42.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178434+0000", 2024-05-11T23:02:42.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030839582000000001 2024-05-11T23:02:42.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:42.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178439+0000", 2024-05-11T23:02:42.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3599999999999998e-06 2024-05-11T23:02:42.883 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.883 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.883 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71340 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147614+0000", 2024-05-11T23:02:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.671409772000001, 2024-05-11T23:02:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030844255000000001, 2024-05-11T23:02:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71340 2024-05-11T23:02:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:42.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147614+0000", 2024-05-11T23:02:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147614+0000", 2024-05-11T23:02:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147615+0000", 2024-05-11T23:02:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4200000000000001e-07 2024-05-11T23:02:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:42.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147617+0000", 2024-05-11T23:02:42.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5150000000000001e-06 2024-05-11T23:02:42.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:42.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147626+0000", 2024-05-11T23:02:42.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1950000000000003e-06 2024-05-11T23:02:42.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:42.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147631+0000", 2024-05-11T23:02:42.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3369999999999999e-06 2024-05-11T23:02:42.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:42.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178454+0000", 2024-05-11T23:02:42.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030823481999999999 2024-05-11T23:02:42.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:42.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178459+0000", 2024-05-11T23:02:42.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.284e-06 2024-05-11T23:02:42.887 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.887 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.887 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.887 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:71341 3.12 3.e5ca9bb2 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-11T23:02:42.887 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:11.147639+0000", 2024-05-11T23:02:42.887 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.671385689000001, 2024-05-11T23:02:42.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030842306999999999, 2024-05-11T23:02:42.888 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:42.888 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:42.888 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:42.888 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:42.888 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:42.888 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:42.888 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 71341 2024-05-11T23:02:42.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.888 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:42.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:42.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147639+0000", 2024-05-11T23:02:42.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:42.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147639+0000", 2024-05-11T23:02:42.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:42.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147639+0000", 2024-05-11T23:02:42.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6600000000000002e-07 2024-05-11T23:02:42.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:42.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147646+0000", 2024-05-11T23:02:42.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8870000000000003e-06 2024-05-11T23:02:42.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:42.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147650+0000", 2024-05-11T23:02:42.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4510000000000002e-06 2024-05-11T23:02:42.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:42.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.147653+0000", 2024-05-11T23:02:42.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8700000000000001e-06 2024-05-11T23:02:42.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:42.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178476+0000", 2024-05-11T23:02:42.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030823350999999999 2024-05-11T23:02:42.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:42.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:11.178481+0000", 2024-05-11T23:02:42.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3819999999999997e-06 2024-05-11T23:02:42.891 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.891 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.891 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.891 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:81250 3.b 3:d7828050:::rbd_data.111e7ffeb370.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:42.891 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:16.371363+0000", 2024-05-11T23:02:42.891 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 26.447661332999999, 2024-05-11T23:02:42.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.037265637999999997, 2024-05-11T23:02:42.891 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:42.891 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:42.891 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:42.891 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:42.892 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:42.892 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:42.892 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 81250 2024-05-11T23:02:42.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.892 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:42.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:42.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.371363+0000", 2024-05-11T23:02:42.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:42.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.371363+0000", 2024-05-11T23:02:42.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:42.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:42.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.371365+0000", 2024-05-11T23:02:42.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1330000000000002e-06 2024-05-11T23:02:42.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:42.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.372770+0000", 2024-05-11T23:02:42.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0014046110000000001 2024-05-11T23:02:42.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:42.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.372790+0000", 2024-05-11T23:02:42.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0530000000000002e-05 2024-05-11T23:02:42.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:42.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.372799+0000", 2024-05-11T23:02:42.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3000000000000007e-06 2024-05-11T23:02:42.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:42.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.372883+0000", 2024-05-11T23:02:42.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3709999999999996e-05 2024-05-11T23:02:42.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:42.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.373076+0000", 2024-05-11T23:02:42.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019249499999999999 2024-05-11T23:02:42.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 4", 2024-05-11T23:02:42.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.380091+0000", 2024-05-11T23:02:42.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.007015533 2024-05-11T23:02:42.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:42.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.408542+0000", 2024-05-11T23:02:42.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028451026000000001 2024-05-11T23:02:42.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:02:42.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.408555+0000", 2024-05-11T23:02:42.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3151e-05 2024-05-11T23:02:42.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:42.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.408592+0000", 2024-05-11T23:02:42.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6463e-05 2024-05-11T23:02:42.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:42.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:42.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:42.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.408629+0000", 2024-05-11T23:02:42.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6686000000000001e-05 2024-05-11T23:02:42.896 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.896 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.896 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.896 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:42.896 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:42.896 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:42.898 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:02:42.901 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:42.898+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:43.000 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:42.999+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:43.048 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:43.049 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:43.049 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:02:43.049 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:43.054 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 3, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:02:43.054 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:02:43.054 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-05-11T23:02:43.055 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-11T23:02:43.059 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:02:43.063 INFO:teuthology.orchestra.run.smithi005.stdout:3878 trunc from 0xac35b50 to 0xb7784d7 2024-05-11T23:02:43.069 INFO:teuthology.orchestra.run.smithi005.stdout:3879 write 0x4ec867a thru 0x4ed733c (0xecc3 bytes) 2024-05-11T23:02:43.086 INFO:teuthology.orchestra.run.smithi005.stdout:3881 write 0x1bb29ee thru 0x1bb40ee (0x1701 bytes) 2024-05-11T23:02:43.100 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:43.099+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:43.108 INFO:teuthology.orchestra.run.smithi005.stdout:3882 trunc from 0xb7784d7 to 0x1083060 2024-05-11T23:02:43.136 INFO:teuthology.orchestra.run.smithi005.stdout:3884 punch from 0x5ff500 to 0x6061e2, (0x6ce2 bytes) 2024-05-11T23:02:43.139 INFO:teuthology.orchestra.run.smithi005.stdout:3886 trunc from 0x1083060 to 0x3e4e307 2024-05-11T23:02:43.161 INFO:teuthology.orchestra.run.smithi005.stdout:3887 writesame 0x99c1b6e thru 0x99c5ff7 data_size 0x2(0x448a bytes) 2024-05-11T23:02:43.161 INFO:teuthology.orchestra.run.smithi005.stdout:3888 writesame 0x5a214a9 thru 0x5a31217 data_size 0x1(0xfd6f bytes) 2024-05-11T23:02:43.181 INFO:teuthology.orchestra.run.smithi005.stdout:3889 read 0x7c21c8a thru 0x7c25e56 (0x41cd bytes) 2024-05-11T23:02:43.192 INFO:teuthology.orchestra.run.smithi005.stdout:3891 write 0xe1101c8 thru 0xe11eeab (0xece4 bytes) 2024-05-11T23:02:43.200 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:43.200 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:43.201 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:43.201 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:43.201 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:43.201 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:43.201+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:43.211 INFO:teuthology.orchestra.run.smithi005.stdout:3894 writesame 0x71f683f thru 0x71ff1f1 data_size 0x1(0x89b3 bytes) 2024-05-11T23:02:43.214 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:02:43.227 INFO:teuthology.orchestra.run.smithi005.stdout:3895 writesame 0xbdcbb8f thru 0xbdd1835 data_size 0x1(0x5ca7 bytes) 2024-05-11T23:02:43.244 INFO:teuthology.orchestra.run.smithi005.stdout:3897 writesame 0x4e23410 thru 0x4e26ec2 data_size 0x1(0x3ab3 bytes) 2024-05-11T23:02:43.261 INFO:teuthology.orchestra.run.smithi005.stdout:3898 writesame 0xde4bdea thru 0xde581f1 data_size 0x66(0xc408 bytes) 2024-05-11T23:02:43.279 INFO:teuthology.orchestra.run.smithi005.stdout:3900 read 0xd2fb04a thru 0xd3086d3 (0xd68a bytes) 2024-05-11T23:02:43.279 INFO:teuthology.orchestra.run.smithi005.stdout:3901 punch from 0xb369dd9 to 0xb36dd78, (0x3f9f bytes) 2024-05-11T23:02:43.280 INFO:teuthology.orchestra.run.smithi005.stdout:3902 write 0x6dbe98d thru 0x6dbff67 (0x15db bytes) 2024-05-11T23:02:43.291 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:43.291 INFO:teuthology.orchestra.run.smithi005.stdout:3904 write 0xc23c1a3 thru 0xc2436ff (0x755d bytes) 2024-05-11T23:02:43.301 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:43.301+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:43.306 INFO:teuthology.orchestra.run.smithi005.stdout:3905 punch from 0x149ee to 0x164c6, (0x1ad8 bytes) 2024-05-11T23:02:43.306 INFO:teuthology.orchestra.run.smithi005.stdout:3906 write 0x1d754d thru 0x1daa6a (0x351e bytes) 2024-05-11T23:02:43.312 INFO:teuthology.orchestra.run.smithi005.stdout:3907 writesame 0x37f1578 thru 0x380067a data_size 0x47(0xf103 bytes) 2024-05-11T23:02:43.330 INFO:teuthology.orchestra.run.smithi005.stdout:3908 read 0xd7c52b thru 0xd87d87 (0xb85d bytes) 2024-05-11T23:02:43.331 INFO:teuthology.orchestra.run.smithi005.stdout:3911 write 0x1b6ef60 thru 0x1b7e531 (0xf5d2 bytes) 2024-05-11T23:02:43.345 INFO:teuthology.orchestra.run.smithi005.stdout:3912 write 0x824cad4 thru 0x8254352 (0x787f bytes) 2024-05-11T23:02:43.359 INFO:teuthology.orchestra.run.smithi005.stdout:3913 write 0x7ad6548 thru 0x7adab45 (0x45fe bytes) 2024-05-11T23:02:43.375 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:43.375 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:43.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:43.375 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:43.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.375 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28445 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:43.375 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:43.375 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.292982031000001, 2024-05-11T23:02:43.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031719429, 2024-05-11T23:02:43.376 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.376 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.376 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:43.376 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:43.376 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:43.376 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:43.376 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28445 2024-05-11T23:02:43.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.376 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:43.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:43.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:43.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-05-11T23:02:43.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081234+0000", 2024-05-11T23:02:43.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.578e-06 2024-05-11T23:02:43.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081237+0000", 2024-05-11T23:02:43.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3340000000000002e-06 2024-05-11T23:02:43.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081240+0000", 2024-05-11T23:02:43.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7429999999999998e-06 2024-05-11T23:02:43.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112828+0000", 2024-05-11T23:02:43.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031587941000000001 2024-05-11T23:02:43.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112878+0000", 2024-05-11T23:02:43.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0535999999999999e-05 2024-05-11T23:02:43.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:43.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112951+0000", 2024-05-11T23:02:43.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3072999999999996e-05 2024-05-11T23:02:43.380 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.380 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.380 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.381 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28448 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 507904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:43.381 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:02:43.381 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.292949530000001, 2024-05-11T23:02:43.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031916923, 2024-05-11T23:02:43.381 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.381 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.381 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:43.381 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:43.381 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:43.381 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:43.381 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28448 2024-05-11T23:02:43.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.381 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:02:43.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:02:43.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081265+0000", 2024-05-11T23:02:43.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2899999999999999e-07 2024-05-11T23:02:43.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081267+0000", 2024-05-11T23:02:43.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6749999999999998e-06 2024-05-11T23:02:43.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081273+0000", 2024-05-11T23:02:43.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4049999999999999e-06 2024-05-11T23:02:43.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081278+0000", 2024-05-11T23:02:43.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0139999999999998e-06 2024-05-11T23:02:43.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112996+0000", 2024-05-11T23:02:43.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031718283 2024-05-11T23:02:43.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113080+0000", 2024-05-11T23:02:43.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4334999999999998e-05 2024-05-11T23:02:43.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:43.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113181+0000", 2024-05-11T23:02:43.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000100882 2024-05-11T23:02:43.384 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.385 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.385 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.385 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28451 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 540672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:43.385 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:43.385 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.292921028000002, 2024-05-11T23:02:43.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032054412999999997, 2024-05-11T23:02:43.385 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.385 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.385 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:43.385 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:43.385 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:43.385 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:43.385 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28451 2024-05-11T23:02:43.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.386 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:43.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:43.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:43.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2499999999999995e-07 2024-05-11T23:02:43.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081300+0000", 2024-05-11T23:02:43.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0929999999999999e-06 2024-05-11T23:02:43.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081306+0000", 2024-05-11T23:02:43.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6290000000000004e-06 2024-05-11T23:02:43.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081311+0000", 2024-05-11T23:02:43.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9289999999999997e-06 2024-05-11T23:02:43.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113208+0000", 2024-05-11T23:02:43.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031897162 2024-05-11T23:02:43.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113277+0000", 2024-05-11T23:02:43.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8811000000000002e-05 2024-05-11T23:02:43.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:43.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113347+0000", 2024-05-11T23:02:43.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0264000000000004e-05 2024-05-11T23:02:43.389 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.389 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.389 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.389 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28454 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 573440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:43.389 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:43.389 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.292846406999999, 2024-05-11T23:02:43.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032210564999999997, 2024-05-11T23:02:43.389 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.389 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:43.390 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:43.390 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:43.390 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:43.390 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28454 2024-05-11T23:02:43.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.390 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:43.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:43.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:43.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.99e-07 2024-05-11T23:02:43.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081371+0000", 2024-05-11T23:02:43.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1200000000000002e-06 2024-05-11T23:02:43.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081378+0000", 2024-05-11T23:02:43.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6089999999999997e-06 2024-05-11T23:02:43.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081384+0000", 2024-05-11T23:02:43.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0859999999999998e-06 2024-05-11T23:02:43.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113388+0000", 2024-05-11T23:02:43.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032004248999999999 2024-05-11T23:02:43.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113470+0000", 2024-05-11T23:02:43.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1871000000000003e-05 2024-05-11T23:02:43.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:43.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113578+0000", 2024-05-11T23:02:43.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000108031 2024-05-11T23:02:43.393 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.393 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.393 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.393 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28457 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 606208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:43.393 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:43.393 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.292816182999999, 2024-05-11T23:02:43.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032410130000000002, 2024-05-11T23:02:43.394 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.394 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.394 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:43.394 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:43.394 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:43.394 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:43.394 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28457 2024-05-11T23:02:43.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.394 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:43.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:43.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:43.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.32e-07 2024-05-11T23:02:43.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081400+0000", 2024-05-11T23:02:43.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9120000000000001e-06 2024-05-11T23:02:43.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081404+0000", 2024-05-11T23:02:43.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7900000000000001e-06 2024-05-11T23:02:43.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081407+0000", 2024-05-11T23:02:43.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8839999999999998e-06 2024-05-11T23:02:43.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113621+0000", 2024-05-11T23:02:43.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032213841 2024-05-11T23:02:43.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113702+0000", 2024-05-11T23:02:43.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1574000000000002e-05 2024-05-11T23:02:43.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:43.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113808+0000", 2024-05-11T23:02:43.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000105697 2024-05-11T23:02:43.397 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.397 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.397 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.397 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28460 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 638976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:43.398 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:43.398 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.292803352, 2024-05-11T23:02:43.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032616804999999999, 2024-05-11T23:02:43.398 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.398 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.398 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:43.398 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:43.398 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:43.398 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:43.398 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28460 2024-05-11T23:02:43.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.398 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:43.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:43.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:43.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5900000000000003e-07 2024-05-11T23:02:43.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081412+0000", 2024-05-11T23:02:43.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1990000000000001e-06 2024-05-11T23:02:43.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081415+0000", 2024-05-11T23:02:43.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.971e-06 2024-05-11T23:02:43.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081418+0000", 2024-05-11T23:02:43.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-05-11T23:02:43.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113853+0000", 2024-05-11T23:02:43.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032435155 2024-05-11T23:02:43.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113936+0000", 2024-05-11T23:02:43.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3441000000000003e-05 2024-05-11T23:02:43.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:43.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114027+0000", 2024-05-11T23:02:43.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1108999999999998e-05 2024-05-11T23:02:43.401 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.401 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.401 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.402 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28463 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 671744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:43.402 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:43.402 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.29278643, 2024-05-11T23:02:43.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032814958999999998, 2024-05-11T23:02:43.402 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.402 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.402 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:43.402 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:43.402 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:43.402 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:43.402 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28463 2024-05-11T23:02:43.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.402 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:43.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:43.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:43.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4999999999999998e-07 2024-05-11T23:02:43.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081430+0000", 2024-05-11T23:02:43.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6109999999999999e-06 2024-05-11T23:02:43.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081433+0000", 2024-05-11T23:02:43.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5590000000000001e-06 2024-05-11T23:02:43.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081437+0000", 2024-05-11T23:02:43.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3950000000000001e-06 2024-05-11T23:02:43.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114061+0000", 2024-05-11T23:02:43.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032624092 2024-05-11T23:02:43.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114142+0000", 2024-05-11T23:02:43.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0893e-05 2024-05-11T23:02:43.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:43.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114242+0000", 2024-05-11T23:02:43.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010095899999999999 2024-05-11T23:02:43.405 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.405 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.406 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.406 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28466 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 704512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:43.406 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:43.406 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.292773896, 2024-05-11T23:02:43.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032933658999999997, 2024-05-11T23:02:43.406 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.406 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.406 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:43.406 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:43.406 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:43.406 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:43.406 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28466 2024-05-11T23:02:43.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.407 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:43.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:43.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:43.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4700000000000002e-07 2024-05-11T23:02:43.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081442+0000", 2024-05-11T23:02:43.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.296e-06 2024-05-11T23:02:43.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081445+0000", 2024-05-11T23:02:43.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9009999999999998e-06 2024-05-11T23:02:43.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081447+0000", 2024-05-11T23:02:43.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4200000000000001e-06 2024-05-11T23:02:43.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114280+0000", 2024-05-11T23:02:43.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032832667000000003 2024-05-11T23:02:43.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114325+0000", 2024-05-11T23:02:43.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5018000000000002e-05 2024-05-11T23:02:43.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:43.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114374+0000", 2024-05-11T23:02:43.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.901e-05 2024-05-11T23:02:43.409 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.410 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.410 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.410 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28468 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 737280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:43.410 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:43.410 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.292753890999997, 2024-05-11T23:02:43.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033118776000000003, 2024-05-11T23:02:43.410 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.410 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.410 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:43.410 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:43.410 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:43.410 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:43.410 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28468 2024-05-11T23:02:43.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.411 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:43.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:43.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:43.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3799999999999998e-07 2024-05-11T23:02:43.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081462+0000", 2024-05-11T23:02:43.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4899999999999999e-06 2024-05-11T23:02:43.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081465+0000", 2024-05-11T23:02:43.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4079999999999998e-06 2024-05-11T23:02:43.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081468+0000", 2024-05-11T23:02:43.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7e-06 2024-05-11T23:02:43.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114415+0000", 2024-05-11T23:02:43.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032946766000000002 2024-05-11T23:02:43.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114497+0000", 2024-05-11T23:02:43.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1854000000000006e-05 2024-05-11T23:02:43.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:43.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114579+0000", 2024-05-11T23:02:43.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2120000000000007e-05 2024-05-11T23:02:43.413 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.414 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.414 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.414 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28471 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 770048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:43.414 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:43.414 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.292739183999998, 2024-05-11T23:02:43.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033234970000000003, 2024-05-11T23:02:43.414 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:43.414 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:43.414 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:43.414 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:43.414 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28471 2024-05-11T23:02:43.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.415 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:43.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:43.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:43.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-05-11T23:02:43.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081476+0000", 2024-05-11T23:02:43.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3120000000000001e-06 2024-05-11T23:02:43.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081480+0000", 2024-05-11T23:02:43.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1470000000000001e-06 2024-05-11T23:02:43.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081482+0000", 2024-05-11T23:02:43.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7480000000000002e-06 2024-05-11T23:02:43.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114616+0000", 2024-05-11T23:02:43.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033133322 2024-05-11T23:02:43.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114662+0000", 2024-05-11T23:02:43.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6020999999999999e-05 2024-05-11T23:02:43.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:43.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114710+0000", 2024-05-11T23:02:43.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8059999999999997e-05 2024-05-11T23:02:43.418 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.418 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.418 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.418 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28474 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:43.418 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:43.418 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.292723109999997, 2024-05-11T23:02:43.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033336431, 2024-05-11T23:02:43.418 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.418 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.418 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:43.418 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:43.419 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:43.419 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:43.419 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28474 2024-05-11T23:02:43.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.419 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:43.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:43.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:43.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.15e-07 2024-05-11T23:02:43.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081493+0000", 2024-05-11T23:02:43.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4950000000000001e-06 2024-05-11T23:02:43.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081496+0000", 2024-05-11T23:02:43.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.309e-06 2024-05-11T23:02:43.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081499+0000", 2024-05-11T23:02:43.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4570000000000001e-06 2024-05-11T23:02:43.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114740+0000", 2024-05-11T23:02:43.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033240619999999999 2024-05-11T23:02:43.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114781+0000", 2024-05-11T23:02:43.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1106999999999998e-05 2024-05-11T23:02:43.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:43.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114827+0000", 2024-05-11T23:02:43.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6028e-05 2024-05-11T23:02:43.422 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.422 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.422 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.422 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28477 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 835584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:43.422 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:43.422 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.292706096000003, 2024-05-11T23:02:43.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033527225000000001, 2024-05-11T23:02:43.422 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.422 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.423 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:43.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:43.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:43.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:43.423 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28477 2024-05-11T23:02:43.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.423 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:43.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:43.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:43.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1300000000000001e-07 2024-05-11T23:02:43.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081510+0000", 2024-05-11T23:02:43.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4610000000000001e-06 2024-05-11T23:02:43.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081513+0000", 2024-05-11T23:02:43.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2559999999999998e-06 2024-05-11T23:02:43.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081516+0000", 2024-05-11T23:02:43.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7099999999999999e-06 2024-05-11T23:02:43.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114869+0000", 2024-05-11T23:02:43.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033353718999999997 2024-05-11T23:02:43.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114954+0000", 2024-05-11T23:02:43.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4969000000000006e-05 2024-05-11T23:02:43.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:43.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115035+0000", 2024-05-11T23:02:43.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0697000000000003e-05 2024-05-11T23:02:43.426 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.426 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.426 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.426 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28480 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 868352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:43.426 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:43.426 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.292689125999999, 2024-05-11T23:02:43.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033695939000000001, 2024-05-11T23:02:43.426 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.427 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:43.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:43.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:43.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:43.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28480 2024-05-11T23:02:43.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.427 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:43.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:43.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:43.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.46e-07 2024-05-11T23:02:43.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081526+0000", 2024-05-11T23:02:43.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.347e-06 2024-05-11T23:02:43.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081530+0000", 2024-05-11T23:02:43.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.348e-06 2024-05-11T23:02:43.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081533+0000", 2024-05-11T23:02:43.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4369999999999999e-06 2024-05-11T23:02:43.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115078+0000", 2024-05-11T23:02:43.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033545063 2024-05-11T23:02:43.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115164+0000", 2024-05-11T23:02:43.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5662999999999996e-05 2024-05-11T23:02:43.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:43.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115221+0000", 2024-05-11T23:02:43.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6734999999999998e-05 2024-05-11T23:02:43.430 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.430 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.430 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.430 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28483 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:43.430 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:43.430 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.292671953000003, 2024-05-11T23:02:43.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033803466999999997, 2024-05-11T23:02:43.431 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.431 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:43.431 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:43.431 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:43.431 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:43.431 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28483 2024-05-11T23:02:43.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.431 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.431 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.431 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:43.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:43.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:43.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4999999999999998e-07 2024-05-11T23:02:43.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081544+0000", 2024-05-11T23:02:43.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:02:43.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081547+0000", 2024-05-11T23:02:43.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3139999999999999e-06 2024-05-11T23:02:43.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081550+0000", 2024-05-11T23:02:43.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7779999999999999e-06 2024-05-11T23:02:43.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115253+0000", 2024-05-11T23:02:43.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033703516000000003 2024-05-11T23:02:43.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115302+0000", 2024-05-11T23:02:43.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8869999999999998e-05 2024-05-11T23:02:43.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:43.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115345+0000", 2024-05-11T23:02:43.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3340999999999997e-05 2024-05-11T23:02:43.434 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.434 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.434 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.435 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28487 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 933888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:43.435 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:43.435 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.292660752000003, 2024-05-11T23:02:43.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033903918999999998, 2024-05-11T23:02:43.435 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.435 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:43.435 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:43.435 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:43.435 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:43.435 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28487 2024-05-11T23:02:43.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.435 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:43.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:43.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:43.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9999999999999999e-07 2024-05-11T23:02:43.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081559+0000", 2024-05-11T23:02:43.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-05-11T23:02:43.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081563+0000", 2024-05-11T23:02:43.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2370000000000003e-06 2024-05-11T23:02:43.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081566+0000", 2024-05-11T23:02:43.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6199999999999999e-06 2024-05-11T23:02:43.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115375+0000", 2024-05-11T23:02:43.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033808887000000003 2024-05-11T23:02:43.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115413+0000", 2024-05-11T23:02:43.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8309000000000002e-05 2024-05-11T23:02:43.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:43.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115457+0000", 2024-05-11T23:02:43.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3809e-05 2024-05-11T23:02:43.438 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.438 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.438 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.439 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28490 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 966656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:43.439 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:43.439 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.292644228, 2024-05-11T23:02:43.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034114903000000002, 2024-05-11T23:02:43.439 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.439 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:43.439 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:43.439 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:43.439 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:43.439 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28490 2024-05-11T23:02:43.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.439 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:43.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:43.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:43.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4400000000000001e-07 2024-05-11T23:02:43.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081571+0000", 2024-05-11T23:02:43.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2130000000000001e-06 2024-05-11T23:02:43.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081574+0000", 2024-05-11T23:02:43.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8270000000000002e-06 2024-05-11T23:02:43.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081577+0000", 2024-05-11T23:02:43.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-05-11T23:02:43.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115507+0000", 2024-05-11T23:02:43.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033929925999999999 2024-05-11T23:02:43.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115599+0000", 2024-05-11T23:02:43.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1571999999999997e-05 2024-05-11T23:02:43.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:43.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115685+0000", 2024-05-11T23:02:43.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6071999999999999e-05 2024-05-11T23:02:43.442 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.442 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.442 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.443 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28493 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 999424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:43.443 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:43.443 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.292632124999997, 2024-05-11T23:02:43.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034232944000000001, 2024-05-11T23:02:43.443 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.443 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.443 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:43.443 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:43.443 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:43.443 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:43.443 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28493 2024-05-11T23:02:43.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.443 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:43.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:43.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:43.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1300000000000001e-07 2024-05-11T23:02:43.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081584+0000", 2024-05-11T23:02:43.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9479999999999998e-06 2024-05-11T23:02:43.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081590+0000", 2024-05-11T23:02:43.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.728e-06 2024-05-11T23:02:43.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081595+0000", 2024-05-11T23:02:43.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8579999999999999e-06 2024-05-11T23:02:43.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115723+0000", 2024-05-11T23:02:43.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034128473999999999 2024-05-11T23:02:43.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115768+0000", 2024-05-11T23:02:43.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5108999999999998e-05 2024-05-11T23:02:43.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:43.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115815+0000", 2024-05-11T23:02:43.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6514000000000002e-05 2024-05-11T23:02:43.447 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.447 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.447 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.447 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28496 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:43.447 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:02:43.447 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.292612665999997, 2024-05-11T23:02:43.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034427754999999997, 2024-05-11T23:02:43.447 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:43.447 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:43.447 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:43.447 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:43.448 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28496 2024-05-11T23:02:43.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.448 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:02:43.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:02:43.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081602+0000", 2024-05-11T23:02:43.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:02:43.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.449 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081604+0000", 2024-05-11T23:02:43.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3999999999999999e-06 2024-05-11T23:02:43.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.449 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081610+0000", 2024-05-11T23:02:43.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.224e-06 2024-05-11T23:02:43.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.449 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081615+0000", 2024-05-11T23:02:43.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8269999999999997e-06 2024-05-11T23:02:43.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115859+0000", 2024-05-11T23:02:43.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034243498999999997 2024-05-11T23:02:43.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115946+0000", 2024-05-11T23:02:43.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7122999999999999e-05 2024-05-11T23:02:43.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:43.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116029+0000", 2024-05-11T23:02:43.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3332999999999996e-05 2024-05-11T23:02:43.451 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.451 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.451 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.451 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28498 3.0 3:05d5189e:::rbd_object_map.111e5e4d8275.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:43.451 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:02:43.451 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.291094014000002, 2024-05-11T23:02:43.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.035035108000000002, 2024-05-11T23:02:43.451 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.451 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.451 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:43.451 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:43.451 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:43.452 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:43.452 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28498 2024-05-11T23:02:43.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.452 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.452 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.452 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:02:43.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.452 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.452 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:02:43.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083121+0000", 2024-05-11T23:02:43.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5700000000000001e-07 2024-05-11T23:02:43.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083126+0000", 2024-05-11T23:02:43.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8860000000000003e-06 2024-05-11T23:02:43.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083135+0000", 2024-05-11T23:02:43.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8369999999999992e-06 2024-05-11T23:02:43.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.454 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.454 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083142+0000", 2024-05-11T23:02:43.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4779999999999996e-06 2024-05-11T23:02:43.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.454 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.454 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116072+0000", 2024-05-11T23:02:43.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032929991999999998 2024-05-11T23:02:43.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.454 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116158+0000", 2024-05-11T23:02:43.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6077999999999998e-05 2024-05-11T23:02:43.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.455 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-05-11T23:02:43.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116513+0000", 2024-05-11T23:02:43.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00035508300000000002 2024-05-11T23:02:43.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.455 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:43.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118025+0000", 2024-05-11T23:02:43.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015118919999999999 2024-05-11T23:02:43.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.455 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:02:43.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118039+0000", 2024-05-11T23:02:43.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4188999999999999e-05 2024-05-11T23:02:43.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:43.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118094+0000", 2024-05-11T23:02:43.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4929000000000002e-05 2024-05-11T23:02:43.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:43.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118155+0000", 2024-05-11T23:02:43.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0887000000000002e-05 2024-05-11T23:02:43.456 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.456 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.456 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.457 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:64049 3.8 3:11d98727:::rbd_object_map.111e8974a681:head [read 0~38] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-05-11T23:02:43.457 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:02:43.457 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.594332704000003, 2024-05-11T23:02:43.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99884604399999999, 2024-05-11T23:02:43.457 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.457 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.457 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:43.457 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:43.457 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:43.457 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:43.457 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 64049 2024-05-11T23:02:43.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.458 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:02:43.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:02:43.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779883+0000", 2024-05-11T23:02:43.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.57e-06 2024-05-11T23:02:43.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779887+0000", 2024-05-11T23:02:43.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5709999999999999e-06 2024-05-11T23:02:43.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779898+0000", 2024-05-11T23:02:43.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.042e-05 2024-05-11T23:02:43.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779905+0000", 2024-05-11T23:02:43.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0110000000000003e-06 2024-05-11T23:02:43.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779962+0000", 2024-05-11T23:02:43.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7395999999999997e-05 2024-05-11T23:02:43.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:02:43.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779969+0000", 2024-05-11T23:02:43.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2459999999999997e-06 2024-05-11T23:02:43.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778379+0000", 2024-05-11T23:02:43.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99840986300000001 2024-05-11T23:02:43.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778564+0000", 2024-05-11T23:02:43.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018420900000000001 2024-05-11T23:02:43.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:43.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778727+0000", 2024-05-11T23:02:43.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016375800000000001 2024-05-11T23:02:43.461 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.461 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.461 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.461 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.462 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.462 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:43.463 INFO:teuthology.orchestra.run.smithi005.stdout:3915 read 0xbee2c2a thru 0xbee5aa1 (0x2e78 bytes) 2024-05-11T23:02:43.463 INFO:teuthology.orchestra.run.smithi005.stdout:3917 read 0x35ef849 thru 0x35fcdfd (0xd5b5 bytes) 2024-05-11T23:02:43.463 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:43.463 INFO:teuthology.orchestra.run.smithi005.stdout:3919 read 0x8018f8 thru 0x8065ae (0x4cb7 bytes) 2024-05-11T23:02:43.464 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:02:43.465 INFO:teuthology.orchestra.run.smithi005.stdout:3920 trunc from 0xe11eeac to 0x5698272 2024-05-11T23:02:43.465 INFO:teuthology.orchestra.run.smithi005.stdout:3921 read 0x3f3b10b thru 0x3f4a75f (0xf655 bytes) 2024-05-11T23:02:43.465 INFO:teuthology.orchestra.run.smithi005.stdout:3923 read 0x1c70787 thru 0x1c7e63d (0xdeb7 bytes) 2024-05-11T23:02:43.465 INFO:teuthology.orchestra.run.smithi005.stdout:3924 read 0x28069de thru 0x28148b5 (0xded8 bytes) 2024-05-11T23:02:43.465 INFO:teuthology.orchestra.run.smithi005.stdout:3926 write 0x4208791 thru 0x4213216 (0xaa86 bytes) 2024-05-11T23:02:43.466 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:02:43.466 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:43.466 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 88, 2024-05-11T23:02:43.466 INFO:teuthology.orchestra.run.smithi005.stdout: "fsid": "34b20022-f11b-4169-a5f6-6386270a2f8f", 2024-05-11T23:02:43.466 INFO:teuthology.orchestra.run.smithi005.stdout: "created": "2024-05-11T23:01:05.977693+0000", 2024-05-11T23:02:43.466 INFO:teuthology.orchestra.run.smithi005.stdout: "modified": "2024-05-11T23:02:43.054261+0000", 2024-05-11T23:02:43.466 INFO:teuthology.orchestra.run.smithi005.stdout: "last_up_change": "2024-05-11T23:01:17.148916+0000", 2024-05-11T23:02:43.466 INFO:teuthology.orchestra.run.smithi005.stdout: "last_in_change": "2024-05-11T23:02:38.028253+0000", 2024-05-11T23:02:43.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-11T23:02:43.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_num": 5806080, 2024-05-11T23:02:43.467 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_set": [ 2024-05-11T23:02:43.467 INFO:teuthology.orchestra.run.smithi005.stdout: "nodeep-scrub", 2024-05-11T23:02:43.467 INFO:teuthology.orchestra.run.smithi005.stdout: "noscrub", 2024-05-11T23:02:43.467 INFO:teuthology.orchestra.run.smithi005.stdout: "pglog_hardlimit", 2024-05-11T23:02:43.467 INFO:teuthology.orchestra.run.smithi005.stdout: "purged_snapdirs", 2024-05-11T23:02:43.467 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_deletes", 2024-05-11T23:02:43.467 INFO:teuthology.orchestra.run.smithi005.stdout: "sortbitwise" 2024-05-11T23:02:43.467 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:02:43.467 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_version": 5, 2024-05-11T23:02:43.467 INFO:teuthology.orchestra.run.smithi005.stdout: "full_ratio": 0.94999998807907104, 2024-05-11T23:02:43.467 INFO:teuthology.orchestra.run.smithi005.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-11T23:02:43.467 INFO:teuthology.orchestra.run.smithi005.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-11T23:02:43.467 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_snapshot": "", 2024-05-11T23:02:43.467 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_max": 3, 2024-05-11T23:02:43.467 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd": 8, 2024-05-11T23:02:43.468 INFO:teuthology.orchestra.run.smithi005.stdout: "require_min_compat_client": "luminous", 2024-05-11T23:02:43.468 INFO:teuthology.orchestra.run.smithi005.stdout: "min_compat_client": "luminous", 2024-05-11T23:02:43.468 INFO:teuthology.orchestra.run.smithi005.stdout: "require_osd_release": "squid", 2024-05-11T23:02:43.468 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_crimson": false, 2024-05-11T23:02:43.468 INFO:teuthology.orchestra.run.smithi005.stdout: "pools": [ 2024-05-11T23:02:43.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.468 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 1, 2024-05-11T23:02:43.468 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": ".mgr", 2024-05-11T23:02:43.468 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-05-11T23:01:15.187614+0000", 2024-05-11T23:02:43.468 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-11T23:02:43.468 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool", 2024-05-11T23:02:43.468 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-05-11T23:02:43.468 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-05-11T23:02:43.468 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-05-11T23:02:43.469 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-05-11T23:02:43.469 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-05-11T23:02:43.469 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-05-11T23:02:43.469 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-05-11T23:02:43.469 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-11T23:02:43.469 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-05-11T23:02:43.469 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-05-11T23:02:43.469 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 1, 2024-05-11T23:02:43.469 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 1, 2024-05-11T23:02:43.469 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 1, 2024-05-11T23:02:43.469 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 1, 2024-05-11T23:02:43.469 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 1, 2024-05-11T23:02:43.469 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-05-11T23:02:43.469 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-05-11T23:02:43.469 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-05-11T23:02:43.470 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-05-11T23:02:43.470 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-05-11T23:02:43.470 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-05-11T23:02:43.470 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-05-11T23:02:43.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.470 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "17", 2024-05-11T23:02:43.470 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-05-11T23:02:43.470 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-11T23:02:43.470 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-05-11T23:02:43.470 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-05-11T23:02:43.470 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-05-11T23:02:43.470 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 0, 2024-05-11T23:02:43.470 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 0, 2024-05-11T23:02:43.470 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-05-11T23:02:43.471 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-05-11T23:02:43.471 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-05-11T23:02:43.471 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-05-11T23:02:43.471 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-05-11T23:02:43.471 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-05-11T23:02:43.471 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-05-11T23:02:43.471 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-05-11T23:02:43.471 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-05-11T23:02:43.471 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-05-11T23:02:43.471 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-05-11T23:02:43.471 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-11T23:02:43.471 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-11T23:02:43.471 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-11T23:02:43.471 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-05-11T23:02:43.472 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-05-11T23:02:43.472 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-05-11T23:02:43.472 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-05-11T23:02:43.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-05-11T23:02:43.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.472 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-05-11T23:02:43.472 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-05-11T23:02:43.472 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-05-11T23:02:43.472 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-05-11T23:02:43.472 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-05-11T23:02:43.472 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-05-11T23:02:43.472 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-05-11T23:02:43.472 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-05-11T23:02:43.472 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-05-11T23:02:43.472 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-05-11T23:02:43.473 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-05-11T23:02:43.473 INFO:teuthology.orchestra.run.smithi005.stdout: "options": { 2024-05-11T23:02:43.473 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_max": 32, 2024-05-11T23:02:43.473 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_min": 1 2024-05-11T23:02:43.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.473 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-05-11T23:02:43.473 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": {} 2024-05-11T23:02:43.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.473 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-05-11T23:02:43.473 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-05-11T23:02:43.473 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 8, 2024-05-11T23:02:43.473 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 8, 2024-05-11T23:02:43.473 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 0.25, 2024-05-11T23:02:43.473 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2, 2024-05-11T23:02:43.473 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2, 2024-05-11T23:02:43.474 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 1, 2024-05-11T23:02:43.474 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 1, 2024-05-11T23:02:43.474 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 1 2024-05-11T23:02:43.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.474 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 2, 2024-05-11T23:02:43.474 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "rbd", 2024-05-11T23:02:43.474 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-05-11T23:01:17.689285+0000", 2024-05-11T23:02:43.474 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-11T23:02:43.474 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool", 2024-05-11T23:02:43.474 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-05-11T23:02:43.474 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-05-11T23:02:43.474 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-05-11T23:02:43.475 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-05-11T23:02:43.475 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-05-11T23:02:43.475 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-05-11T23:02:43.475 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-05-11T23:02:43.475 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-11T23:02:43.475 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-05-11T23:02:43.475 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-05-11T23:02:43.475 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 8, 2024-05-11T23:02:43.475 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 8, 2024-05-11T23:02:43.475 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 8, 2024-05-11T23:02:43.475 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 8, 2024-05-11T23:02:43.475 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 8, 2024-05-11T23:02:43.475 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-05-11T23:02:43.475 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-05-11T23:02:43.476 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-05-11T23:02:43.476 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-05-11T23:02:43.476 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-05-11T23:02:43.476 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-05-11T23:02:43.476 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-05-11T23:02:43.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.476 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "18", 2024-05-11T23:02:43.476 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-05-11T23:02:43.476 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-11T23:02:43.476 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-05-11T23:02:43.476 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-05-11T23:02:43.476 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-05-11T23:02:43.476 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 0, 2024-05-11T23:02:43.476 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 0, 2024-05-11T23:02:43.476 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-05-11T23:02:43.477 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-05-11T23:02:43.477 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-05-11T23:02:43.477 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-05-11T23:02:43.477 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-05-11T23:02:43.477 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-05-11T23:02:43.477 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-05-11T23:02:43.477 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-05-11T23:02:43.477 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-05-11T23:02:43.477 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-05-11T23:02:43.477 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-05-11T23:02:43.477 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-11T23:02:43.477 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-11T23:02:43.477 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-11T23:02:43.477 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-05-11T23:02:43.477 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-05-11T23:02:43.478 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-05-11T23:02:43.478 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-05-11T23:02:43.478 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-05-11T23:02:43.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.478 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-05-11T23:02:43.478 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-05-11T23:02:43.478 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-05-11T23:02:43.478 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-05-11T23:02:43.478 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-05-11T23:02:43.478 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-05-11T23:02:43.478 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-05-11T23:02:43.478 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-05-11T23:02:43.478 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-05-11T23:02:43.478 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-05-11T23:02:43.478 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-05-11T23:02:43.479 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-05-11T23:02:43.479 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-05-11T23:02:43.479 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-05-11T23:02:43.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.479 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-05-11T23:02:43.479 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-05-11T23:02:43.479 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 1.9900000095367432, 2024-05-11T23:02:43.479 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 1.9900000095367432, 2024-05-11T23:02:43.479 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 0.87999999523162842, 2024-05-11T23:02:43.479 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 1.75, 2024-05-11T23:02:43.479 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 1.75, 2024-05-11T23:02:43.479 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 1, 2024-05-11T23:02:43.479 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 1, 2024-05-11T23:02:43.479 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 1 2024-05-11T23:02:43.479 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 3, 2024-05-11T23:02:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "pool_client.0", 2024-05-11T23:02:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-05-11T23:01:25.743347+0000", 2024-05-11T23:02:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-05-11T23:02:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-11T23:02:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-05-11T23:02:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-05-11T23:02:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-05-11T23:02:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-05-11T23:02:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-05-11T23:02:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-05-11T23:02:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-05-11T23:02:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-11T23:02:43.480 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-05-11T23:02:43.481 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-05-11T23:02:43.481 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 32, 2024-05-11T23:02:43.481 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 32, 2024-05-11T23:02:43.481 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 32, 2024-05-11T23:02:43.481 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 32, 2024-05-11T23:02:43.481 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 32, 2024-05-11T23:02:43.481 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-05-11T23:02:43.481 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-05-11T23:02:43.481 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-05-11T23:02:43.481 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-05-11T23:02:43.481 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-05-11T23:02:43.481 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-05-11T23:02:43.481 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-05-11T23:02:43.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.481 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "82", 2024-05-11T23:02:43.482 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-05-11T23:02:43.482 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-11T23:02:43.482 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-05-11T23:02:43.482 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-05-11T23:02:43.482 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-05-11T23:02:43.482 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 41, 2024-05-11T23:02:43.482 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 82, 2024-05-11T23:02:43.482 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-05-11T23:02:43.482 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-05-11T23:02:43.482 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-05-11T23:02:43.482 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-05-11T23:02:43.482 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-05-11T23:02:43.482 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-05-11T23:02:43.482 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-05-11T23:02:43.483 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-05-11T23:02:43.483 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-05-11T23:02:43.483 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-05-11T23:02:43.483 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-05-11T23:02:43.483 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-11T23:02:43.483 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-11T23:02:43.483 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-11T23:02:43.483 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-05-11T23:02:43.483 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-05-11T23:02:43.483 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-05-11T23:02:43.483 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-05-11T23:02:43.483 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-05-11T23:02:43.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.483 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-05-11T23:02:43.483 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-05-11T23:02:43.483 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-05-11T23:02:43.484 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-05-11T23:02:43.484 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-05-11T23:02:43.484 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-05-11T23:02:43.484 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-05-11T23:02:43.484 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-05-11T23:02:43.484 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-05-11T23:02:43.484 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-05-11T23:02:43.484 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-05-11T23:02:43.484 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-05-11T23:02:43.484 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-05-11T23:02:43.484 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-05-11T23:02:43.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.484 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-05-11T23:02:43.484 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-05-11T23:02:43.484 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 1.5, 2024-05-11T23:02:43.485 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 1.5, 2024-05-11T23:02:43.485 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 1, 2024-05-11T23:02:43.485 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 1.5, 2024-05-11T23:02:43.485 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 1.5, 2024-05-11T23:02:43.485 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 1, 2024-05-11T23:02:43.485 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 1, 2024-05-11T23:02:43.485 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 1 2024-05-11T23:02:43.485 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.485 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.485 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:02:43.485 INFO:teuthology.orchestra.run.smithi005.stdout: "osds": [ 2024-05-11T23:02:43.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.485 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-05-11T23:02:43.485 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "57ab1c70-9478-4a26-bb41-d753b4f3eb79", 2024-05-11T23:02:43.485 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:02:43.486 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:02:43.486 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 0.95001220703125, 2024-05-11T23:02:43.486 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:02:43.486 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:02:43.486 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:02:43.486 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 13, 2024-05-11T23:02:43.486 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 85, 2024-05-11T23:02:43.486 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:02:43.486 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:02:43.486 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:02:43.486 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.486 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.486 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-05-11T23:02:43.487 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:02:43.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.487 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.487 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-05-11T23:02:43.487 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:02:43.487 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.487 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.487 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:02:43.487 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.487 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.487 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6802", 2024-05-11T23:02:43.487 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:02:43.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.488 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.488 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6803", 2024-05-11T23:02:43.488 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:02:43.488 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.488 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.488 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:02:43.488 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.488 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.488 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6806", 2024-05-11T23:02:43.488 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:02:43.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.489 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.489 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6808", 2024-05-11T23:02:43.489 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:02:43.489 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.489 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.489 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:02:43.489 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.489 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.489 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6804", 2024-05-11T23:02:43.489 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:02:43.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.490 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.490 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6805", 2024-05-11T23:02:43.490 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:02:43.490 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.490 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.490 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6801/3452725643", 2024-05-11T23:02:43.490 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6803/3452725643", 2024-05-11T23:02:43.490 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6808/3452725643", 2024-05-11T23:02:43.490 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6805/3452725643", 2024-05-11T23:02:43.490 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:02:43.490 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:02:43.490 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:02:43.490 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.491 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-05-11T23:02:43.491 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "05ece110-4a99-46f0-b061-5980f7ef2a55", 2024-05-11T23:02:43.491 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:02:43.491 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:02:43.491 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 0.95001220703125, 2024-05-11T23:02:43.491 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:02:43.491 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:02:43.491 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:02:43.491 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 14, 2024-05-11T23:02:43.491 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 86, 2024-05-11T23:02:43.491 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:02:43.491 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:02:43.492 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:02:43.492 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.492 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6824", 2024-05-11T23:02:43.492 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:02:43.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.492 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6825", 2024-05-11T23:02:43.492 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:02:43.492 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.492 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.492 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:02:43.493 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.493 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.493 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6826", 2024-05-11T23:02:43.493 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:02:43.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.493 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.493 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6827", 2024-05-11T23:02:43.493 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:02:43.493 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.493 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.493 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:02:43.493 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.494 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.494 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6830", 2024-05-11T23:02:43.494 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:02:43.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.494 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.494 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6831", 2024-05-11T23:02:43.494 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:02:43.494 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.494 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.494 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:02:43.494 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.495 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.495 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6828", 2024-05-11T23:02:43.495 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:02:43.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.495 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.495 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6829", 2024-05-11T23:02:43.495 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:02:43.495 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.495 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.495 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6825/4028727138", 2024-05-11T23:02:43.495 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6827/4028727138", 2024-05-11T23:02:43.495 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6831/4028727138", 2024-05-11T23:02:43.496 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6829/4028727138", 2024-05-11T23:02:43.496 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:02:43.496 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:02:43.496 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:02:43.496 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.496 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-05-11T23:02:43.496 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5", 2024-05-11T23:02:43.496 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:02:43.496 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:02:43.496 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:02:43.496 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:02:43.496 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:02:43.496 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:02:43.497 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 13, 2024-05-11T23:02:43.497 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 73, 2024-05-11T23:02:43.497 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:02:43.497 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:02:43.497 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:02:43.497 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.497 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.497 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6807", 2024-05-11T23:02:43.497 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:02:43.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.497 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.497 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6809", 2024-05-11T23:02:43.497 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:02:43.497 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.498 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.498 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:02:43.498 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.498 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.498 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6810", 2024-05-11T23:02:43.498 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:02:43.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.498 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.498 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6811", 2024-05-11T23:02:43.498 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:02:43.498 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.499 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.499 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:02:43.499 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.499 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.499 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6814", 2024-05-11T23:02:43.499 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:02:43.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.499 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.499 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6815", 2024-05-11T23:02:43.499 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:02:43.499 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.499 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.500 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:02:43.500 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.500 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.500 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6812", 2024-05-11T23:02:43.500 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:02:43.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.500 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.500 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6813", 2024-05-11T23:02:43.500 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:02:43.500 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.500 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.501 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6809/2059959509", 2024-05-11T23:02:43.501 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6811/2059959509", 2024-05-11T23:02:43.501 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6815/2059959509", 2024-05-11T23:02:43.501 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6813/2059959509", 2024-05-11T23:02:43.501 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:02:43.501 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:02:43.501 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:02:43.501 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.501 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-05-11T23:02:43.501 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a", 2024-05-11T23:02:43.501 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:02:43.501 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:02:43.501 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 0.9000244140625, 2024-05-11T23:02:43.502 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:02:43.502 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:02:43.502 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:02:43.502 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 14, 2024-05-11T23:02:43.502 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 87, 2024-05-11T23:02:43.502 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:02:43.502 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:02:43.502 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:02:43.502 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.502 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.502 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6816", 2024-05-11T23:02:43.502 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:02:43.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.503 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.503 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6817", 2024-05-11T23:02:43.503 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:02:43.503 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.503 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.503 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:02:43.503 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.503 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.503 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6818", 2024-05-11T23:02:43.503 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:02:43.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.504 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6819", 2024-05-11T23:02:43.504 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:02:43.504 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.504 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.504 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:02:43.504 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.504 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6822", 2024-05-11T23:02:43.504 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:02:43.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.504 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6823", 2024-05-11T23:02:43.505 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:02:43.505 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.505 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.505 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:02:43.505 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.505 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.505 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6820", 2024-05-11T23:02:43.505 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:02:43.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.505 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.505 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6821", 2024-05-11T23:02:43.505 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:02:43.506 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.506 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.506 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6817/296930146", 2024-05-11T23:02:43.506 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6819/296930146", 2024-05-11T23:02:43.506 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6823/296930146", 2024-05-11T23:02:43.506 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6821/296930146", 2024-05-11T23:02:43.506 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:02:43.506 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:02:43.506 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:02:43.506 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-05-11T23:02:43.507 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "4e87e4f0-288e-4ee6-919e-9337925a5d5e", 2024-05-11T23:02:43.507 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:02:43.507 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:02:43.507 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:02:43.507 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:02:43.507 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:02:43.507 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:02:43.507 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 15, 2024-05-11T23:02:43.507 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 75, 2024-05-11T23:02:43.507 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:02:43.507 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:02:43.507 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:02:43.507 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.508 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6800", 2024-05-11T23:02:43.508 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:02:43.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.508 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6801", 2024-05-11T23:02:43.508 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:02:43.508 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.508 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.508 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:02:43.508 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.509 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6802", 2024-05-11T23:02:43.509 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:02:43.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.509 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.509 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6803", 2024-05-11T23:02:43.509 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:02:43.509 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.509 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.509 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:02:43.509 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.509 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.509 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6806", 2024-05-11T23:02:43.509 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:02:43.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.510 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.510 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6807", 2024-05-11T23:02:43.510 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:02:43.510 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.510 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.510 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:02:43.510 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.510 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.510 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6804", 2024-05-11T23:02:43.510 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:02:43.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.511 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.511 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6805", 2024-05-11T23:02:43.511 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:02:43.511 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.511 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.511 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.115:6801/495703077", 2024-05-11T23:02:43.511 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.115:6803/495703077", 2024-05-11T23:02:43.511 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.115:6807/495703077", 2024-05-11T23:02:43.511 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.115:6805/495703077", 2024-05-11T23:02:43.511 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:02:43.511 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:02:43.511 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:02:43.512 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.512 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-05-11T23:02:43.512 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5", 2024-05-11T23:02:43.512 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:02:43.512 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:02:43.512 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:02:43.512 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:02:43.512 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:02:43.512 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:02:43.512 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 15, 2024-05-11T23:02:43.512 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 77, 2024-05-11T23:02:43.512 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:02:43.512 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:02:43.513 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:02:43.513 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.513 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.513 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6811", 2024-05-11T23:02:43.513 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:02:43.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.513 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.513 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6813", 2024-05-11T23:02:43.513 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:02:43.513 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.513 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.513 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:02:43.514 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.514 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.514 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6815", 2024-05-11T23:02:43.514 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:02:43.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.514 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.514 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6817", 2024-05-11T23:02:43.514 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:02:43.514 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.514 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.514 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:02:43.515 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.515 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.515 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6822", 2024-05-11T23:02:43.515 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:02:43.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.515 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.515 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6823", 2024-05-11T23:02:43.515 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:02:43.515 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.515 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.515 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:02:43.515 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.516 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.516 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6819", 2024-05-11T23:02:43.516 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:02:43.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.516 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.516 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6821", 2024-05-11T23:02:43.516 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:02:43.516 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.516 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.516 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.115:6813/902573489", 2024-05-11T23:02:43.516 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.115:6817/902573489", 2024-05-11T23:02:43.517 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.115:6823/902573489", 2024-05-11T23:02:43.517 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.115:6821/902573489", 2024-05-11T23:02:43.517 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:02:43.517 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:02:43.517 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:02:43.517 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.517 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 6, 2024-05-11T23:02:43.517 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "edb686ec-e1a2-40d6-975a-e32e0e7760ff", 2024-05-11T23:02:43.517 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:02:43.517 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:02:43.517 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:02:43.517 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:02:43.517 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:02:43.518 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:02:43.518 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 15, 2024-05-11T23:02:43.518 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 87, 2024-05-11T23:02:43.518 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:02:43.518 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:02:43.518 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:02:43.518 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.518 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.518 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6824", 2024-05-11T23:02:43.518 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:02:43.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.518 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.519 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6825", 2024-05-11T23:02:43.519 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:02:43.519 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.519 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.519 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:02:43.519 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.519 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6826", 2024-05-11T23:02:43.519 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:02:43.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.520 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6827", 2024-05-11T23:02:43.520 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:02:43.520 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.520 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.520 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:02:43.520 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.520 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.520 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6830", 2024-05-11T23:02:43.520 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:02:43.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.520 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.520 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6831", 2024-05-11T23:02:43.520 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:02:43.521 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.521 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.521 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:02:43.521 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.521 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6828", 2024-05-11T23:02:43.521 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:02:43.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.521 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6829", 2024-05-11T23:02:43.521 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:02:43.521 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.115:6825/2749332272", 2024-05-11T23:02:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.115:6827/2749332272", 2024-05-11T23:02:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.115:6831/2749332272", 2024-05-11T23:02:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.115:6829/2749332272", 2024-05-11T23:02:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:02:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:02:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:02:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 7, 2024-05-11T23:02:43.522 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "7aa14dbb-c6cb-4540-b160-3fd346f6a046", 2024-05-11T23:02:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:02:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:02:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:02:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:02:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:02:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:02:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 15, 2024-05-11T23:02:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 76, 2024-05-11T23:02:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:02:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:02:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:02:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.523 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6808", 2024-05-11T23:02:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:02:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6809", 2024-05-11T23:02:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:02:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:02:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6810", 2024-05-11T23:02:43.524 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:02:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6812", 2024-05-11T23:02:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:02:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:02:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6818", 2024-05-11T23:02:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:02:43.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6820", 2024-05-11T23:02:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:02:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:02:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:02:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:02:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6814", 2024-05-11T23:02:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:02:43.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:02:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6816", 2024-05-11T23:02:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:02:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.115:6809/1043826337", 2024-05-11T23:02:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.115:6812/1043826337", 2024-05-11T23:02:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.115:6820/1043826337", 2024-05-11T23:02:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.115:6816/1043826337", 2024-05-11T23:02:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:02:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:02:43.527 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:02:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:02:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_xinfo": [ 2024-05-11T23:02:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-05-11T23:02:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:02:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:02:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:02:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:02:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:02:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.113232+0000", 2024-05-11T23:02:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:02:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-05-11T23:02:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:02:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:02:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:02:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:02:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:02:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.174679+0000", 2024-05-11T23:02:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:02:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-05-11T23:02:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:02:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:02:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:02:43.529 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:02:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:02:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.131118+0000", 2024-05-11T23:02:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:02:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-05-11T23:02:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:02:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:02:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:02:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:02:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:02:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.207668+0000", 2024-05-11T23:02:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:02:43.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-05-11T23:02:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:02:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:02:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:02:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:02:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:02:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.124542+0000", 2024-05-11T23:02:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:02:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-05-11T23:02:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:02:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:02:43.531 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:02:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:02:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:02:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.200832+0000", 2024-05-11T23:02:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:02:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 6, 2024-05-11T23:02:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:02:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:02:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:02:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:02:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:02:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.169560+0000", 2024-05-11T23:02:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:02:43.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 7, 2024-05-11T23:02:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:02:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:02:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:02:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:02:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:02:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.182465+0000", 2024-05-11T23:02:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:02:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:02:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap": [], 2024-05-11T23:02:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_items": [ 2024-05-11T23:02:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.533 INFO:teuthology.orchestra.run.smithi005.stdout: "pgid": "3.3", 2024-05-11T23:02:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: "mappings": [ 2024-05-11T23:02:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: "from": 4, 2024-05-11T23:02:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: "to": 2 2024-05-11T23:02:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:02:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_primaries": [], 2024-05-11T23:02:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_temp": [], 2024-05-11T23:02:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_temp": [], 2024-05-11T23:02:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklist": {}, 2024-05-11T23:02:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: "range_blocklist": {}, 2024-05-11T23:02:43.534 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profiles": { 2024-05-11T23:02:43.535 INFO:teuthology.orchestra.run.smithi005.stdout: "default": { 2024-05-11T23:02:43.535 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-failure-domain": "osd", 2024-05-11T23:02:43.535 INFO:teuthology.orchestra.run.smithi005.stdout: "k": "2", 2024-05-11T23:02:43.535 INFO:teuthology.orchestra.run.smithi005.stdout: "m": "1", 2024-05-11T23:02:43.535 INFO:teuthology.orchestra.run.smithi005.stdout: "plugin": "jerasure", 2024-05-11T23:02:43.535 INFO:teuthology.orchestra.run.smithi005.stdout: "technique": "reed_sol_van" 2024-05-11T23:02:43.535 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.535 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps_queue": [], 2024-05-11T23:02:43.535 INFO:teuthology.orchestra.run.smithi005.stdout: "new_removed_snaps": [], 2024-05-11T23:02:43.535 INFO:teuthology.orchestra.run.smithi005.stdout: "new_purged_snaps": [], 2024-05-11T23:02:43.535 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_node_flags": {}, 2024-05-11T23:02:43.535 INFO:teuthology.orchestra.run.smithi005.stdout: "device_class_flags": {}, 2024-05-11T23:02:43.535 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode": { 2024-05-11T23:02:43.535 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_enabled": false, 2024-05-11T23:02:43.535 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_bucket_count": 0, 2024-05-11T23:02:43.536 INFO:teuthology.orchestra.run.smithi005.stdout: "degraded_stretch_mode": 0, 2024-05-11T23:02:43.536 INFO:teuthology.orchestra.run.smithi005.stdout: "recovering_stretch_mode": 0, 2024-05-11T23:02:43.536 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_bucket": 0 2024-05-11T23:02:43.536 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.536 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:43.537 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:43.462+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:43.537 INFO:teuthology.orchestra.run.smithi005.stdout:3927 trunc from 0x5698272 to 0xec5e62e 2024-05-11T23:02:43.537 INFO:teuthology.orchestra.run.smithi005.stdout:3929 read 0xe5cca61 thru 0xe5d6bb5 (0xa155 bytes) 2024-05-11T23:02:43.537 INFO:teuthology.orchestra.run.smithi005.stdout:3930 writesame 0x6c62403 thru 0x6c6931f data_size 0x1(0x6f1d bytes) 2024-05-11T23:02:43.539 INFO:tasks.thrashosds.thrasher:j is {'epoch': 88, 'fsid': '34b20022-f11b-4169-a5f6-6386270a2f8f', 'created': '2024-05-11T23:01:05.977693+0000', 'modified': '2024-05-11T23:02:43.054261+0000', 'last_up_change': '2024-05-11T23:01:17.148916+0000', 'last_in_change': '2024-05-11T23:02:38.028253+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-11T23:01:15.187614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-05-11T23:01:17.689285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-11T23:01:25.743347+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 41, 'snap_epoch': 82, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '57ab1c70-9478-4a26-bb41-d753b4f3eb79', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 85, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6800', 'nonce': 3452725643}, {'type': 'v1', 'addr': '172.21.15.5:6801', 'nonce': 3452725643}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6802', 'nonce': 3452725643}, {'type': 'v1', 'addr': '172.21.15.5:6803', 'nonce': 3452725643}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6806', 'nonce': 3452725643}, {'type': 'v1', 'addr': '172.21.15.5:6808', 'nonce': 3452725643}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6804', 'nonce': 3452725643}, {'type': 'v1', 'addr': '172.21.15.5:6805', 'nonce': 3452725643}]}, 'public_addr': '172.21.15.5:6801/3452725643', 'cluster_addr': '172.21.15.5:6803/3452725643', 'heartbeat_back_addr': '172.21.15.5:6808/3452725643', 'heartbeat_front_addr': '172.21.15.5:6805/3452725643', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '05ece110-4a99-46f0-b061-5980f7ef2a55', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 86, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6824', 'nonce': 4028727138}, {'type': 'v1', 'addr': '172.21.15.5:6825', 'nonce': 4028727138}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6826', 'nonce': 4028727138}, {'type': 'v1', 'addr': '172.21.15.5:6827', 'nonce': 4028727138}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6830', 'nonce': 4028727138}, {'type': 'v1', 'addr': '172.21.15.5:6831', 'nonce': 4028727138}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6828', 'nonce': 4028727138}, {'type': 'v1', 'addr': '172.21.15.5:6829', 'nonce': 4028727138}]}, 'public_addr': '172.21.15.5:6825/4028727138', 'cluster_addr': '172.21.15.5:6827/4028727138', 'heartbeat_back_addr': '172.21.15.5:6831/4028727138', 'heartbeat_front_addr': '172.21.15.5:6829/4028727138', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 73, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6807', 'nonce': 2059959509}, {'type': 'v1', 'addr': '172.21.15.5:6809', 'nonce': 2059959509}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6810', 'nonce': 2059959509}, {'type': 'v1', 'addr': '172.21.15.5:6811', 'nonce': 2059959509}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6814', 'nonce': 2059959509}, {'type': 'v1', 'addr': '172.21.15.5:6815', 'nonce': 2059959509}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6812', 'nonce': 2059959509}, {'type': 'v1', 'addr': '172.21.15.5:6813', 'nonce': 2059959509}]}, 'public_addr': '172.21.15.5:6809/2059959509', 'cluster_addr': '172.21.15.5:6811/2059959509', 'heartbeat_back_addr': '172.21.15.5:6815/2059959509', 'heartbeat_front_addr': '172.21.15.5:6813/2059959509', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a', 'up': 1, 'in': 1, 'weight': 0.9000244140625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6816', 'nonce': 296930146}, {'type': 'v1', 'addr': '172.21.15.5:6817', 'nonce': 296930146}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6818', 'nonce': 296930146}, {'type': 'v1', 'addr': '172.21.15.5:6819', 'nonce': 296930146}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6822', 'nonce': 296930146}, {'type': 'v1', 'addr': '172.21.15.5:6823', 'nonce': 296930146}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6820', 'nonce': 296930146}, {'type': 'v1', 'addr': '172.21.15.5:6821', 'nonce': 296930146}]}, 'public_addr': '172.21.15.5:6817/296930146', 'cluster_addr': '172.21.15.5:6819/296930146', 'heartbeat_back_addr': '172.21.15.5:6823/296930146', 'heartbeat_front_addr': '172.21.15.5:6821/296930146', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '4e87e4f0-288e-4ee6-919e-9337925a5d5e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 495703077}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 495703077}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 495703077}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 495703077}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 495703077}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 495703077}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 495703077}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 495703077}]}, 'public_addr': '172.21.15.115:6801/495703077', 'cluster_addr': '172.21.15.115:6803/495703077', 'heartbeat_back_addr': '172.21.15.115:6807/495703077', 'heartbeat_front_addr': '172.21.15.115:6805/495703077', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6811', 'nonce': 902573489}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 902573489}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6815', 'nonce': 902573489}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 902573489}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 902573489}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 902573489}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6819', 'nonce': 902573489}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 902573489}]}, 'public_addr': '172.21.15.115:6813/902573489', 'cluster_addr': '172.21.15.115:6817/902573489', 'heartbeat_back_addr': '172.21.15.115:6823/902573489', 'heartbeat_front_addr': '172.21.15.115:6821/902573489', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'edb686ec-e1a2-40d6-975a-e32e0e7760ff', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6824', 'nonce': 2749332272}, {'type': 'v1', 'addr': '172.21.15.115:6825', 'nonce': 2749332272}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 2749332272}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 2749332272}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 2749332272}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 2749332272}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 2749332272}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 2749332272}]}, 'public_addr': '172.21.15.115:6825/2749332272', 'cluster_addr': '172.21.15.115:6827/2749332272', 'heartbeat_back_addr': '172.21.15.115:6831/2749332272', 'heartbeat_front_addr': '172.21.15.115:6829/2749332272', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '7aa14dbb-c6cb-4540-b160-3fd346f6a046', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 1043826337}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 1043826337}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 1043826337}, {'type': 'v1', 'addr': '172.21.15.115:6812', 'nonce': 1043826337}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 1043826337}, {'type': 'v1', 'addr': '172.21.15.115:6820', 'nonce': 1043826337}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 1043826337}, {'type': 'v1', 'addr': '172.21.15.115:6816', 'nonce': 1043826337}]}, 'public_addr': '172.21.15.115:6809/1043826337', 'cluster_addr': '172.21.15.115:6812/1043826337', 'heartbeat_back_addr': '172.21.15.115:6820/1043826337', 'heartbeat_front_addr': '172.21.15.115:6816/1043826337', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.113232+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.174679+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.131118+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.207668+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.124542+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.200832+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.169560+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.182465+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.3', 'mappings': [{'from': 4, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-11T23:02:43.539 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T23:02:43.540 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:43.540 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:43.540 INFO:teuthology.orchestra.run.smithi005.stdout:3933 write 0xb8469c2 thru 0xb84ab3f (0x417e bytes) 2024-05-11T23:02:43.540 INFO:teuthology.orchestra.run.smithi005.stdout:3935 writesame 0xa594400 thru 0xa59a3a8 data_size 0x9(0x5fa9 bytes) 2024-05-11T23:02:43.540 INFO:teuthology.orchestra.run.smithi005.stdout:3936 writesame 0xee10076 thru 0xee16880 data_size 0x7(0x680b bytes) 2024-05-11T23:02:43.540 INFO:teuthology.orchestra.run.smithi005.stdout:3937 write 0xdb1367f thru 0xdb13ab3 (0x435 bytes) 2024-05-11T23:02:43.540 INFO:teuthology.orchestra.run.smithi005.stdout:3938 write 0x689c1de thru 0x68a20b2 (0x5ed5 bytes) 2024-05-11T23:02:43.551 INFO:teuthology.orchestra.run.smithi005.stdout:3939 writesame 0xabdbc57 thru 0xabe5674 data_size 0x1(0x9a1e bytes) 2024-05-11T23:02:43.564 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:43.563+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:43.569 INFO:teuthology.orchestra.run.smithi005.stdout:3940 read 0xa0471c2 thru 0xa05278d (0xb5cc bytes) 2024-05-11T23:02:43.569 INFO:teuthology.orchestra.run.smithi005.stdout:3941 write 0xe21af98 thru 0xe220a8a (0x5af3 bytes) 2024-05-11T23:02:43.589 INFO:teuthology.orchestra.run.smithi005.stdout:3942 read 0x18bd37b thru 0x18c4cf2 (0x7978 bytes) 2024-05-11T23:02:43.590 INFO:teuthology.orchestra.run.smithi005.stdout:3943 read 0x5568998 thru 0x556d854 (0x4ebd bytes) 2024-05-11T23:02:43.590 INFO:teuthology.orchestra.run.smithi005.stdout:3944 write 0x3a9442 thru 0x3b2f44 (0x9b03 bytes) 2024-05-11T23:02:43.595 INFO:teuthology.orchestra.run.smithi005.stdout:3947 read 0x47cf2f8 thru 0x47cf94a (0x653 bytes) 2024-05-11T23:02:43.596 INFO:teuthology.orchestra.run.smithi005.stdout:3948 write 0x2dbada thru 0x2e2959 (0x6e80 bytes) 2024-05-11T23:02:43.603 INFO:teuthology.orchestra.run.smithi005.stdout:3950 read 0xb588a78 thru 0xb59212a (0x96b3 bytes) 2024-05-11T23:02:43.603 INFO:teuthology.orchestra.run.smithi005.stdout:3951 read 0xc32bbb4 thru 0xc33ac9f (0xf0ec bytes) 2024-05-11T23:02:43.603 INFO:teuthology.orchestra.run.smithi005.stdout:3952 write 0xb0263fc thru 0xb02be2a (0x5a2f bytes) 2024-05-11T23:02:43.622 INFO:teuthology.orchestra.run.smithi005.stdout:3954 read 0x8e42aee thru 0x8e4f869 (0xcd7c bytes) 2024-05-11T23:02:43.622 INFO:teuthology.orchestra.run.smithi005.stdout:3955 write 0x13ee8d6 thru 0x13f6191 (0x78bc bytes) 2024-05-11T23:02:43.637 INFO:teuthology.orchestra.run.smithi005.stdout:3956 read 0xbde84c5 thru 0xbdf2a2f (0xa56b bytes) 2024-05-11T23:02:43.637 INFO:teuthology.orchestra.run.smithi005.stdout:3957 read 0x15b8636 thru 0x15bd441 (0x4e0c bytes) 2024-05-11T23:02:43.637 INFO:teuthology.orchestra.run.smithi005.stdout:3958 read 0x69a4266 thru 0x69b3459 (0xf1f4 bytes) 2024-05-11T23:02:43.638 INFO:teuthology.orchestra.run.smithi005.stdout:3959 read 0x3d9a3d thru 0x3da548 (0xb0c bytes) 2024-05-11T23:02:43.640 INFO:teuthology.orchestra.run.smithi005.stdout:3960 trunc from 0xee16881 to 0xd9aff3e 2024-05-11T23:02:43.659 INFO:teuthology.orchestra.run.smithi005.stdout:3961 write 0x3cd03aa thru 0x3cd9740 (0x9397 bytes) 2024-05-11T23:02:43.666 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:43.665+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:43.674 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:43.674 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:43.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.675 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:140085 3.9 e88/19 3:92f8701c:::rbd_data.111ec437b352.00000000000000f9:head v 88'331, mlcod=88'329)", 2024-05-11T23:02:43.675 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:43.673368+0000", 2024-05-11T23:02:43.675 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.00078490099999999996, 2024-05-11T23:02:43.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00080111099999999997, 2024-05-11T23:02:43.675 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:43.675 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:43.675 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:43.675 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:43.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.675 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:43.675 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.673368+0000", 2024-05-11T23:02:43.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:43.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.673368+0000", 2024-05-11T23:02:43.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:43.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:43.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.673369+0000", 2024-05-11T23:02:43.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5910000000000001e-06 2024-05-11T23:02:43.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:43.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.673380+0000", 2024-05-11T23:02:43.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0268999999999999e-05 2024-05-11T23:02:43.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:43.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.673396+0000", 2024-05-11T23:02:43.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5937999999999999e-05 2024-05-11T23:02:43.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:43.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.673404+0000", 2024-05-11T23:02:43.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1089999999999998e-06 2024-05-11T23:02:43.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:43.678 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.673460+0000", 2024-05-11T23:02:43.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6268999999999999e-05 2024-05-11T23:02:43.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:43.678 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.673475+0000", 2024-05-11T23:02:43.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5413e-05 2024-05-11T23:02:43.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:43.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:43.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.674171+0000", 2024-05-11T23:02:43.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00069588400000000002 2024-05-11T23:02:43.679 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.679 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:43.679 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.679 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:43.679 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:02:43.679 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 1 2024-05-11T23:02:43.679 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:43.679 INFO:teuthology.orchestra.run.smithi005.stdout:3962 write 0xbd97099 thru 0xbda562e (0xe596 bytes) 2024-05-11T23:02:43.686 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:02:43.688 INFO:teuthology.orchestra.run.smithi005.stdout:3968 punch from 0xc755d6a to 0xc761261, (0xb4f7 bytes) 2024-05-11T23:02:43.688 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:43.689 INFO:teuthology.orchestra.run.smithi005.stdout:3970 read 0x4e6eac3 thru 0x4e7679e (0x7cdc bytes) 2024-05-11T23:02:43.702 INFO:teuthology.orchestra.run.smithi005.stdout:3971 trunc from 0xd9aff3e to 0x3912b61 2024-05-11T23:02:43.767 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:43.766+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:43.784 INFO:teuthology.orchestra.run.smithi005.stdout:3974 writesame 0xeae2d5d thru 0xeaefe01 data_size 0x1(0xd0a5 bytes) 2024-05-11T23:02:43.785 INFO:teuthology.orchestra.run.smithi005.stdout:3978 punch from 0xa21c1dd to 0xa228d88, (0xcbab bytes) 2024-05-11T23:02:43.785 INFO:teuthology.orchestra.run.smithi005.stdout:3979 writesame 0xda346a2 thru 0xda3c452 data_size 0x1(0x7db1 bytes) 2024-05-11T23:02:43.803 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:43.804 INFO:teuthology.orchestra.run.smithi005.stdout:3982 writesame 0x9cbf059 thru 0x9ccc586 data_size 0xd(0xd52e bytes) 2024-05-11T23:02:43.817 INFO:teuthology.orchestra.run.smithi005.stdout:3983 read 0x4eb5ab thru 0x4f00f2 (0x4b48 bytes) 2024-05-11T23:02:43.819 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:43.826 INFO:teuthology.orchestra.run.smithi005.stdout:3985 trunc from 0xeaefe02 to 0x8ff2146 2024-05-11T23:02:43.828 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:43.828 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:43.828 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:43.828 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:43.828 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:43.840 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:02:43.843 INFO:teuthology.orchestra.run.smithi005.stdout:3986 punch from 0x324385a to 0x324c418, (0x8bbe bytes) 2024-05-11T23:02:43.855 INFO:teuthology.orchestra.run.smithi005.stdout:3989 write 0xe011a12 thru 0xe01430d (0x28fc bytes) 2024-05-11T23:02:43.868 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:43.868+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:43.870 INFO:teuthology.orchestra.run.smithi005.stdout:3991 punch from 0x749fd68 to 0x74aab6c, (0xae04 bytes) 2024-05-11T23:02:43.870 INFO:teuthology.orchestra.run.smithi005.stdout:3992 writesame 0xbfd8f4d thru 0xbfdf893 data_size 0x1(0x6947 bytes) 2024-05-11T23:02:43.887 INFO:teuthology.orchestra.run.smithi005.stdout:3993 writesame 0x68bbeee thru 0x68bf016 data_size 0x5(0x3129 bytes) 2024-05-11T23:02:43.907 INFO:teuthology.orchestra.run.smithi005.stdout:3995 writesame 0x3736bf7 thru 0x3743a15 data_size 0x7b(0xce1f bytes) 2024-05-11T23:02:43.915 INFO:teuthology.orchestra.run.smithi005.stdout:3996 write 0xd26beb thru 0xd30c11 (0xa027 bytes) 2024-05-11T23:02:43.961 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:02:43.962 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-11T23:02:43.969 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:43.969+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:43.974 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:43.978 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-05-11T23:02:43.151381+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1294475073,"num_objects":2694,"num_object_clones":510,"num_object_copies":5388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":61,"num_objects_unfound":0,"num_objects_dirty":2694,"num_whiteouts":1617,"num_read":164263,"num_read_kb":776775,"num_write":13803,"num_write_kb":403259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2036,"num_bytes_recovered":183766029,"num_keys_recovered":363,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12376,"ondisk_log_size":12376,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":370500,"kb_used_data":68292,"kb_used_omap":12,"kb_used_meta":302067,"kb_avail":754604220,"statfs":{"total":773094113280,"available":772714721280,"internally_reserved":0,"allocated":69931008,"data_stored":723256507,"data_compressed":38865698,"data_compressed_allocated":42545152,"data_compressed_original":702600241,"omap_allocated":12752,"internal_metadata":309317168},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8386891,"num_objects":162,"num_object_clones":9,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":61,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":126,"num_read":5794,"num_read_kb":5702,"num_write":250,"num_write_kb":5530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":22096122,"num_keys_recovered":101,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000653"},"pg_stats":[{"pgid":"3.1f","version":"82'302","reported_seq":3241,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T23:02:40.045875+0000","last_change":"2024-05-11T23:01:29.462665+0000","last_active":"2024-05-11T23:02:40.045875+0000","last_peered":"2024-05-11T23:02:40.045875+0000","last_clean":"2024-05-11T23:02:40.045875+0000","last_became_active":"2024-05-11T23:01:27.459623+0000","last_became_peered":"2024-05-11T23:01:27.459623+0000","last_unstale":"2024-05-11T23:02:40.045875+0000","last_undegraded":"2024-05-11T23:02:40.045875+0000","last_fullsized":"2024-05-11T23:02:40.045875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:15:08.557892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096814299999999998,"stat_sum":{"num_bytes":20780748,"num_objects":84,"num_object_clones":15,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":55,"num_read":3796,"num_read_kb":11030,"num_write":276,"num_write_kb":17289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"85'436","reported_seq":4339,"reported_epoch":86,"state":"active+recovering+undersized+remapped","last_fresh":"2024-05-11T23:02:41.155361+0000","last_change":"2024-05-11T23:02:41.033828+0000","last_active":"2024-05-11T23:02:41.155361+0000","last_peered":"2024-05-11T23:02:41.155361+0000","last_clean":"2024-05-11T23:02:38.036233+0000","last_became_active":"2024-05-11T23:02:40.848050+0000","last_became_peered":"2024-05-11T23:02:40.848050+0000","last_unstale":"2024-05-11T23:02:41.155361+0000","last_undegraded":"2024-05-11T23:02:41.155361+0000","last_fullsized":"2024-05-11T23:02:40.047284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062673000000000004,"stat_sum":{"num_bytes":58345025,"num_objects":96,"num_object_clones":22,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":61,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":58,"num_read":4507,"num_read_kb":41419,"num_write":449,"num_write_kb":19154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":43745359,"num_keys_recovered":96,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":61},{"shards":"3","objects":35}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"82'251","reported_seq":1825,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T23:02:38.044091+0000","last_change":"2024-05-11T23:01:29.474888+0000","last_active":"2024-05-11T23:02:38.044091+0000","last_peered":"2024-05-11T23:02:38.044091+0000","last_clean":"2024-05-11T23:02:38.044091+0000","last_became_active":"2024-05-11T23:01:27.458054+0000","last_became_peered":"2024-05-11T23:01:27.458054+0000","last_unstale":"2024-05-11T23:02:38.044091+0000","last_undegraded":"2024-05-11T23:02:38.044091+0000","last_fullsized":"2024-05-11T23:02:38.044091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:34:13.960353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001000658,"stat_sum":{"num_bytes":25530492,"num_objects":70,"num_object_clones":10,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":52,"num_read":1548,"num_read_kb":11929,"num_write":216,"num_write_kb":8826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"82'373","reported_seq":5964,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T23:02:38.043417+0000","last_change":"2024-05-11T23:01:29.474685+0000","last_active":"2024-05-11T23:02:38.043417+0000","last_peered":"2024-05-11T23:02:38.043417+0000","last_clean":"2024-05-11T23:02:38.043417+0000","last_became_active":"2024-05-11T23:01:27.463341+0000","last_became_peered":"2024-05-11T23:01:27.463341+0000","last_unstale":"2024-05-11T23:02:38.043417+0000","last_undegraded":"2024-05-11T23:02:38.043417+0000","last_fullsized":"2024-05-11T23:02:38.043417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:37:33.400273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091161300000000003,"stat_sum":{"num_bytes":40205343,"num_objects":92,"num_object_clones":14,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":59,"num_read":6383,"num_read_kb":30350,"num_write":300,"num_write_kb":5956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'317","reported_seq":7734,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T23:02:38.043766+0000","last_change":"2024-05-11T23:01:39.800279+0000","last_active":"2024-05-11T23:02:38.043766+0000","last_peered":"2024-05-11T23:02:38.043766+0000","last_clean":"2024-05-11T23:02:38.043766+0000","last_became_active":"2024-05-11T23:01:39.787165+0000","last_became_peered":"2024-05-11T23:01:39.787165+0000","last_unstale":"2024-05-11T23:02:38.043766+0000","last_undegraded":"2024-05-11T23:02:38.043766+0000","last_fullsized":"2024-05-11T23:02:38.043766+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:25:09.702891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065062200000000001,"stat_sum":{"num_bytes":88431067,"num_objects":89,"num_object_clones":17,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":53,"num_read":7587,"num_read_kb":61991,"num_write":375,"num_write_kb":34461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"82'246","reported_seq":7659,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T23:02:38.044369+0000","last_change":"2024-05-11T23:01:29.466255+0000","last_active":"2024-05-11T23:02:38.044369+0000","last_peered":"2024-05-11T23:02:38.044369+0000","last_clean":"2024-05-11T23:02:38.044369+0000","last_became_active":"2024-05-11T23:01:27.460541+0000","last_became_peered":"2024-05-11T23:01:27.460541+0000","last_unstale":"2024-05-11T23:02:38.044369+0000","last_undegraded":"2024-05-11T23:02:38.044369+0000","last_fullsized":"2024-05-11T23:02:38.044369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:36:25.295053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087678099999999998,"stat_sum":{"num_bytes":69436353,"num_objects":80,"num_object_clones":10,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":44,"num_read":7568,"num_read_kb":28957,"num_write":325,"num_write_kb":17748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"85'730","reported_seq":4891,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T23:02:41.026948+0000","last_change":"2024-05-11T23:02:31.781713+0000","last_active":"2024-05-11T23:02:41.026948+0000","last_peered":"2024-05-11T23:02:41.026948+0000","last_clean":"2024-05-11T23:02:41.026948+0000","last_became_active":"2024-05-11T23:02:31.780769+0000","last_became_peered":"2024-05-11T23:02:31.780769+0000","last_unstale":"2024-05-11T23:02:41.026948+0000","last_undegraded":"2024-05-11T23:02:41.026948+0000","last_fullsized":"2024-05-11T23:02:41.026948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:44:20.009805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000433275,"stat_sum":{"num_bytes":66841922,"num_objects":95,"num_object_clones":15,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":51,"num_read":6410,"num_read_kb":35184,"num_write":1095,"num_write_kb":12724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":13589947,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"82'265","reported_seq":1813,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T23:02:41.054245+0000","last_change":"2024-05-11T23:02:31.781325+0000","last_active":"2024-05-11T23:02:41.054245+0000","last_peered":"2024-05-11T23:02:41.054245+0000","last_clean":"2024-05-11T23:02:41.054245+0000","last_became_active":"2024-05-11T23:02:31.779496+0000","last_became_peered":"2024-05-11T23:02:31.779496+0000","last_unstale":"2024-05-11T23:02:41.054245+0000","last_undegraded":"2024-05-11T23:02:41.054245+0000","last_fullsized":"2024-05-11T23:02:41.054245+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:46:54.204294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079167699999999996,"stat_sum":{"num_bytes":19063972,"num_objects":73,"num_object_clones":14,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":47,"num_read":1618,"num_read_kb":3234,"num_write":223,"num_write_kb":12311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":4822628,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"82'295","reported_seq":2517,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T23:02:40.045990+0000","last_change":"2024-05-11T23:01:29.462988+0000","last_active":"2024-05-11T23:02:40.045990+0000","last_peered":"2024-05-11T23:02:40.045990+0000","last_clean":"2024-05-11T23:02:40.045990+0000","last_became_active":"2024-05-11T23:01:27.460088+0000","last_became_peered":"2024-05-11T23:01:27.460088+0000","last_unstale":"2024-05-11T23:02:40.045990+0000","last_undegraded":"2024-05-11T23:02:40.045990+0000","last_fullsized":"2024-05-11T23:02:40.045990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:03:07.062697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045912,"stat_sum":{"num_bytes":8666005,"num_objects":59,"num_object_clones":6,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":42,"num_read":2724,"num_read_kb":11650,"num_write":345,"num_write_kb":2654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"82'257","reported_seq":3331,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T23:02:38.043355+0000","last_change":"2024-05-11T23:01:29.474960+0000","last_active":"2024-05-11T23:02:38.043355+0000","last_peered":"2024-05-11T23:02:38.043355+0000","last_clean":"2024-05-11T23:02:38.043355+0000","last_became_active":"2024-05-11T23:01:27.463623+0000","last_became_peered":"2024-05-11T23:01:27.463623+0000","last_unstale":"2024-05-11T23:02:38.043355+0000","last_undegraded":"2024-05-11T23:02:38.043355+0000","last_fullsized":"2024-05-11T23:02:38.043355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:40:52.565279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001047748,"stat_sum":{"num_bytes":38994882,"num_objects":76,"num_object_clones":12,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":51,"num_read":3218,"num_read_kb":34590,"num_write":241,"num_write_kb":15329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"82'294","reported_seq":5180,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T23:02:38.044165+0000","last_change":"2024-05-11T23:02:30.784538+0000","last_active":"2024-05-11T23:02:38.044165+0000","last_peered":"2024-05-11T23:02:38.044165+0000","last_clean":"2024-05-11T23:02:38.044165+0000","last_became_active":"2024-05-11T23:02:30.783475+0000","last_became_peered":"2024-05-11T23:02:30.783475+0000","last_unstale":"2024-05-11T23:02:38.044165+0000","last_undegraded":"2024-05-11T23:02:38.044165+0000","last_fullsized":"2024-05-11T23:02:38.044165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T09:00:10.779677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039810900000000001,"stat_sum":{"num_bytes":52068573,"num_objects":91,"num_object_clones":19,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":58,"num_read":3691,"num_read_kb":16579,"num_write":316,"num_write_kb":11285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":12363304,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"82'313","reported_seq":7929,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T23:02:38.043945+0000","last_change":"2024-05-11T23:01:29.466206+0000","last_active":"2024-05-11T23:02:38.043945+0000","last_peered":"2024-05-11T23:02:38.043945+0000","last_clean":"2024-05-11T23:02:38.043945+0000","last_became_active":"2024-05-11T23:01:27.465952+0000","last_became_peered":"2024-05-11T23:01:27.465952+0000","last_unstale":"2024-05-11T23:02:38.043945+0000","last_undegraded":"2024-05-11T23:02:38.043945+0000","last_fullsized":"2024-05-11T23:02:38.043945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:19:31.283980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066166100000000002,"stat_sum":{"num_bytes":25858948,"num_objects":77,"num_object_clones":15,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":44,"num_read":8464,"num_read_kb":27854,"num_write":279,"num_write_kb":4721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-11T23:02:42.057576+0000","last_change":"2024-05-11T23:02:39.044686+0000","last_active":"2024-05-11T23:02:42.057576+0000","last_peered":"2024-05-11T23:02:42.057576+0000","last_clean":"2024-05-11T23:02:42.057576+0000","last_became_active":"2024-05-11T23:02:39.044216+0000","last_became_peered":"2024-05-11T23:02:39.044216+0000","last_unstale":"2024-05-11T23:02:42.057576+0000","last_undegraded":"2024-05-11T23:02:42.057576+0000","last_fullsized":"2024-05-11T23:02:42.057576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:31:03.826529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"82'609","reported_seq":3328,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T23:02:40.046025+0000","last_change":"2024-05-11T23:01:29.462499+0000","last_active":"2024-05-11T23:02:40.046025+0000","last_peered":"2024-05-11T23:02:40.046025+0000","last_clean":"2024-05-11T23:02:40.046025+0000","last_became_active":"2024-05-11T23:01:27.458177+0000","last_became_peered":"2024-05-11T23:01:27.458177+0000","last_unstale":"2024-05-11T23:02:40.046025+0000","last_undegraded":"2024-05-11T23:02:40.046025+0000","last_fullsized":"2024-05-11T23:02:40.046025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:08:32.390281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099453400000000004,"stat_sum":{"num_bytes":27019410,"num_objects":81,"num_object_clones":8,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":54,"num_read":5595,"num_read_kb":16867,"num_write":830,"num_write_kb":9605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":156,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T23:02:40.048177+0000","last_change":"2024-05-11T23:02:39.047593+0000","last_active":"2024-05-11T23:02:40.048177+0000","last_peered":"2024-05-11T23:02:40.048177+0000","last_clean":"2024-05-11T23:02:40.048177+0000","last_became_active":"2024-05-11T23:02:39.047225+0000","last_became_peered":"2024-05-11T23:02:39.047225+0000","last_unstale":"2024-05-11T23:02:40.048177+0000","last_undegraded":"2024-05-11T23:02:40.048177+0000","last_fullsized":"2024-05-11T23:02:40.048177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:34:13.738178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"82'611","reported_seq":4499,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T23:02:41.054414+0000","last_change":"2024-05-11T23:02:31.782882+0000","last_active":"2024-05-11T23:02:41.054414+0000","last_peered":"2024-05-11T23:02:41.054414+0000","last_clean":"2024-05-11T23:02:41.054414+0000","last_became_active":"2024-05-11T23:02:31.781966+0000","last_became_peered":"2024-05-11T23:02:31.781966+0000","last_unstale":"2024-05-11T23:02:41.054414+0000","last_undegraded":"2024-05-11T23:02:41.054414+0000","last_fullsized":"2024-05-11T23:02:41.054414+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:50:57.354839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058786099999999996,"stat_sum":{"num_bytes":29418993,"num_objects":78,"num_object_clones":12,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":47,"num_read":6224,"num_read_kb":16637,"num_write":815,"num_write_kb":4639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":4415701,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T23:02:38.043950+0000","last_change":"2024-05-11T23:01:19.216700+0000","last_active":"2024-05-11T23:02:38.043950+0000","last_peered":"2024-05-11T23:02:38.043950+0000","last_clean":"2024-05-11T23:02:38.043950+0000","last_became_active":"2024-05-11T23:01:19.216413+0000","last_became_peered":"2024-05-11T23:01:19.216413+0000","last_unstale":"2024-05-11T23:02:38.043950+0000","last_undegraded":"2024-05-11T23:02:38.043950+0000","last_fullsized":"2024-05-11T23:02:38.043950+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:38.639409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"85'234","reported_seq":4048,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T23:02:40.924507+0000","last_change":"2024-05-11T23:01:29.468176+0000","last_active":"2024-05-11T23:02:40.924507+0000","last_peered":"2024-05-11T23:02:40.924507+0000","last_clean":"2024-05-11T23:02:40.924507+0000","last_became_active":"2024-05-11T23:01:27.460866+0000","last_became_peered":"2024-05-11T23:01:27.460866+0000","last_unstale":"2024-05-11T23:02:40.924507+0000","last_undegraded":"2024-05-11T23:02:40.924507+0000","last_fullsized":"2024-05-11T23:02:40.924507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:02:28.463885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050028300000000002,"stat_sum":{"num_bytes":64776863,"num_objects":79,"num_object_clones":18,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":47,"num_read":3833,"num_read_kb":45735,"num_write":251,"num_write_kb":31676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":146,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T23:02:40.048286+0000","last_change":"2024-05-11T23:01:19.220232+0000","last_active":"2024-05-11T23:02:40.048286+0000","last_peered":"2024-05-11T23:02:40.048286+0000","last_clean":"2024-05-11T23:02:40.048286+0000","last_became_active":"2024-05-11T23:01:19.219917+0000","last_became_peered":"2024-05-11T23:01:19.219917+0000","last_unstale":"2024-05-11T23:02:40.048286+0000","last_undegraded":"2024-05-11T23:02:40.048286+0000","last_fullsized":"2024-05-11T23:02:40.048286+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:44:02.868252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"82'257","reported_seq":3579,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T23:02:38.044192+0000","last_change":"2024-05-11T23:02:30.785107+0000","last_active":"2024-05-11T23:02:38.044192+0000","last_peered":"2024-05-11T23:02:38.044192+0000","last_clean":"2024-05-11T23:02:38.044192+0000","last_became_active":"2024-05-11T23:02:30.784580+0000","last_became_peered":"2024-05-11T23:02:30.784580+0000","last_unstale":"2024-05-11T23:02:38.044192+0000","last_undegraded":"2024-05-11T23:02:38.044192+0000","last_fullsized":"2024-05-11T23:02:38.044192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:38:25.085518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012391380000000001,"stat_sum":{"num_bytes":47842829,"num_objects":90,"num_object_clones":23,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":51,"num_read":3299,"num_read_kb":21013,"num_write":212,"num_write_kb":9990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":13712581,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"82'430","reported_seq":4390,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T23:02:40.045399+0000","last_change":"2024-05-11T23:02:15.933489+0000","last_active":"2024-05-11T23:02:40.045399+0000","last_peered":"2024-05-11T23:02:40.045399+0000","last_clean":"2024-05-11T23:02:40.045399+0000","last_became_active":"2024-05-11T23:02:15.932604+0000","last_became_peered":"2024-05-11T23:02:15.932604+0000","last_unstale":"2024-05-11T23:02:40.045399+0000","last_undegraded":"2024-05-11T23:02:40.045399+0000","last_fullsized":"2024-05-11T23:02:40.045399+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:26:27.729528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038468999999999998,"stat_sum":{"num_bytes":63114675,"num_objects":90,"num_object_clones":15,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":54,"num_read":5608,"num_read_kb":27123,"num_write":466,"num_write_kb":11314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":3164613,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T23:02:38.043372+0000","last_change":"2024-05-11T23:01:19.221118+0000","last_active":"2024-05-11T23:02:38.043372+0000","last_peered":"2024-05-11T23:02:38.043372+0000","last_clean":"2024-05-11T23:02:38.043372+0000","last_became_active":"2024-05-11T23:01:19.220855+0000","last_became_peered":"2024-05-11T23:01:19.220855+0000","last_unstale":"2024-05-11T23:02:38.043372+0000","last_undegraded":"2024-05-11T23:02:38.043372+0000","last_fullsized":"2024-05-11T23:02:38.043372+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:01:56.076963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"82'401","reported_seq":3805,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T23:02:40.048045+0000","last_change":"2024-05-11T23:01:29.467921+0000","last_active":"2024-05-11T23:02:40.048045+0000","last_peered":"2024-05-11T23:02:40.048045+0000","last_clean":"2024-05-11T23:02:40.048045+0000","last_became_active":"2024-05-11T23:01:27.459593+0000","last_became_peered":"2024-05-11T23:01:27.459593+0000","last_unstale":"2024-05-11T23:02:40.048045+0000","last_undegraded":"2024-05-11T23:02:40.048045+0000","last_fullsized":"2024-05-11T23:02:40.048045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T09:43:05.844890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047314299999999998,"stat_sum":{"num_bytes":15516889,"num_objects":77,"num_object_clones":15,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":48,"num_read":5311,"num_read_kb":12854,"num_write":458,"num_write_kb":2546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":156,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T23:02:40.045427+0000","last_change":"2024-05-11T23:02:27.995538+0000","last_active":"2024-05-11T23:02:40.045427+0000","last_peered":"2024-05-11T23:02:40.045427+0000","last_clean":"2024-05-11T23:02:40.045427+0000","last_became_active":"2024-05-11T23:02:27.995242+0000","last_became_peered":"2024-05-11T23:02:27.995242+0000","last_unstale":"2024-05-11T23:02:40.045427+0000","last_undegraded":"2024-05-11T23:02:40.045427+0000","last_fullsized":"2024-05-11T23:02:40.045427+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T09:00:41.429605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"82'264","reported_seq":2099,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T23:02:40.046048+0000","last_change":"2024-05-11T23:01:29.462388+0000","last_active":"2024-05-11T23:02:40.046048+0000","last_peered":"2024-05-11T23:02:40.046048+0000","last_clean":"2024-05-11T23:02:40.046048+0000","last_became_active":"2024-05-11T23:01:27.460113+0000","last_became_peered":"2024-05-11T23:01:27.460113+0000","last_unstale":"2024-05-11T23:02:40.046048+0000","last_undegraded":"2024-05-11T23:02:40.046048+0000","last_fullsized":"2024-05-11T23:02:40.046048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:49:44.366264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062926200000000001,"stat_sum":{"num_bytes":18129329,"num_objects":77,"num_object_clones":15,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":48,"num_read":2442,"num_read_kb":12414,"num_write":217,"num_write_kb":4748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T23:02:38.043368+0000","last_change":"2024-05-11T23:01:19.219755+0000","last_active":"2024-05-11T23:02:38.043368+0000","last_peered":"2024-05-11T23:02:38.043368+0000","last_clean":"2024-05-11T23:02:38.043368+0000","last_became_active":"2024-05-11T23:01:19.219471+0000","last_became_peered":"2024-05-11T23:01:19.219471+0000","last_unstale":"2024-05-11T23:02:38.043368+0000","last_undegraded":"2024-05-11T23:02:38.043368+0000","last_fullsized":"2024-05-11T23:02:38.043368+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:39:05.284213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"82'369","reported_seq":2022,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T23:02:41.054290+0000","last_change":"2024-05-11T23:02:30.783741+0000","last_active":"2024-05-11T23:02:41.054290+0000","last_peered":"2024-05-11T23:02:41.054290+0000","last_clean":"2024-05-11T23:02:41.054290+0000","last_became_active":"2024-05-11T23:02:30.782707+0000","last_became_peered":"2024-05-11T23:02:30.782707+0000","last_unstale":"2024-05-11T23:02:41.054290+0000","last_undegraded":"2024-05-11T23:02:41.054290+0000","last_fullsized":"2024-05-11T23:02:41.054290+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:27:03.335746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000407852,"stat_sum":{"num_bytes":34336607,"num_objects":88,"num_object_clones":15,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":57,"num_read":1911,"num_read_kb":9427,"num_write":395,"num_write_kb":5877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":10719266,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T23:02:38.043372+0000","last_change":"2024-05-11T23:01:19.219128+0000","last_active":"2024-05-11T23:02:38.043372+0000","last_peered":"2024-05-11T23:02:38.043372+0000","last_clean":"2024-05-11T23:02:38.043372+0000","last_became_active":"2024-05-11T23:01:19.218846+0000","last_became_peered":"2024-05-11T23:01:19.218846+0000","last_unstale":"2024-05-11T23:02:38.043372+0000","last_undegraded":"2024-05-11T23:02:38.043372+0000","last_fullsized":"2024-05-11T23:02:38.043372+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:41:09.111000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":294,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T23:02:38.043453+0000","last_change":"2024-05-11T23:01:18.164452+0000","last_active":"2024-05-11T23:02:38.043453+0000","last_peered":"2024-05-11T23:02:38.043453+0000","last_clean":"2024-05-11T23:02:38.043453+0000","last_became_active":"2024-05-11T23:01:18.163431+0000","last_became_peered":"2024-05-11T23:01:18.163431+0000","last_unstale":"2024-05-11T23:02:38.043453+0000","last_undegraded":"2024-05-11T23:02:38.043453+0000","last_fullsized":"2024-05-11T23:02:38.043453+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_clean_scrub_stamp":"2024-05-11T23:01:16.147564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:40:59.974639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"82'665","reported_seq":4308,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T23:02:41.054245+0000","last_change":"2024-05-11T23:02:31.782579+0000","last_active":"2024-05-11T23:02:41.054245+0000","last_peered":"2024-05-11T23:02:41.054245+0000","last_clean":"2024-05-11T23:02:41.054245+0000","last_became_active":"2024-05-11T23:02:31.781467+0000","last_became_peered":"2024-05-11T23:02:31.781467+0000","last_unstale":"2024-05-11T23:02:41.054245+0000","last_undegraded":"2024-05-11T23:02:41.054245+0000","last_fullsized":"2024-05-11T23:02:41.054245+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:49:06.523294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061400699999999996,"stat_sum":{"num_bytes":38751080,"num_objects":87,"num_object_clones":15,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":53,"num_read":5546,"num_read_kb":14163,"num_write":900,"num_write_kb":7001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":4220183,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"82'311","reported_seq":6142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T23:02:38.043739+0000","last_change":"2024-05-11T23:01:29.465998+0000","last_active":"2024-05-11T23:02:38.043739+0000","last_peered":"2024-05-11T23:02:38.043739+0000","last_clean":"2024-05-11T23:02:38.043739+0000","last_became_active":"2024-05-11T23:01:27.461824+0000","last_became_peered":"2024-05-11T23:01:27.461824+0000","last_unstale":"2024-05-11T23:02:38.043739+0000","last_undegraded":"2024-05-11T23:02:38.043739+0000","last_fullsized":"2024-05-11T23:02:38.043739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:55:11.406361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055969500000000003,"stat_sum":{"num_bytes":72382950,"num_objects":95,"num_object_clones":20,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":50,"num_read":6612,"num_read_kb":40301,"num_write":277,"num_write_kb":26501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"86'348","reported_seq":3477,"reported_epoch":87,"state":"creating+peering","last_fresh":"2024-05-11T23:02:41.524077+0000","last_change":"2024-05-11T23:02:42.167739+0000","last_active":"2024-05-11T23:02:42.167739+0000","last_peered":"2024-05-11T23:02:41.524077+0000","last_clean":"2024-05-11T23:02:37.394938+0000","last_became_active":"2024-05-11T23:02:40.848253+0000","last_became_peered":"2024-05-11T23:02:40.848253+0000","last_unstale":"2024-05-11T23:02:41.524077+0000","last_undegraded":"2024-05-11T23:02:41.524077+0000","last_fullsized":"2024-05-11T23:02:41.524077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030522600000000002,"stat_sum":{"num_bytes":30030424,"num_objects":95,"num_object_clones":19,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":53,"num_read":3391,"num_read_kb":15724,"num_write":312,"num_write_kb":6080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":8102656,"num_keys_recovered":69,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"82'333","reported_seq":7628,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-05-11T23:02:41.054497+0000","last_change":"2024-05-11T23:02:30.783606+0000","last_active":"2024-05-11T23:02:41.054497+0000","last_peered":"2024-05-11T23:02:41.054497+0000","last_clean":"2024-05-11T23:02:41.054497+0000","last_became_active":"2024-05-11T23:02:30.779706+0000","last_became_peered":"2024-05-11T23:02:30.779706+0000","last_unstale":"2024-05-11T23:02:41.054497+0000","last_undegraded":"2024-05-11T23:02:41.054497+0000","last_fullsized":"2024-05-11T23:02:41.054497+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:16:59.482174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042056999999999999,"stat_sum":{"num_bytes":73561194,"num_objects":90,"num_object_clones":23,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":40,"num_read":7312,"num_read_kb":44684,"num_write":357,"num_write_kb":41075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":30909049,"num_keys_recovered":11,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"82'337","reported_seq":3153,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T23:02:38.043470+0000","last_change":"2024-05-11T23:02:31.781981+0000","last_active":"2024-05-11T23:02:38.043470+0000","last_peered":"2024-05-11T23:02:38.043470+0000","last_clean":"2024-05-11T23:02:38.043470+0000","last_became_active":"2024-05-11T23:02:31.781419+0000","last_became_peered":"2024-05-11T23:02:31.781419+0000","last_unstale":"2024-05-11T23:02:38.043470+0000","last_undegraded":"2024-05-11T23:02:38.043470+0000","last_fullsized":"2024-05-11T23:02:38.043470+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T09:56:22.231418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084388499999999995,"stat_sum":{"num_bytes":14001193,"num_objects":70,"num_object_clones":10,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":49,"num_read":3713,"num_read_kb":20788,"num_write":352,"num_write_kb":9288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":10615766,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"82'345","reported_seq":7059,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T23:02:38.043850+0000","last_change":"2024-05-11T23:01:29.864324+0000","last_active":"2024-05-11T23:02:38.043850+0000","last_peered":"2024-05-11T23:02:38.043850+0000","last_clean":"2024-05-11T23:02:38.043850+0000","last_became_active":"2024-05-11T23:01:27.465570+0000","last_became_peered":"2024-05-11T23:01:27.465570+0000","last_unstale":"2024-05-11T23:02:38.043850+0000","last_undegraded":"2024-05-11T23:02:38.043850+0000","last_fullsized":"2024-05-11T23:02:38.043850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:31:19.112542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062217299999999995,"stat_sum":{"num_bytes":33343749,"num_objects":86,"num_object_clones":21,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":47,"num_read":7327,"num_read_kb":27145,"num_write":319,"num_write_kb":9330,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"82'688","reported_seq":11358,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T23:02:38.044331+0000","last_change":"2024-05-11T23:01:29.864587+0000","last_active":"2024-05-11T23:02:38.044331+0000","last_peered":"2024-05-11T23:02:38.044331+0000","last_clean":"2024-05-11T23:02:38.044331+0000","last_became_active":"2024-05-11T23:01:27.466572+0000","last_became_peered":"2024-05-11T23:01:27.466572+0000","last_unstale":"2024-05-11T23:02:38.044331+0000","last_undegraded":"2024-05-11T23:02:38.044331+0000","last_fullsized":"2024-05-11T23:02:38.044331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:26:07.508038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087175200000000005,"stat_sum":{"num_bytes":37297283,"num_objects":112,"num_object_clones":27,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":63,"num_read":14167,"num_read_kb":36506,"num_write":829,"num_write_kb":16784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"82'300","reported_seq":4600,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T23:02:38.044613+0000","last_change":"2024-05-11T23:01:29.864577+0000","last_active":"2024-05-11T23:02:38.044613+0000","last_peered":"2024-05-11T23:02:38.044613+0000","last_clean":"2024-05-11T23:02:38.044613+0000","last_became_active":"2024-05-11T23:01:27.466778+0000","last_became_peered":"2024-05-11T23:01:27.466778+0000","last_unstale":"2024-05-11T23:02:38.044613+0000","last_undegraded":"2024-05-11T23:02:38.044613+0000","last_fullsized":"2024-05-11T23:02:38.044613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:20:37.285013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080844100000000002,"stat_sum":{"num_bytes":30721531,"num_objects":100,"num_object_clones":22,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":66,"num_read":4461,"num_read_kb":27155,"num_write":258,"num_write_kb":11779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"87'340","reported_seq":4782,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-11T23:02:42.062941+0000","last_change":"2024-05-11T23:02:42.060650+0000","last_active":"2024-05-11T23:02:42.062941+0000","last_peered":"2024-05-11T23:02:42.062941+0000","last_clean":"2024-05-11T23:02:42.062941+0000","last_became_active":"2024-05-11T23:02:42.059228+0000","last_became_peered":"2024-05-11T23:02:42.059228+0000","last_unstale":"2024-05-11T23:02:42.062941+0000","last_undegraded":"2024-05-11T23:02:42.062941+0000","last_fullsized":"2024-05-11T23:02:42.062941+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T09:18:11.896324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033171200000000001,"stat_sum":{"num_bytes":39812286,"num_objects":85,"num_object_clones":19,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":42,"num_read":5926,"num_read_kb":21647,"num_write":370,"num_write_kb":11332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":14890358,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"82'450","reported_seq":3639,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T23:02:38.043560+0000","last_change":"2024-05-11T23:02:24.979353+0000","last_active":"2024-05-11T23:02:38.043560+0000","last_peered":"2024-05-11T23:02:38.043560+0000","last_clean":"2024-05-11T23:02:38.043560+0000","last_became_active":"2024-05-11T23:02:24.978284+0000","last_became_peered":"2024-05-11T23:02:24.978284+0000","last_unstale":"2024-05-11T23:02:38.043560+0000","last_undegraded":"2024-05-11T23:02:38.043560+0000","last_fullsized":"2024-05-11T23:02:38.043560+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:10:48.912301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044654299999999998,"stat_sum":{"num_bytes":23814552,"num_objects":81,"num_object_clones":17,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":47,"num_read":5153,"num_read_kb":18014,"num_write":523,"num_write_kb":5280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":8486426,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"82'671","reported_seq":3624,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-11T23:02:40.045884+0000","last_change":"2024-05-11T23:01:29.462751+0000","last_active":"2024-05-11T23:02:40.045884+0000","last_peered":"2024-05-11T23:02:40.045884+0000","last_clean":"2024-05-11T23:02:40.045884+0000","last_became_active":"2024-05-11T23:01:27.459137+0000","last_became_peered":"2024-05-11T23:01:27.459137+0000","last_unstale":"2024-05-11T23:02:40.045884+0000","last_undegraded":"2024-05-11T23:02:40.045884+0000","last_fullsized":"2024-05-11T23:02:40.045884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T09:05:33.366823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010326510000000001,"stat_sum":{"num_bytes":70495727,"num_objects":81,"num_object_clones":15,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":42,"num_read":5215,"num_read_kb":20876,"num_write":893,"num_write_kb":10121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"82'217","reported_seq":3261,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T23:02:38.044019+0000","last_change":"2024-05-11T23:01:29.864290+0000","last_active":"2024-05-11T23:02:38.044019+0000","last_peered":"2024-05-11T23:02:38.044019+0000","last_clean":"2024-05-11T23:02:38.044019+0000","last_became_active":"2024-05-11T23:01:27.465953+0000","last_became_peered":"2024-05-11T23:01:27.465953+0000","last_unstale":"2024-05-11T23:02:38.044019+0000","last_undegraded":"2024-05-11T23:02:38.044019+0000","last_fullsized":"2024-05-11T23:02:38.044019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:42:51.046751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065432399999999996,"stat_sum":{"num_bytes":15293811,"num_objects":68,"num_object_clones":12,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":45,"num_read":3587,"num_read_kb":12835,"num_write":170,"num_write_kb":3419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1293884705,"num_objects":2692,"num_object_clones":510,"num_object_copies":5384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":61,"num_objects_unfound":0,"num_objects_dirty":2692,"num_whiteouts":1617,"num_read":164147,"num_read_kb":776675,"num_write":13644,"num_write_kb":400843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2036,"num_bytes_recovered":183766029,"num_keys_recovered":363,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62087168,"data_stored":719763077,"data_compressed":38877324,"data_compressed_allocated":42557440,"data_compressed_original":702764081,"omap_allocated":0,"internal_metadata":0},"log_size":12259,"ondisk_log_size":12259,"up":64,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65020,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56260,"kb_used_data":11184,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94315580,"statfs":{"total":96636764160,"available":96579153920,"internally_reserved":0,"allocated":11452416,"data_stored":114631747,"data_compressed":6554429,"data_compressed_allocated":6873088,"data_compressed_original":110958720,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47940,"kb_used_data":8432,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323900,"statfs":{"total":96636764160,"available":96587673600,"internally_reserved":0,"allocated":8634368,"data_stored":88453792,"data_compressed":5048208,"data_compressed_allocated":5459968,"data_compressed_original":86127679,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42520,"kb_used_data":11344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329320,"statfs":{"total":96636764160,"available":96593223680,"internally_reserved":0,"allocated":11616256,"data_stored":118214448,"data_compressed":6265276,"data_compressed_allocated":6991872,"data_compressed_original":114459092,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39696,"kb_used_data":4352,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332144,"statfs":{"total":96636764160,"available":96596115456,"internally_reserved":0,"allocated":4456448,"data_stored":37185789,"data_compressed":2047971,"data_compressed_allocated":2379776,"data_compressed_original":35741696,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41828,"kb_used_data":10648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330012,"statfs":{"total":96636764160,"available":96593932288,"internally_reserved":0,"allocated":10903552,"data_stored":117760212,"data_compressed":6145825,"data_compressed_allocated":6778880,"data_compressed_original":114616968,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56236,"kb_used_data":7000,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94315604,"statfs":{"total":96636764160,"available":96579178496,"internally_reserved":0,"allocated":7168000,"data_stored":64332598,"data_compressed":3268963,"data_compressed_allocated":4087808,"data_compressed_original":62111857,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46672,"kb_used_data":7168,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325168,"statfs":{"total":96636764160,"available":96588972032,"internally_reserved":0,"allocated":7340032,"data_stored":84703483,"data_compressed":4365749,"data_compressed_allocated":4763648,"data_compressed_original":82856920,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39348,"kb_used_data":8164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332492,"statfs":{"total":96636764160,"available":96596471808,"internally_reserved":0,"allocated":8359936,"data_stored":97974438,"data_compressed":5169277,"data_compressed_allocated":5210112,"data_compressed_original":95727309,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7311360,"data_stored":96927208,"data_compressed":5136767,"data_compressed_allocated":5165056,"data_compressed_original":95153869,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6197248,"data_stored":63882866,"data_compressed":3268963,"data_compressed_allocated":4087808,"data_compressed_original":62111857,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6356992,"data_stored":84246621,"data_compressed":4365749,"data_compressed_allocated":4763648,"data_compressed_original":82856920,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9924608,"data_stored":117306440,"data_compressed":6145825,"data_compressed_allocated":6778880,"data_compressed_original":114616968,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10743808,"data_stored":119077102,"data_compressed":6341922,"data_compressed_allocated":7094272,"data_compressed_original":115769812,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7663616,"data_stored":88002914,"data_compressed":5048208,"data_compressed_allocated":5459968,"data_compressed_original":86127679,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3473408,"data_stored":36729425,"data_compressed":2047971,"data_compressed_allocated":2379776,"data_compressed_original":35741696,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10416128,"data_stored":113590501,"data_compressed":6521919,"data_compressed_allocated":6828032,"data_compressed_original":110385280,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T23:02:43.982 INFO:tasks.thrashosds.thrasher:Setting 3.11 to [7, 4, 6, 1] 2024-05-11T23:02:43.982 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.11', '7', '4', '6', '1'] 2024-05-11T23:02:43.982 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.11 7 4 6 1 2024-05-11T23:02:43.985 INFO:teuthology.orchestra.run.smithi005.stdout:3998 trunc from 0xe01430e to 0x4ae6dfe 2024-05-11T23:02:43.999 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60471 3.4 e43/19 3:2079ea20:::rbd_data.111e1df04b77.0000000000000000:head v 43'84, mlcod=42'81)", 2024-05-11T23:02:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.071208321, 2024-05-11T23:02:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0079043050000000004, 2024-05-11T23:02:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-05-11T23:02:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926880+0000", 2024-05-11T23:02:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:02:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926893+0000", 2024-05-11T23:02:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3256e-05 2024-05-11T23:02:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926899+0000", 2024-05-11T23:02:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9850000000000003e-06 2024-05-11T23:02:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926982+0000", 2024-05-11T23:02:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2997000000000005e-05 2024-05-11T23:02:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927005+0000", 2024-05-11T23:02:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3428e-05 2024-05-11T23:02:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.934747+0000", 2024-05-11T23:02:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0077415820000000003 2024-05-11T23:02:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.934778+0000", 2024-05-11T23:02:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1479999999999997e-05 2024-05-11T23:02:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:64399 3.1e e48/47 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head v 48'297, mlcod=0'0)", 2024-05-11T23:02:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:02.104619+0000", 2024-05-11T23:02:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.893463208, 2024-05-11T23:02:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014840949000000001, 2024-05-11T23:02:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.104619+0000", 2024-05-11T23:02:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.104619+0000", 2024-05-11T23:02:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.104620+0000", 2024-05-11T23:02:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.299e-06 2024-05-11T23:02:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.112866+0000", 2024-05-11T23:02:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082450949999999992 2024-05-11T23:02:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.112893+0000", 2024-05-11T23:02:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7131000000000002e-05 2024-05-11T23:02:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.112902+0000", 2024-05-11T23:02:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4089999999999992e-06 2024-05-11T23:02:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.112980+0000", 2024-05-11T23:02:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8375999999999996e-05 2024-05-11T23:02:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.113009+0000", 2024-05-11T23:02:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8713000000000001e-05 2024-05-11T23:02:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.119403+0000", 2024-05-11T23:02:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0063940580000000002 2024-05-11T23:02:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.119460+0000", 2024-05-11T23:02:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6867999999999998e-05 2024-05-11T23:02:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "MOSDPGPushReply(3.1e 52/51 [PushReplyOp(3:7943f929:::rbd_data.111ef7a115a1.0000000000000050:head)])", 2024-05-11T23:02:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:06.114699+0000", 2024-05-11T23:02:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.883383444000003, 2024-05-11T23:02:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0064609460000000004, 2024-05-11T23:02:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114699+0000", 2024-05-11T23:02:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114699+0000", 2024-05-11T23:02:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114700+0000", 2024-05-11T23:02:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0780000000000001e-06 2024-05-11T23:02:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114706+0000", 2024-05-11T23:02:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4200000000000004e-06 2024-05-11T23:02:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114724+0000", 2024-05-11T23:02:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7142e-05 2024-05-11T23:02:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114732+0000", 2024-05-11T23:02:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7630000000000006e-06 2024-05-11T23:02:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.120983+0000", 2024-05-11T23:02:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0062507250000000004 2024-05-11T23:02:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.121160+0000", 2024-05-11T23:02:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000176818 2024-05-11T23:02:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:77128 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_get in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-11T23:02:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:02:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.982080659000001, 2024-05-11T23:02:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91737855899999998, 2024-05-11T23:02:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 77128 2024-05-11T23:02:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:02:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:02:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016003+0000", 2024-05-11T23:02:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5379999999999999e-06 2024-05-11T23:02:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016011+0000", 2024-05-11T23:02:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6410000000000002e-06 2024-05-11T23:02:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016026+0000", 2024-05-11T23:02:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4779e-05 2024-05-11T23:02:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016032+0000", 2024-05-11T23:02:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.066e-06 2024-05-11T23:02:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016093+0000", 2024-05-11T23:02:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0878000000000002e-05 2024-05-11T23:02:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:02:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016100+0000", 2024-05-11T23:02:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6799999999999993e-06 2024-05-11T23:02:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933302+0000", 2024-05-11T23:02:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91720135300000005 2024-05-11T23:02:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933380+0000", 2024-05-11T23:02:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8623999999999998e-05 2024-05-11T23:02:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:78890 3.3 3:c289f39e:::rbd_data.111ea74f268d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:16.008252+0000", 2024-05-11T23:02:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 27.989830519000002, 2024-05-11T23:02:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0070432100000000003, 2024-05-11T23:02:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 78890 2024-05-11T23:02:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.008252+0000", 2024-05-11T23:02:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.008252+0000", 2024-05-11T23:02:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.008253+0000", 2024-05-11T23:02:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-05-11T23:02:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009306+0000", 2024-05-11T23:02:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001053327 2024-05-11T23:02:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009329+0000", 2024-05-11T23:02:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3113999999999999e-05 2024-05-11T23:02:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009345+0000", 2024-05-11T23:02:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5136e-05 2024-05-11T23:02:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009447+0000", 2024-05-11T23:02:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010199799999999999 2024-05-11T23:02:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009640+0000", 2024-05-11T23:02:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019346000000000001 2024-05-11T23:02:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-05-11T23:02:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009832+0000", 2024-05-11T23:02:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019212799999999999 2024-05-11T23:02:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:02:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.014184+0000", 2024-05-11T23:02:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0043513909999999996 2024-05-11T23:02:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.015200+0000", 2024-05-11T23:02:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001016926 2024-05-11T23:02:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.015250+0000", 2024-05-11T23:02:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9162999999999997e-05 2024-05-11T23:02:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.015295+0000", 2024-05-11T23:02:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5425e-05 2024-05-11T23:02:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:97486 3.9 e71/19 3:966aeabc:::rbd_data.111ed003fd7f.0000000000000011:head v 71'192, mlcod=69'190)", 2024-05-11T23:02:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:25.042193+0000", 2024-05-11T23:02:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.955889524, 2024-05-11T23:02:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01046652, 2024-05-11T23:02:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:25.042193+0000", 2024-05-11T23:02:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:25.042193+0000", 2024-05-11T23:02:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:25.042193+0000", 2024-05-11T23:02:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.92e-07 2024-05-11T23:02:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:25.047425+0000", 2024-05-11T23:02:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.005232123 2024-05-11T23:02:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:25.047444+0000", 2024-05-11T23:02:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8805e-05 2024-05-11T23:02:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:25.047454+0000", 2024-05-11T23:02:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0361e-05 2024-05-11T23:02:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:25.047534+0000", 2024-05-11T23:02:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9466000000000001e-05 2024-05-11T23:02:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:25.047561+0000", 2024-05-11T23:02:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6956000000000001e-05 2024-05-11T23:02:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:25.052626+0000", 2024-05-11T23:02:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0050649170000000004 2024-05-11T23:02:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:25.052659+0000", 2024-05-11T23:02:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3699999999999999e-05 2024-05-11T23:02:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98950 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:02:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.865999769999998, 2024-05-11T23:02:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021419438999999998, 2024-05-11T23:02:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98950 2024-05-11T23:02:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:02:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:02:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:02:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.34e-07 2024-05-11T23:02:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132085+0000", 2024-05-11T23:02:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0049999999999999e-06 2024-05-11T23:02:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132093+0000", 2024-05-11T23:02:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6660000000000008e-06 2024-05-11T23:02:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132097+0000", 2024-05-11T23:02:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2590000000000002e-06 2024-05-11T23:02:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135114+0000", 2024-05-11T23:02:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003017573 2024-05-11T23:02:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153416+0000", 2024-05-11T23:02:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018301884000000001 2024-05-11T23:02:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153502+0000", 2024-05-11T23:02:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5717999999999995e-05 2024-05-11T23:02:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98959 3.1e 3:7807da99:::rbd_data.111ed2934718.000000000000006d:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:02:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.865507033, 2024-05-11T23:02:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020525928999999998, 2024-05-11T23:02:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98959 2024-05-11T23:02:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:02:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:02:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132576+0000", 2024-05-11T23:02:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4199999999999995e-07 2024-05-11T23:02:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132580+0000", 2024-05-11T23:02:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2910000000000001e-06 2024-05-11T23:02:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132588+0000", 2024-05-11T23:02:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7970000000000001e-06 2024-05-11T23:02:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132602+0000", 2024-05-11T23:02:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3743999999999999e-05 2024-05-11T23:02:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132655+0000", 2024-05-11T23:02:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2714999999999999e-05 2024-05-11T23:02:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132787+0000", 2024-05-11T23:02:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013273899999999999 2024-05-11T23:02:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153101+0000", 2024-05-11T23:02:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020313900999999999 2024-05-11T23:02:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98965 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:02:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.862983397000001, 2024-05-11T23:02:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018608983999999999, 2024-05-11T23:02:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98965 2024-05-11T23:02:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:02:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:02:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135100+0000", 2024-05-11T23:02:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.17e-06 2024-05-11T23:02:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135105+0000", 2024-05-11T23:02:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.899e-06 2024-05-11T23:02:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135122+0000", 2024-05-11T23:02:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7346999999999998e-05 2024-05-11T23:02:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135130+0000", 2024-05-11T23:02:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0639999999999994e-06 2024-05-11T23:02:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153533+0000", 2024-05-11T23:02:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018402753000000001 2024-05-11T23:02:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153615+0000", 2024-05-11T23:02:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1720999999999999e-05 2024-05-11T23:02:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153708+0000", 2024-05-11T23:02:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3029999999999995e-05 2024-05-11T23:02:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98966 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3760128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:02:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.862911271000002, 2024-05-11T23:02:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018716535999999999, 2024-05-11T23:02:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98966 2024-05-11T23:02:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:02:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:02:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135172+0000", 2024-05-11T23:02:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-05-11T23:02:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135175+0000", 2024-05-11T23:02:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-05-11T23:02:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135184+0000", 2024-05-11T23:02:44.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1840000000000005e-06 2024-05-11T23:02:44.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152903+0000", 2024-05-11T23:02:44.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017718357000000001 2024-05-11T23:02:44.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153736+0000", 2024-05-11T23:02:44.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00083324799999999997 2024-05-11T23:02:44.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153809+0000", 2024-05-11T23:02:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2995000000000002e-05 2024-05-11T23:02:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153888+0000", 2024-05-11T23:02:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8695999999999993e-05 2024-05-11T23:02:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "pg_update_log_missing_reply(3.1e epoch 72/53 rep_tid 104 lcod 72'363)", 2024-05-11T23:02:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:02:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.862848374999999, 2024-05-11T23:02:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017871951000000001, 2024-05-11T23:02:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:02:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:02:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135235+0000", 2024-05-11T23:02:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8899999999999999e-07 2024-05-11T23:02:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135238+0000", 2024-05-11T23:02:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.27e-06 2024-05-11T23:02:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152878+0000", 2024-05-11T23:02:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01763986 2024-05-11T23:02:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152896+0000", 2024-05-11T23:02:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8573e-05 2024-05-11T23:02:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152968+0000", 2024-05-11T23:02:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1780999999999996e-05 2024-05-11T23:02:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153106+0000", 2024-05-11T23:02:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013777799999999999 2024-05-11T23:02:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:99788 3.3 3:c495528a:::rbd_header.111ed2934718:head [watch reconnect cookie 139809507265552 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.200323+0000", 2024-05-11T23:02:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.797758952999999, 2024-05-11T23:02:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0064864149999999997, 2024-05-11T23:02:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 99788 2024-05-11T23:02:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.200323+0000", 2024-05-11T23:02:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.200323+0000", 2024-05-11T23:02:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.200324+0000", 2024-05-11T23:02:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8800000000000003e-07 2024-05-11T23:02:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.200326+0000", 2024-05-11T23:02:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2280000000000001e-06 2024-05-11T23:02:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.200333+0000", 2024-05-11T23:02:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7750000000000004e-06 2024-05-11T23:02:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.200339+0000", 2024-05-11T23:02:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2400000000000004e-06 2024-05-11T23:02:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.201371+0000", 2024-05-11T23:02:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001031786 2024-05-11T23:02:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.201459+0000", 2024-05-11T23:02:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7862999999999995e-05 2024-05-11T23:02:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.206810+0000", 2024-05-11T23:02:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0053510349999999996 2024-05-11T23:02:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:100474 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.235591+0000", 2024-05-11T23:02:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.762491349000001, 2024-05-11T23:02:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0066337169999999999, 2024-05-11T23:02:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 100474 2024-05-11T23:02:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.235591+0000", 2024-05-11T23:02:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.235591+0000", 2024-05-11T23:02:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.235591+0000", 2024-05-11T23:02:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1399999999999997e-07 2024-05-11T23:02:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.235598+0000", 2024-05-11T23:02:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3169999999999998e-06 2024-05-11T23:02:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.235604+0000", 2024-05-11T23:02:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4180000000000002e-06 2024-05-11T23:02:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.235609+0000", 2024-05-11T23:02:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0549999999999999e-06 2024-05-11T23:02:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.242005+0000", 2024-05-11T23:02:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0063958440000000004 2024-05-11T23:02:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.242113+0000", 2024-05-11T23:02:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107549 2024-05-11T23:02:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.242225+0000", 2024-05-11T23:02:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011212 2024-05-11T23:02:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:100788 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [sparse-read 11796480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.389416+0000", 2024-05-11T23:02:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.608665938000001, 2024-05-11T23:02:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0064764569999999997, 2024-05-11T23:02:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 100788 2024-05-11T23:02:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389416+0000", 2024-05-11T23:02:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389416+0000", 2024-05-11T23:02:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389417+0000", 2024-05-11T23:02:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.75e-07 2024-05-11T23:02:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389419+0000", 2024-05-11T23:02:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1969999999999999e-06 2024-05-11T23:02:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389425+0000", 2024-05-11T23:02:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2820000000000002e-06 2024-05-11T23:02:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389430+0000", 2024-05-11T23:02:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0379999999999999e-06 2024-05-11T23:02:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.395683+0000", 2024-05-11T23:02:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0062525009999999997 2024-05-11T23:02:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.395778+0000", 2024-05-11T23:02:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5305999999999999e-05 2024-05-11T23:02:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.395893+0000", 2024-05-11T23:02:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000114558 2024-05-11T23:02:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:100807 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [sparse-read 1835008~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.389437+0000", 2024-05-11T23:02:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.608645036999999, 2024-05-11T23:02:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.007388113, 2024-05-11T23:02:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 100807 2024-05-11T23:02:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389437+0000", 2024-05-11T23:02:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389437+0000", 2024-05-11T23:02:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389438+0000", 2024-05-11T23:02:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7399999999999998e-07 2024-05-11T23:02:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389441+0000", 2024-05-11T23:02:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7179999999999998e-06 2024-05-11T23:02:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389448+0000", 2024-05-11T23:02:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5919999999999997e-06 2024-05-11T23:02:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389453+0000", 2024-05-11T23:02:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:02:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.395935+0000", 2024-05-11T23:02:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0064818300000000001 2024-05-11T23:02:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.396027+0000", 2024-05-11T23:02:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2015999999999994e-05 2024-05-11T23:02:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.396825+0000", 2024-05-11T23:02:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00079866799999999999 2024-05-11T23:02:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:100809 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [sparse-read 3407872~196608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.389460+0000", 2024-05-11T23:02:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.608622667999999, 2024-05-11T23:02:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0080309130000000006, 2024-05-11T23:02:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 100809 2024-05-11T23:02:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389460+0000", 2024-05-11T23:02:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389460+0000", 2024-05-11T23:02:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389460+0000", 2024-05-11T23:02:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-05-11T23:02:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389463+0000", 2024-05-11T23:02:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.446e-06 2024-05-11T23:02:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389468+0000", 2024-05-11T23:02:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6890000000000001e-06 2024-05-11T23:02:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389473+0000", 2024-05-11T23:02:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5979999999999999e-06 2024-05-11T23:02:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.396887+0000", 2024-05-11T23:02:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0074143660000000004 2024-05-11T23:02:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.397017+0000", 2024-05-11T23:02:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013025299999999999 2024-05-11T23:02:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.397491+0000", 2024-05-11T23:02:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00047314500000000001 2024-05-11T23:02:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:100810 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.389480+0000", 2024-05-11T23:02:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.608602734000002, 2024-05-11T23:02:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0084564610000000002, 2024-05-11T23:02:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 100810 2024-05-11T23:02:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389480+0000", 2024-05-11T23:02:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389480+0000", 2024-05-11T23:02:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389480+0000", 2024-05-11T23:02:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:02:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389482+0000", 2024-05-11T23:02:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.294e-06 2024-05-11T23:02:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389488+0000", 2024-05-11T23:02:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0419999999999999e-06 2024-05-11T23:02:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389493+0000", 2024-05-11T23:02:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8029999999999996e-06 2024-05-11T23:02:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.397532+0000", 2024-05-11T23:02:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0080390170000000007 2024-05-11T23:02:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.397818+0000", 2024-05-11T23:02:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00028542200000000002 2024-05-11T23:02:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.397936+0000", 2024-05-11T23:02:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011846200000000001 2024-05-11T23:02:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:100811 3.3 3.76f4d2e3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.389500+0000", 2024-05-11T23:02:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.608582496, 2024-05-11T23:02:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0084682569999999999, 2024-05-11T23:02:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 100811 2024-05-11T23:02:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389500+0000", 2024-05-11T23:02:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389500+0000", 2024-05-11T23:02:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389500+0000", 2024-05-11T23:02:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7700000000000005e-07 2024-05-11T23:02:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389502+0000", 2024-05-11T23:02:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1619999999999998e-06 2024-05-11T23:02:44.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389509+0000", 2024-05-11T23:02:44.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.083e-06 2024-05-11T23:02:44.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389513+0000", 2024-05-11T23:02:44.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.707e-06 2024-05-11T23:02:44.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.397962+0000", 2024-05-11T23:02:44.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0084483300000000004 2024-05-11T23:02:44.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.397968+0000", 2024-05-11T23:02:44.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4980000000000003e-06 2024-05-11T23:02:44.075 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.075 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.075 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.075 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:100830 3.3 3.76f4d2e3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:44.075 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.389599+0000", 2024-05-11T23:02:44.075 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.608483197999998, 2024-05-11T23:02:44.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0083954479999999998, 2024-05-11T23:02:44.076 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.076 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.076 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:44.076 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.076 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.076 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.076 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 100830 2024-05-11T23:02:44.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.076 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389599+0000", 2024-05-11T23:02:44.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389599+0000", 2024-05-11T23:02:44.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389600+0000", 2024-05-11T23:02:44.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1100000000000005e-07 2024-05-11T23:02:44.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389604+0000", 2024-05-11T23:02:44.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0720000000000001e-06 2024-05-11T23:02:44.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389613+0000", 2024-05-11T23:02:44.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4369999999999996e-06 2024-05-11T23:02:44.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389620+0000", 2024-05-11T23:02:44.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2480000000000001e-06 2024-05-11T23:02:44.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.397989+0000", 2024-05-11T23:02:44.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.008368973 2024-05-11T23:02:44.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.397995+0000", 2024-05-11T23:02:44.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9070000000000004e-06 2024-05-11T23:02:44.079 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.079 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.079 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.080 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:139214 3.3 3:c4c3de8a:::rbd_data.111ec437b352.0000000000000162:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-05-11T23:02:44.080 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:43.168656+0000", 2024-05-11T23:02:44.080 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.82942601000000005, 2024-05-11T23:02:44.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0073981209999999997, 2024-05-11T23:02:44.080 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.080 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.080 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:44.080 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.080 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.080 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.080 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 139214 2024-05-11T23:02:44.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.080 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.168656+0000", 2024-05-11T23:02:44.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.168656+0000", 2024-05-11T23:02:44.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.168658+0000", 2024-05-11T23:02:44.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9429999999999999e-06 2024-05-11T23:02:44.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.168668+0000", 2024-05-11T23:02:44.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5829999999999996e-06 2024-05-11T23:02:44.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.168689+0000", 2024-05-11T23:02:44.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1012000000000001e-05 2024-05-11T23:02:44.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.168699+0000", 2024-05-11T23:02:44.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0277e-05 2024-05-11T23:02:44.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.168816+0000", 2024-05-11T23:02:44.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000116564 2024-05-11T23:02:44.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.168972+0000", 2024-05-11T23:02:44.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015664100000000001 2024-05-11T23:02:44.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-05-11T23:02:44.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.169611+0000", 2024-05-11T23:02:44.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00063818700000000002 2024-05-11T23:02:44.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:44.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.175899+0000", 2024-05-11T23:02:44.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0062883410000000002 2024-05-11T23:02:44.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:02:44.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.175958+0000", 2024-05-11T23:02:44.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8727999999999997e-05 2024-05-11T23:02:44.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:44.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.176009+0000", 2024-05-11T23:02:44.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1243999999999999e-05 2024-05-11T23:02:44.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.176054+0000", 2024-05-11T23:02:44.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5600999999999999e-05 2024-05-11T23:02:44.085 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.085 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.085 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.085 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.085 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.085 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:44.085 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-11T23:02:44.086 INFO:teuthology.orchestra.run.smithi005.stdout:4000 writesame 0xfc322e thru 0xfccba1 data_size 0x3d(0x9974 bytes) 2024-05-11T23:02:44.087 INFO:teuthology.orchestra.run.smithi005.stdout:4002 read 0x42aa05c thru 0x42b44b7 (0xa45c bytes) 2024-05-11T23:02:44.087 INFO:teuthology.orchestra.run.smithi005.stdout:4003 write 0x170d174 thru 0x1715150 (0x7fdd bytes) 2024-05-11T23:02:44.088 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:02:44.088 INFO:teuthology.orchestra.run.smithi005.stdout:4004 trunc from 0x4ae6dfe to 0xc6df5 2024-05-11T23:02:44.088 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:44.089 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:44.086+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:44.095 INFO:teuthology.orchestra.run.smithi005.stdout:4006 write 0x8b87686 thru 0x8b8cafd (0x5478 bytes) 2024-05-11T23:02:44.110 INFO:teuthology.orchestra.run.smithi005.stdout:4007 read 0x2979d1a thru 0x2988597 (0xe87e bytes) 2024-05-11T23:02:44.121 INFO:teuthology.orchestra.run.smithi005.stdout:4008 write 0xd7fe58a thru 0xd8092a4 (0xad1b bytes) 2024-05-11T23:02:44.144 INFO:teuthology.orchestra.run.smithi005.stdout:4009 trunc from 0xd8092a5 to 0x923ad1a 2024-05-11T23:02:44.172 INFO:teuthology.orchestra.run.smithi005.stdout:4011 write 0xc65fcff thru 0xc661eba (0x21bc bytes) 2024-05-11T23:02:44.184 INFO:teuthology.orchestra.run.smithi005.stdout:4012 write 0x76a5336 thru 0x76ad2b8 (0x7f83 bytes) 2024-05-11T23:02:44.187 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:44.187+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:44.203 INFO:teuthology.orchestra.run.smithi005.stdout:4013 punch from 0x7b298f9 to 0x7b33d78, (0xa47f bytes) 2024-05-11T23:02:44.203 INFO:teuthology.orchestra.run.smithi005.stdout:4014 write 0x9220da7 thru 0x9220f94 (0x1ee bytes) 2024-05-11T23:02:44.208 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:44.208 INFO:teuthology.orchestra.run.smithi005.stdout:4016 write 0x84f188a thru 0x84f7f36 (0x66ad bytes) 2024-05-11T23:02:44.222 INFO:teuthology.orchestra.run.smithi005.stdout:4019 read 0xc336400 thru 0xc3385a2 (0x21a3 bytes) 2024-05-11T23:02:44.223 INFO:teuthology.orchestra.run.smithi005.stdout:4020 write 0xc5cac73 thru 0xc5d6285 (0xb613 bytes) 2024-05-11T23:02:44.232 INFO:teuthology.orchestra.run.smithi005.stdout:4022 read 0x62a5f8 thru 0x637fab (0xd9b4 bytes) 2024-05-11T23:02:44.244 INFO:teuthology.orchestra.run.smithi005.stdout:4023 trunc from 0xc661ebb to 0x1ecd5df 2024-05-11T23:02:44.263 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:44.263 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:44.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.263 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:140993 3.15 e88/76 3:abd8f7f0:::rbd_data.111ec437b352.000000000000022d:head v 88'333, mlcod=88'331)", 2024-05-11T23:02:44.263 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:44.259872+0000", 2024-05-11T23:02:44.263 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0012178779999999999, 2024-05-11T23:02:44.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0012354989999999999, 2024-05-11T23:02:44.263 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.264 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:44.264 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:44.259872+0000", 2024-05-11T23:02:44.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:44.259872+0000", 2024-05-11T23:02:44.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:44.259873+0000", 2024-05-11T23:02:44.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2900000000000002e-07 2024-05-11T23:02:44.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:44.259880+0000", 2024-05-11T23:02:44.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0600000000000002e-06 2024-05-11T23:02:44.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:44.259891+0000", 2024-05-11T23:02:44.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1109e-05 2024-05-11T23:02:44.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:44.259898+0000", 2024-05-11T23:02:44.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4840000000000001e-06 2024-05-11T23:02:44.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:44.259976+0000", 2024-05-11T23:02:44.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8226000000000006e-05 2024-05-11T23:02:44.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:44.259996+0000", 2024-05-11T23:02:44.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0058999999999999e-05 2024-05-11T23:02:44.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:44.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:44.261068+0000", 2024-05-11T23:02:44.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0010717190000000001 2024-05-11T23:02:44.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:44.261108+0000", 2024-05-11T23:02:44.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0012999999999998e-05 2024-05-11T23:02:44.267 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.267 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.268 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.268 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.268 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:02:44.268 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 1 2024-05-11T23:02:44.268 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:44.273 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:02:44.276 INFO:teuthology.orchestra.run.smithi005.stdout:4024 read 0x5215fb thru 0x528033 (0x6a39 bytes) 2024-05-11T23:02:44.276 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:44.287 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:44.287+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:44.290 INFO:teuthology.orchestra.run.smithi005.stdout:4026 write 0x91820e6 thru 0x9185760 (0x367b bytes) 2024-05-11T23:02:44.309 INFO:teuthology.orchestra.run.smithi005.stdout:4027 write 0x9070db8 thru 0x907c41c (0xb665 bytes) 2024-05-11T23:02:44.317 INFO:teuthology.orchestra.run.smithi005.stdout:4029 write 0x3de8a99 thru 0x3df1107 (0x866f bytes) 2024-05-11T23:02:44.334 INFO:teuthology.orchestra.run.smithi005.stdout:4030 write 0x6e6bbbc thru 0x6e72060 (0x64a5 bytes) 2024-05-11T23:02:44.350 INFO:teuthology.orchestra.run.smithi005.stdout:4032 punch from 0x8dd6a8 to 0x8e622c, (0x8b84 bytes) 2024-05-11T23:02:44.351 INFO:teuthology.orchestra.run.smithi005.stdout:4033 writesame 0x6d5d925 thru 0x6d652fe data_size 0x9(0x79da bytes) 2024-05-11T23:02:44.356 INFO:teuthology.orchestra.run.smithi005.stdout:4035 write 0x4dcf730 thru 0x4ddb31e (0xbbef bytes) 2024-05-11T23:02:44.375 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:44.377 INFO:teuthology.orchestra.run.smithi005.stdout:4038 trunc from 0x9185761 to 0xb129af2 2024-05-11T23:02:44.383 INFO:teuthology.orchestra.run.smithi005.stdout:4041 read 0xaa0cda9 thru 0xaa152ef (0x8547 bytes) 2024-05-11T23:02:44.383 INFO:teuthology.orchestra.run.smithi005.stdout:4042 write 0x8513098 thru 0x851fe8b (0xcdf4 bytes) 2024-05-11T23:02:44.387 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:44.387+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:44.411 INFO:teuthology.orchestra.run.smithi005.stdout:4043 write 0xda167ee thru 0xda181fe (0x1a11 bytes) 2024-05-11T23:02:44.414 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:44.414 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:44.421 INFO:teuthology.orchestra.run.smithi005.stdout:4044 write 0x18fa88d thru 0x19082d7 (0xda4b bytes) 2024-05-11T23:02:44.426 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:02:44.450 INFO:teuthology.orchestra.run.smithi005.stdout:4045 trunc from 0xda181ff to 0x2b967c1 2024-05-11T23:02:44.483 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:44.483 INFO:teuthology.orchestra.run.smithi005.stdout:4048 read 0xb31bb4 thru 0xb3a641 (0x8a8e bytes) 2024-05-11T23:02:44.484 INFO:teuthology.orchestra.run.smithi005.stdout:4049 punch from 0x4c8206 to 0x4cae5b, (0x2c55 bytes) 2024-05-11T23:02:44.489 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:44.488+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:44.494 INFO:teuthology.orchestra.run.smithi005.stdout:4050 writesame 0x7690dae thru 0x769eeb2 data_size 0x29(0xe105 bytes) 2024-05-11T23:02:44.494 INFO:teuthology.orchestra.run.smithi005.stdout:4052 punch from 0x520151f to 0x5202c5a, (0x173b bytes) 2024-05-11T23:02:44.495 INFO:teuthology.orchestra.run.smithi005.stdout:4053 punch from 0x3fc66a3 to 0x3fd207b, (0xb9d8 bytes) 2024-05-11T23:02:44.503 INFO:teuthology.orchestra.run.smithi005.stdout:4054 writesame 0x922e583 thru 0x922fd10 data_size 0x12(0x178e bytes) 2024-05-11T23:02:44.503 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:44.503 INFO:teuthology.orchestra.run.smithi005.stdout:4056 punch from 0xec5094 to 0xecf0d5, (0xa041 bytes) 2024-05-11T23:02:44.503 INFO:teuthology.orchestra.run.smithi005.stdout:4057 read 0x7c61d3d thru 0x7c6f177 (0xd43b bytes) 2024-05-11T23:02:44.503 INFO:teuthology.orchestra.run.smithi005.stdout:4058 write 0x387fdcd thru 0x3885d59 (0x5f8d bytes) 2024-05-11T23:02:44.517 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:44.517 INFO:teuthology.orchestra.run.smithi005.stdout:4060 writesame 0x765c1d8 thru 0x766669c data_size 0x1(0xa4c5 bytes) 2024-05-11T23:02:44.539 INFO:teuthology.orchestra.run.smithi005.stdout:4061 write 0x93931a7 thru 0x939e42b (0xb285 bytes) 2024-05-11T23:02:44.558 INFO:teuthology.orchestra.run.smithi005.stdout:4063 punch from 0x8f12ed3 to 0x8f17f09, (0x5036 bytes) 2024-05-11T23:02:44.559 INFO:teuthology.orchestra.run.smithi005.stdout:4064 write 0x77aba14 thru 0x77ad6da (0x1cc7 bytes) 2024-05-11T23:02:44.584 INFO:teuthology.orchestra.run.smithi005.stdout:4065 write 0xd34f2f4 thru 0xd355b3e (0x684b bytes) 2024-05-11T23:02:44.589 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36828 3.b 3:d385bc8e:::rbd_data.111e5e4d8275.0000000000000020:11 [read 512000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.655487088000001, 2024-05-11T23:02:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051481218000000002, 2024-05-11T23:02:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36828 2024-05-11T23:02:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934915+0000", 2024-05-11T23:02:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00181814 2024-05-11T23:02:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934919+0000", 2024-05-11T23:02:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6739999999999996e-06 2024-05-11T23:02:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934934+0000", 2024-05-11T23:02:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4415e-05 2024-05-11T23:02:44.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934942+0000", 2024-05-11T23:02:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8619999999999992e-06 2024-05-11T23:02:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984433+0000", 2024-05-11T23:02:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049490658999999999 2024-05-11T23:02:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984508+0000", 2024-05-11T23:02:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4394999999999995e-05 2024-05-11T23:02:44.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984578+0000", 2024-05-11T23:02:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0073000000000005e-05 2024-05-11T23:02:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36950 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 520192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.651181321999999, 2024-05-11T23:02:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051450856000000003, 2024-05-11T23:02:44.594 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36950 2024-05-11T23:02:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938914+0000", 2024-05-11T23:02:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015121480000000001 2024-05-11T23:02:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938918+0000", 2024-05-11T23:02:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4149999999999999e-06 2024-05-11T23:02:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938925+0000", 2024-05-11T23:02:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0779999999999998e-06 2024-05-11T23:02:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938929+0000", 2024-05-11T23:02:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2930000000000002e-06 2024-05-11T23:02:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988675+0000", 2024-05-11T23:02:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049746394999999999 2024-05-11T23:02:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988756+0000", 2024-05-11T23:02:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0499000000000003e-05 2024-05-11T23:02:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988853+0000", 2024-05-11T23:02:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7027999999999993e-05 2024-05-11T23:02:44.598 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36957 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 778240~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.649559533000001, 2024-05-11T23:02:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051328273000000001, 2024-05-11T23:02:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.599 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36957 2024-05-11T23:02:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2099999999999998e-07 2024-05-11T23:02:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939026+0000", 2024-05-11T23:02:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5069999999999999e-06 2024-05-11T23:02:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939030+0000", 2024-05-11T23:02:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7919999999999999e-06 2024-05-11T23:02:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939032+0000", 2024-05-11T23:02:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9129999999999999e-06 2024-05-11T23:02:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990178+0000", 2024-05-11T23:02:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051145373000000001 2024-05-11T23:02:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990255+0000", 2024-05-11T23:02:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7195999999999997e-05 2024-05-11T23:02:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990352+0000", 2024-05-11T23:02:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7170999999999995e-05 2024-05-11T23:02:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36958 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 815104~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:44.603 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.649547214000002, 2024-05-11T23:02:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051531625999999997, 2024-05-11T23:02:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36958 2024-05-11T23:02:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:44.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939037+0000", 2024-05-11T23:02:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0400000000000002e-07 2024-05-11T23:02:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939038+0000", 2024-05-11T23:02:44.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.164e-06 2024-05-11T23:02:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939041+0000", 2024-05-11T23:02:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1109999999999999e-06 2024-05-11T23:02:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939043+0000", 2024-05-11T23:02:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.554e-06 2024-05-11T23:02:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990382+0000", 2024-05-11T23:02:44.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051338953999999999 2024-05-11T23:02:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990465+0000", 2024-05-11T23:02:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2461999999999998e-05 2024-05-11T23:02:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990568+0000", 2024-05-11T23:02:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000103077 2024-05-11T23:02:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36959 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 851968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.649535317999998, 2024-05-11T23:02:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051758936999999998, 2024-05-11T23:02:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36959 2024-05-11T23:02:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.16e-07 2024-05-11T23:02:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939050+0000", 2024-05-11T23:02:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.176e-06 2024-05-11T23:02:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939053+0000", 2024-05-11T23:02:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1140000000000002e-06 2024-05-11T23:02:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939055+0000", 2024-05-11T23:02:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.621e-06 2024-05-11T23:02:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990610+0000", 2024-05-11T23:02:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051554244999999999 2024-05-11T23:02:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990700+0000", 2024-05-11T23:02:44.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0284999999999994e-05 2024-05-11T23:02:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990807+0000", 2024-05-11T23:02:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010728 2024-05-11T23:02:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36960 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 888832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.649525083, 2024-05-11T23:02:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051886831000000001, 2024-05-11T23:02:44.612 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36960 2024-05-11T23:02:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940913+0000", 2024-05-11T23:02:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001855104 2024-05-11T23:02:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940916+0000", 2024-05-11T23:02:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0000000000000001e-06 2024-05-11T23:02:44.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940924+0000", 2024-05-11T23:02:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0999999999999998e-06 2024-05-11T23:02:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940927+0000", 2024-05-11T23:02:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7160000000000001e-06 2024-05-11T23:02:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990826+0000", 2024-05-11T23:02:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049898832999999997 2024-05-11T23:02:44.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990872+0000", 2024-05-11T23:02:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5717000000000003e-05 2024-05-11T23:02:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990945+0000", 2024-05-11T23:02:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3361000000000005e-05 2024-05-11T23:02:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36986 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 528384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:44.616 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.647518865000002, 2024-05-11T23:02:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051341053999999997, 2024-05-11T23:02:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36986 2024-05-11T23:02:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:02:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941066+0000", 2024-05-11T23:02:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1629999999999999e-06 2024-05-11T23:02:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941069+0000", 2024-05-11T23:02:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.118e-06 2024-05-11T23:02:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941072+0000", 2024-05-11T23:02:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5720000000000001e-06 2024-05-11T23:02:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992292+0000", 2024-05-11T23:02:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051220620000000001 2024-05-11T23:02:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992344+0000", 2024-05-11T23:02:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1737000000000003e-05 2024-05-11T23:02:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992406+0000", 2024-05-11T23:02:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1623000000000003e-05 2024-05-11T23:02:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36987 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 565248~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.647508744, 2024-05-11T23:02:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051568725000000003, 2024-05-11T23:02:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36987 2024-05-11T23:02:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.621 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.91e-07 2024-05-11T23:02:44.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941076+0000", 2024-05-11T23:02:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1379999999999999e-06 2024-05-11T23:02:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941079+0000", 2024-05-11T23:02:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9739999999999998e-06 2024-05-11T23:02:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941082+0000", 2024-05-11T23:02:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-05-11T23:02:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992446+0000", 2024-05-11T23:02:44.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051364058999999997 2024-05-11T23:02:44.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992536+0000", 2024-05-11T23:02:44.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0085000000000002e-05 2024-05-11T23:02:44.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992643+0000", 2024-05-11T23:02:44.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107377 2024-05-11T23:02:44.624 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.625 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.625 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.625 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36988 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 602112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:44.625 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:44.625 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.647498374999998, 2024-05-11T23:02:44.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051700007999999999, 2024-05-11T23:02:44.625 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.625 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.625 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.625 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.625 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.625 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.625 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36988 2024-05-11T23:02:44.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.626 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:44.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:44.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:44.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.05e-07 2024-05-11T23:02:44.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941086+0000", 2024-05-11T23:02:44.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.125e-06 2024-05-11T23:02:44.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941089+0000", 2024-05-11T23:02:44.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0599999999999999e-06 2024-05-11T23:02:44.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941092+0000", 2024-05-11T23:02:44.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6529999999999998e-06 2024-05-11T23:02:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992667+0000", 2024-05-11T23:02:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051574587999999998 2024-05-11T23:02:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992723+0000", 2024-05-11T23:02:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6682999999999999e-05 2024-05-11T23:02:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992785+0000", 2024-05-11T23:02:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1693999999999996e-05 2024-05-11T23:02:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.629 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.629 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.629 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36989 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 638976~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:44.629 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:44.629 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.647488185999997, 2024-05-11T23:02:44.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05192811, 2024-05-11T23:02:44.629 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.629 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.629 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.629 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.629 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.629 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.629 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36989 2024-05-11T23:02:44.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.630 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:44.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:44.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942916+0000", 2024-05-11T23:02:44.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001820835 2024-05-11T23:02:44.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942920+0000", 2024-05-11T23:02:44.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5310000000000002e-06 2024-05-11T23:02:44.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942931+0000", 2024-05-11T23:02:44.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1839e-05 2024-05-11T23:02:44.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942940+0000", 2024-05-11T23:02:44.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2309999999999996e-06 2024-05-11T23:02:44.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992820+0000", 2024-05-11T23:02:44.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049880751000000001 2024-05-11T23:02:44.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992918+0000", 2024-05-11T23:02:44.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7051000000000003e-05 2024-05-11T23:02:44.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.993023+0000", 2024-05-11T23:02:44.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010587200000000001 2024-05-11T23:02:44.633 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.633 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.633 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.633 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36997 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 933888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:44.633 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:44.633 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.645454350999998, 2024-05-11T23:02:44.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051390602000000001, 2024-05-11T23:02:44.633 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.633 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.633 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.633 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.633 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.634 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.634 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36997 2024-05-11T23:02:44.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.634 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:44.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:44.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943130+0000", 2024-05-11T23:02:44.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0200000000000003e-07 2024-05-11T23:02:44.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943132+0000", 2024-05-11T23:02:44.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.305e-06 2024-05-11T23:02:44.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943138+0000", 2024-05-11T23:02:44.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4219999999999997e-06 2024-05-11T23:02:44.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943143+0000", 2024-05-11T23:02:44.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0610000000000003e-06 2024-05-11T23:02:44.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994323+0000", 2024-05-11T23:02:44.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051179592000000003 2024-05-11T23:02:44.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994412+0000", 2024-05-11T23:02:44.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8709999999999996e-05 2024-05-11T23:02:44.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994520+0000", 2024-05-11T23:02:44.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010811000000000001 2024-05-11T23:02:44.637 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.637 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.637 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.637 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36998 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 970752~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:44.637 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:44.637 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.645433472000001, 2024-05-11T23:02:44.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051517669000000002, 2024-05-11T23:02:44.637 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.637 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.637 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.637 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.638 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.638 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.638 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36998 2024-05-11T23:02:44.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.638 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:44.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:44.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:44.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:02:44.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943153+0000", 2024-05-11T23:02:44.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.306e-06 2024-05-11T23:02:44.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943160+0000", 2024-05-11T23:02:44.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.452e-06 2024-05-11T23:02:44.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943165+0000", 2024-05-11T23:02:44.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.152e-06 2024-05-11T23:02:44.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994561+0000", 2024-05-11T23:02:44.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051395765000000003 2024-05-11T23:02:44.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994607+0000", 2024-05-11T23:02:44.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6415000000000002e-05 2024-05-11T23:02:44.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994668+0000", 2024-05-11T23:02:44.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0179000000000001e-05 2024-05-11T23:02:44.641 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.641 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.641 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.641 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36999 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 1007616~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:44.641 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:44.641 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.645388591, 2024-05-11T23:02:44.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051696919000000001, 2024-05-11T23:02:44.641 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.641 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.642 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.642 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.642 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.642 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.642 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36999 2024-05-11T23:02:44.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.642 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:44.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:44.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:44.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:02:44.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943197+0000", 2024-05-11T23:02:44.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1280000000000002e-06 2024-05-11T23:02:44.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943202+0000", 2024-05-11T23:02:44.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8910000000000003e-06 2024-05-11T23:02:44.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943206+0000", 2024-05-11T23:02:44.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3550000000000001e-06 2024-05-11T23:02:44.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994703+0000", 2024-05-11T23:02:44.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051497045999999998 2024-05-11T23:02:44.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994789+0000", 2024-05-11T23:02:44.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6458000000000005e-05 2024-05-11T23:02:44.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994892+0000", 2024-05-11T23:02:44.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102705 2024-05-11T23:02:44.645 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.645 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.645 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.645 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37000 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 1044480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:44.645 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:44.646 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.645372645999998, 2024-05-11T23:02:44.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051842371999999998, 2024-05-11T23:02:44.646 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.646 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.646 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.646 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.646 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.646 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.646 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37000 2024-05-11T23:02:44.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.646 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:44.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:44.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944916+0000", 2024-05-11T23:02:44.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001705097 2024-05-11T23:02:44.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944921+0000", 2024-05-11T23:02:44.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5809999999999998e-06 2024-05-11T23:02:44.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944930+0000", 2024-05-11T23:02:44.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5529999999999998e-06 2024-05-11T23:02:44.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944938+0000", 2024-05-11T23:02:44.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2519999999999999e-06 2024-05-11T23:02:44.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994911+0000", 2024-05-11T23:02:44.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049973113999999999 2024-05-11T23:02:44.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994982+0000", 2024-05-11T23:02:44.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0544999999999996e-05 2024-05-11T23:02:44.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.995053+0000", 2024-05-11T23:02:44.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1229999999999994e-05 2024-05-11T23:02:44.649 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.649 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.649 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.649 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37041 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 618496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:44.649 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:44.650 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.643528666999998, 2024-05-11T23:02:44.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051352768999999999, 2024-05-11T23:02:44.650 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.650 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.650 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.650 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.650 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.650 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37041 2024-05-11T23:02:44.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.650 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:44.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:44.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:44.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:02:44.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945056+0000", 2024-05-11T23:02:44.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:02:44.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945059+0000", 2024-05-11T23:02:44.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.965e-06 2024-05-11T23:02:44.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945062+0000", 2024-05-11T23:02:44.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:02:44.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996213+0000", 2024-05-11T23:02:44.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051151532999999999 2024-05-11T23:02:44.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996301+0000", 2024-05-11T23:02:44.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8143e-05 2024-05-11T23:02:44.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996408+0000", 2024-05-11T23:02:44.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010623700000000001 2024-05-11T23:02:44.653 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.653 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.653 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.653 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37042 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 655360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:44.653 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:44.654 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.643518682, 2024-05-11T23:02:44.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051462075000000003, 2024-05-11T23:02:44.654 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.654 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.654 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.654 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.654 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.654 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.654 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37042 2024-05-11T23:02:44.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.654 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:44.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0599999999999999e-07 2024-05-11T23:02:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945066+0000", 2024-05-11T23:02:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-05-11T23:02:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945069+0000", 2024-05-11T23:02:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0079999999999998e-06 2024-05-11T23:02:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945072+0000", 2024-05-11T23:02:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6740000000000001e-06 2024-05-11T23:02:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996422+0000", 2024-05-11T23:02:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051349999 2024-05-11T23:02:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996467+0000", 2024-05-11T23:02:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.528e-05 2024-05-11T23:02:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996527+0000", 2024-05-11T23:02:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9759999999999997e-05 2024-05-11T23:02:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.658 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.658 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37043 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 692224~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:44.658 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:44.658 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.643506608999999, 2024-05-11T23:02:44.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051679535999999998, 2024-05-11T23:02:44.658 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.658 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.658 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.658 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.658 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.658 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37043 2024-05-11T23:02:44.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.659 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:44.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:44.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:44.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8999999999999997e-07 2024-05-11T23:02:44.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945079+0000", 2024-05-11T23:02:44.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.001e-06 2024-05-11T23:02:44.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945087+0000", 2024-05-11T23:02:44.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1749999999999999e-06 2024-05-11T23:02:44.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945093+0000", 2024-05-11T23:02:44.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9639999999999997e-06 2024-05-11T23:02:44.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996563+0000", 2024-05-11T23:02:44.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051469050000000002 2024-05-11T23:02:44.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996651+0000", 2024-05-11T23:02:44.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8356000000000006e-05 2024-05-11T23:02:44.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996756+0000", 2024-05-11T23:02:44.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0001055 2024-05-11T23:02:44.661 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.661 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.662 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.662 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37044 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 729088~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:44.662 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:44.662 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.643472697, 2024-05-11T23:02:44.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051763445999999998, 2024-05-11T23:02:44.662 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.662 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.662 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.662 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.662 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.662 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.662 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37044 2024-05-11T23:02:44.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.663 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:44.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:44.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946915+0000", 2024-05-11T23:02:44.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018042760000000001 2024-05-11T23:02:44.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946918+0000", 2024-05-11T23:02:44.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.36e-06 2024-05-11T23:02:44.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946926+0000", 2024-05-11T23:02:44.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3130000000000001e-06 2024-05-11T23:02:44.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946930+0000", 2024-05-11T23:02:44.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5249999999999999e-06 2024-05-11T23:02:44.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996772+0000", 2024-05-11T23:02:44.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049841868999999997 2024-05-11T23:02:44.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996817+0000", 2024-05-11T23:02:44.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5351e-05 2024-05-11T23:02:44.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996874+0000", 2024-05-11T23:02:44.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6752000000000001e-05 2024-05-11T23:02:44.665 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.665 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.666 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.666 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37052 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 1024000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:44.666 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:44.666 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.641565845999999, 2024-05-11T23:02:44.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051326140999999999, 2024-05-11T23:02:44.666 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.666 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.666 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.666 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.666 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.666 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37052 2024-05-11T23:02:44.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.666 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:44.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:44.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:44.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:44.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947019+0000", 2024-05-11T23:02:44.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.218e-06 2024-05-11T23:02:44.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947022+0000", 2024-05-11T23:02:44.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1889999999999999e-06 2024-05-11T23:02:44.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947025+0000", 2024-05-11T23:02:44.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.464e-06 2024-05-11T23:02:44.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998237+0000", 2024-05-11T23:02:44.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051212494999999997 2024-05-11T23:02:44.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998284+0000", 2024-05-11T23:02:44.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7246000000000001e-05 2024-05-11T23:02:44.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998344+0000", 2024-05-11T23:02:44.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9320000000000001e-05 2024-05-11T23:02:44.669 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.669 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.669 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.670 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37071 3.b 3:d385bc8e:::rbd_data.111e5e4d8275.0000000000000020:11 [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:44.670 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:44.670 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.641555637000003, 2024-05-11T23:02:44.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051528796000000002, 2024-05-11T23:02:44.670 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:44.670 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:44.670 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:44.670 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:44.670 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:44.670 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:44.670 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37071 2024-05-11T23:02:44.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.671 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:44.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:44.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:44.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:44.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:44.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:44.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:44.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948913+0000", 2024-05-11T23:02:44.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018854970000000001 2024-05-11T23:02:44.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:44.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948917+0000", 2024-05-11T23:02:44.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8089999999999999e-06 2024-05-11T23:02:44.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:44.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948926+0000", 2024-05-11T23:02:44.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8389999999999996e-06 2024-05-11T23:02:44.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:44.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948930+0000", 2024-05-11T23:02:44.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4519999999999999e-06 2024-05-11T23:02:44.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:44.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998379+0000", 2024-05-11T23:02:44.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049449011000000001 2024-05-11T23:02:44.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:44.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998462+0000", 2024-05-11T23:02:44.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2739999999999997e-05 2024-05-11T23:02:44.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:44.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:44.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:44.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998557+0000", 2024-05-11T23:02:44.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4448000000000001e-05 2024-05-11T23:02:44.674 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.674 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.674 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.674 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:44.674 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:44.675 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:44.675 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:44.589+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:44.676 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:02:44.677 INFO:teuthology.orchestra.run.smithi005.stdout:4067 write 0x8264391 thru 0x826694a (0x25ba bytes) 2024-05-11T23:02:44.677 INFO:teuthology.orchestra.run.smithi005.stdout:4068 read 0x611ef1c thru 0x612c743 (0xd828 bytes) 2024-05-11T23:02:44.677 INFO:teuthology.orchestra.run.smithi005.stdout:4071 write 0x9043b84 thru 0x90491df (0x565c bytes) 2024-05-11T23:02:44.677 INFO:teuthology.orchestra.run.smithi005.stdout:4072 trunc from 0xd355b3f to 0x9bde554 2024-05-11T23:02:44.677 INFO:teuthology.orchestra.run.smithi005.stdout:4073 write 0x8b41d02 thru 0x8b4899e (0x6c9d bytes) 2024-05-11T23:02:44.677 INFO:teuthology.orchestra.run.smithi005.stdout:4074 read 0x787d79b thru 0x7889979 (0xc1df bytes) 2024-05-11T23:02:44.678 INFO:teuthology.orchestra.run.smithi005.stdout:4075 read 0x7117591 thru 0x711f188 (0x7bf8 bytes) 2024-05-11T23:02:44.678 INFO:teuthology.orchestra.run.smithi005.stdout:4076 writesame 0x5d170f0 thru 0x5d21908 data_size 0x1(0xa819 bytes) 2024-05-11T23:02:44.684 INFO:teuthology.orchestra.run.smithi005.stdout:4077 read 0x9a92ae5 thru 0x9a981ec (0x5708 bytes) 2024-05-11T23:02:44.684 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:44.684 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:44.684 INFO:teuthology.orchestra.run.smithi005.stdout:4080 punch from 0x7e449f0 to 0x7e44e7b, (0x48b bytes) 2024-05-11T23:02:44.689 INFO:teuthology.orchestra.run.smithi005.stdout:4081 trunc from 0x9bde554 to 0xe750a43 2024-05-11T23:02:44.695 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:44.695 INFO:teuthology.orchestra.run.smithi005.stdout:4084 punch from 0xb4579f0 to 0xb4635f2, (0xbc02 bytes) 2024-05-11T23:02:44.695 INFO:teuthology.orchestra.run.smithi005.stdout:4085 writesame 0xa1b8b4c thru 0xa1bcf3a data_size 0x5d(0x43ef bytes) 2024-05-11T23:02:44.715 INFO:teuthology.orchestra.run.smithi005.stdout:4086 write 0xe703b70 thru 0xe7076dd (0x3b6e bytes) 2024-05-11T23:02:44.722 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:44.721+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:44.736 INFO:teuthology.orchestra.run.smithi005.stdout:4087 punch from 0x280f0c3 to 0x2811642, (0x257f bytes) 2024-05-11T23:02:44.736 INFO:teuthology.orchestra.run.smithi005.stdout:4088 read 0xb9f1c07 thru 0xb9f3e57 (0x2251 bytes) 2024-05-11T23:02:44.736 INFO:teuthology.orchestra.run.smithi005.stdout:4089 read 0xa5c9f9f thru 0xa5cc086 (0x20e8 bytes) 2024-05-11T23:02:44.736 INFO:teuthology.orchestra.run.smithi005.stdout:4090 punch from 0x6812caa to 0x681b4bd, (0x8813 bytes) 2024-05-11T23:02:44.737 INFO:teuthology.orchestra.run.smithi005.stdout:4091 read 0x54151ed thru 0x54199ea (0x47fe bytes) 2024-05-11T23:02:44.737 INFO:teuthology.orchestra.run.smithi005.stdout:4092 writesame 0x1fe6b2f thru 0x1fed0fa data_size 0xa(0x65cc bytes) 2024-05-11T23:02:44.751 INFO:teuthology.orchestra.run.smithi005.stdout:4093 punch from 0x5cd1a3d to 0x5cde472, (0xca35 bytes) 2024-05-11T23:02:44.752 INFO:teuthology.orchestra.run.smithi005.stdout:4094 write 0x1eaf1a thru 0x1f166c (0x6753 bytes) 2024-05-11T23:02:44.754 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:02:44.755 INFO:teuthology.orchestra.run.smithi005.stdout:4095 read 0x97a45f2 thru 0x97aa197 (0x5ba6 bytes) 2024-05-11T23:02:44.755 INFO:teuthology.orchestra.run.smithi005.stdout:4096 write 0x95f270b thru 0x95fc104 (0x99fa bytes) 2024-05-11T23:02:44.758 INFO:teuthology.orchestra.run.smithi005.stdout:4097 read 0x615d5eb thru 0x6164be4 (0x75fa bytes) 2024-05-11T23:02:44.767 INFO:teuthology.orchestra.run.smithi005.stdout:4098 trunc from 0xe750a43 to 0x6ca3726 2024-05-11T23:02:44.790 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:44.790+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:44.801 INFO:teuthology.orchestra.run.smithi005.stdout:4099 punch from 0x6961f89 to 0x69682cb, (0x6342 bytes) 2024-05-11T23:02:44.801 INFO:teuthology.orchestra.run.smithi005.stdout:4101 punch from 0x11ac0b5 to 0x11ace1c, (0xd67 bytes) 2024-05-11T23:02:44.801 INFO:teuthology.orchestra.run.smithi005.stdout:4103 punch from 0x14da80 to 0x153f0f, (0x648f bytes) 2024-05-11T23:02:44.802 INFO:teuthology.orchestra.run.smithi005.stdout:4105 flatten 2024-05-11T23:02:44.808 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:44.808 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:44.808 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:44.809 INFO:teuthology.orchestra.run.smithi005.stdout:4106 read 0x55f20bd thru 0x560047a (0xe3be bytes) 2024-05-11T23:02:44.809 INFO:teuthology.orchestra.run.smithi005.stdout:4107 punch from 0x1d84ed9 to 0x1d863d3, (0x14fa bytes) 2024-05-11T23:02:44.818 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:02:44.819 INFO:teuthology.orchestra.run.smithi005.stdout:4108 write 0x96137be thru 0x9620011 (0xc854 bytes) 2024-05-11T23:02:44.835 INFO:teuthology.orchestra.run.smithi005.stdout:4109 read 0x1111cb1 thru 0x111ba11 (0x9d61 bytes) 2024-05-11T23:02:44.835 INFO:teuthology.orchestra.run.smithi005.stdout:4110 read 0x5ee0551 thru 0x5ee23bf (0x1e6f bytes) 2024-05-11T23:02:44.835 INFO:teuthology.orchestra.run.smithi005.stdout:4111 write 0x740fde6 thru 0x7413b26 (0x3d41 bytes) 2024-05-11T23:02:44.852 INFO:teuthology.orchestra.run.smithi005.stdout:4112 trunc from 0x9620012 to 0x83b10b3 2024-05-11T23:02:44.865 INFO:teuthology.orchestra.run.smithi005.stdout:4113 punch from 0xa02d27 to 0xa0c5cb, (0x98a4 bytes) 2024-05-11T23:02:44.865 INFO:teuthology.orchestra.run.smithi005.stdout:4115 write 0x66aa0f2 thru 0x66acf17 (0x2e26 bytes) 2024-05-11T23:02:44.879 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:44.892 INFO:teuthology.orchestra.run.smithi005.stdout:4117 writesame 0xd7ffc64 thru 0xd80da03 data_size 0x12(0xdda0 bytes) 2024-05-11T23:02:44.893 INFO:teuthology.orchestra.run.smithi005.stdout:4120 punch from 0x4998363 to 0x49996ef, (0x138c bytes) 2024-05-11T23:02:44.893 INFO:teuthology.orchestra.run.smithi005.stdout:4123 read 0x815f318 thru 0x816d5a7 (0xe290 bytes) 2024-05-11T23:02:44.893 INFO:teuthology.orchestra.run.smithi005.stdout:4124 punch from 0x7ee969c to 0x7ef353b, (0x9e9f bytes) 2024-05-11T23:02:44.893 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:44.893 INFO:teuthology.orchestra.run.smithi005.stdout:4128 write 0xca4aa2e thru 0xca51037 (0x660a bytes) 2024-05-11T23:02:44.902 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:44.902+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:44.913 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:44.913 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:44.915 INFO:teuthology.orchestra.run.smithi005.stdout:4129 trunc from 0xd80da04 to 0x9f752a2 2024-05-11T23:02:44.930 INFO:teuthology.orchestra.run.smithi005.stdout:4130 punch from 0x666265e to 0x666edcf, (0xc771 bytes) 2024-05-11T23:02:44.930 INFO:teuthology.orchestra.run.smithi005.stdout:4131 writesame 0x4a4719f thru 0x4a53519 data_size 0x7(0xc37b bytes) 2024-05-11T23:02:44.933 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:44.934 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:44.947 INFO:teuthology.orchestra.run.smithi005.stdout:truncating to largest ever: 0xee1ae0b 2024-05-11T23:02:44.952 INFO:teuthology.orchestra.run.smithi005.stdout:4132 trunc from 0x9f752a2 to 0xee1ae0b 2024-05-11T23:02:44.955 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:44.955 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:44.959 INFO:teuthology.orchestra.run.smithi005.stdout:4134 write 0xe761ea9 thru 0xe77161a (0xf772 bytes) 2024-05-11T23:02:44.978 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:44.978 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:44.984 INFO:teuthology.orchestra.run.smithi005.stdout:4135 trunc from 0xee1ae0b to 0xa0196e2 2024-05-11T23:02:44.986 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:44.986 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:44.986 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:44.991 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:44.991+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:44.997 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:02:44.997 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:44.998 INFO:teuthology.orchestra.run.smithi005.stdout:4136 read 0x9c10001 thru 0x9c1da4f (0xda4f bytes) 2024-05-11T23:02:44.998 INFO:teuthology.orchestra.run.smithi005.stdout:4137 writesame 0x45eb85a thru 0x45f6a3a data_size 0x2b(0xb1e1 bytes) 2024-05-11T23:02:45.020 INFO:teuthology.orchestra.run.smithi005.stdout:4139 trunc from 0xa0196e2 to 0x664cb4c 2024-05-11T23:02:45.020 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:45.036 INFO:teuthology.orchestra.run.smithi005.stdout:4140 punch from 0x4143793 to 0x414d148, (0x99b5 bytes) 2024-05-11T23:02:45.036 INFO:teuthology.orchestra.run.smithi005.stdout:4141 read 0x21dacdc thru 0x21dc6e4 (0x1a09 bytes) 2024-05-11T23:02:45.036 INFO:teuthology.orchestra.run.smithi005.stdout:4142 read 0x2d4fb04 thru 0x2d566e6 (0x6be3 bytes) 2024-05-11T23:02:45.037 INFO:teuthology.orchestra.run.smithi005.stdout:4143 read 0x58c6ec5 thru 0x58cf543 (0x867f bytes) 2024-05-11T23:02:45.037 INFO:teuthology.orchestra.run.smithi005.stdout:4144 write 0x2df4e98 thru 0x2df7473 (0x25dc bytes) 2024-05-11T23:02:45.042 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:45.043 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:45.050 INFO:teuthology.orchestra.run.smithi005.stdout:4147 read 0x55219dd thru 0x55243a5 (0x29c9 bytes) 2024-05-11T23:02:45.050 INFO:teuthology.orchestra.run.smithi005.stdout:4148 write 0x42794e7 thru 0x4280d79 (0x7893 bytes) 2024-05-11T23:02:45.067 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:45.067 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:45.070 INFO:teuthology.orchestra.run.smithi005.stdout:4149 punch from 0xbbfe9d to 0xbc3106, (0x3269 bytes) 2024-05-11T23:02:45.070 INFO:teuthology.orchestra.run.smithi005.stdout:4150 read 0x261a8da thru 0x261ec7a (0x43a1 bytes) 2024-05-11T23:02:45.072 INFO:teuthology.orchestra.run.smithi005.stderr:set 3.11 pg_upmap_items mapping to [7->4,6->1] 2024-05-11T23:02:45.076 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-05-11T23:02:45.084 INFO:teuthology.orchestra.run.smithi005.stdout:4152 write 0xb904b23 thru 0xb905d41 (0x121f bytes) 2024-05-11T23:02:45.092 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:45.092+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:45.095 INFO:teuthology.orchestra.run.smithi005.stdout:4153 write 0x2e5a96 thru 0x2ee1ca (0x8735 bytes) 2024-05-11T23:02:45.106 INFO:teuthology.orchestra.run.smithi005.stdout:4154 writesame 0xcba9d0c thru 0xcbae2d8 data_size 0x1(0x45cd bytes) 2024-05-11T23:02:45.106 INFO:teuthology.orchestra.run.smithi005.stdout:4155 punch from 0x59ff36c to 0x5a05989, (0x661d bytes) 2024-05-11T23:02:45.106 INFO:teuthology.orchestra.run.smithi005.stdout:4156 punch from 0x7c7a835 to 0x7c879bf, (0xd18a bytes) 2024-05-11T23:02:45.113 INFO:teuthology.orchestra.run.smithi005.stdout:4157 write 0xd76c4bf thru 0xd76fbab (0x36ed bytes) 2024-05-11T23:02:45.133 INFO:teuthology.orchestra.run.smithi005.stdout:4158 write 0x32761cc thru 0x327b746 (0x557b bytes) 2024-05-11T23:02:45.154 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:45.154 INFO:teuthology.orchestra.run.smithi005.stdout:4161 punch from 0x4f0a74e to 0x4f15714, (0xafc6 bytes) 2024-05-11T23:02:45.154 INFO:teuthology.orchestra.run.smithi005.stdout:4162 write 0x843422c thru 0x8442fb7 (0xed8c bytes) 2024-05-11T23:02:45.167 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:45.167 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:45.176 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:02:45.177 INFO:teuthology.orchestra.run.smithi005.stdout:4163 punch from 0x4243d46 to 0x4244330, (0x5ea bytes) 2024-05-11T23:02:45.177 INFO:teuthology.orchestra.run.smithi005.stdout:4164 read 0x8d7aebd thru 0x8d85dfc (0xaf40 bytes) 2024-05-11T23:02:45.177 INFO:teuthology.orchestra.run.smithi005.stdout:4165 write 0x9c74d7f thru 0x9c76b37 (0x1db9 bytes) 2024-05-11T23:02:45.194 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:45.194+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:45.200 INFO:teuthology.orchestra.run.smithi005.stdout:4167 trunc from 0xd76fbac to 0x1984e7f 2024-05-11T23:02:45.251 INFO:teuthology.orchestra.run.smithi005.stdout:4168 trunc from 0x1984e7f to 0x2c61731 2024-05-11T23:02:45.273 INFO:teuthology.orchestra.run.smithi005.stdout:4169 writesame 0xbe786ca thru 0xbe79e83 data_size 0x1(0x17ba bytes) 2024-05-11T23:02:45.274 INFO:teuthology.orchestra.run.smithi005.stdout:4170 writesame 0x8f0f84f thru 0x8f17457 data_size 0x1(0x7c09 bytes) 2024-05-11T23:02:45.295 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:45.294+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:45.296 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:45.305 INFO:teuthology.orchestra.run.smithi005.stdout:4173 trunc from 0xbe79e84 to 0x480b432 2024-05-11T23:02:45.322 INFO:teuthology.orchestra.run.smithi005.stdout:4174 trunc from 0x480b432 to 0x14181b1 2024-05-11T23:02:45.340 INFO:teuthology.orchestra.run.smithi005.stdout:4176 trunc from 0x14181b1 to 0x5ac72b2 2024-05-11T23:02:45.345 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:45.345 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:45.346 INFO:teuthology.orchestra.run.smithi005.stdout:4177 read 0x48b7124 thru 0x48c2aa2 (0xb97f bytes) 2024-05-11T23:02:45.347 INFO:teuthology.orchestra.run.smithi005.stdout:4178 writesame 0x524d06c thru 0x524f4b2 data_size 0x1(0x2447 bytes) 2024-05-11T23:02:45.358 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:02:45.367 INFO:teuthology.orchestra.run.smithi005.stdout:4179 write 0x7f89558 thru 0x7f8b27c (0x1d25 bytes) 2024-05-11T23:02:45.384 INFO:teuthology.orchestra.run.smithi005.stdout:4180 trunc from 0x7f8b27d to 0x458632f 2024-05-11T23:02:45.396 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:45.396+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:45.401 INFO:teuthology.orchestra.run.smithi005.stdout:4181 read 0x2b60c3f thru 0x2b60e87 (0x249 bytes) 2024-05-11T23:02:45.401 INFO:teuthology.orchestra.run.smithi005.stdout:4182 clone 40 order 19 su 2048 sc 5 2024-05-11T23:02:45.496 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:45.496+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:45.503 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:45.503 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:45.512 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:02:45.597 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:45.597+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:45.617 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:45.617 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:45.625 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:02:45.698 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:45.698+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:45.732 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:45.732 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:45.740 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:02:45.824 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:45.824+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:45.885 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:45.885 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:45.893 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:02:45.934 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:45.935+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:46.000 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:46.001+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:46.038 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:46.038 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:46.047 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:02:46.101 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:46.101+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:46.161 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:46.161 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:46.170 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:02:46.203 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:46.202+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:46.272 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-05-11T23:02:46.286 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:46.287 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:46.296 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:02:46.304 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:46.304+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:46.405 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:46.405+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:46.438 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:46.438 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:46.447 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:02:46.506 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:46.506+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:46.594 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:46.594 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:46.604 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:02:46.645 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:46.645+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:46.707 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:46.707+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:46.760 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:46.760 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:46.770 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:02:46.814 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:46.814+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:46.900 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:46.900 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:46.909 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:46.908+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:46.911 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:02:47.009 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:47.009+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:47.069 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:47.070 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:47.079 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:02:47.085 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:02:47.097 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-11T23:02:47.123 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:47.123+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:47.205 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:47.205 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:47.211 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:47.211+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:47.215 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:02:47.264 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:47.264 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:47.289 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:47.290 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:47.307 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:47.308 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:47.312 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:47.311+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:47.327 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:47.328 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:47.348 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:47.369 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:47.369 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:47.391 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:47.398 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28445 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.312938637999999, 2024-05-11T23:02:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031719429, 2024-05-11T23:02:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28445 2024-05-11T23:02:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081232+0000", 2024-05-11T23:02:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2399999999999999e-07 2024-05-11T23:02:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081234+0000", 2024-05-11T23:02:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.578e-06 2024-05-11T23:02:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081237+0000", 2024-05-11T23:02:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3340000000000002e-06 2024-05-11T23:02:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081240+0000", 2024-05-11T23:02:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7429999999999998e-06 2024-05-11T23:02:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112828+0000", 2024-05-11T23:02:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031587941000000001 2024-05-11T23:02:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112878+0000", 2024-05-11T23:02:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0535999999999999e-05 2024-05-11T23:02:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112951+0000", 2024-05-11T23:02:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3072999999999996e-05 2024-05-11T23:02:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28448 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 507904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:02:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.312906136999999, 2024-05-11T23:02:47.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031916923, 2024-05-11T23:02:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28448 2024-05-11T23:02:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:02:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081264+0000", 2024-05-11T23:02:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081265+0000", 2024-05-11T23:02:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2899999999999999e-07 2024-05-11T23:02:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081267+0000", 2024-05-11T23:02:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6749999999999998e-06 2024-05-11T23:02:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081273+0000", 2024-05-11T23:02:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4049999999999999e-06 2024-05-11T23:02:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081278+0000", 2024-05-11T23:02:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0139999999999998e-06 2024-05-11T23:02:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.112996+0000", 2024-05-11T23:02:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031718283 2024-05-11T23:02:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113080+0000", 2024-05-11T23:02:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4334999999999998e-05 2024-05-11T23:02:47.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113181+0000", 2024-05-11T23:02:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000100882 2024-05-11T23:02:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28451 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 540672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.312877635, 2024-05-11T23:02:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032054412999999997, 2024-05-11T23:02:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.406 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28451 2024-05-11T23:02:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:47.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081293+0000", 2024-05-11T23:02:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2499999999999995e-07 2024-05-11T23:02:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081300+0000", 2024-05-11T23:02:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0929999999999999e-06 2024-05-11T23:02:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081306+0000", 2024-05-11T23:02:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6290000000000004e-06 2024-05-11T23:02:47.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081311+0000", 2024-05-11T23:02:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9289999999999997e-06 2024-05-11T23:02:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113208+0000", 2024-05-11T23:02:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031897162 2024-05-11T23:02:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113277+0000", 2024-05-11T23:02:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8811000000000002e-05 2024-05-11T23:02:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113347+0000", 2024-05-11T23:02:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0264000000000004e-05 2024-05-11T23:02:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28454 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 573440~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.312803013999996, 2024-05-11T23:02:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032210564999999997, 2024-05-11T23:02:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.410 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28454 2024-05-11T23:02:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081368+0000", 2024-05-11T23:02:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.99e-07 2024-05-11T23:02:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081371+0000", 2024-05-11T23:02:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1200000000000002e-06 2024-05-11T23:02:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081378+0000", 2024-05-11T23:02:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6089999999999997e-06 2024-05-11T23:02:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081384+0000", 2024-05-11T23:02:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0859999999999998e-06 2024-05-11T23:02:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113388+0000", 2024-05-11T23:02:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032004248999999999 2024-05-11T23:02:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113470+0000", 2024-05-11T23:02:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1871000000000003e-05 2024-05-11T23:02:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113578+0000", 2024-05-11T23:02:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000108031 2024-05-11T23:02:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28457 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 606208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:47.414 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.312772789999997, 2024-05-11T23:02:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032410130000000002, 2024-05-11T23:02:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28457 2024-05-11T23:02:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081398+0000", 2024-05-11T23:02:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.32e-07 2024-05-11T23:02:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081400+0000", 2024-05-11T23:02:47.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9120000000000001e-06 2024-05-11T23:02:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081404+0000", 2024-05-11T23:02:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7900000000000001e-06 2024-05-11T23:02:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081407+0000", 2024-05-11T23:02:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8839999999999998e-06 2024-05-11T23:02:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113621+0000", 2024-05-11T23:02:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032213841 2024-05-11T23:02:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113702+0000", 2024-05-11T23:02:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1574000000000002e-05 2024-05-11T23:02:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113808+0000", 2024-05-11T23:02:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000105697 2024-05-11T23:02:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.418 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28460 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 638976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.312759958999997, 2024-05-11T23:02:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032616804999999999, 2024-05-11T23:02:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28460 2024-05-11T23:02:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081411+0000", 2024-05-11T23:02:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5900000000000003e-07 2024-05-11T23:02:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081412+0000", 2024-05-11T23:02:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1990000000000001e-06 2024-05-11T23:02:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081415+0000", 2024-05-11T23:02:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.971e-06 2024-05-11T23:02:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081418+0000", 2024-05-11T23:02:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.571e-06 2024-05-11T23:02:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113853+0000", 2024-05-11T23:02:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032435155 2024-05-11T23:02:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.113936+0000", 2024-05-11T23:02:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3441000000000003e-05 2024-05-11T23:02:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114027+0000", 2024-05-11T23:02:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1108999999999998e-05 2024-05-11T23:02:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.422 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28463 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 671744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.312743036999997, 2024-05-11T23:02:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032814958999999998, 2024-05-11T23:02:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.423 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28463 2024-05-11T23:02:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081428+0000", 2024-05-11T23:02:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4999999999999998e-07 2024-05-11T23:02:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081430+0000", 2024-05-11T23:02:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6109999999999999e-06 2024-05-11T23:02:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081433+0000", 2024-05-11T23:02:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5590000000000001e-06 2024-05-11T23:02:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081437+0000", 2024-05-11T23:02:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3950000000000001e-06 2024-05-11T23:02:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114061+0000", 2024-05-11T23:02:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032624092 2024-05-11T23:02:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114142+0000", 2024-05-11T23:02:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0893e-05 2024-05-11T23:02:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114242+0000", 2024-05-11T23:02:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010095899999999999 2024-05-11T23:02:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28466 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 704512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.312730502999997, 2024-05-11T23:02:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032933658999999997, 2024-05-11T23:02:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:47.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28466 2024-05-11T23:02:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081440+0000", 2024-05-11T23:02:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4700000000000002e-07 2024-05-11T23:02:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081442+0000", 2024-05-11T23:02:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.296e-06 2024-05-11T23:02:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081445+0000", 2024-05-11T23:02:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9009999999999998e-06 2024-05-11T23:02:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081447+0000", 2024-05-11T23:02:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4200000000000001e-06 2024-05-11T23:02:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114280+0000", 2024-05-11T23:02:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032832667000000003 2024-05-11T23:02:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114325+0000", 2024-05-11T23:02:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5018000000000002e-05 2024-05-11T23:02:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114374+0000", 2024-05-11T23:02:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.901e-05 2024-05-11T23:02:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28468 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 737280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.312710498000001, 2024-05-11T23:02:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033118776000000003, 2024-05-11T23:02:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:47.431 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28468 2024-05-11T23:02:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081460+0000", 2024-05-11T23:02:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3799999999999998e-07 2024-05-11T23:02:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081462+0000", 2024-05-11T23:02:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4899999999999999e-06 2024-05-11T23:02:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081465+0000", 2024-05-11T23:02:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4079999999999998e-06 2024-05-11T23:02:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081468+0000", 2024-05-11T23:02:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7e-06 2024-05-11T23:02:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114415+0000", 2024-05-11T23:02:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032946766000000002 2024-05-11T23:02:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114497+0000", 2024-05-11T23:02:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1854000000000006e-05 2024-05-11T23:02:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114579+0000", 2024-05-11T23:02:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2120000000000007e-05 2024-05-11T23:02:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28471 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 770048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.312695791000003, 2024-05-11T23:02:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033234970000000003, 2024-05-11T23:02:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.435 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28471 2024-05-11T23:02:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081475+0000", 2024-05-11T23:02:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-05-11T23:02:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081476+0000", 2024-05-11T23:02:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3120000000000001e-06 2024-05-11T23:02:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081480+0000", 2024-05-11T23:02:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1470000000000001e-06 2024-05-11T23:02:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081482+0000", 2024-05-11T23:02:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7480000000000002e-06 2024-05-11T23:02:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114616+0000", 2024-05-11T23:02:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033133322 2024-05-11T23:02:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114662+0000", 2024-05-11T23:02:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6020999999999999e-05 2024-05-11T23:02:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114710+0000", 2024-05-11T23:02:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8059999999999997e-05 2024-05-11T23:02:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28474 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 802816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.312679717000002, 2024-05-11T23:02:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033336431, 2024-05-11T23:02:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28474 2024-05-11T23:02:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:47.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081491+0000", 2024-05-11T23:02:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.15e-07 2024-05-11T23:02:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081493+0000", 2024-05-11T23:02:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4950000000000001e-06 2024-05-11T23:02:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081496+0000", 2024-05-11T23:02:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.309e-06 2024-05-11T23:02:47.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081499+0000", 2024-05-11T23:02:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4570000000000001e-06 2024-05-11T23:02:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114740+0000", 2024-05-11T23:02:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033240619999999999 2024-05-11T23:02:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114781+0000", 2024-05-11T23:02:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1106999999999998e-05 2024-05-11T23:02:47.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114827+0000", 2024-05-11T23:02:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6028e-05 2024-05-11T23:02:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28477 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 835584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.312662703000001, 2024-05-11T23:02:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033527225000000001, 2024-05-11T23:02:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.443 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28477 2024-05-11T23:02:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081508+0000", 2024-05-11T23:02:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1300000000000001e-07 2024-05-11T23:02:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081510+0000", 2024-05-11T23:02:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4610000000000001e-06 2024-05-11T23:02:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081513+0000", 2024-05-11T23:02:47.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2559999999999998e-06 2024-05-11T23:02:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081516+0000", 2024-05-11T23:02:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7099999999999999e-06 2024-05-11T23:02:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114869+0000", 2024-05-11T23:02:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033353718999999997 2024-05-11T23:02:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.114954+0000", 2024-05-11T23:02:47.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4969000000000006e-05 2024-05-11T23:02:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115035+0000", 2024-05-11T23:02:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0697000000000003e-05 2024-05-11T23:02:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28480 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 868352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.312645732999997, 2024-05-11T23:02:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033695939000000001, 2024-05-11T23:02:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.448 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28480 2024-05-11T23:02:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081525+0000", 2024-05-11T23:02:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.46e-07 2024-05-11T23:02:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081526+0000", 2024-05-11T23:02:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.347e-06 2024-05-11T23:02:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081530+0000", 2024-05-11T23:02:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.348e-06 2024-05-11T23:02:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081533+0000", 2024-05-11T23:02:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4369999999999999e-06 2024-05-11T23:02:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115078+0000", 2024-05-11T23:02:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033545063 2024-05-11T23:02:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115164+0000", 2024-05-11T23:02:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5662999999999996e-05 2024-05-11T23:02:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115221+0000", 2024-05-11T23:02:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6734999999999998e-05 2024-05-11T23:02:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28483 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 901120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.31262856, 2024-05-11T23:02:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033803466999999997, 2024-05-11T23:02:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.452 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28483 2024-05-11T23:02:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081542+0000", 2024-05-11T23:02:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4999999999999998e-07 2024-05-11T23:02:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081544+0000", 2024-05-11T23:02:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:02:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081547+0000", 2024-05-11T23:02:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3139999999999999e-06 2024-05-11T23:02:47.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081550+0000", 2024-05-11T23:02:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7779999999999999e-06 2024-05-11T23:02:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115253+0000", 2024-05-11T23:02:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033703516000000003 2024-05-11T23:02:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115302+0000", 2024-05-11T23:02:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8869999999999998e-05 2024-05-11T23:02:47.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115345+0000", 2024-05-11T23:02:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3340999999999997e-05 2024-05-11T23:02:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28487 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 933888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.312617359000001, 2024-05-11T23:02:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033903918999999998, 2024-05-11T23:02:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.456 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28487 2024-05-11T23:02:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:47.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081553+0000", 2024-05-11T23:02:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9999999999999999e-07 2024-05-11T23:02:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081559+0000", 2024-05-11T23:02:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7570000000000001e-06 2024-05-11T23:02:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081563+0000", 2024-05-11T23:02:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2370000000000003e-06 2024-05-11T23:02:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081566+0000", 2024-05-11T23:02:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6199999999999999e-06 2024-05-11T23:02:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115375+0000", 2024-05-11T23:02:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033808887000000003 2024-05-11T23:02:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115413+0000", 2024-05-11T23:02:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8309000000000002e-05 2024-05-11T23:02:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115457+0000", 2024-05-11T23:02:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3809e-05 2024-05-11T23:02:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28490 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 966656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.312600834999998, 2024-05-11T23:02:47.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034114903000000002, 2024-05-11T23:02:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28490 2024-05-11T23:02:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081570+0000", 2024-05-11T23:02:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4400000000000001e-07 2024-05-11T23:02:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081571+0000", 2024-05-11T23:02:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2130000000000001e-06 2024-05-11T23:02:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081574+0000", 2024-05-11T23:02:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8270000000000002e-06 2024-05-11T23:02:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081577+0000", 2024-05-11T23:02:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-05-11T23:02:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115507+0000", 2024-05-11T23:02:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033929925999999999 2024-05-11T23:02:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115599+0000", 2024-05-11T23:02:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1571999999999997e-05 2024-05-11T23:02:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115685+0000", 2024-05-11T23:02:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6071999999999999e-05 2024-05-11T23:02:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28493 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 999424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:47.464 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.312588732000002, 2024-05-11T23:02:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034232944000000001, 2024-05-11T23:02:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28493 2024-05-11T23:02:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081582+0000", 2024-05-11T23:02:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1300000000000001e-07 2024-05-11T23:02:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081584+0000", 2024-05-11T23:02:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9479999999999998e-06 2024-05-11T23:02:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081590+0000", 2024-05-11T23:02:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.728e-06 2024-05-11T23:02:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081595+0000", 2024-05-11T23:02:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8579999999999999e-06 2024-05-11T23:02:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115723+0000", 2024-05-11T23:02:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034128473999999999 2024-05-11T23:02:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115768+0000", 2024-05-11T23:02:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5108999999999998e-05 2024-05-11T23:02:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115815+0000", 2024-05-11T23:02:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6514000000000002e-05 2024-05-11T23:02:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28496 3.0 3:02a4c49b:::rbd_data.111e4e122501.000000000000001b:f [read 1032192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:02:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.312569273000001, 2024-05-11T23:02:47.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034427754999999997, 2024-05-11T23:02:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28496 2024-05-11T23:02:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:02:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081601+0000", 2024-05-11T23:02:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081602+0000", 2024-05-11T23:02:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:02:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081604+0000", 2024-05-11T23:02:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3999999999999999e-06 2024-05-11T23:02:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081610+0000", 2024-05-11T23:02:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.224e-06 2024-05-11T23:02:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.081615+0000", 2024-05-11T23:02:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8269999999999997e-06 2024-05-11T23:02:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115859+0000", 2024-05-11T23:02:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034243498999999997 2024-05-11T23:02:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.115946+0000", 2024-05-11T23:02:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7122999999999999e-05 2024-05-11T23:02:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116029+0000", 2024-05-11T23:02:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3332999999999996e-05 2024-05-11T23:02:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:28498 3.0 3:05d5189e:::rbd_object_map.111e5e4d8275.0000000000000011:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:02:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.311050621, 2024-05-11T23:02:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.035035108000000002, 2024-05-11T23:02:47.472 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 28498 2024-05-11T23:02:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:02:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083120+0000", 2024-05-11T23:02:47.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083121+0000", 2024-05-11T23:02:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5700000000000001e-07 2024-05-11T23:02:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083126+0000", 2024-05-11T23:02:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8860000000000003e-06 2024-05-11T23:02:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083135+0000", 2024-05-11T23:02:47.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8369999999999992e-06 2024-05-11T23:02:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.083142+0000", 2024-05-11T23:02:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4779999999999996e-06 2024-05-11T23:02:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116072+0000", 2024-05-11T23:02:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032929991999999998 2024-05-11T23:02:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116158+0000", 2024-05-11T23:02:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6077999999999998e-05 2024-05-11T23:02:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-05-11T23:02:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.116513+0000", 2024-05-11T23:02:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00035508300000000002 2024-05-11T23:02:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118025+0000", 2024-05-11T23:02:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015118919999999999 2024-05-11T23:02:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:02:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118039+0000", 2024-05-11T23:02:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4188999999999999e-05 2024-05-11T23:02:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118094+0000", 2024-05-11T23:02:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4929000000000002e-05 2024-05-11T23:02:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.118155+0000", 2024-05-11T23:02:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0887000000000002e-05 2024-05-11T23:02:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:64049 3.8 3:11d98727:::rbd_object_map.111e8974a681:head [read 0~38] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-05-11T23:02:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:02:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 46.614289311, 2024-05-11T23:02:47.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99884604399999999, 2024-05-11T23:02:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 64049 2024-05-11T23:02:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:02:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:02:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779883+0000", 2024-05-11T23:02:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.57e-06 2024-05-11T23:02:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779887+0000", 2024-05-11T23:02:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5709999999999999e-06 2024-05-11T23:02:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779898+0000", 2024-05-11T23:02:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.042e-05 2024-05-11T23:02:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779905+0000", 2024-05-11T23:02:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0110000000000003e-06 2024-05-11T23:02:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779962+0000", 2024-05-11T23:02:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7395999999999997e-05 2024-05-11T23:02:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:02:47.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779969+0000", 2024-05-11T23:02:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2459999999999997e-06 2024-05-11T23:02:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778379+0000", 2024-05-11T23:02:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99840986300000001 2024-05-11T23:02:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778564+0000", 2024-05-11T23:02:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018420900000000001 2024-05-11T23:02:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778727+0000", 2024-05-11T23:02:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016375800000000001 2024-05-11T23:02:47.481 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.482 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.482 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:47.483 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:02:47.484 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:47.484 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:47.485 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:47.483+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:47.583 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:47.583+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:47.638 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:47.638 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:47.639 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:02:47.639 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:47.647 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:02:47.688 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:47.688+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:47.786 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:47.786+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:47.802 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:47.806 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:47.806 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:47.806 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:47.806 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:47.812 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:02:47.886 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:47.887+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:47.986 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:47.986 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:47.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:47.986 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:47.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.986 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:60471 3.4 e43/19 3:2079ea20:::rbd_data.111e1df04b77.0000000000000000:head v 43'84, mlcod=42'81)", 2024-05-11T23:02:47.986 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:47.987 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.058258918, 2024-05-11T23:02:47.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0079043050000000004, 2024-05-11T23:02:47.987 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.987 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.987 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:47.987 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:47.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:47.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926874+0000", 2024-05-11T23:02:47.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-05-11T23:02:47.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926880+0000", 2024-05-11T23:02:47.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:02:47.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926893+0000", 2024-05-11T23:02:47.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3256e-05 2024-05-11T23:02:47.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926899+0000", 2024-05-11T23:02:47.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9850000000000003e-06 2024-05-11T23:02:47.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.926982+0000", 2024-05-11T23:02:47.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2997000000000005e-05 2024-05-11T23:02:47.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.927005+0000", 2024-05-11T23:02:47.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3428e-05 2024-05-11T23:02:47.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:47.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.934747+0000", 2024-05-11T23:02:47.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0077415820000000003 2024-05-11T23:02:47.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:56.934778+0000", 2024-05-11T23:02:47.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1479999999999997e-05 2024-05-11T23:02:47.990 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.990 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.990 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.991 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:64399 3.1e e48/47 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head v 48'297, mlcod=0'0)", 2024-05-11T23:02:47.991 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:02.104619+0000", 2024-05-11T23:02:47.991 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.880513805, 2024-05-11T23:02:47.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014840949000000001, 2024-05-11T23:02:47.991 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.991 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:47.991 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.104619+0000", 2024-05-11T23:02:47.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.104619+0000", 2024-05-11T23:02:47.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.104620+0000", 2024-05-11T23:02:47.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.299e-06 2024-05-11T23:02:47.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.112866+0000", 2024-05-11T23:02:47.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0082450949999999992 2024-05-11T23:02:47.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.112893+0000", 2024-05-11T23:02:47.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7131000000000002e-05 2024-05-11T23:02:47.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.112902+0000", 2024-05-11T23:02:47.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4089999999999992e-06 2024-05-11T23:02:47.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.112980+0000", 2024-05-11T23:02:47.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8375999999999996e-05 2024-05-11T23:02:47.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:47.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.113009+0000", 2024-05-11T23:02:47.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8713000000000001e-05 2024-05-11T23:02:47.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:47.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.119403+0000", 2024-05-11T23:02:47.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0063940580000000002 2024-05-11T23:02:47.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:02.119460+0000", 2024-05-11T23:02:47.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6867999999999998e-05 2024-05-11T23:02:47.995 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.995 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.995 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.995 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "MOSDPGPushReply(3.1e 52/51 [PushReplyOp(3:7943f929:::rbd_data.111ef7a115a1.0000000000000050:head)])", 2024-05-11T23:02:47.995 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:06.114699+0000", 2024-05-11T23:02:47.995 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.870434041000003, 2024-05-11T23:02:47.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0064609460000000004, 2024-05-11T23:02:47.995 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.995 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.995 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:47.995 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114699+0000", 2024-05-11T23:02:47.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:47.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114699+0000", 2024-05-11T23:02:47.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:47.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114700+0000", 2024-05-11T23:02:47.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0780000000000001e-06 2024-05-11T23:02:47.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:47.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114706+0000", 2024-05-11T23:02:47.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4200000000000004e-06 2024-05-11T23:02:47.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:47.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114724+0000", 2024-05-11T23:02:47.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7142e-05 2024-05-11T23:02:47.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:47.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.114732+0000", 2024-05-11T23:02:47.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7630000000000006e-06 2024-05-11T23:02:47.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:47.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.120983+0000", 2024-05-11T23:02:47.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0062507250000000004 2024-05-11T23:02:47.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:47.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:06.121160+0000", 2024-05-11T23:02:47.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000176818 2024-05-11T23:02:47.998 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.998 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:47.998 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:47.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.998 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:77128 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_get in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-11T23:02:47.998 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:02:47.998 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.969131255999997, 2024-05-11T23:02:47.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91737855899999998, 2024-05-11T23:02:47.999 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:47.999 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:47.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:47.999 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:47.999 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:47.999 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:47.999 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 77128 2024-05-11T23:02:47.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.999 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:47.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:47.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:47.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:02:47.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:47.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:47.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:02:48.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016003+0000", 2024-05-11T23:02:48.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5379999999999999e-06 2024-05-11T23:02:48.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016011+0000", 2024-05-11T23:02:48.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6410000000000002e-06 2024-05-11T23:02:48.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016026+0000", 2024-05-11T23:02:48.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4779e-05 2024-05-11T23:02:48.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016032+0000", 2024-05-11T23:02:48.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.066e-06 2024-05-11T23:02:48.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016093+0000", 2024-05-11T23:02:48.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0878000000000002e-05 2024-05-11T23:02:48.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:02:48.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016100+0000", 2024-05-11T23:02:48.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6799999999999993e-06 2024-05-11T23:02:48.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933302+0000", 2024-05-11T23:02:48.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91720135300000005 2024-05-11T23:02:48.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933380+0000", 2024-05-11T23:02:48.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8623999999999998e-05 2024-05-11T23:02:48.002 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.002 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.002 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.003 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:78890 3.3 3:c289f39e:::rbd_data.111ea74f268d.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:48.003 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:16.008252+0000", 2024-05-11T23:02:48.003 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.976881116000001, 2024-05-11T23:02:48.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0070432100000000003, 2024-05-11T23:02:48.003 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.003 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.003 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:48.003 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.003 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.003 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.003 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 78890 2024-05-11T23:02:48.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.003 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.008252+0000", 2024-05-11T23:02:48.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.008252+0000", 2024-05-11T23:02:48.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.008253+0000", 2024-05-11T23:02:48.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.142e-06 2024-05-11T23:02:48.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009306+0000", 2024-05-11T23:02:48.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001053327 2024-05-11T23:02:48.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009329+0000", 2024-05-11T23:02:48.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3113999999999999e-05 2024-05-11T23:02:48.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009345+0000", 2024-05-11T23:02:48.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5136e-05 2024-05-11T23:02:48.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009447+0000", 2024-05-11T23:02:48.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010199799999999999 2024-05-11T23:02:48.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009640+0000", 2024-05-11T23:02:48.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019346000000000001 2024-05-11T23:02:48.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-05-11T23:02:48.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.009832+0000", 2024-05-11T23:02:48.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00019212799999999999 2024-05-11T23:02:48.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:02:48.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.014184+0000", 2024-05-11T23:02:48.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0043513909999999996 2024-05-11T23:02:48.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:48.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.015200+0000", 2024-05-11T23:02:48.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001016926 2024-05-11T23:02:48.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:48.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.015250+0000", 2024-05-11T23:02:48.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9162999999999997e-05 2024-05-11T23:02:48.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:16.015295+0000", 2024-05-11T23:02:48.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5425e-05 2024-05-11T23:02:48.011 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.011 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.011 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.011 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:97486 3.9 e71/19 3:966aeabc:::rbd_data.111ed003fd7f.0000000000000011:head v 71'192, mlcod=69'190)", 2024-05-11T23:02:48.011 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:25.042193+0000", 2024-05-11T23:02:48.011 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.942940120999999, 2024-05-11T23:02:48.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01046652, 2024-05-11T23:02:48.011 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.011 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:48.012 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:25.042193+0000", 2024-05-11T23:02:48.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:25.042193+0000", 2024-05-11T23:02:48.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:25.042193+0000", 2024-05-11T23:02:48.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.92e-07 2024-05-11T23:02:48.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:25.047425+0000", 2024-05-11T23:02:48.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.005232123 2024-05-11T23:02:48.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:25.047444+0000", 2024-05-11T23:02:48.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8805e-05 2024-05-11T23:02:48.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:25.047454+0000", 2024-05-11T23:02:48.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0361e-05 2024-05-11T23:02:48.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:25.047534+0000", 2024-05-11T23:02:48.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9466000000000001e-05 2024-05-11T23:02:48.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:25.047561+0000", 2024-05-11T23:02:48.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6956000000000001e-05 2024-05-11T23:02:48.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:48.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:25.052626+0000", 2024-05-11T23:02:48.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0050649170000000004 2024-05-11T23:02:48.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:25.052659+0000", 2024-05-11T23:02:48.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3699999999999999e-05 2024-05-11T23:02:48.015 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.015 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.015 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.016 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98950 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:48.016 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:02:48.016 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.853050367000002, 2024-05-11T23:02:48.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021419438999999998, 2024-05-11T23:02:48.016 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.016 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.016 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.016 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.016 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.016 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.016 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98950 2024-05-11T23:02:48.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.016 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:02:48.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:02:48.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:02:48.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.34e-07 2024-05-11T23:02:48.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132085+0000", 2024-05-11T23:02:48.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0049999999999999e-06 2024-05-11T23:02:48.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132093+0000", 2024-05-11T23:02:48.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6660000000000008e-06 2024-05-11T23:02:48.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132097+0000", 2024-05-11T23:02:48.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2590000000000002e-06 2024-05-11T23:02:48.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135114+0000", 2024-05-11T23:02:48.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003017573 2024-05-11T23:02:48.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153416+0000", 2024-05-11T23:02:48.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018301884000000001 2024-05-11T23:02:48.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153502+0000", 2024-05-11T23:02:48.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5717999999999995e-05 2024-05-11T23:02:48.019 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.019 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.020 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.020 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98959 3.1e 3:7807da99:::rbd_data.111ed2934718.000000000000006d:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:48.020 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:02:48.020 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.85255763, 2024-05-11T23:02:48.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020525928999999998, 2024-05-11T23:02:48.020 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.020 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.020 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.020 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.020 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.020 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.020 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98959 2024-05-11T23:02:48.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.021 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:02:48.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:02:48.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132576+0000", 2024-05-11T23:02:48.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4199999999999995e-07 2024-05-11T23:02:48.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132580+0000", 2024-05-11T23:02:48.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2910000000000001e-06 2024-05-11T23:02:48.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132588+0000", 2024-05-11T23:02:48.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7970000000000001e-06 2024-05-11T23:02:48.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132602+0000", 2024-05-11T23:02:48.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3743999999999999e-05 2024-05-11T23:02:48.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132655+0000", 2024-05-11T23:02:48.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2714999999999999e-05 2024-05-11T23:02:48.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132787+0000", 2024-05-11T23:02:48.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013273899999999999 2024-05-11T23:02:48.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153101+0000", 2024-05-11T23:02:48.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020313900999999999 2024-05-11T23:02:48.023 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.024 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.024 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.024 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98965 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:48.024 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:02:48.024 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.850033994, 2024-05-11T23:02:48.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018608983999999999, 2024-05-11T23:02:48.024 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.024 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.024 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.024 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.024 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.024 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.024 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98965 2024-05-11T23:02:48.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.025 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:02:48.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:02:48.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135100+0000", 2024-05-11T23:02:48.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.17e-06 2024-05-11T23:02:48.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135105+0000", 2024-05-11T23:02:48.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.899e-06 2024-05-11T23:02:48.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135122+0000", 2024-05-11T23:02:48.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7346999999999998e-05 2024-05-11T23:02:48.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135130+0000", 2024-05-11T23:02:48.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0639999999999994e-06 2024-05-11T23:02:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153533+0000", 2024-05-11T23:02:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018402753000000001 2024-05-11T23:02:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153615+0000", 2024-05-11T23:02:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1720999999999999e-05 2024-05-11T23:02:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153708+0000", 2024-05-11T23:02:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3029999999999995e-05 2024-05-11T23:02:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.028 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.028 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.028 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98966 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3760128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:48.028 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:02:48.028 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.849961868000001, 2024-05-11T23:02:48.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018716535999999999, 2024-05-11T23:02:48.028 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.028 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.028 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.028 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.028 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.028 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98966 2024-05-11T23:02:48.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.029 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:02:48.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:02:48.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135172+0000", 2024-05-11T23:02:48.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-05-11T23:02:48.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135175+0000", 2024-05-11T23:02:48.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-05-11T23:02:48.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135184+0000", 2024-05-11T23:02:48.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1840000000000005e-06 2024-05-11T23:02:48.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152903+0000", 2024-05-11T23:02:48.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017718357000000001 2024-05-11T23:02:48.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153736+0000", 2024-05-11T23:02:48.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00083324799999999997 2024-05-11T23:02:48.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153809+0000", 2024-05-11T23:02:48.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2995000000000002e-05 2024-05-11T23:02:48.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153888+0000", 2024-05-11T23:02:48.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8695999999999993e-05 2024-05-11T23:02:48.031 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.031 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.031 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.032 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "pg_update_log_missing_reply(3.1e epoch 72/53 rep_tid 104 lcod 72'363)", 2024-05-11T23:02:48.032 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:02:48.032 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.849898971999998, 2024-05-11T23:02:48.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017871951000000001, 2024-05-11T23:02:48.032 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.032 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.032 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:48.032 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:02:48.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:02:48.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135235+0000", 2024-05-11T23:02:48.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8899999999999999e-07 2024-05-11T23:02:48.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135238+0000", 2024-05-11T23:02:48.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.27e-06 2024-05-11T23:02:48.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152878+0000", 2024-05-11T23:02:48.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01763986 2024-05-11T23:02:48.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152896+0000", 2024-05-11T23:02:48.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8573e-05 2024-05-11T23:02:48.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152968+0000", 2024-05-11T23:02:48.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1780999999999996e-05 2024-05-11T23:02:48.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153106+0000", 2024-05-11T23:02:48.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013777799999999999 2024-05-11T23:02:48.034 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:99788 3.3 3:c495528a:::rbd_header.111ed2934718:head [watch reconnect cookie 139809507265552 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.200323+0000", 2024-05-11T23:02:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.784809549999999, 2024-05-11T23:02:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0064864149999999997, 2024-05-11T23:02:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.035 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 99788 2024-05-11T23:02:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.200323+0000", 2024-05-11T23:02:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.200323+0000", 2024-05-11T23:02:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.200324+0000", 2024-05-11T23:02:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8800000000000003e-07 2024-05-11T23:02:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.200326+0000", 2024-05-11T23:02:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2280000000000001e-06 2024-05-11T23:02:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.200333+0000", 2024-05-11T23:02:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7750000000000004e-06 2024-05-11T23:02:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.200339+0000", 2024-05-11T23:02:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2400000000000004e-06 2024-05-11T23:02:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.201371+0000", 2024-05-11T23:02:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001031786 2024-05-11T23:02:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.201459+0000", 2024-05-11T23:02:48.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7862999999999995e-05 2024-05-11T23:02:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.206810+0000", 2024-05-11T23:02:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0053510349999999996 2024-05-11T23:02:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:100474 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 2875392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.235591+0000", 2024-05-11T23:02:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.749541946000001, 2024-05-11T23:02:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0066337169999999999, 2024-05-11T23:02:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.039 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 100474 2024-05-11T23:02:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.235591+0000", 2024-05-11T23:02:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.235591+0000", 2024-05-11T23:02:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.235591+0000", 2024-05-11T23:02:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1399999999999997e-07 2024-05-11T23:02:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.235598+0000", 2024-05-11T23:02:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3169999999999998e-06 2024-05-11T23:02:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.235604+0000", 2024-05-11T23:02:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4180000000000002e-06 2024-05-11T23:02:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.235609+0000", 2024-05-11T23:02:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0549999999999999e-06 2024-05-11T23:02:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.242005+0000", 2024-05-11T23:02:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0063958440000000004 2024-05-11T23:02:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.242113+0000", 2024-05-11T23:02:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107549 2024-05-11T23:02:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.242225+0000", 2024-05-11T23:02:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011212 2024-05-11T23:02:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:100788 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [sparse-read 11796480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.389416+0000", 2024-05-11T23:02:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.595716535000001, 2024-05-11T23:02:48.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0064764569999999997, 2024-05-11T23:02:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 100788 2024-05-11T23:02:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389416+0000", 2024-05-11T23:02:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389416+0000", 2024-05-11T23:02:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389417+0000", 2024-05-11T23:02:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.75e-07 2024-05-11T23:02:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389419+0000", 2024-05-11T23:02:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1969999999999999e-06 2024-05-11T23:02:48.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389425+0000", 2024-05-11T23:02:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2820000000000002e-06 2024-05-11T23:02:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389430+0000", 2024-05-11T23:02:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0379999999999999e-06 2024-05-11T23:02:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.395683+0000", 2024-05-11T23:02:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0062525009999999997 2024-05-11T23:02:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.395778+0000", 2024-05-11T23:02:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5305999999999999e-05 2024-05-11T23:02:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.395893+0000", 2024-05-11T23:02:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000114558 2024-05-11T23:02:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:100807 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [sparse-read 1835008~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.389437+0000", 2024-05-11T23:02:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.595695633999998, 2024-05-11T23:02:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.007388113, 2024-05-11T23:02:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 100807 2024-05-11T23:02:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389437+0000", 2024-05-11T23:02:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389437+0000", 2024-05-11T23:02:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389438+0000", 2024-05-11T23:02:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7399999999999998e-07 2024-05-11T23:02:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389441+0000", 2024-05-11T23:02:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7179999999999998e-06 2024-05-11T23:02:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389448+0000", 2024-05-11T23:02:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5919999999999997e-06 2024-05-11T23:02:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389453+0000", 2024-05-11T23:02:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:02:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.395935+0000", 2024-05-11T23:02:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0064818300000000001 2024-05-11T23:02:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.396027+0000", 2024-05-11T23:02:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2015999999999994e-05 2024-05-11T23:02:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.396825+0000", 2024-05-11T23:02:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00079866799999999999 2024-05-11T23:02:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:100809 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [sparse-read 3407872~196608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.389460+0000", 2024-05-11T23:02:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.595673264999999, 2024-05-11T23:02:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0080309130000000006, 2024-05-11T23:02:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 100809 2024-05-11T23:02:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389460+0000", 2024-05-11T23:02:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389460+0000", 2024-05-11T23:02:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389460+0000", 2024-05-11T23:02:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-05-11T23:02:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389463+0000", 2024-05-11T23:02:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.446e-06 2024-05-11T23:02:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389468+0000", 2024-05-11T23:02:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6890000000000001e-06 2024-05-11T23:02:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389473+0000", 2024-05-11T23:02:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5979999999999999e-06 2024-05-11T23:02:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.396887+0000", 2024-05-11T23:02:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0074143660000000004 2024-05-11T23:02:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.397017+0000", 2024-05-11T23:02:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013025299999999999 2024-05-11T23:02:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.397491+0000", 2024-05-11T23:02:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00047314500000000001 2024-05-11T23:02:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.055 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:100810 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [sparse-read 8388608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.389480+0000", 2024-05-11T23:02:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.595653331000001, 2024-05-11T23:02:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0084564610000000002, 2024-05-11T23:02:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 100810 2024-05-11T23:02:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.056 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389480+0000", 2024-05-11T23:02:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389480+0000", 2024-05-11T23:02:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389480+0000", 2024-05-11T23:02:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:02:48.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389482+0000", 2024-05-11T23:02:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.294e-06 2024-05-11T23:02:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389488+0000", 2024-05-11T23:02:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0419999999999999e-06 2024-05-11T23:02:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389493+0000", 2024-05-11T23:02:48.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8029999999999996e-06 2024-05-11T23:02:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.397532+0000", 2024-05-11T23:02:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0080390170000000007 2024-05-11T23:02:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.397818+0000", 2024-05-11T23:02:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00028542200000000002 2024-05-11T23:02:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.397936+0000", 2024-05-11T23:02:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00011846200000000001 2024-05-11T23:02:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.059 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:100811 3.3 3.76f4d2e3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.389500+0000", 2024-05-11T23:02:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.595633093, 2024-05-11T23:02:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0084682569999999999, 2024-05-11T23:02:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 100811 2024-05-11T23:02:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389500+0000", 2024-05-11T23:02:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389500+0000", 2024-05-11T23:02:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389500+0000", 2024-05-11T23:02:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7700000000000005e-07 2024-05-11T23:02:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389502+0000", 2024-05-11T23:02:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1619999999999998e-06 2024-05-11T23:02:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389509+0000", 2024-05-11T23:02:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.083e-06 2024-05-11T23:02:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389513+0000", 2024-05-11T23:02:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.707e-06 2024-05-11T23:02:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.397962+0000", 2024-05-11T23:02:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0084483300000000004 2024-05-11T23:02:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.397968+0000", 2024-05-11T23:02:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4980000000000003e-06 2024-05-11T23:02:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:100830 3.3 3.76f4d2e3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:02:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.389599+0000", 2024-05-11T23:02:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 21.595533795000001, 2024-05-11T23:02:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0083954479999999998, 2024-05-11T23:02:48.063 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 100830 2024-05-11T23:02:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389599+0000", 2024-05-11T23:02:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389599+0000", 2024-05-11T23:02:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389600+0000", 2024-05-11T23:02:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1100000000000005e-07 2024-05-11T23:02:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389604+0000", 2024-05-11T23:02:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0720000000000001e-06 2024-05-11T23:02:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389613+0000", 2024-05-11T23:02:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4369999999999996e-06 2024-05-11T23:02:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.389620+0000", 2024-05-11T23:02:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2480000000000001e-06 2024-05-11T23:02:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.397989+0000", 2024-05-11T23:02:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.008368973 2024-05-11T23:02:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.397995+0000", 2024-05-11T23:02:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9070000000000004e-06 2024-05-11T23:02:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:139214 3.3 3:c4c3de8a:::rbd_data.111ec437b352.0000000000000162:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-05-11T23:02:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:43.168656+0000", 2024-05-11T23:02:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.8164766070000002, 2024-05-11T23:02:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0073981209999999997, 2024-05-11T23:02:48.067 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 139214 2024-05-11T23:02:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.168656+0000", 2024-05-11T23:02:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.168656+0000", 2024-05-11T23:02:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.168658+0000", 2024-05-11T23:02:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9429999999999999e-06 2024-05-11T23:02:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.168668+0000", 2024-05-11T23:02:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5829999999999996e-06 2024-05-11T23:02:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.168689+0000", 2024-05-11T23:02:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1012000000000001e-05 2024-05-11T23:02:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.168699+0000", 2024-05-11T23:02:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0277e-05 2024-05-11T23:02:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.168816+0000", 2024-05-11T23:02:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000116564 2024-05-11T23:02:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.168972+0000", 2024-05-11T23:02:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00015664100000000001 2024-05-11T23:02:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 0", 2024-05-11T23:02:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.169611+0000", 2024-05-11T23:02:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00063818700000000002 2024-05-11T23:02:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:48.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.175899+0000", 2024-05-11T23:02:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0062883410000000002 2024-05-11T23:02:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:02:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.175958+0000", 2024-05-11T23:02:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8727999999999997e-05 2024-05-11T23:02:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "commit_sent", 2024-05-11T23:02:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.176009+0000", 2024-05-11T23:02:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1243999999999999e-05 2024-05-11T23:02:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:43.176054+0000", 2024-05-11T23:02:48.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5600999999999999e-05 2024-05-11T23:02:48.073 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.073 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.073 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.073 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.073 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.073 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:48.075 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:02:48.075 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:48.074+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:48.085 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-05-11T23:02:48.136 INFO:teuthology.orchestra.run.smithi005.stdout:4183 write 0x1efb3a3 thru 0x1f0667c (0xb2da bytes) 2024-05-11T23:02:48.155 INFO:teuthology.orchestra.run.smithi005.stdout:4184 read 0x871d37 thru 0x87d4a4 (0xb76e bytes) 2024-05-11T23:02:48.155 INFO:teuthology.orchestra.run.smithi005.stdout:4185 punch from 0x3bc56d9 to 0x3bc7db3, (0x26da bytes) 2024-05-11T23:02:48.155 INFO:teuthology.orchestra.run.smithi005.stdout:4186 writesame 0x9fff0e thru 0xa02d6b data_size 0x5(0x2e5e bytes) 2024-05-11T23:02:48.171 INFO:teuthology.orchestra.run.smithi005.stdout:4187 write 0x301236c thru 0x301be17 (0x9aac bytes) 2024-05-11T23:02:48.175 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:48.175+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:48.185 INFO:teuthology.orchestra.run.smithi005.stdout:4188 trunc from 0x458632f to 0x23236f5 2024-05-11T23:02:48.205 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:48.205 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:48.205 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:02:48.205 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:48.214 INFO:teuthology.orchestra.run.smithi005.stdout:4190 trunc from 0x23236f5 to 0x839071 2024-05-11T23:02:48.215 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:02:48.238 INFO:teuthology.orchestra.run.smithi005.stdout:4192 trunc from 0x839071 to 0x288bf6b 2024-05-11T23:02:48.244 INFO:teuthology.orchestra.run.smithi005.stdout:4193 read 0x44445f thru 0x44dbd7 (0x9779 bytes) 2024-05-11T23:02:48.244 INFO:teuthology.orchestra.run.smithi005.stdout:4194 read 0x22bf98b thru 0x22c02c0 (0x936 bytes) 2024-05-11T23:02:48.244 INFO:teuthology.orchestra.run.smithi005.stdout:4195 punch from 0x1390803 to 0x139bbab, (0xb3a8 bytes) 2024-05-11T23:02:48.250 INFO:teuthology.orchestra.run.smithi005.stdout:4197 writesame 0x4404675 thru 0x440853e data_size 0x39(0x3eca bytes) 2024-05-11T23:02:48.252 INFO:teuthology.orchestra.run.smithi005.stdout:4198 trunc from 0x440853f to 0x56050ec 2024-05-11T23:02:48.257 INFO:teuthology.orchestra.run.smithi005.stdout:4199 write 0x43767a5 thru 0x437828f (0x1aeb bytes) 2024-05-11T23:02:48.276 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:48.276+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:48.277 INFO:teuthology.orchestra.run.smithi005.stdout:4200 writesame 0xcf6d66a thru 0xcf756a8 data_size 0x1(0x803f bytes) 2024-05-11T23:02:48.277 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:48.277 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:48.277 INFO:teuthology.orchestra.run.smithi005.stdout:4204 read 0x40cb162 thru 0x40d6bec (0xba8b bytes) 2024-05-11T23:02:48.277 INFO:teuthology.orchestra.run.smithi005.stdout:4205 writesame 0x695605 thru 0x6982b6 data_size 0x3(0x2cb2 bytes) 2024-05-11T23:02:48.290 INFO:teuthology.orchestra.run.smithi005.stdout:4207 punch from 0x9906801 to 0x9915a32, (0xf231 bytes) 2024-05-11T23:02:48.290 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:48.290 INFO:teuthology.orchestra.run.smithi005.stdout:4210 read 0xb6cd28b thru 0xb6d8fc6 (0xbd3c bytes) 2024-05-11T23:02:48.297 INFO:teuthology.orchestra.run.smithi005.stdout:4211 writesame 0xe71cbae thru 0xe72775b data_size 0xf(0xabae bytes) 2024-05-11T23:02:48.297 INFO:teuthology.orchestra.run.smithi005.stdout:4212 write 0xdf9a10b thru 0xdf9c903 (0x27f9 bytes) 2024-05-11T23:02:48.312 INFO:teuthology.orchestra.run.smithi005.stdout:4214 trunc from 0xe72775c to 0xbc44065 2024-05-11T23:02:48.322 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:48.322 INFO:teuthology.orchestra.run.smithi005.stdout:4217 trunc from 0xbc44065 to 0xbfd334b 2024-05-11T23:02:48.327 INFO:teuthology.orchestra.run.smithi005.stdout:4218 read 0x4a150ba thru 0x4a1b114 (0x605b bytes) 2024-05-11T23:02:48.327 INFO:teuthology.orchestra.run.smithi005.stdout:4219 writesame 0x37133c2 thru 0x37231b6 data_size 0xd(0xfdf5 bytes) 2024-05-11T23:02:48.338 INFO:teuthology.orchestra.run.smithi005.stdout:4220 trunc from 0xbfd334b to 0xecbb26c 2024-05-11T23:02:48.344 INFO:teuthology.orchestra.run.smithi005.stdout:4222 write 0x4b422e1 thru 0x4b50fc9 (0xece9 bytes) 2024-05-11T23:02:48.351 INFO:teuthology.orchestra.run.smithi005.stdout:4224 writesame 0x522424e thru 0x52269e4 data_size 0x1(0x2797 bytes) 2024-05-11T23:02:48.361 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:48.361 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:48.361 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:48.361 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:48.361 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:48.363 INFO:teuthology.orchestra.run.smithi005.stdout:4226 trunc from 0xecbb26c to 0xcddb7dd 2024-05-11T23:02:48.369 INFO:teuthology.orchestra.run.smithi005.stdout:4227 punch from 0x5e457dd to 0x5e4b8f4, (0x6117 bytes) 2024-05-11T23:02:48.369 INFO:teuthology.orchestra.run.smithi005.stdout:4228 read 0xa485ad thru 0xa50e0c (0x8860 bytes) 2024-05-11T23:02:48.369 INFO:teuthology.orchestra.run.smithi005.stdout:4229 write 0x98706a6 thru 0x9872cd5 (0x2630 bytes) 2024-05-11T23:02:48.372 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:02:48.380 INFO:teuthology.orchestra.run.smithi005.stdout:4230 read 0x528d8a3 thru 0x529bbfc (0xe35a bytes) 2024-05-11T23:02:48.380 INFO:teuthology.orchestra.run.smithi005.stdout:4231 read 0x23a13d4 thru 0x23a21bb (0xde8 bytes) 2024-05-11T23:02:48.380 INFO:teuthology.orchestra.run.smithi005.stdout:4232 write 0x39e7b6e thru 0x39e92d2 (0x1765 bytes) 2024-05-11T23:02:48.380 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:48.380+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:48.391 INFO:teuthology.orchestra.run.smithi005.stdout:4233 write 0x5c75bd9 thru 0x5c7cd42 (0x716a bytes) 2024-05-11T23:02:48.402 INFO:teuthology.orchestra.run.smithi005.stdout:4234 punch from 0x81b53f0 to 0x81b6219, (0xe29 bytes) 2024-05-11T23:02:48.403 INFO:teuthology.orchestra.run.smithi005.stdout:4235 punch from 0x689f712 to 0x68abe47, (0xc735 bytes) 2024-05-11T23:02:48.415 INFO:teuthology.orchestra.run.smithi005.stdout:4236 trunc from 0xcddb7dd to 0x1baa432 2024-05-11T23:02:48.434 INFO:teuthology.orchestra.run.smithi005.stdout:4238 read 0x697e59 thru 0x69f064 (0x720c bytes) 2024-05-11T23:02:48.434 INFO:teuthology.orchestra.run.smithi005.stdout:4239 punch from 0xe2257a to 0xe2a224, (0x7caa bytes) 2024-05-11T23:02:48.449 INFO:teuthology.orchestra.run.smithi005.stdout:4240 writesame 0x85609fa thru 0x8562384 data_size 0x1(0x198b bytes) 2024-05-11T23:02:48.456 INFO:teuthology.orchestra.run.smithi005.stdout:4243 write 0xa4a47f7 thru 0xa4b3680 (0xee8a bytes) 2024-05-11T23:02:48.472 INFO:teuthology.orchestra.run.smithi005.stdout:4244 writesame 0xa7c98f7 thru 0xa7d90f2 data_size 0x4(0xf7fc bytes) 2024-05-11T23:02:48.472 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:48.472 INFO:teuthology.orchestra.run.smithi005.stdout:4246 punch from 0x8cf0bce to 0x8cf54a1, (0x48d3 bytes) 2024-05-11T23:02:48.472 INFO:teuthology.orchestra.run.smithi005.stdout:4247 writesame 0x65c4c1d thru 0x65ce8ac data_size 0x3c(0x9c90 bytes) 2024-05-11T23:02:48.476 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:48.477+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:48.481 INFO:teuthology.orchestra.run.smithi005.stdout:4248 punch from 0x1e52630 to 0x1e5e5b4, (0xbf84 bytes) 2024-05-11T23:02:48.482 INFO:teuthology.orchestra.run.smithi005.stdout:4249 read 0x94e0922 thru 0x94f00e6 (0xf7c5 bytes) 2024-05-11T23:02:48.482 INFO:teuthology.orchestra.run.smithi005.stdout:4250 writesame 0x1aca7af thru 0x1accdc1 data_size 0x1b(0x2613 bytes) 2024-05-11T23:02:48.491 INFO:teuthology.orchestra.run.smithi005.stdout:4251 read 0x8f82228 thru 0x8f8c42b (0xa204 bytes) 2024-05-11T23:02:48.491 INFO:teuthology.orchestra.run.smithi005.stdout:4252 writesame 0xfbd717 thru 0xfc94ec data_size 0x2f(0xbdd6 bytes) 2024-05-11T23:02:48.498 INFO:teuthology.orchestra.run.smithi005.stdout:4253 punch from 0x1d654cc to 0x1d688d6, (0x340a bytes) 2024-05-11T23:02:48.504 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:48.504 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:48.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:48.504 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36828 3.b 3:d385bc8e:::rbd_data.111e5e4d8275.0000000000000020:11 [read 512000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.570259602999997, 2024-05-11T23:02:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051481218000000002, 2024-05-11T23:02:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36828 2024-05-11T23:02:48.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934915+0000", 2024-05-11T23:02:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00181814 2024-05-11T23:02:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934919+0000", 2024-05-11T23:02:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6739999999999996e-06 2024-05-11T23:02:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934934+0000", 2024-05-11T23:02:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4415e-05 2024-05-11T23:02:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934942+0000", 2024-05-11T23:02:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8619999999999992e-06 2024-05-11T23:02:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984433+0000", 2024-05-11T23:02:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049490658999999999 2024-05-11T23:02:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984508+0000", 2024-05-11T23:02:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4394999999999995e-05 2024-05-11T23:02:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984578+0000", 2024-05-11T23:02:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0073000000000005e-05 2024-05-11T23:02:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.509 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36950 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 520192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.565953837000002, 2024-05-11T23:02:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051450856000000003, 2024-05-11T23:02:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36950 2024-05-11T23:02:48.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938914+0000", 2024-05-11T23:02:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015121480000000001 2024-05-11T23:02:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938918+0000", 2024-05-11T23:02:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4149999999999999e-06 2024-05-11T23:02:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938925+0000", 2024-05-11T23:02:48.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0779999999999998e-06 2024-05-11T23:02:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938929+0000", 2024-05-11T23:02:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2930000000000002e-06 2024-05-11T23:02:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988675+0000", 2024-05-11T23:02:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049746394999999999 2024-05-11T23:02:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988756+0000", 2024-05-11T23:02:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0499000000000003e-05 2024-05-11T23:02:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988853+0000", 2024-05-11T23:02:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7027999999999993e-05 2024-05-11T23:02:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36957 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 778240~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:48.514 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.564332047999997, 2024-05-11T23:02:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051328273000000001, 2024-05-11T23:02:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36957 2024-05-11T23:02:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2099999999999998e-07 2024-05-11T23:02:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939026+0000", 2024-05-11T23:02:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5069999999999999e-06 2024-05-11T23:02:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939030+0000", 2024-05-11T23:02:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7919999999999999e-06 2024-05-11T23:02:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939032+0000", 2024-05-11T23:02:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9129999999999999e-06 2024-05-11T23:02:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990178+0000", 2024-05-11T23:02:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051145373000000001 2024-05-11T23:02:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990255+0000", 2024-05-11T23:02:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7195999999999997e-05 2024-05-11T23:02:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990352+0000", 2024-05-11T23:02:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7170999999999995e-05 2024-05-11T23:02:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36958 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 815104~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.564319728999997, 2024-05-11T23:02:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051531625999999997, 2024-05-11T23:02:48.519 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36958 2024-05-11T23:02:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:48.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939037+0000", 2024-05-11T23:02:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0400000000000002e-07 2024-05-11T23:02:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939038+0000", 2024-05-11T23:02:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.164e-06 2024-05-11T23:02:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939041+0000", 2024-05-11T23:02:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1109999999999999e-06 2024-05-11T23:02:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939043+0000", 2024-05-11T23:02:48.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.554e-06 2024-05-11T23:02:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990382+0000", 2024-05-11T23:02:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051338953999999999 2024-05-11T23:02:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990465+0000", 2024-05-11T23:02:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2461999999999998e-05 2024-05-11T23:02:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990568+0000", 2024-05-11T23:02:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000103077 2024-05-11T23:02:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36959 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 851968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.564307833000001, 2024-05-11T23:02:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051758936999999998, 2024-05-11T23:02:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.524 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36959 2024-05-11T23:02:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.16e-07 2024-05-11T23:02:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939050+0000", 2024-05-11T23:02:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.176e-06 2024-05-11T23:02:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939053+0000", 2024-05-11T23:02:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1140000000000002e-06 2024-05-11T23:02:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939055+0000", 2024-05-11T23:02:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.621e-06 2024-05-11T23:02:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990610+0000", 2024-05-11T23:02:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051554244999999999 2024-05-11T23:02:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990700+0000", 2024-05-11T23:02:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0284999999999994e-05 2024-05-11T23:02:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990807+0000", 2024-05-11T23:02:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010728 2024-05-11T23:02:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36960 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 888832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.564297598000003, 2024-05-11T23:02:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051886831000000001, 2024-05-11T23:02:48.529 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36960 2024-05-11T23:02:48.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940913+0000", 2024-05-11T23:02:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001855104 2024-05-11T23:02:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940916+0000", 2024-05-11T23:02:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0000000000000001e-06 2024-05-11T23:02:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940924+0000", 2024-05-11T23:02:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0999999999999998e-06 2024-05-11T23:02:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940927+0000", 2024-05-11T23:02:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7160000000000001e-06 2024-05-11T23:02:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990826+0000", 2024-05-11T23:02:48.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049898832999999997 2024-05-11T23:02:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990872+0000", 2024-05-11T23:02:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5717000000000003e-05 2024-05-11T23:02:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990945+0000", 2024-05-11T23:02:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3361000000000005e-05 2024-05-11T23:02:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.534 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36986 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 528384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.562291379999998, 2024-05-11T23:02:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051341053999999997, 2024-05-11T23:02:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.535 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36986 2024-05-11T23:02:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:02:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941066+0000", 2024-05-11T23:02:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1629999999999999e-06 2024-05-11T23:02:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941069+0000", 2024-05-11T23:02:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.118e-06 2024-05-11T23:02:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941072+0000", 2024-05-11T23:02:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5720000000000001e-06 2024-05-11T23:02:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992292+0000", 2024-05-11T23:02:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051220620000000001 2024-05-11T23:02:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992344+0000", 2024-05-11T23:02:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1737000000000003e-05 2024-05-11T23:02:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992406+0000", 2024-05-11T23:02:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1623000000000003e-05 2024-05-11T23:02:48.539 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36987 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 565248~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.562281259000002, 2024-05-11T23:02:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051568725000000003, 2024-05-11T23:02:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.540 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36987 2024-05-11T23:02:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.91e-07 2024-05-11T23:02:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941076+0000", 2024-05-11T23:02:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1379999999999999e-06 2024-05-11T23:02:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941079+0000", 2024-05-11T23:02:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9739999999999998e-06 2024-05-11T23:02:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941082+0000", 2024-05-11T23:02:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-05-11T23:02:48.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992446+0000", 2024-05-11T23:02:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051364058999999997 2024-05-11T23:02:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992536+0000", 2024-05-11T23:02:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0085000000000002e-05 2024-05-11T23:02:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992643+0000", 2024-05-11T23:02:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107377 2024-05-11T23:02:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36988 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 602112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.562270890000001, 2024-05-11T23:02:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051700007999999999, 2024-05-11T23:02:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.545 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36988 2024-05-11T23:02:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.05e-07 2024-05-11T23:02:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941086+0000", 2024-05-11T23:02:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.125e-06 2024-05-11T23:02:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941089+0000", 2024-05-11T23:02:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0599999999999999e-06 2024-05-11T23:02:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941092+0000", 2024-05-11T23:02:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6529999999999998e-06 2024-05-11T23:02:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992667+0000", 2024-05-11T23:02:48.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051574587999999998 2024-05-11T23:02:48.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992723+0000", 2024-05-11T23:02:48.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6682999999999999e-05 2024-05-11T23:02:48.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992785+0000", 2024-05-11T23:02:48.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1693999999999996e-05 2024-05-11T23:02:48.550 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.550 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.550 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.550 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36989 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 638976~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:48.550 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:48.550 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.562260701, 2024-05-11T23:02:48.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05192811, 2024-05-11T23:02:48.550 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.550 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.551 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.551 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.551 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.551 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36989 2024-05-11T23:02:48.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.551 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:48.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:48.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942916+0000", 2024-05-11T23:02:48.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001820835 2024-05-11T23:02:48.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942920+0000", 2024-05-11T23:02:48.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5310000000000002e-06 2024-05-11T23:02:48.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942931+0000", 2024-05-11T23:02:48.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1839e-05 2024-05-11T23:02:48.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942940+0000", 2024-05-11T23:02:48.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2309999999999996e-06 2024-05-11T23:02:48.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992820+0000", 2024-05-11T23:02:48.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049880751000000001 2024-05-11T23:02:48.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992918+0000", 2024-05-11T23:02:48.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7051000000000003e-05 2024-05-11T23:02:48.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.993023+0000", 2024-05-11T23:02:48.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010587200000000001 2024-05-11T23:02:48.555 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.555 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.555 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.555 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36997 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 933888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:48.555 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:48.555 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.560226866000001, 2024-05-11T23:02:48.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051390602000000001, 2024-05-11T23:02:48.555 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.556 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.556 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.556 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.556 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36997 2024-05-11T23:02:48.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.556 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:48.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:48.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943130+0000", 2024-05-11T23:02:48.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0200000000000003e-07 2024-05-11T23:02:48.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943132+0000", 2024-05-11T23:02:48.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.305e-06 2024-05-11T23:02:48.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943138+0000", 2024-05-11T23:02:48.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4219999999999997e-06 2024-05-11T23:02:48.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943143+0000", 2024-05-11T23:02:48.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0610000000000003e-06 2024-05-11T23:02:48.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994323+0000", 2024-05-11T23:02:48.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051179592000000003 2024-05-11T23:02:48.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994412+0000", 2024-05-11T23:02:48.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8709999999999996e-05 2024-05-11T23:02:48.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994520+0000", 2024-05-11T23:02:48.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010811000000000001 2024-05-11T23:02:48.560 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.560 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.560 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.560 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36998 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 970752~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:48.560 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:48.560 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.560205987000003, 2024-05-11T23:02:48.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051517669000000002, 2024-05-11T23:02:48.561 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.561 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.561 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.561 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.561 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.561 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.561 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36998 2024-05-11T23:02:48.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.561 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:48.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:48.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:48.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:02:48.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943153+0000", 2024-05-11T23:02:48.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.306e-06 2024-05-11T23:02:48.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943160+0000", 2024-05-11T23:02:48.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.452e-06 2024-05-11T23:02:48.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943165+0000", 2024-05-11T23:02:48.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.152e-06 2024-05-11T23:02:48.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994561+0000", 2024-05-11T23:02:48.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051395765000000003 2024-05-11T23:02:48.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994607+0000", 2024-05-11T23:02:48.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6415000000000002e-05 2024-05-11T23:02:48.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994668+0000", 2024-05-11T23:02:48.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0179000000000001e-05 2024-05-11T23:02:48.565 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.565 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.565 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.565 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36999 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 1007616~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:48.565 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:48.565 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.560161106000002, 2024-05-11T23:02:48.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051696919000000001, 2024-05-11T23:02:48.565 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.565 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.565 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.566 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.566 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.566 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.566 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36999 2024-05-11T23:02:48.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.566 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:48.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:48.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:48.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:02:48.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943197+0000", 2024-05-11T23:02:48.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1280000000000002e-06 2024-05-11T23:02:48.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943202+0000", 2024-05-11T23:02:48.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8910000000000003e-06 2024-05-11T23:02:48.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943206+0000", 2024-05-11T23:02:48.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3550000000000001e-06 2024-05-11T23:02:48.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994703+0000", 2024-05-11T23:02:48.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051497045999999998 2024-05-11T23:02:48.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994789+0000", 2024-05-11T23:02:48.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6458000000000005e-05 2024-05-11T23:02:48.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994892+0000", 2024-05-11T23:02:48.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102705 2024-05-11T23:02:48.570 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.570 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.570 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.570 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37000 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 1044480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:48.570 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:48.570 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.560145161000001, 2024-05-11T23:02:48.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051842371999999998, 2024-05-11T23:02:48.570 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.571 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.571 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.571 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.571 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.571 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37000 2024-05-11T23:02:48.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.571 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:48.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:48.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944916+0000", 2024-05-11T23:02:48.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001705097 2024-05-11T23:02:48.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944921+0000", 2024-05-11T23:02:48.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5809999999999998e-06 2024-05-11T23:02:48.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944930+0000", 2024-05-11T23:02:48.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5529999999999998e-06 2024-05-11T23:02:48.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944938+0000", 2024-05-11T23:02:48.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2519999999999999e-06 2024-05-11T23:02:48.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994911+0000", 2024-05-11T23:02:48.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049973113999999999 2024-05-11T23:02:48.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994982+0000", 2024-05-11T23:02:48.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0544999999999996e-05 2024-05-11T23:02:48.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.995053+0000", 2024-05-11T23:02:48.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1229999999999994e-05 2024-05-11T23:02:48.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.575 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.575 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37041 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 618496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:48.575 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:48.575 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.558301182000001, 2024-05-11T23:02:48.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051352768999999999, 2024-05-11T23:02:48.575 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.575 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.575 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.576 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.576 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.576 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.576 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37041 2024-05-11T23:02:48.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.576 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:48.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:48.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:48.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:02:48.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945056+0000", 2024-05-11T23:02:48.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:02:48.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945059+0000", 2024-05-11T23:02:48.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.965e-06 2024-05-11T23:02:48.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945062+0000", 2024-05-11T23:02:48.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:02:48.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996213+0000", 2024-05-11T23:02:48.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051151532999999999 2024-05-11T23:02:48.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996301+0000", 2024-05-11T23:02:48.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8143e-05 2024-05-11T23:02:48.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996408+0000", 2024-05-11T23:02:48.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010623700000000001 2024-05-11T23:02:48.579 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.579 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.580 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37042 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 655360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:48.580 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:48.580 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.558291197000003, 2024-05-11T23:02:48.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051462075000000003, 2024-05-11T23:02:48.580 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.580 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.580 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.580 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.580 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.580 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.581 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37042 2024-05-11T23:02:48.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.581 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:48.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:48.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:48.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0599999999999999e-07 2024-05-11T23:02:48.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945066+0000", 2024-05-11T23:02:48.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-05-11T23:02:48.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945069+0000", 2024-05-11T23:02:48.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0079999999999998e-06 2024-05-11T23:02:48.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945072+0000", 2024-05-11T23:02:48.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6740000000000001e-06 2024-05-11T23:02:48.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996422+0000", 2024-05-11T23:02:48.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051349999 2024-05-11T23:02:48.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996467+0000", 2024-05-11T23:02:48.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.528e-05 2024-05-11T23:02:48.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996527+0000", 2024-05-11T23:02:48.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9759999999999997e-05 2024-05-11T23:02:48.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.584 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.585 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37043 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 692224~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:48.585 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:48.585 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.558279124000002, 2024-05-11T23:02:48.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051679535999999998, 2024-05-11T23:02:48.585 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37043 2024-05-11T23:02:48.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.585 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:48.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:48.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:48.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8999999999999997e-07 2024-05-11T23:02:48.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945079+0000", 2024-05-11T23:02:48.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.001e-06 2024-05-11T23:02:48.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945087+0000", 2024-05-11T23:02:48.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1749999999999999e-06 2024-05-11T23:02:48.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945093+0000", 2024-05-11T23:02:48.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9639999999999997e-06 2024-05-11T23:02:48.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996563+0000", 2024-05-11T23:02:48.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051469050000000002 2024-05-11T23:02:48.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996651+0000", 2024-05-11T23:02:48.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8356000000000006e-05 2024-05-11T23:02:48.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996756+0000", 2024-05-11T23:02:48.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0001055 2024-05-11T23:02:48.589 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.589 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.589 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.589 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37044 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 729088~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:48.589 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:48.590 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.558245212000003, 2024-05-11T23:02:48.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051763445999999998, 2024-05-11T23:02:48.590 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.590 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.590 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.590 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.590 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.590 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37044 2024-05-11T23:02:48.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.590 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:48.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:48.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946915+0000", 2024-05-11T23:02:48.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018042760000000001 2024-05-11T23:02:48.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946918+0000", 2024-05-11T23:02:48.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.36e-06 2024-05-11T23:02:48.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946926+0000", 2024-05-11T23:02:48.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3130000000000001e-06 2024-05-11T23:02:48.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946930+0000", 2024-05-11T23:02:48.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5249999999999999e-06 2024-05-11T23:02:48.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996772+0000", 2024-05-11T23:02:48.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049841868999999997 2024-05-11T23:02:48.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996817+0000", 2024-05-11T23:02:48.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5351e-05 2024-05-11T23:02:48.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996874+0000", 2024-05-11T23:02:48.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6752000000000001e-05 2024-05-11T23:02:48.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.594 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.594 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37052 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 1024000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:48.594 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:48.595 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.556338361000002, 2024-05-11T23:02:48.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051326140999999999, 2024-05-11T23:02:48.595 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.595 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37052 2024-05-11T23:02:48.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.595 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:48.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:48.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:48.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:48.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947019+0000", 2024-05-11T23:02:48.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.218e-06 2024-05-11T23:02:48.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947022+0000", 2024-05-11T23:02:48.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1889999999999999e-06 2024-05-11T23:02:48.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947025+0000", 2024-05-11T23:02:48.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.464e-06 2024-05-11T23:02:48.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998237+0000", 2024-05-11T23:02:48.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051212494999999997 2024-05-11T23:02:48.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998284+0000", 2024-05-11T23:02:48.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7246000000000001e-05 2024-05-11T23:02:48.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998344+0000", 2024-05-11T23:02:48.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9320000000000001e-05 2024-05-11T23:02:48.599 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.599 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.599 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.599 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37071 3.b 3:d385bc8e:::rbd_data.111e5e4d8275.0000000000000020:11 [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:48.599 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:48.599 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.556328151999999, 2024-05-11T23:02:48.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051528796000000002, 2024-05-11T23:02:48.600 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:48.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:48.600 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:48.600 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:48.600 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:48.600 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:48.600 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37071 2024-05-11T23:02:48.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.600 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:48.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:48.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:48.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:48.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:48.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:48.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:48.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948913+0000", 2024-05-11T23:02:48.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018854970000000001 2024-05-11T23:02:48.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:48.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948917+0000", 2024-05-11T23:02:48.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8089999999999999e-06 2024-05-11T23:02:48.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:48.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948926+0000", 2024-05-11T23:02:48.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8389999999999996e-06 2024-05-11T23:02:48.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:48.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948930+0000", 2024-05-11T23:02:48.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4519999999999999e-06 2024-05-11T23:02:48.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:48.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998379+0000", 2024-05-11T23:02:48.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049449011000000001 2024-05-11T23:02:48.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:48.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998462+0000", 2024-05-11T23:02:48.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2739999999999997e-05 2024-05-11T23:02:48.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:48.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:48.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:48.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998557+0000", 2024-05-11T23:02:48.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4448000000000001e-05 2024-05-11T23:02:48.604 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.604 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.604 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.604 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:48.604 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:48.604 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:48.606 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:48.606+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:48.606 INFO:teuthology.orchestra.run.smithi005.stdout:4255 writesame 0xbdf05f1 thru 0xbdf55d3 data_size 0x11(0x4fe3 bytes) 2024-05-11T23:02:48.606 INFO:teuthology.orchestra.run.smithi005.stdout:4257 read 0x7358dba thru 0x7359c33 (0xe7a bytes) 2024-05-11T23:02:48.606 INFO:teuthology.orchestra.run.smithi005.stdout:4258 read 0x207cae9 thru 0x2087927 (0xae3f bytes) 2024-05-11T23:02:48.606 INFO:teuthology.orchestra.run.smithi005.stdout:4259 trunc from 0xbdf55d4 to 0x17704f3 2024-05-11T23:02:48.608 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:02:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:4260 write 0xb1b7198 thru 0xb1c42e2 (0xd14b bytes) 2024-05-11T23:02:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:4261 read 0x117f991 thru 0x118aafc (0xb16c bytes) 2024-05-11T23:02:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:4263 read 0x35bea8c thru 0x35c9ea7 (0xb41c bytes) 2024-05-11T23:02:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:4265 write 0x7482f20 thru 0x7490f6a (0xe04b bytes) 2024-05-11T23:02:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:4266 punch from 0x705d493 to 0x705e2bb, (0xe28 bytes) 2024-05-11T23:02:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:4267 trunc from 0xb1c42e3 to 0x27ca5c9 2024-05-11T23:02:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:4269 write 0xfdc179 thru 0xfdf9c2 (0x384a bytes) 2024-05-11T23:02:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:4271 writesame 0xc9d81a thru 0xca5032 data_size 0xd(0x7819 bytes) 2024-05-11T23:02:48.610 INFO:teuthology.orchestra.run.smithi005.stdout:4272 write 0x125641c thru 0x1257a70 (0x1655 bytes) 2024-05-11T23:02:48.619 INFO:teuthology.orchestra.run.smithi005.stdout:4274 punch from 0x170d093 to 0x1712b09, (0x5a76 bytes) 2024-05-11T23:02:48.627 INFO:teuthology.orchestra.run.smithi005.stdout:4275 trunc from 0x27ca5c9 to 0x999f5df 2024-05-11T23:02:48.638 INFO:teuthology.orchestra.run.smithi005.stdout:4276 trunc from 0x999f5df to 0xec26b4a 2024-05-11T23:02:48.644 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:48.644 INFO:teuthology.orchestra.run.smithi005.stdout:4278 punch from 0x1c51b47 to 0x1c615dd, (0xfa96 bytes) 2024-05-11T23:02:48.644 INFO:teuthology.orchestra.run.smithi005.stdout:4280 writesame 0x14b0018 thru 0x14b58af data_size 0x2(0x5898 bytes) 2024-05-11T23:02:48.652 INFO:teuthology.orchestra.run.smithi005.stdout:4281 punch from 0xce66db6 to 0xce6dbc8, (0x6e12 bytes) 2024-05-11T23:02:48.652 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:48.654 INFO:teuthology.orchestra.run.smithi005.stdout:4283 trunc from 0xec26b4a to 0xd0e167d 2024-05-11T23:02:48.660 INFO:teuthology.orchestra.run.smithi005.stdout:4284 read 0xbe5089b thru 0xbe5359e (0x2d04 bytes) 2024-05-11T23:02:48.660 INFO:teuthology.orchestra.run.smithi005.stdout:4285 writesame 0x3d6deb3 thru 0x3d75eb2 data_size 0x40(0x8000 bytes) 2024-05-11T23:02:48.667 INFO:teuthology.orchestra.run.smithi005.stdout:4286 read 0x619debb thru 0x61a9697 (0xb7dd bytes) 2024-05-11T23:02:48.667 INFO:teuthology.orchestra.run.smithi005.stdout:4289 read 0x4fbe592 thru 0x4fcc07f (0xdaee bytes) 2024-05-11T23:02:48.667 INFO:teuthology.orchestra.run.smithi005.stdout:4290 write 0x7241de4 thru 0x7246848 (0x4a65 bytes) 2024-05-11T23:02:48.674 INFO:teuthology.orchestra.run.smithi005.stdout:4291 read 0xbb96244 thru 0xbb9981d (0x35da bytes) 2024-05-11T23:02:48.686 INFO:teuthology.orchestra.run.smithi005.stdout:4292 trunc from 0xd0e167d to 0x2b7ae32 2024-05-11T23:02:48.698 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:48.706 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:48.706+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:48.708 INFO:teuthology.orchestra.run.smithi005.stdout:4294 trunc from 0x2b7ae32 to 0xb954a31 2024-05-11T23:02:48.716 INFO:teuthology.orchestra.run.smithi005.stdout:4295 writesame 0xb4fa7c4 thru 0xb501c1e data_size 0x1(0x745b bytes) 2024-05-11T23:02:48.727 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:48.727 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:48.728 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:48.728 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:48.728 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:48.732 INFO:teuthology.orchestra.run.smithi005.stdout:4297 trunc from 0xb954a31 to 0x6dce611 2024-05-11T23:02:48.737 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:02:48.745 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:48.745 INFO:teuthology.orchestra.run.smithi005.stdout:4300 punch from 0x4323702 to 0x4323900, (0x1fe bytes) 2024-05-11T23:02:48.745 INFO:teuthology.orchestra.run.smithi005.stdout:4302 read 0x3fa4e2f thru 0x3fab142 (0x6314 bytes) 2024-05-11T23:02:48.745 INFO:teuthology.orchestra.run.smithi005.stdout:4303 write 0x5156353 thru 0x515fdb1 (0x9a5f bytes) 2024-05-11T23:02:48.752 INFO:teuthology.orchestra.run.smithi005.stdout:4304 punch from 0x4c45417 to 0x4c48f52, (0x3b3b bytes) 2024-05-11T23:02:48.765 INFO:teuthology.orchestra.run.smithi005.stdout:4307 write 0xbe322a5 thru 0xbe3cbf2 (0xa94e bytes) 2024-05-11T23:02:48.776 INFO:teuthology.orchestra.run.smithi005.stdout:4308 write 0x6d09f49 thru 0x6d0ea81 (0x4b39 bytes) 2024-05-11T23:02:48.787 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:48.787 INFO:teuthology.orchestra.run.smithi005.stdout:4310 read 0x87c480f thru 0x87cadad (0x659f bytes) 2024-05-11T23:02:48.788 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:48.788 INFO:teuthology.orchestra.run.smithi005.stdout:4313 write 0x15139c6 thru 0x151ba0d (0x8048 bytes) 2024-05-11T23:02:48.790 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:48.798 INFO:teuthology.orchestra.run.smithi005.stdout:4315 writesame 0xd7c88d2 thru 0xd7c9a18 data_size 0x1(0x1147 bytes) 2024-05-11T23:02:48.798 INFO:teuthology.orchestra.run.smithi005.stdout:4316 read 0x650b93d thru 0x650ee99 (0x355d bytes) 2024-05-11T23:02:48.798 INFO:teuthology.orchestra.run.smithi005.stdout:4319 write 0x8902989 thru 0x890f44f (0xcac7 bytes) 2024-05-11T23:02:48.807 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:48.808+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:48.818 INFO:teuthology.orchestra.run.smithi005.stdout:4320 write 0xe8ac042 thru 0xe8b9f28 (0xdee7 bytes) 2024-05-11T23:02:48.827 INFO:teuthology.orchestra.run.smithi005.stdout:4321 read 0x111947f thru 0x11211f1 (0x7d73 bytes) 2024-05-11T23:02:48.828 INFO:teuthology.orchestra.run.smithi005.stdout:4323 read 0x8f78c92 thru 0x8f7bae5 (0x2e54 bytes) 2024-05-11T23:02:48.828 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:48.828 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:48.840 INFO:teuthology.orchestra.run.smithi005.stdout:4326 trunc from 0xe8b9f29 to 0x395e30f 2024-05-11T23:02:48.857 INFO:teuthology.orchestra.run.smithi005.stdout:4327 read 0x2592a1e thru 0x2598950 (0x5f33 bytes) 2024-05-11T23:02:48.857 INFO:teuthology.orchestra.run.smithi005.stdout:4328 read 0x2bc1338 thru 0x2bcac98 (0x9961 bytes) 2024-05-11T23:02:48.858 INFO:teuthology.orchestra.run.smithi005.stdout:4329 read 0xd3f835 thru 0xd4c5e9 (0xcdb5 bytes) 2024-05-11T23:02:48.858 INFO:teuthology.orchestra.run.smithi005.stdout:4332 writesame 0x2bf697f thru 0x2bf7b32 data_size 0xb(0x11b4 bytes) 2024-05-11T23:02:48.860 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:48.860 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:48.866 INFO:teuthology.orchestra.run.smithi005.stdout:4333 read 0x6c6aa7 thru 0x6c8c86 (0x21e0 bytes) 2024-05-11T23:02:48.866 INFO:teuthology.orchestra.run.smithi005.stdout:4334 punch from 0x27fc4c to 0x287be6, (0x7f9a bytes) 2024-05-11T23:02:48.866 INFO:teuthology.orchestra.run.smithi005.stdout:4338 read 0x32f7a7c thru 0x32fa002 (0x2587 bytes) 2024-05-11T23:02:48.869 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:02:48.873 INFO:teuthology.orchestra.run.smithi005.stdout:4340 writesame 0x48dfde4 thru 0x48e781f data_size 0x2(0x7a3c bytes) 2024-05-11T23:02:48.884 INFO:teuthology.orchestra.run.smithi005.stdout:4341 trunc from 0x48e7820 to 0xe31d3f4 2024-05-11T23:02:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:4342 write 0xac08dae thru 0xac0d152 (0x43a5 bytes) 2024-05-11T23:02:48.897 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:48.897 INFO:teuthology.orchestra.run.smithi005.stdout:4344 write 0x3472507 thru 0x347dae4 (0xb5de bytes) 2024-05-11T23:02:48.903 INFO:teuthology.orchestra.run.smithi005.stdout:4345 clone 41 order 21 su 131072 sc 13 2024-05-11T23:02:48.910 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:48.911+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:49.008 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:49.009+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:49.015 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:49.015 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:49.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:49.016 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:49.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.016 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116446 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8273920~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:49.016 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:02:49.016 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.639410632000001, 2024-05-11T23:02:49.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045535299000000001, 2024-05-11T23:02:49.016 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.016 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.016 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.016 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.016 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.016 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.016 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116446 2024-05-11T23:02:49.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.017 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:02:49.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:02:49.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375547+0000", 2024-05-11T23:02:49.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4000000000000001e-07 2024-05-11T23:02:49.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375549+0000", 2024-05-11T23:02:49.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.368e-06 2024-05-11T23:02:49.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375556+0000", 2024-05-11T23:02:49.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1840000000000004e-06 2024-05-11T23:02:49.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375560+0000", 2024-05-11T23:02:49.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.741e-06 2024-05-11T23:02:49.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420896+0000", 2024-05-11T23:02:49.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045335435 2024-05-11T23:02:49.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420987+0000", 2024-05-11T23:02:49.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1033999999999996e-05 2024-05-11T23:02:49.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421082+0000", 2024-05-11T23:02:49.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4896999999999996e-05 2024-05-11T23:02:49.020 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.020 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.020 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.020 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116447 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6316032~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:49.020 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:02:49.020 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.639389122000001, 2024-05-11T23:02:49.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045745830000000001, 2024-05-11T23:02:49.020 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.020 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.020 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.020 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.020 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.021 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116447 2024-05-11T23:02:49.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.021 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:02:49.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:02:49.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:02:49.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.58e-07 2024-05-11T23:02:49.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375570+0000", 2024-05-11T23:02:49.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:02:49.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375576+0000", 2024-05-11T23:02:49.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7939999999999999e-06 2024-05-11T23:02:49.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375581+0000", 2024-05-11T23:02:49.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9280000000000001e-06 2024-05-11T23:02:49.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421127+0000", 2024-05-11T23:02:49.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045545785999999998 2024-05-11T23:02:49.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421214+0000", 2024-05-11T23:02:49.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7039000000000005e-05 2024-05-11T23:02:49.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421314+0000", 2024-05-11T23:02:49.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9772999999999995e-05 2024-05-11T23:02:49.024 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.024 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.024 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.024 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116448 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8314880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:49.024 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:02:49.024 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.639370048, 2024-05-11T23:02:49.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045941753000000002, 2024-05-11T23:02:49.024 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.024 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.025 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.025 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116448 2024-05-11T23:02:49.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.025 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:02:49.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:02:49.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:02:49.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.77e-07 2024-05-11T23:02:49.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375589+0000", 2024-05-11T23:02:49.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.091e-06 2024-05-11T23:02:49.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375596+0000", 2024-05-11T23:02:49.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9179999999999997e-06 2024-05-11T23:02:49.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375601+0000", 2024-05-11T23:02:49.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0030000000000002e-06 2024-05-11T23:02:49.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421363+0000", 2024-05-11T23:02:49.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045761352999999998 2024-05-11T23:02:49.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421440+0000", 2024-05-11T23:02:49.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7585999999999999e-05 2024-05-11T23:02:49.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421529+0000", 2024-05-11T23:02:49.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8424999999999994e-05 2024-05-11T23:02:49.031 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.031 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.032 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.032 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116449 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6438912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:49.032 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:02:49.032 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.639349613, 2024-05-11T23:02:49.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046082208999999999, 2024-05-11T23:02:49.032 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.032 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.032 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.032 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.032 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.032 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.032 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116449 2024-05-11T23:02:49.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.033 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:02:49.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:02:49.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377225+0000", 2024-05-11T23:02:49.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016177030000000001 2024-05-11T23:02:49.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377229+0000", 2024-05-11T23:02:49.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8e-06 2024-05-11T23:02:49.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377238+0000", 2024-05-11T23:02:49.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2229999999999996e-06 2024-05-11T23:02:49.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377244+0000", 2024-05-11T23:02:49.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4060000000000002e-06 2024-05-11T23:02:49.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421520+0000", 2024-05-11T23:02:49.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044275222000000003 2024-05-11T23:02:49.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421595+0000", 2024-05-11T23:02:49.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4838000000000004e-05 2024-05-11T23:02:49.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421690+0000", 2024-05-11T23:02:49.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5017000000000002e-05 2024-05-11T23:02:49.036 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.036 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.036 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.036 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116458 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7094272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:49.036 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:02:49.036 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.637535396000001, 2024-05-11T23:02:49.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045706877999999999, 2024-05-11T23:02:49.036 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.036 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.036 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.036 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.036 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.037 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.037 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116458 2024-05-11T23:02:49.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.037 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:02:49.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:02:49.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:02:49.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9400000000000005e-07 2024-05-11T23:02:49.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377424+0000", 2024-05-11T23:02:49.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1859999999999999e-06 2024-05-11T23:02:49.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377431+0000", 2024-05-11T23:02:49.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4049999999999997e-06 2024-05-11T23:02:49.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377436+0000", 2024-05-11T23:02:49.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1399999999999999e-06 2024-05-11T23:02:49.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.422945+0000", 2024-05-11T23:02:49.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045509326000000003 2024-05-11T23:02:49.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423036+0000", 2024-05-11T23:02:49.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0649999999999994e-05 2024-05-11T23:02:49.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423128+0000", 2024-05-11T23:02:49.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2577000000000005e-05 2024-05-11T23:02:49.040 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.040 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.040 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.040 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116459 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7258112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:49.040 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:02:49.040 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.637514663999999, 2024-05-11T23:02:49.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045831945999999998, 2024-05-11T23:02:49.040 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.040 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.041 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.041 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.041 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.041 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.041 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116459 2024-05-11T23:02:49.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.041 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:02:49.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:02:49.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377443+0000", 2024-05-11T23:02:49.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999999e-07 2024-05-11T23:02:49.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377445+0000", 2024-05-11T23:02:49.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0209999999999998e-06 2024-05-11T23:02:49.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377451+0000", 2024-05-11T23:02:49.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.071e-06 2024-05-11T23:02:49.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377456+0000", 2024-05-11T23:02:49.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8289999999999998e-06 2024-05-11T23:02:49.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423155+0000", 2024-05-11T23:02:49.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045699447999999997 2024-05-11T23:02:49.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423200+0000", 2024-05-11T23:02:49.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5083000000000002e-05 2024-05-11T23:02:49.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423274+0000", 2024-05-11T23:02:49.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4046000000000002e-05 2024-05-11T23:02:49.045 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.045 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.045 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.045 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116460 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7299072~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:49.045 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:02:49.045 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.637495602, 2024-05-11T23:02:49.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046010987000000003, 2024-05-11T23:02:49.045 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.045 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.045 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.045 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.046 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.046 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.046 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116460 2024-05-11T23:02:49.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.046 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:02:49.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:02:49.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377462+0000", 2024-05-11T23:02:49.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:02:49.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377464+0000", 2024-05-11T23:02:49.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1600000000000001e-06 2024-05-11T23:02:49.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377469+0000", 2024-05-11T23:02:49.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5430000000000001e-06 2024-05-11T23:02:49.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377474+0000", 2024-05-11T23:02:49.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8509999999999998e-06 2024-05-11T23:02:49.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423312+0000", 2024-05-11T23:02:49.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045838080000000003 2024-05-11T23:02:49.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423401+0000", 2024-05-11T23:02:49.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8170000000000005e-05 2024-05-11T23:02:49.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423472+0000", 2024-05-11T23:02:49.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1847000000000005e-05 2024-05-11T23:02:49.049 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.049 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.049 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.049 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116461 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7421952~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:49.049 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:02:49.049 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.637476832000001, 2024-05-11T23:02:49.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046130749999999998, 2024-05-11T23:02:49.049 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.050 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.050 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.050 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.050 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.050 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.050 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116461 2024-05-11T23:02:49.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.050 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:02:49.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:02:49.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:02:49.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4700000000000002e-07 2024-05-11T23:02:49.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377483+0000", 2024-05-11T23:02:49.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0219999999999999e-06 2024-05-11T23:02:49.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377489+0000", 2024-05-11T23:02:49.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4420000000000003e-06 2024-05-11T23:02:49.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377494+0000", 2024-05-11T23:02:49.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9239999999999998e-06 2024-05-11T23:02:49.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423500+0000", 2024-05-11T23:02:49.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046006054999999997 2024-05-11T23:02:49.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423557+0000", 2024-05-11T23:02:49.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7169000000000001e-05 2024-05-11T23:02:49.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423611+0000", 2024-05-11T23:02:49.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3791e-05 2024-05-11T23:02:49.053 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.053 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.053 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.053 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116462 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7462912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:49.053 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:02:49.054 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.637457213999999, 2024-05-11T23:02:49.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046254666, 2024-05-11T23:02:49.054 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.054 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.054 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.054 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.054 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.054 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.054 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116462 2024-05-11T23:02:49.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.054 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:02:49.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:02:49.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:02:49.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:02:49.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377502+0000", 2024-05-11T23:02:49.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-05-11T23:02:49.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377508+0000", 2024-05-11T23:02:49.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4500000000000003e-06 2024-05-11T23:02:49.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377512+0000", 2024-05-11T23:02:49.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7700000000000001e-06 2024-05-11T23:02:49.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423649+0000", 2024-05-11T23:02:49.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046136859000000002 2024-05-11T23:02:49.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423704+0000", 2024-05-11T23:02:49.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4938000000000002e-05 2024-05-11T23:02:49.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423754+0000", 2024-05-11T23:02:49.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0145999999999997e-05 2024-05-11T23:02:49.057 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.057 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.058 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.058 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116463 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7585792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:49.058 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:02:49.058 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.637438788000001, 2024-05-11T23:02:49.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361763, 2024-05-11T23:02:49.058 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.058 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.058 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.058 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.058 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.058 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.058 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116463 2024-05-11T23:02:49.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.059 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:02:49.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:02:49.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377519+0000", 2024-05-11T23:02:49.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2600000000000002e-07 2024-05-11T23:02:49.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377521+0000", 2024-05-11T23:02:49.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1710000000000001e-06 2024-05-11T23:02:49.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377527+0000", 2024-05-11T23:02:49.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9649999999999996e-06 2024-05-11T23:02:49.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377532+0000", 2024-05-11T23:02:49.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.78e-06 2024-05-11T23:02:49.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423779+0000", 2024-05-11T23:02:49.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046247706 2024-05-11T23:02:49.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423829+0000", 2024-05-11T23:02:49.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9188999999999999e-05 2024-05-11T23:02:49.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423880+0000", 2024-05-11T23:02:49.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1425999999999999e-05 2024-05-11T23:02:49.061 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.062 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.062 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.062 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116464 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7749632~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:49.062 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:02:49.062 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.637419552000001, 2024-05-11T23:02:49.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046491719000000001, 2024-05-11T23:02:49.062 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.062 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.062 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.062 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.062 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.062 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.062 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116464 2024-05-11T23:02:49.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.063 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:02:49.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:02:49.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377538+0000", 2024-05-11T23:02:49.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4499999999999998e-07 2024-05-11T23:02:49.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377540+0000", 2024-05-11T23:02:49.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2340000000000001e-06 2024-05-11T23:02:49.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377546+0000", 2024-05-11T23:02:49.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8289999999999996e-06 2024-05-11T23:02:49.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377551+0000", 2024-05-11T23:02:49.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8559999999999998e-06 2024-05-11T23:02:49.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423913+0000", 2024-05-11T23:02:49.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361885999999998 2024-05-11T23:02:49.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423958+0000", 2024-05-11T23:02:49.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5513999999999998e-05 2024-05-11T23:02:49.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424029+0000", 2024-05-11T23:02:49.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1055000000000003e-05 2024-05-11T23:02:49.066 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.066 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.066 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.066 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116465 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7626752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:49.066 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:02:49.066 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.637400403999999, 2024-05-11T23:02:49.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046600057, 2024-05-11T23:02:49.066 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.066 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.066 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.066 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.066 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.066 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.067 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116465 2024-05-11T23:02:49.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.067 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:02:49.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:02:49.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:02:49.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3599999999999999e-07 2024-05-11T23:02:49.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377559+0000", 2024-05-11T23:02:49.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9640000000000002e-06 2024-05-11T23:02:49.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377572+0000", 2024-05-11T23:02:49.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3188e-05 2024-05-11T23:02:49.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377578+0000", 2024-05-11T23:02:49.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.942e-06 2024-05-11T23:02:49.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424058+0000", 2024-05-11T23:02:49.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046480021000000003 2024-05-11T23:02:49.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424103+0000", 2024-05-11T23:02:49.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5021999999999997e-05 2024-05-11T23:02:49.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424157+0000", 2024-05-11T23:02:49.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3483999999999998e-05 2024-05-11T23:02:49.070 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.070 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.070 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.070 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116466 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7790592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:49.070 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:02:49.070 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.637371822, 2024-05-11T23:02:49.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046743741999999998, 2024-05-11T23:02:49.070 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.070 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.070 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.070 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.070 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.070 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.071 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116466 2024-05-11T23:02:49.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.071 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:02:49.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:02:49.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377586+0000", 2024-05-11T23:02:49.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:02:49.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377588+0000", 2024-05-11T23:02:49.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3039999999999999e-06 2024-05-11T23:02:49.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377594+0000", 2024-05-11T23:02:49.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-05-11T23:02:49.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377600+0000", 2024-05-11T23:02:49.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7699999999999998e-06 2024-05-11T23:02:49.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424185+0000", 2024-05-11T23:02:49.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046585891999999997 2024-05-11T23:02:49.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424235+0000", 2024-05-11T23:02:49.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9356e-05 2024-05-11T23:02:49.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424329+0000", 2024-05-11T23:02:49.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4036999999999994e-05 2024-05-11T23:02:49.074 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.074 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.074 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.074 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116467 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7954432~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:49.074 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:02:49.074 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.637351535000001, 2024-05-11T23:02:49.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046930134999999998, 2024-05-11T23:02:49.074 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.074 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.074 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.074 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.075 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.075 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.075 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116467 2024-05-11T23:02:49.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.075 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:02:49.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:02:49.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379161+0000", 2024-05-11T23:02:49.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015555009999999999 2024-05-11T23:02:49.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379166+0000", 2024-05-11T23:02:49.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6689999999999997e-06 2024-05-11T23:02:49.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379176+0000", 2024-05-11T23:02:49.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0549e-05 2024-05-11T23:02:49.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379183+0000", 2024-05-11T23:02:49.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1199999999999996e-06 2024-05-11T23:02:49.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424359+0000", 2024-05-11T23:02:49.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045175949999999999 2024-05-11T23:02:49.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424433+0000", 2024-05-11T23:02:49.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3824000000000003e-05 2024-05-11T23:02:49.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424536+0000", 2024-05-11T23:02:49.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010252200000000001 2024-05-11T23:02:49.078 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.078 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.078 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.078 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116468 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7913472~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:49.078 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:02:49.078 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.635761316, 2024-05-11T23:02:49.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045525364999999998, 2024-05-11T23:02:49.078 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.079 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.079 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.079 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.079 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.079 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.079 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116468 2024-05-11T23:02:49.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.079 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:02:49.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:02:49.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:02:49.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3300000000000002e-07 2024-05-11T23:02:49.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379198+0000", 2024-05-11T23:02:49.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2670000000000001e-06 2024-05-11T23:02:49.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379206+0000", 2024-05-11T23:02:49.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7149999999999998e-06 2024-05-11T23:02:49.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379228+0000", 2024-05-11T23:02:49.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1429000000000001e-05 2024-05-11T23:02:49.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424574+0000", 2024-05-11T23:02:49.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045346209999999998 2024-05-11T23:02:49.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424627+0000", 2024-05-11T23:02:49.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3352e-05 2024-05-11T23:02:49.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424721+0000", 2024-05-11T23:02:49.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3858999999999997e-05 2024-05-11T23:02:49.082 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.082 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.082 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.082 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116469 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8077312~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:49.082 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:02:49.083 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.635711496000001, 2024-05-11T23:02:49.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045636687000000002, 2024-05-11T23:02:49.083 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.083 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.083 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.083 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.083 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116469 2024-05-11T23:02:49.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.083 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:02:49.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:02:49.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379246+0000", 2024-05-11T23:02:49.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.32e-07 2024-05-11T23:02:49.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379254+0000", 2024-05-11T23:02:49.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.807e-06 2024-05-11T23:02:49.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379261+0000", 2024-05-11T23:02:49.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.985e-06 2024-05-11T23:02:49.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379264+0000", 2024-05-11T23:02:49.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6480000000000001e-06 2024-05-11T23:02:49.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424744+0000", 2024-05-11T23:02:49.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045479638000000003 2024-05-11T23:02:49.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424797+0000", 2024-05-11T23:02:49.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2725000000000001e-05 2024-05-11T23:02:49.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424882+0000", 2024-05-11T23:02:49.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5451999999999994e-05 2024-05-11T23:02:49.086 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.086 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.086 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.086 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116470 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8118272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:49.087 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:02:49.087 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.635686284, 2024-05-11T23:02:49.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045755745, 2024-05-11T23:02:49.087 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.087 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.087 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.087 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.087 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.087 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.087 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116470 2024-05-11T23:02:49.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.087 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:02:49.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:02:49.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:02:49.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3999999999999997e-07 2024-05-11T23:02:49.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379273+0000", 2024-05-11T23:02:49.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-05-11T23:02:49.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379277+0000", 2024-05-11T23:02:49.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4020000000000004e-06 2024-05-11T23:02:49.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379280+0000", 2024-05-11T23:02:49.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-05-11T23:02:49.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424922+0000", 2024-05-11T23:02:49.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045642598999999999 2024-05-11T23:02:49.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424973+0000", 2024-05-11T23:02:49.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0674999999999998e-05 2024-05-11T23:02:49.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425026+0000", 2024-05-11T23:02:49.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3492000000000002e-05 2024-05-11T23:02:49.090 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.090 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.090 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.091 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116471 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8241152~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:49.091 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:02:49.091 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.635673500999999, 2024-05-11T23:02:49.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045867374000000002, 2024-05-11T23:02:49.091 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.091 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.091 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.091 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.091 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.091 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.091 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116471 2024-05-11T23:02:49.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.091 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:02:49.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:02:49.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379284+0000", 2024-05-11T23:02:49.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.53e-07 2024-05-11T23:02:49.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379285+0000", 2024-05-11T23:02:49.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.223e-06 2024-05-11T23:02:49.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379288+0000", 2024-05-11T23:02:49.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5379999999999998e-06 2024-05-11T23:02:49.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379291+0000", 2024-05-11T23:02:49.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6699999999999998e-06 2024-05-11T23:02:49.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425055+0000", 2024-05-11T23:02:49.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045763497 2024-05-11T23:02:49.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425099+0000", 2024-05-11T23:02:49.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3899000000000001e-05 2024-05-11T23:02:49.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425151+0000", 2024-05-11T23:02:49.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2293999999999998e-05 2024-05-11T23:02:49.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.094 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.094 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116472 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8282112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:49.095 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:02:49.095 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.635662502000001, 2024-05-11T23:02:49.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045973778999999999, 2024-05-11T23:02:49.095 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.095 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.095 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.095 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.095 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.095 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.095 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116472 2024-05-11T23:02:49.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.095 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:02:49.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:02:49.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381226+0000", 2024-05-11T23:02:49.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001931627 2024-05-11T23:02:49.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381231+0000", 2024-05-11T23:02:49.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8679999999999998e-06 2024-05-11T23:02:49.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381242+0000", 2024-05-11T23:02:49.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0702e-05 2024-05-11T23:02:49.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381249+0000", 2024-05-11T23:02:49.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9800000000000001e-06 2024-05-11T23:02:49.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425175+0000", 2024-05-11T23:02:49.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043926805999999999 2024-05-11T23:02:49.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425215+0000", 2024-05-11T23:02:49.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9737999999999998e-05 2024-05-11T23:02:49.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425268+0000", 2024-05-11T23:02:49.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3057999999999999e-05 2024-05-11T23:02:49.098 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.098 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.099 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.099 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116482 3.1b 3:db6e8843:::rbd_object_map.111e67f27da3.0000000000000028:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:49.099 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:02:49.099 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 13.633693959, 2024-05-11T23:02:49.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046526076, 2024-05-11T23:02:49.099 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.099 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.099 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:49.099 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.099 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.099 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.099 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116482 2024-05-11T23:02:49.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.099 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:02:49.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:02:49.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381264+0000", 2024-05-11T23:02:49.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5300000000000004e-07 2024-05-11T23:02:49.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381267+0000", 2024-05-11T23:02:49.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1999999999999999e-06 2024-05-11T23:02:49.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381274+0000", 2024-05-11T23:02:49.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7109999999999995e-06 2024-05-11T23:02:49.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381285+0000", 2024-05-11T23:02:49.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0856000000000001e-05 2024-05-11T23:02:49.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425292+0000", 2024-05-11T23:02:49.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044006376 2024-05-11T23:02:49.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425337+0000", 2024-05-11T23:02:49.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4907999999999998e-05 2024-05-11T23:02:49.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:02:49.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425507+0000", 2024-05-11T23:02:49.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017029999999999999 2024-05-11T23:02:49.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:49.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427668+0000", 2024-05-11T23:02:49.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021612430000000002 2024-05-11T23:02:49.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:49.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427683+0000", 2024-05-11T23:02:49.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5016e-05 2024-05-11T23:02:49.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:49.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427742+0000", 2024-05-11T23:02:49.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8811999999999999e-05 2024-05-11T23:02:49.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427789+0000", 2024-05-11T23:02:49.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7101000000000002e-05 2024-05-11T23:02:49.104 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.104 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.104 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.104 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.104 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.104 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:49.105 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:02:49.149 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:49.150+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:49.156 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone40 from 0xe31d3f4 (overlap 0x839071) to 0x0 2024-05-11T23:02:49.209 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:49.209+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:49.244 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:49.245 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:49.245 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:49.245 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:49.256 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:02:49.310 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:49.310+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:49.380 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-05-11T23:02:49.411 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:49.412+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:49.416 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:49.417 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:49.426 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:02:49.476 INFO:teuthology.orchestra.run.smithi005.stdout:4346 punch from 0xaf9a997 to 0xafa911b, (0xe784 bytes) 2024-05-11T23:02:49.481 INFO:teuthology.orchestra.run.smithi005.stdout:4348 write 0xd45e73 thru 0xd518bf (0xba4d bytes) 2024-05-11T23:02:49.486 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:02:49.491 INFO:teuthology.orchestra.run.smithi005.stdout:4350 writesame 0xcf0a07b thru 0xcf0b417 data_size 0x1(0x139d bytes) 2024-05-11T23:02:49.496 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:49.496 INFO:teuthology.orchestra.run.smithi005.stdout:4353 writesame 0x2bda28d thru 0x2bdc7fc data_size 0x8(0x2570 bytes) 2024-05-11T23:02:49.501 INFO:teuthology.orchestra.run.smithi005.stdout:4354 read 0x7b0fd9e thru 0x7b127cc (0x2a2f bytes) 2024-05-11T23:02:49.501 INFO:teuthology.orchestra.run.smithi005.stdout:4355 writesame 0x9c5c07f thru 0x9c5d096 data_size 0x14(0x1018 bytes) 2024-05-11T23:02:49.505 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:49.505 INFO:teuthology.orchestra.run.smithi005.stdout:4358 read 0x5bb3bc8 thru 0x5bb760d (0x3a46 bytes) 2024-05-11T23:02:49.505 INFO:teuthology.orchestra.run.smithi005.stdout:4359 write 0x97d1ed8 thru 0x97de9b2 (0xcadb bytes) 2024-05-11T23:02:49.509 INFO:teuthology.orchestra.run.smithi005.stdout:4360 punch from 0xa6a7a42 to 0xa6ab831, (0x3def bytes) 2024-05-11T23:02:49.509 INFO:teuthology.orchestra.run.smithi005.stdout:4361 write 0xc88439 thru 0xc8ed5a (0x6922 bytes) 2024-05-11T23:02:49.512 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:49.512+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:49.531 INFO:teuthology.orchestra.run.smithi005.stdout:4362 trunc from 0xe31d3f4 to 0x48f6289 2024-05-11T23:02:49.557 INFO:teuthology.orchestra.run.smithi005.stdout:4364 trunc from 0x48f6289 to 0x52839ce 2024-05-11T23:02:49.563 INFO:teuthology.orchestra.run.smithi005.stdout:4366 read 0x3508119 thru 0x350cac4 (0x49ac bytes) 2024-05-11T23:02:49.563 INFO:teuthology.orchestra.run.smithi005.stdout:4367 read 0x502f768 thru 0x503686e (0x7107 bytes) 2024-05-11T23:02:49.573 INFO:teuthology.orchestra.run.smithi005.stdout:4368 trunc from 0x52839ce to 0xdc51465 2024-05-11T23:02:49.578 INFO:teuthology.orchestra.run.smithi005.stdout:4369 writesame 0x1e5730 thru 0x1f4d4c data_size 0x1(0xf61d bytes) 2024-05-11T23:02:49.587 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67817 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.966978333999997, 2024-05-11T23:02:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044148845999999999, 2024-05-11T23:02:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.588 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67817 2024-05-11T23:02:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620227+0000", 2024-05-11T23:02:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00046792099999999999 2024-05-11T23:02:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620232+0000", 2024-05-11T23:02:49.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3050000000000003e-06 2024-05-11T23:02:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620243+0000", 2024-05-11T23:02:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1289e-05 2024-05-11T23:02:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620250+0000", 2024-05-11T23:02:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2130000000000002e-06 2024-05-11T23:02:49.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663745+0000", 2024-05-11T23:02:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043494865000000001 2024-05-11T23:02:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663815+0000", 2024-05-11T23:02:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0284999999999995e-05 2024-05-11T23:02:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663908+0000", 2024-05-11T23:02:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2967999999999995e-05 2024-05-11T23:02:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67820 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4902912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:49.593 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.966428827000001, 2024-05-11T23:02:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044096781000000002, 2024-05-11T23:02:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67820 2024-05-11T23:02:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620310+0000", 2024-05-11T23:02:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9800000000000004e-07 2024-05-11T23:02:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620312+0000", 2024-05-11T23:02:49.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.272e-06 2024-05-11T23:02:49.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620319+0000", 2024-05-11T23:02:49.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9310000000000002e-06 2024-05-11T23:02:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620323+0000", 2024-05-11T23:02:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:02:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664284+0000", 2024-05-11T23:02:49.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043960908999999999 2024-05-11T23:02:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664327+0000", 2024-05-11T23:02:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2811000000000001e-05 2024-05-11T23:02:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664406+0000", 2024-05-11T23:02:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8588e-05 2024-05-11T23:02:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67821 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4927488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.966408416999997, 2024-05-11T23:02:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044203340000000001, 2024-05-11T23:02:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67821 2024-05-11T23:02:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620330+0000", 2024-05-11T23:02:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:02:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620332+0000", 2024-05-11T23:02:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:02:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620338+0000", 2024-05-11T23:02:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6250000000000004e-06 2024-05-11T23:02:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620342+0000", 2024-05-11T23:02:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7609999999999998e-06 2024-05-11T23:02:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664433+0000", 2024-05-11T23:02:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044090455000000001 2024-05-11T23:02:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664476+0000", 2024-05-11T23:02:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2861999999999997e-05 2024-05-11T23:02:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664533+0000", 2024-05-11T23:02:49.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7156e-05 2024-05-11T23:02:49.603 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.603 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.603 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67822 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4976640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.966389378999999, 2024-05-11T23:02:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044409416, 2024-05-11T23:02:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67822 2024-05-11T23:02:49.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620349+0000", 2024-05-11T23:02:49.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.51e-07 2024-05-11T23:02:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620351+0000", 2024-05-11T23:02:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.181e-06 2024-05-11T23:02:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620358+0000", 2024-05-11T23:02:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3060000000000003e-06 2024-05-11T23:02:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620362+0000", 2024-05-11T23:02:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7949999999999998e-06 2024-05-11T23:02:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664572+0000", 2024-05-11T23:02:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044209318999999997 2024-05-11T23:02:49.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664654+0000", 2024-05-11T23:02:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2367999999999995e-05 2024-05-11T23:02:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664758+0000", 2024-05-11T23:02:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103896 2024-05-11T23:02:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67823 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4952064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.966369329000003, 2024-05-11T23:02:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044623293000000001, 2024-05-11T23:02:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.609 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67823 2024-05-11T23:02:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622228+0000", 2024-05-11T23:02:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018592280000000001 2024-05-11T23:02:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622232+0000", 2024-05-11T23:02:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.403e-06 2024-05-11T23:02:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622248+0000", 2024-05-11T23:02:49.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5449000000000001e-05 2024-05-11T23:02:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622254+0000", 2024-05-11T23:02:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5819999999999998e-06 2024-05-11T23:02:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664800+0000", 2024-05-11T23:02:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042546048000000003 2024-05-11T23:02:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664886+0000", 2024-05-11T23:02:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5525000000000005e-05 2024-05-11T23:02:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664992+0000", 2024-05-11T23:02:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010605800000000001 2024-05-11T23:02:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67833 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5099520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.964262855999998, 2024-05-11T23:02:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044076606999999997, 2024-05-11T23:02:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.614 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67833 2024-05-11T23:02:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8099999999999998e-07 2024-05-11T23:02:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622477+0000", 2024-05-11T23:02:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0700000000000001e-06 2024-05-11T23:02:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622483+0000", 2024-05-11T23:02:49.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0780000000000003e-06 2024-05-11T23:02:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622487+0000", 2024-05-11T23:02:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.352e-06 2024-05-11T23:02:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666460+0000", 2024-05-11T23:02:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043973085000000002 2024-05-11T23:02:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666498+0000", 2024-05-11T23:02:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8441e-05 2024-05-11T23:02:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666552+0000", 2024-05-11T23:02:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3199999999999999e-05 2024-05-11T23:02:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67834 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3944448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.964224578, 2024-05-11T23:02:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044166551999999998, 2024-05-11T23:02:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67834 2024-05-11T23:02:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.619 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622514+0000", 2024-05-11T23:02:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1100000000000002e-07 2024-05-11T23:02:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622515+0000", 2024-05-11T23:02:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:02:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622519+0000", 2024-05-11T23:02:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8260000000000003e-06 2024-05-11T23:02:49.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622522+0000", 2024-05-11T23:02:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8899999999999999e-06 2024-05-11T23:02:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666580+0000", 2024-05-11T23:02:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044058235000000001 2024-05-11T23:02:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666624+0000", 2024-05-11T23:02:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3501999999999998e-05 2024-05-11T23:02:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666680+0000", 2024-05-11T23:02:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6042999999999999e-05 2024-05-11T23:02:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67835 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:49.623 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.964212042, 2024-05-11T23:02:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044272019000000003, 2024-05-11T23:02:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67835 2024-05-11T23:02:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1699999999999999e-07 2024-05-11T23:02:49.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622527+0000", 2024-05-11T23:02:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2109999999999999e-06 2024-05-11T23:02:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622531+0000", 2024-05-11T23:02:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4910000000000001e-06 2024-05-11T23:02:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622534+0000", 2024-05-11T23:02:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7080000000000002e-06 2024-05-11T23:02:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666703+0000", 2024-05-11T23:02:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044169377000000003 2024-05-11T23:02:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666745+0000", 2024-05-11T23:02:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1616000000000003e-05 2024-05-11T23:02:49.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666798+0000", 2024-05-11T23:02:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3298999999999999e-05 2024-05-11T23:02:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67836 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.964201381999999, 2024-05-11T23:02:49.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044373692999999999, 2024-05-11T23:02:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67836 2024-05-11T23:02:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622537+0000", 2024-05-11T23:02:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:49.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622538+0000", 2024-05-11T23:02:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1510000000000001e-06 2024-05-11T23:02:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622541+0000", 2024-05-11T23:02:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:02:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622543+0000", 2024-05-11T23:02:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4870000000000001e-06 2024-05-11T23:02:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666821+0000", 2024-05-11T23:02:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044278038999999998 2024-05-11T23:02:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666859+0000", 2024-05-11T23:02:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7224000000000001e-05 2024-05-11T23:02:49.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666910+0000", 2024-05-11T23:02:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1629999999999999e-05 2024-05-11T23:02:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67837 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:49.633 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.964191655, 2024-05-11T23:02:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044477883000000003, 2024-05-11T23:02:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67837 2024-05-11T23:02:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:49.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622548+0000", 2024-05-11T23:02:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:02:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622550+0000", 2024-05-11T23:02:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-05-11T23:02:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622553+0000", 2024-05-11T23:02:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.43e-06 2024-05-11T23:02:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666932+0000", 2024-05-11T23:02:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044379584 2024-05-11T23:02:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666971+0000", 2024-05-11T23:02:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8427000000000003e-05 2024-05-11T23:02:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667024+0000", 2024-05-11T23:02:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3226000000000002e-05 2024-05-11T23:02:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.638 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67838 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.964182137999998, 2024-05-11T23:02:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044580374999999998, 2024-05-11T23:02:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67838 2024-05-11T23:02:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.639 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:02:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622557+0000", 2024-05-11T23:02:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.102e-06 2024-05-11T23:02:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622560+0000", 2024-05-11T23:02:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:02:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622562+0000", 2024-05-11T23:02:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:02:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667045+0000", 2024-05-11T23:02:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044482936000000001 2024-05-11T23:02:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667083+0000", 2024-05-11T23:02:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7443e-05 2024-05-11T23:02:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667136+0000", 2024-05-11T23:02:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3257000000000002e-05 2024-05-11T23:02:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67839 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5394432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:49.643 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.964172396000002, 2024-05-11T23:02:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04470263, 2024-05-11T23:02:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67839 2024-05-11T23:02:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622566+0000", 2024-05-11T23:02:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-05-11T23:02:49.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622572+0000", 2024-05-11T23:02:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5610000000000004e-06 2024-05-11T23:02:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622576+0000", 2024-05-11T23:02:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3210000000000001e-06 2024-05-11T23:02:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622578+0000", 2024-05-11T23:02:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4710000000000001e-06 2024-05-11T23:02:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667160+0000", 2024-05-11T23:02:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044582220999999998 2024-05-11T23:02:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667205+0000", 2024-05-11T23:02:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5049000000000002e-05 2024-05-11T23:02:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667268+0000", 2024-05-11T23:02:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2788999999999998e-05 2024-05-11T23:02:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.648 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67840 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5419008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.964156766000002, 2024-05-11T23:02:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044796006999999999, 2024-05-11T23:02:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67840 2024-05-11T23:02:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.649 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:02:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622583+0000", 2024-05-11T23:02:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4389999999999999e-06 2024-05-11T23:02:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622586+0000", 2024-05-11T23:02:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:02:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622588+0000", 2024-05-11T23:02:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4540000000000001e-06 2024-05-11T23:02:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667287+0000", 2024-05-11T23:02:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044698882000000002 2024-05-11T23:02:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667324+0000", 2024-05-11T23:02:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7196000000000001e-05 2024-05-11T23:02:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667377+0000", 2024-05-11T23:02:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2927999999999999e-05 2024-05-11T23:02:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.653 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67841 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5443584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.964146993, 2024-05-11T23:02:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044907337999999998, 2024-05-11T23:02:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67841 2024-05-11T23:02:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622592+0000", 2024-05-11T23:02:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-05-11T23:02:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622595+0000", 2024-05-11T23:02:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:02:49.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622598+0000", 2024-05-11T23:02:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5909999999999998e-06 2024-05-11T23:02:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667401+0000", 2024-05-11T23:02:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044802980999999999 2024-05-11T23:02:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667439+0000", 2024-05-11T23:02:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8840000000000001e-05 2024-05-11T23:02:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667498+0000", 2024-05-11T23:02:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8709000000000003e-05 2024-05-11T23:02:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67842 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5468160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:49.658 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.964137385999997, 2024-05-11T23:02:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045005876, 2024-05-11T23:02:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67842 2024-05-11T23:02:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622601+0000", 2024-05-11T23:02:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622602+0000", 2024-05-11T23:02:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0780000000000001e-06 2024-05-11T23:02:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622605+0000", 2024-05-11T23:02:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.914e-06 2024-05-11T23:02:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622607+0000", 2024-05-11T23:02:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:02:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667516+0000", 2024-05-11T23:02:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044909156999999998 2024-05-11T23:02:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667554+0000", 2024-05-11T23:02:49.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7920000000000003e-05 2024-05-11T23:02:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667606+0000", 2024-05-11T23:02:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2111000000000003e-05 2024-05-11T23:02:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67843 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4067328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:49.663 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.964127859999998, 2024-05-11T23:02:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045119014999999998, 2024-05-11T23:02:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67843 2024-05-11T23:02:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624228+0000", 2024-05-11T23:02:49.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001617867 2024-05-11T23:02:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624232+0000", 2024-05-11T23:02:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-05-11T23:02:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624240+0000", 2024-05-11T23:02:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4589999999999998e-06 2024-05-11T23:02:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624247+0000", 2024-05-11T23:02:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-05-11T23:02:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667629+0000", 2024-05-11T23:02:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043382012999999997 2024-05-11T23:02:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667674+0000", 2024-05-11T23:02:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5172000000000001e-05 2024-05-11T23:02:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667729+0000", 2024-05-11T23:02:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5086000000000001e-05 2024-05-11T23:02:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67849 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4190208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:49.668 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.962369739000003, 2024-05-11T23:02:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044061433999999997, 2024-05-11T23:02:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67849 2024-05-11T23:02:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624369+0000", 2024-05-11T23:02:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5900000000000002e-07 2024-05-11T23:02:49.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624371+0000", 2024-05-11T23:02:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0310000000000001e-06 2024-05-11T23:02:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624376+0000", 2024-05-11T23:02:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4029999999999998e-06 2024-05-11T23:02:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624381+0000", 2024-05-11T23:02:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4349999999999996e-06 2024-05-11T23:02:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668331+0000", 2024-05-11T23:02:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043949191999999998 2024-05-11T23:02:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668370+0000", 2024-05-11T23:02:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9227999999999998e-05 2024-05-11T23:02:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668430+0000", 2024-05-11T23:02:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9685999999999997e-05 2024-05-11T23:02:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.673 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67850 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.962348507000002, 2024-05-11T23:02:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044151953000000001, 2024-05-11T23:02:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67850 2024-05-11T23:02:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.674 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624390+0000", 2024-05-11T23:02:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:02:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624392+0000", 2024-05-11T23:02:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1880000000000001e-06 2024-05-11T23:02:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624399+0000", 2024-05-11T23:02:49.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7390000000000002e-06 2024-05-11T23:02:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624405+0000", 2024-05-11T23:02:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1059999999999996e-06 2024-05-11T23:02:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668450+0000", 2024-05-11T23:02:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044045147999999999 2024-05-11T23:02:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668487+0000", 2024-05-11T23:02:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7450000000000002e-05 2024-05-11T23:02:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668541+0000", 2024-05-11T23:02:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3952000000000001e-05 2024-05-11T23:02:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67851 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5541888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.962321924999998, 2024-05-11T23:02:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044264413000000002, 2024-05-11T23:02:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67851 2024-05-11T23:02:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:49.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-05-11T23:02:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624419+0000", 2024-05-11T23:02:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.21e-06 2024-05-11T23:02:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624424+0000", 2024-05-11T23:02:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9050000000000002e-06 2024-05-11T23:02:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624430+0000", 2024-05-11T23:02:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-05-11T23:02:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668566+0000", 2024-05-11T23:02:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044136208000000003 2024-05-11T23:02:49.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668614+0000", 2024-05-11T23:02:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8022000000000002e-05 2024-05-11T23:02:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668680+0000", 2024-05-11T23:02:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6181999999999998e-05 2024-05-11T23:02:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67852 3.16 3.1a4a4af6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.962301457999999, 2024-05-11T23:02:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044278287999999999, 2024-05-11T23:02:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67852 2024-05-11T23:02:49.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:49.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626226+0000", 2024-05-11T23:02:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017893 2024-05-11T23:02:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626229+0000", 2024-05-11T23:02:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0709999999999999e-06 2024-05-11T23:02:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626239+0000", 2024-05-11T23:02:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0507e-05 2024-05-11T23:02:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626244+0000", 2024-05-11T23:02:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4880000000000001e-06 2024-05-11T23:02:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668706+0000", 2024-05-11T23:02:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042462054999999999 2024-05-11T23:02:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668715+0000", 2024-05-11T23:02:49.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.867e-06 2024-05-11T23:02:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:49.688 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:49.688 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:49.690 INFO:teuthology.orchestra.run.smithi005.stdout:4370 trunc from 0xdc51465 to 0xa9ae837 2024-05-11T23:02:49.690 INFO:teuthology.orchestra.run.smithi005.stdout:4373 write 0xcd3bfd7 thru 0xcd4729c (0xb2c6 bytes) 2024-05-11T23:02:49.690 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:49.691 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:49.692 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:02:49.692 INFO:teuthology.orchestra.run.smithi005.stdout:4375 trunc from 0xcd4729d to 0xbfe535f 2024-05-11T23:02:49.692 INFO:teuthology.orchestra.run.smithi005.stdout:4377 write 0x4cb4e75 thru 0x4cb7844 (0x29d0 bytes) 2024-05-11T23:02:49.693 INFO:teuthology.orchestra.run.smithi005.stdout:4378 writesame 0x8492f29 thru 0x849b475 data_size 0x41(0x854d bytes) 2024-05-11T23:02:49.693 INFO:teuthology.orchestra.run.smithi005.stdout:4379 read 0x2ffc929 thru 0x2ffdcbb (0x1393 bytes) 2024-05-11T23:02:49.693 INFO:teuthology.orchestra.run.smithi005.stdout:4381 read 0xb9498f9 thru 0xb94db73 (0x427b bytes) 2024-05-11T23:02:49.693 INFO:teuthology.orchestra.run.smithi005.stdout:4382 clone 42 order 22 su 131072 sc 6 2024-05-11T23:02:49.693 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:49.694 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:49.695 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:49.695 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:49.689+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:49.698 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:49.716 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:49.716 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:49.734 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:49.734 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:49.752 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:49.752 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:49.791 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:49.791+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:49.801 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:49.801 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:49.810 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:02:49.892 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:49.893+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:49.955 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:49.956 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:49.964 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:02:49.993 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:49.994+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:50.070 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:50.070 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:50.078 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:02:50.094 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:50.094+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:50.095 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 3, 6] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:02:50.096 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:02:50.096 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [0, 1, 2, 4, 5, 7, 3, 6] 2024-05-11T23:02:50.096 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 4 2024-05-11T23:02:50.102 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-11T23:02:50.160 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone41 from 0xbfe535f (overlap 0x48f6289) to 0xe97a1b 2024-05-11T23:02:50.195 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:50.195+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:50.200 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:50.200 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:50.215 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:02:50.296 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:50.296+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:50.298 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-05-11T23:02:50.300 INFO:teuthology.orchestra.run.smithi005.stdout:4383 read 0x9922960 thru 0x992df84 (0xb625 bytes) 2024-05-11T23:02:50.300 INFO:teuthology.orchestra.run.smithi005.stdout:4385 writesame 0x72d6bf6 thru 0x72e50a6 data_size 0xf(0xe4b1 bytes) 2024-05-11T23:02:50.308 INFO:teuthology.orchestra.run.smithi005.stdout:4386 write 0x3b8be8f thru 0x3b8e345 (0x24b7 bytes) 2024-05-11T23:02:50.314 INFO:teuthology.orchestra.run.smithi005.stdout:4388 writesame 0x2375a67 thru 0x23805c7 data_size 0x1(0xab61 bytes) 2024-05-11T23:02:50.330 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:50.330 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:50.340 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:02:50.340 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:50.340 INFO:teuthology.orchestra.run.smithi005.stdout:4391 punch from 0x9ce2e58 to 0x9ce4afa, (0x1ca2 bytes) 2024-05-11T23:02:50.341 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:50.341 INFO:teuthology.orchestra.run.smithi005.stdout:4393 trunc from 0xbfe535f to 0xa9af01e 2024-05-11T23:02:50.356 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:50.356 INFO:teuthology.orchestra.run.smithi005.stdout:4395 punch from 0x3146431 to 0x3150660, (0xa22f bytes) 2024-05-11T23:02:50.369 INFO:teuthology.orchestra.run.smithi005.stdout:4396 trunc from 0xa9af01e to 0xfad5b 2024-05-11T23:02:50.411 INFO:teuthology.orchestra.run.smithi005.stdout:4397 punch from 0xa8f45 to 0xace8e, (0x3f49 bytes) 2024-05-11T23:02:50.424 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:50.425+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:50.430 INFO:teuthology.orchestra.run.smithi005.stdout:4400 write 0xb7421cf thru 0xb749062 (0x6e94 bytes) 2024-05-11T23:02:50.433 INFO:teuthology.orchestra.run.smithi005.stdout:4402 read 0x2b0e7e thru 0x2bf9d9 (0xeb5c bytes) 2024-05-11T23:02:50.434 INFO:teuthology.orchestra.run.smithi005.stdout:4403 write 0x85cab93 thru 0x85d9146 (0xe5b4 bytes) 2024-05-11T23:02:50.438 INFO:teuthology.orchestra.run.smithi005.stdout:4405 punch from 0x3446aae to 0x34514ae, (0xaa00 bytes) 2024-05-11T23:02:50.439 INFO:teuthology.orchestra.run.smithi005.stdout:4406 trunc from 0xb749063 to 0xaa0cf60 2024-05-11T23:02:50.445 INFO:teuthology.orchestra.run.smithi005.stdout:4408 writesame 0x4b30f4c thru 0x4b38f71 data_size 0x2f(0x8026 bytes) 2024-05-11T23:02:50.448 INFO:teuthology.orchestra.run.smithi005.stdout:4411 read 0x450c701 thru 0x450de52 (0x1752 bytes) 2024-05-11T23:02:50.449 INFO:teuthology.orchestra.run.smithi005.stdout:4412 write 0x619ba0d thru 0x61a6046 (0xa63a bytes) 2024-05-11T23:02:50.454 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:50.454 INFO:teuthology.orchestra.run.smithi005.stdout:4416 read 0x478ae43 thru 0x478bd57 (0xf15 bytes) 2024-05-11T23:02:50.463 INFO:teuthology.orchestra.run.smithi005.stdout:4418 writesame 0xd21d9a4 thru 0xd22c248 data_size 0x1(0xe8a5 bytes) 2024-05-11T23:02:50.468 INFO:teuthology.orchestra.run.smithi005.stdout:4419 trunc from 0xd22c249 to 0x96c5ada 2024-05-11T23:02:50.479 INFO:teuthology.orchestra.run.smithi005.stdout:4420 read 0x55c63b4 thru 0x55cd6be (0x730b bytes) 2024-05-11T23:02:50.480 INFO:teuthology.orchestra.run.smithi005.stdout:4421 trunc from 0x96c5ada to 0x7fdcc1c 2024-05-11T23:02:50.491 INFO:teuthology.orchestra.run.smithi005.stdout:4422 trunc from 0x7fdcc1c to 0x43ba126 2024-05-11T23:02:50.493 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:50.493 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:50.496 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:50.497+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:50.503 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:02:50.505 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:50.505 INFO:teuthology.orchestra.run.smithi005.stdout:4424 read 0x22bc9c8 thru 0x22c21d8 (0x5811 bytes) 2024-05-11T23:02:50.505 INFO:teuthology.orchestra.run.smithi005.stdout:4425 writesame 0xfa0d6 thru 0x103f32 data_size 0x1(0x9e5d bytes) 2024-05-11T23:02:50.519 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-11T23:02:50.519+0000 7fd16247e640 -1 mon.a@0(leader).osd e93 definitely_dead 0 2024-05-11T23:02:50.523 INFO:teuthology.orchestra.run.smithi005.stdout:4426 writesame 0x5f9fbce thru 0x5fa8af8 data_size 0x3(0x8f2b bytes) 2024-05-11T23:02:50.523 INFO:teuthology.orchestra.run.smithi005.stdout:4428 writesame 0x324076c thru 0x32411b7 data_size 0x2(0xa4c bytes) 2024-05-11T23:02:50.528 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:50.528 INFO:teuthology.orchestra.run.smithi005.stdout:4430 write 0x5ce6014 thru 0x5cf41a1 (0xe18e bytes) 2024-05-11T23:02:50.532 INFO:teuthology.orchestra.run.smithi005.stdout:4431 punch from 0x154013 to 0x15ca06, (0x89f3 bytes) 2024-05-11T23:02:50.532 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:50.541 INFO:teuthology.orchestra.run.smithi005.stdout:4434 write 0x843984a thru 0x8444f3f (0xb6f6 bytes) 2024-05-11T23:02:50.547 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:50.548 INFO:teuthology.orchestra.run.smithi005.stdout:4436 punch from 0x6c0099f to 0x6c07208, (0x6869 bytes) 2024-05-11T23:02:50.560 INFO:teuthology.orchestra.run.smithi005.stdout:4437 write 0xe6b9e40 thru 0xe6bfcf1 (0x5eb2 bytes) 2024-05-11T23:02:50.564 INFO:teuthology.orchestra.run.smithi005.stdout:4439 read 0x7a05fda thru 0x7a11320 (0xb347 bytes) 2024-05-11T23:02:50.564 INFO:teuthology.orchestra.run.smithi005.stdout:4440 punch from 0x2ce03bf to 0x2ceb642, (0xb283 bytes) 2024-05-11T23:02:50.564 INFO:teuthology.orchestra.run.smithi005.stdout:4442 read 0x642b22e thru 0x6433866 (0x8639 bytes) 2024-05-11T23:02:50.564 INFO:teuthology.orchestra.run.smithi005.stdout:4443 write 0xd0e6ac7 thru 0xd0f484a (0xdd84 bytes) 2024-05-11T23:02:50.566 INFO:teuthology.orchestra.run.smithi005.stdout:4445 write 0x6c9bc98 thru 0x6c9eb4e (0x2eb7 bytes) 2024-05-11T23:02:50.569 INFO:teuthology.orchestra.run.smithi005.stdout:4446 write 0x7c929ce thru 0x7c941b9 (0x17ec bytes) 2024-05-11T23:02:50.571 INFO:teuthology.orchestra.run.smithi005.stdout:4448 write 0x43beb14 thru 0x43c69c6 (0x7eb3 bytes) 2024-05-11T23:02:50.577 INFO:teuthology.orchestra.run.smithi005.stdout:4449 punch from 0x21e9546 to 0x21f82ea, (0xeda4 bytes) 2024-05-11T23:02:50.577 INFO:teuthology.orchestra.run.smithi005.stdout:4451 writesame 0x21f099c thru 0x21f3792 data_size 0x7(0x2df7 bytes) 2024-05-11T23:02:50.581 INFO:teuthology.orchestra.run.smithi005.stdout:4452 punch from 0x4a1efd7 to 0x4a26fb2, (0x7fdb bytes) 2024-05-11T23:02:50.581 INFO:teuthology.orchestra.run.smithi005.stdout:4455 punch from 0x67c51c7 to 0x67c71f5, (0x202e bytes) 2024-05-11T23:02:50.581 INFO:teuthology.orchestra.run.smithi005.stdout:4458 write 0x9b60738 thru 0x9b6936a (0x8c33 bytes) 2024-05-11T23:02:50.584 INFO:teuthology.orchestra.run.smithi005.stdout:4459 write 0x73f50d9 thru 0x7404539 (0xf461 bytes) 2024-05-11T23:02:50.596 INFO:teuthology.orchestra.run.smithi005.stdout:4460 trunc from 0xe6bfcf2 to 0x6a6969a 2024-05-11T23:02:50.597 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:50.598+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:50.607 INFO:teuthology.orchestra.run.smithi005.stdout:4461 read 0x1d59c6b thru 0x1d63919 (0x9caf bytes) 2024-05-11T23:02:50.610 INFO:teuthology.orchestra.run.smithi005.stdout:4462 trunc from 0x6a6969a to 0x8ad8e5e 2024-05-11T23:02:50.615 INFO:teuthology.orchestra.run.smithi005.stdout:4463 punch from 0x15a460b to 0x15a981f, (0x5214 bytes) 2024-05-11T23:02:50.626 INFO:teuthology.orchestra.run.smithi005.stdout:4464 write 0xdca32e0 thru 0xdca6c1f (0x3940 bytes) 2024-05-11T23:02:50.628 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:50.628 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:50.630 INFO:teuthology.orchestra.run.smithi005.stdout:4465 write 0x91e7b31 thru 0x91f5e02 (0xe2d2 bytes) 2024-05-11T23:02:50.633 INFO:teuthology.orchestra.run.smithi005.stdout:4466 punch from 0xf80637 to 0xf8f9cd, (0xf396 bytes) 2024-05-11T23:02:50.633 INFO:teuthology.orchestra.run.smithi005.stdout:4467 writesame 0x67031f1 thru 0x670e65d data_size 0x13(0xb46d bytes) 2024-05-11T23:02:50.638 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:02:50.638 INFO:teuthology.orchestra.run.smithi005.stdout:4468 writesame 0x70ab2ef thru 0x70af022 data_size 0x2(0x3d34 bytes) 2024-05-11T23:02:50.643 INFO:teuthology.orchestra.run.smithi005.stdout:4470 read 0x441d9b5 thru 0x441e083 (0x6cf bytes) 2024-05-11T23:02:50.643 INFO:teuthology.orchestra.run.smithi005.stdout:4471 writesame 0xc373988 thru 0xc37c45a data_size 0x1(0x8ad3 bytes) 2024-05-11T23:02:50.668 INFO:teuthology.orchestra.run.smithi005.stdout:4473 trunc from 0xdca6c20 to 0x83aa60c 2024-05-11T23:02:50.677 INFO:teuthology.orchestra.run.smithi005.stdout:4474 writesame 0x6c7b72a thru 0x6c7f99a data_size 0x1(0x4271 bytes) 2024-05-11T23:02:50.696 INFO:teuthology.orchestra.run.smithi005.stdout:4475 write 0xc8fff91 thru 0xc904f02 (0x4f72 bytes) 2024-05-11T23:02:50.698 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:50.699+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:50.699 INFO:teuthology.orchestra.run.smithi005.stdout:4476 writesame 0x12dafae thru 0x12e2a34 data_size 0x1(0x7a87 bytes) 2024-05-11T23:02:50.709 INFO:teuthology.orchestra.run.smithi005.stdout:4477 punch from 0x302be65 to 0x303a3b1, (0xe54c bytes) 2024-05-11T23:02:50.723 INFO:teuthology.orchestra.run.smithi005.stdout:4478 trunc from 0xc904f03 to 0x1bb7823 2024-05-11T23:02:50.735 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:50.758 INFO:teuthology.orchestra.run.smithi005.stdout:4480 writesame 0xe573191 thru 0xe57c625 data_size 0x1f(0x9495 bytes) 2024-05-11T23:02:50.758 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:50.758 INFO:teuthology.orchestra.run.smithi005.stdout:4484 read 0x588b7a4 thru 0x5898ab1 (0xd30e bytes) 2024-05-11T23:02:50.758 INFO:teuthology.orchestra.run.smithi005.stdout:4485 write 0x1c02a3d thru 0x1c0bc4a (0x920e bytes) 2024-05-11T23:02:50.766 INFO:teuthology.orchestra.run.smithi005.stdout:4486 trunc from 0xe57c626 to 0xa21a19c 2024-05-11T23:02:50.772 INFO:teuthology.orchestra.run.smithi005.stdout:4488 write 0x79b30c thru 0x7a9489 (0xe17e bytes) 2024-05-11T23:02:50.774 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:50.774 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:50.774 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:50.774 INFO:teuthology.orchestra.run.smithi005.stdout:4492 punch from 0x4deb288 to 0x4df4a30, (0x97a8 bytes) 2024-05-11T23:02:50.774 INFO:teuthology.orchestra.run.smithi005.stdout:4493 punch from 0x349f522 to 0x34aa1ac, (0xac8a bytes) 2024-05-11T23:02:50.774 INFO:teuthology.orchestra.run.smithi005.stdout:4494 read 0x4d408c3 thru 0x4d4865c (0x7d9a bytes) 2024-05-11T23:02:50.775 INFO:teuthology.orchestra.run.smithi005.stdout:4495 writesame 0x25e810b thru 0x25f139e data_size 0x9f(0x9294 bytes) 2024-05-11T23:02:50.776 INFO:teuthology.orchestra.run.smithi005.stdout:4496 read 0x71ec379 thru 0x71f6efe (0xab86 bytes) 2024-05-11T23:02:50.777 INFO:teuthology.orchestra.run.smithi005.stdout:4497 writesame 0x96f53a5 thru 0x97016dc data_size 0x4(0xc338 bytes) 2024-05-11T23:02:50.789 INFO:teuthology.orchestra.run.smithi005.stdout:4499 trunc from 0xa21a19c to 0x6dcffb7 2024-05-11T23:02:50.792 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:50.792 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:50.799 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:50.799+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:50.800 INFO:teuthology.orchestra.run.smithi005.stdout:4500 read 0x5854778 thru 0x585bcdb (0x7564 bytes) 2024-05-11T23:02:50.801 INFO:teuthology.orchestra.run.smithi005.stdout:4502 writesame 0x202b959 thru 0x2031052 data_size 0x9(0x56fa bytes) 2024-05-11T23:02:50.802 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:02:50.804 INFO:teuthology.orchestra.run.smithi005.stdout:4503 read 0x1c02d0e thru 0x1c07331 (0x4624 bytes) 2024-05-11T23:02:50.805 INFO:teuthology.orchestra.run.smithi005.stdout:4504 punch from 0x477eb63 to 0x4787b0e, (0x8fab bytes) 2024-05-11T23:02:50.805 INFO:teuthology.orchestra.run.smithi005.stdout:4506 write 0x43f02f9 thru 0x43f3cee (0x39f6 bytes) 2024-05-11T23:02:50.808 INFO:teuthology.orchestra.run.smithi005.stdout:4508 read 0x17d472b thru 0x17dbef8 (0x77ce bytes) 2024-05-11T23:02:50.821 INFO:teuthology.orchestra.run.smithi005.stdout:4509 writesame 0xcf96b30 thru 0xcf99f99 data_size 0x1(0x346a bytes) 2024-05-11T23:02:50.821 INFO:teuthology.orchestra.run.smithi005.stdout:4510 writesame 0xb4060f2 thru 0xb408281 data_size 0x10(0x2190 bytes) 2024-05-11T23:02:50.824 INFO:teuthology.orchestra.run.smithi005.stdout:4512 write 0xc71159 thru 0xc73e57 (0x2cff bytes) 2024-05-11T23:02:50.825 INFO:teuthology.orchestra.run.smithi005.stdout:4513 read 0x7011615 thru 0x701b5c4 (0x9fb0 bytes) 2024-05-11T23:02:50.825 INFO:teuthology.orchestra.run.smithi005.stdout:4514 clone 43 order 23 su 1048576 sc 5 2024-05-11T23:02:50.901 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:50.901+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:50.915 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:50.915 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:50.924 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:02:51.001 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:51.002+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:51.074 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:51.074 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:51.083 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:02:51.098 INFO:teuthology.orchestra.run.smithi005.stderr:marked out osd.4. 2024-05-11T23:02:51.103 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-05-11T23:02:51.104 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:51.104+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:51.199 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:51.199 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:51.203 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:51.204+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:51.207 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:02:51.304 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:51.304+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:51.316 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:51.317 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:51.325 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:02:51.405 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:51.406+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:51.476 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:51.476 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:51.485 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:02:51.529 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:51.529+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:51.594 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:51.595 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:51.603 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:02:51.644 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:51.644+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:51.708 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:51.709+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:51.753 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:51.753 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:51.762 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:02:51.764 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:02:51.808 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:51.808+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:51.874 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:51.875 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:51.884 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:02:51.890 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:51.908 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:51.909 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:51.909 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:51.910+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:51.926 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:51.926 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:51.945 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:51.945 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:51.963 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:51.963 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:51.981 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:51.981 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:51.999 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:52.001 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:52.001 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:52.002 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:52.002 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:52.002 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:52.010 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:52.010+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:52.011 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:02:52.017 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:52.017 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:52.111 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:52.111+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:52.132 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36828 3.b 3:d385bc8e:::rbd_data.111e5e4d8275.0000000000000020:11 [read 512000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.198177370000003, 2024-05-11T23:02:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051481218000000002, 2024-05-11T23:02:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36828 2024-05-11T23:02:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:02:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934915+0000", 2024-05-11T23:02:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00181814 2024-05-11T23:02:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934919+0000", 2024-05-11T23:02:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6739999999999996e-06 2024-05-11T23:02:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934934+0000", 2024-05-11T23:02:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4415e-05 2024-05-11T23:02:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934942+0000", 2024-05-11T23:02:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8619999999999992e-06 2024-05-11T23:02:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984433+0000", 2024-05-11T23:02:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049490658999999999 2024-05-11T23:02:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984508+0000", 2024-05-11T23:02:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4394999999999995e-05 2024-05-11T23:02:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984578+0000", 2024-05-11T23:02:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0073000000000005e-05 2024-05-11T23:02:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36950 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 520192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.193871604000002, 2024-05-11T23:02:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051450856000000003, 2024-05-11T23:02:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36950 2024-05-11T23:02:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:02:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938914+0000", 2024-05-11T23:02:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015121480000000001 2024-05-11T23:02:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938918+0000", 2024-05-11T23:02:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4149999999999999e-06 2024-05-11T23:02:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938925+0000", 2024-05-11T23:02:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0779999999999998e-06 2024-05-11T23:02:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938929+0000", 2024-05-11T23:02:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2930000000000002e-06 2024-05-11T23:02:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988675+0000", 2024-05-11T23:02:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049746394999999999 2024-05-11T23:02:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988756+0000", 2024-05-11T23:02:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0499000000000003e-05 2024-05-11T23:02:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988853+0000", 2024-05-11T23:02:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7027999999999993e-05 2024-05-11T23:02:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36957 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 778240~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.192249814999997, 2024-05-11T23:02:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051328273000000001, 2024-05-11T23:02:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36957 2024-05-11T23:02:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:02:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2099999999999998e-07 2024-05-11T23:02:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939026+0000", 2024-05-11T23:02:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5069999999999999e-06 2024-05-11T23:02:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939030+0000", 2024-05-11T23:02:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7919999999999999e-06 2024-05-11T23:02:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939032+0000", 2024-05-11T23:02:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9129999999999999e-06 2024-05-11T23:02:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990178+0000", 2024-05-11T23:02:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051145373000000001 2024-05-11T23:02:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990255+0000", 2024-05-11T23:02:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7195999999999997e-05 2024-05-11T23:02:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990352+0000", 2024-05-11T23:02:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7170999999999995e-05 2024-05-11T23:02:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36958 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 815104~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.192237495999997, 2024-05-11T23:02:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051531625999999997, 2024-05-11T23:02:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36958 2024-05-11T23:02:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:02:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939037+0000", 2024-05-11T23:02:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0400000000000002e-07 2024-05-11T23:02:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939038+0000", 2024-05-11T23:02:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.164e-06 2024-05-11T23:02:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939041+0000", 2024-05-11T23:02:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1109999999999999e-06 2024-05-11T23:02:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939043+0000", 2024-05-11T23:02:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.554e-06 2024-05-11T23:02:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990382+0000", 2024-05-11T23:02:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051338953999999999 2024-05-11T23:02:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990465+0000", 2024-05-11T23:02:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2461999999999998e-05 2024-05-11T23:02:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990568+0000", 2024-05-11T23:02:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000103077 2024-05-11T23:02:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36959 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 851968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.1922256, 2024-05-11T23:02:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051758936999999998, 2024-05-11T23:02:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36959 2024-05-11T23:02:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:02:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.16e-07 2024-05-11T23:02:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939050+0000", 2024-05-11T23:02:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.176e-06 2024-05-11T23:02:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939053+0000", 2024-05-11T23:02:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1140000000000002e-06 2024-05-11T23:02:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939055+0000", 2024-05-11T23:02:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.621e-06 2024-05-11T23:02:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990610+0000", 2024-05-11T23:02:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051554244999999999 2024-05-11T23:02:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990700+0000", 2024-05-11T23:02:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0284999999999994e-05 2024-05-11T23:02:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990807+0000", 2024-05-11T23:02:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010728 2024-05-11T23:02:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36960 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 888832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.192215365000003, 2024-05-11T23:02:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051886831000000001, 2024-05-11T23:02:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36960 2024-05-11T23:02:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:02:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940913+0000", 2024-05-11T23:02:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001855104 2024-05-11T23:02:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940916+0000", 2024-05-11T23:02:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0000000000000001e-06 2024-05-11T23:02:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940924+0000", 2024-05-11T23:02:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0999999999999998e-06 2024-05-11T23:02:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940927+0000", 2024-05-11T23:02:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7160000000000001e-06 2024-05-11T23:02:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990826+0000", 2024-05-11T23:02:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049898832999999997 2024-05-11T23:02:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990872+0000", 2024-05-11T23:02:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5717000000000003e-05 2024-05-11T23:02:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:52.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990945+0000", 2024-05-11T23:02:52.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3361000000000005e-05 2024-05-11T23:02:52.157 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.157 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.157 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.157 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36986 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 528384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:52.157 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:52.157 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.190209146999997, 2024-05-11T23:02:52.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051341053999999997, 2024-05-11T23:02:52.157 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:52.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:52.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:52.157 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:52.158 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:52.158 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.158 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36986 2024-05-11T23:02:52.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.158 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:52.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:52.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:52.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:52.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:52.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:52.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:02:52.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:02:52.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:52.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941066+0000", 2024-05-11T23:02:52.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1629999999999999e-06 2024-05-11T23:02:52.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:52.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941069+0000", 2024-05-11T23:02:52.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.118e-06 2024-05-11T23:02:52.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941072+0000", 2024-05-11T23:02:52.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5720000000000001e-06 2024-05-11T23:02:52.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:52.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992292+0000", 2024-05-11T23:02:52.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051220620000000001 2024-05-11T23:02:52.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:52.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992344+0000", 2024-05-11T23:02:52.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1737000000000003e-05 2024-05-11T23:02:52.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:52.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992406+0000", 2024-05-11T23:02:52.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1623000000000003e-05 2024-05-11T23:02:52.161 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.161 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.161 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.161 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36987 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 565248~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:52.161 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:52.161 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.190199026000002, 2024-05-11T23:02:52.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051568725000000003, 2024-05-11T23:02:52.161 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:52.162 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:52.162 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:52.162 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:52.162 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:52.162 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.162 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36987 2024-05-11T23:02:52.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.162 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:52.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:52.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:52.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:52.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:52.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:52.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:02:52.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.91e-07 2024-05-11T23:02:52.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:52.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941076+0000", 2024-05-11T23:02:52.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1379999999999999e-06 2024-05-11T23:02:52.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:52.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941079+0000", 2024-05-11T23:02:52.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9739999999999998e-06 2024-05-11T23:02:52.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941082+0000", 2024-05-11T23:02:52.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-05-11T23:02:52.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:52.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992446+0000", 2024-05-11T23:02:52.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051364058999999997 2024-05-11T23:02:52.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:52.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992536+0000", 2024-05-11T23:02:52.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0085000000000002e-05 2024-05-11T23:02:52.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:52.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992643+0000", 2024-05-11T23:02:52.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107377 2024-05-11T23:02:52.165 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.165 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.165 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.165 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36988 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 602112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:52.165 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:52.166 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.190188657, 2024-05-11T23:02:52.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051700007999999999, 2024-05-11T23:02:52.166 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:52.166 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:52.166 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:52.166 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:52.166 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:52.166 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.166 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36988 2024-05-11T23:02:52.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.166 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:52.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:52.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:52.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:52.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:52.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:52.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:02:52.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.05e-07 2024-05-11T23:02:52.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:52.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941086+0000", 2024-05-11T23:02:52.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.125e-06 2024-05-11T23:02:52.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:52.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941089+0000", 2024-05-11T23:02:52.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0599999999999999e-06 2024-05-11T23:02:52.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941092+0000", 2024-05-11T23:02:52.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6529999999999998e-06 2024-05-11T23:02:52.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:52.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992667+0000", 2024-05-11T23:02:52.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051574587999999998 2024-05-11T23:02:52.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:52.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992723+0000", 2024-05-11T23:02:52.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6682999999999999e-05 2024-05-11T23:02:52.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:52.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992785+0000", 2024-05-11T23:02:52.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1693999999999996e-05 2024-05-11T23:02:52.169 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.169 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.169 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.170 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36989 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 638976~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:52.170 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:52.170 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.190178467999999, 2024-05-11T23:02:52.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05192811, 2024-05-11T23:02:52.170 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:52.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:52.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:52.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:52.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:52.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.170 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36989 2024-05-11T23:02:52.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.170 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:52.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:52.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:52.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:52.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:02:52.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:52.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942916+0000", 2024-05-11T23:02:52.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001820835 2024-05-11T23:02:52.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:52.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942920+0000", 2024-05-11T23:02:52.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5310000000000002e-06 2024-05-11T23:02:52.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:52.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942931+0000", 2024-05-11T23:02:52.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1839e-05 2024-05-11T23:02:52.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942940+0000", 2024-05-11T23:02:52.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2309999999999996e-06 2024-05-11T23:02:52.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:52.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992820+0000", 2024-05-11T23:02:52.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049880751000000001 2024-05-11T23:02:52.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:52.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992918+0000", 2024-05-11T23:02:52.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7051000000000003e-05 2024-05-11T23:02:52.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:52.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.993023+0000", 2024-05-11T23:02:52.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010587200000000001 2024-05-11T23:02:52.173 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.173 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.173 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.174 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36997 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 933888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:52.174 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:52.174 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.188144633, 2024-05-11T23:02:52.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051390602000000001, 2024-05-11T23:02:52.174 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:52.174 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:52.174 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:52.174 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:52.174 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:52.174 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36997 2024-05-11T23:02:52.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.175 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:52.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:52.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:52.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:52.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:02:52.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:52.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943130+0000", 2024-05-11T23:02:52.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0200000000000003e-07 2024-05-11T23:02:52.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:52.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943132+0000", 2024-05-11T23:02:52.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.305e-06 2024-05-11T23:02:52.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:52.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943138+0000", 2024-05-11T23:02:52.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4219999999999997e-06 2024-05-11T23:02:52.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943143+0000", 2024-05-11T23:02:52.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0610000000000003e-06 2024-05-11T23:02:52.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:52.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994323+0000", 2024-05-11T23:02:52.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051179592000000003 2024-05-11T23:02:52.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:52.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994412+0000", 2024-05-11T23:02:52.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8709999999999996e-05 2024-05-11T23:02:52.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:52.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994520+0000", 2024-05-11T23:02:52.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010811000000000001 2024-05-11T23:02:52.178 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.178 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.178 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.178 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36998 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 970752~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:52.178 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:52.178 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.188123754000003, 2024-05-11T23:02:52.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051517669000000002, 2024-05-11T23:02:52.178 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:52.178 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:52.178 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:52.178 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:52.178 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:52.178 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.179 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36998 2024-05-11T23:02:52.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.179 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:52.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:52.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:52.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:52.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:52.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:52.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:02:52.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:02:52.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:52.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943153+0000", 2024-05-11T23:02:52.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.306e-06 2024-05-11T23:02:52.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:52.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943160+0000", 2024-05-11T23:02:52.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.452e-06 2024-05-11T23:02:52.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943165+0000", 2024-05-11T23:02:52.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.152e-06 2024-05-11T23:02:52.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:52.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994561+0000", 2024-05-11T23:02:52.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051395765000000003 2024-05-11T23:02:52.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:52.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994607+0000", 2024-05-11T23:02:52.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6415000000000002e-05 2024-05-11T23:02:52.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:52.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994668+0000", 2024-05-11T23:02:52.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0179000000000001e-05 2024-05-11T23:02:52.182 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.182 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.182 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.182 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36999 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 1007616~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:52.182 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:52.182 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.188078873000002, 2024-05-11T23:02:52.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051696919000000001, 2024-05-11T23:02:52.182 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:52.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:52.182 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:52.182 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:52.183 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:52.183 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.183 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36999 2024-05-11T23:02:52.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.183 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:52.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:52.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:52.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:52.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:52.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:52.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:02:52.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:02:52.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:52.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943197+0000", 2024-05-11T23:02:52.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1280000000000002e-06 2024-05-11T23:02:52.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:52.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943202+0000", 2024-05-11T23:02:52.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8910000000000003e-06 2024-05-11T23:02:52.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943206+0000", 2024-05-11T23:02:52.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3550000000000001e-06 2024-05-11T23:02:52.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:52.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994703+0000", 2024-05-11T23:02:52.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051497045999999998 2024-05-11T23:02:52.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:52.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994789+0000", 2024-05-11T23:02:52.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6458000000000005e-05 2024-05-11T23:02:52.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:52.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994892+0000", 2024-05-11T23:02:52.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102705 2024-05-11T23:02:52.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.186 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.186 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37000 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 1044480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:52.186 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:52.186 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.188062928000001, 2024-05-11T23:02:52.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051842371999999998, 2024-05-11T23:02:52.186 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:52.186 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:52.186 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:52.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:52.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:52.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.187 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37000 2024-05-11T23:02:52.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.187 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:52.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:52.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:52.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:52.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:02:52.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:52.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944916+0000", 2024-05-11T23:02:52.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001705097 2024-05-11T23:02:52.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:52.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944921+0000", 2024-05-11T23:02:52.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5809999999999998e-06 2024-05-11T23:02:52.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:52.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944930+0000", 2024-05-11T23:02:52.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5529999999999998e-06 2024-05-11T23:02:52.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944938+0000", 2024-05-11T23:02:52.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2519999999999999e-06 2024-05-11T23:02:52.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:52.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994911+0000", 2024-05-11T23:02:52.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049973113999999999 2024-05-11T23:02:52.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:52.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994982+0000", 2024-05-11T23:02:52.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0544999999999996e-05 2024-05-11T23:02:52.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:52.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.995053+0000", 2024-05-11T23:02:52.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1229999999999994e-05 2024-05-11T23:02:52.190 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.190 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.190 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.190 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37041 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 618496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:52.190 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:52.190 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.186218949000001, 2024-05-11T23:02:52.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051352768999999999, 2024-05-11T23:02:52.190 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:52.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:52.190 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:52.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:52.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:52.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37041 2024-05-11T23:02:52.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.191 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:52.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:52.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:52.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:52.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:52.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:52.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:02:52.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:02:52.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:52.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945056+0000", 2024-05-11T23:02:52.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:02:52.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:52.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945059+0000", 2024-05-11T23:02:52.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.965e-06 2024-05-11T23:02:52.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945062+0000", 2024-05-11T23:02:52.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:02:52.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:52.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996213+0000", 2024-05-11T23:02:52.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051151532999999999 2024-05-11T23:02:52.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:52.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996301+0000", 2024-05-11T23:02:52.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8143e-05 2024-05-11T23:02:52.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:52.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996408+0000", 2024-05-11T23:02:52.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010623700000000001 2024-05-11T23:02:52.194 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.194 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.194 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.194 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37042 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 655360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:52.194 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:52.194 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.186208964000002, 2024-05-11T23:02:52.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051462075000000003, 2024-05-11T23:02:52.194 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:52.194 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:52.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:52.194 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:52.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:52.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.195 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37042 2024-05-11T23:02:52.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.195 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:52.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:52.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:52.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:52.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:52.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:52.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:02:52.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0599999999999999e-07 2024-05-11T23:02:52.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:52.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945066+0000", 2024-05-11T23:02:52.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-05-11T23:02:52.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:52.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945069+0000", 2024-05-11T23:02:52.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0079999999999998e-06 2024-05-11T23:02:52.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945072+0000", 2024-05-11T23:02:52.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6740000000000001e-06 2024-05-11T23:02:52.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:52.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996422+0000", 2024-05-11T23:02:52.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051349999 2024-05-11T23:02:52.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:52.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996467+0000", 2024-05-11T23:02:52.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.528e-05 2024-05-11T23:02:52.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:52.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996527+0000", 2024-05-11T23:02:52.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9759999999999997e-05 2024-05-11T23:02:52.198 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.198 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.198 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.198 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37043 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 692224~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:52.198 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:52.198 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.186196891000002, 2024-05-11T23:02:52.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051679535999999998, 2024-05-11T23:02:52.198 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:52.199 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:52.199 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:52.199 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:52.199 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:52.199 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.199 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37043 2024-05-11T23:02:52.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.199 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:52.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:52.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:52.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:52.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:52.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:52.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:02:52.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8999999999999997e-07 2024-05-11T23:02:52.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:52.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945079+0000", 2024-05-11T23:02:52.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.001e-06 2024-05-11T23:02:52.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:52.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945087+0000", 2024-05-11T23:02:52.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1749999999999999e-06 2024-05-11T23:02:52.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945093+0000", 2024-05-11T23:02:52.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9639999999999997e-06 2024-05-11T23:02:52.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:52.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996563+0000", 2024-05-11T23:02:52.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051469050000000002 2024-05-11T23:02:52.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:52.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996651+0000", 2024-05-11T23:02:52.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8356000000000006e-05 2024-05-11T23:02:52.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:52.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996756+0000", 2024-05-11T23:02:52.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0001055 2024-05-11T23:02:52.202 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.202 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.202 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.202 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37044 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 729088~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:52.202 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:52.202 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.186162979000002, 2024-05-11T23:02:52.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051763445999999998, 2024-05-11T23:02:52.203 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:52.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:52.203 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:52.203 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:52.203 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:52.203 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.203 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37044 2024-05-11T23:02:52.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.203 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:52.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:52.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:52.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:52.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:02:52.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:52.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946915+0000", 2024-05-11T23:02:52.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018042760000000001 2024-05-11T23:02:52.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:52.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946918+0000", 2024-05-11T23:02:52.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.36e-06 2024-05-11T23:02:52.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:52.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946926+0000", 2024-05-11T23:02:52.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3130000000000001e-06 2024-05-11T23:02:52.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946930+0000", 2024-05-11T23:02:52.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5249999999999999e-06 2024-05-11T23:02:52.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:52.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996772+0000", 2024-05-11T23:02:52.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049841868999999997 2024-05-11T23:02:52.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:52.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996817+0000", 2024-05-11T23:02:52.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5351e-05 2024-05-11T23:02:52.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:52.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996874+0000", 2024-05-11T23:02:52.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6752000000000001e-05 2024-05-11T23:02:52.206 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.206 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.206 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.206 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37052 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 1024000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:52.207 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:52.207 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.184256128000001, 2024-05-11T23:02:52.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051326140999999999, 2024-05-11T23:02:52.207 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:52.207 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:52.207 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:52.207 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:52.207 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:52.207 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37052 2024-05-11T23:02:52.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.207 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:52.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:52.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:52.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:52.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:52.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:52.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:02:52.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:52.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:52.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947019+0000", 2024-05-11T23:02:52.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.218e-06 2024-05-11T23:02:52.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:52.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947022+0000", 2024-05-11T23:02:52.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1889999999999999e-06 2024-05-11T23:02:52.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947025+0000", 2024-05-11T23:02:52.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.464e-06 2024-05-11T23:02:52.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:52.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998237+0000", 2024-05-11T23:02:52.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051212494999999997 2024-05-11T23:02:52.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:52.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998284+0000", 2024-05-11T23:02:52.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7246000000000001e-05 2024-05-11T23:02:52.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:52.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998344+0000", 2024-05-11T23:02:52.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9320000000000001e-05 2024-05-11T23:02:52.210 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.210 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.210 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.211 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37071 3.b 3:d385bc8e:::rbd_data.111e5e4d8275.0000000000000020:11 [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:52.211 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:52.211 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 59.184245918999999, 2024-05-11T23:02:52.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051528796000000002, 2024-05-11T23:02:52.211 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:52.211 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:52.211 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:52.211 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:52.211 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:52.211 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.211 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37071 2024-05-11T23:02:52.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.211 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:52.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:52.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:52.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:52.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:02:52.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:52.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:52.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948913+0000", 2024-05-11T23:02:52.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018854970000000001 2024-05-11T23:02:52.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948917+0000", 2024-05-11T23:02:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8089999999999999e-06 2024-05-11T23:02:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948926+0000", 2024-05-11T23:02:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8389999999999996e-06 2024-05-11T23:02:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948930+0000", 2024-05-11T23:02:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4519999999999999e-06 2024-05-11T23:02:52.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998379+0000", 2024-05-11T23:02:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049449011000000001 2024-05-11T23:02:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998462+0000", 2024-05-11T23:02:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2739999999999997e-05 2024-05-11T23:02:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998557+0000", 2024-05-11T23:02:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4448000000000001e-05 2024-05-11T23:02:52.214 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.215 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.215 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.215 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:52.215 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:52.215 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:52.216 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:02:52.217 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:52.217+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:52.317 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:52.317+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:52.334 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:52.334 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:52.334 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:52.334 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:52.343 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:02:52.418 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:52.418+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:52.450 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:52.450 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:52.450 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:52.450 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:52.450 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:52.458 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:02:52.519 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:52.519+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:52.565 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:52.565 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:52.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116446 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8273920~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:02:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.189072564, 2024-05-11T23:02:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045535299000000001, 2024-05-11T23:02:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116446 2024-05-11T23:02:52.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:02:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:02:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375547+0000", 2024-05-11T23:02:52.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4000000000000001e-07 2024-05-11T23:02:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375549+0000", 2024-05-11T23:02:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.368e-06 2024-05-11T23:02:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375556+0000", 2024-05-11T23:02:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1840000000000004e-06 2024-05-11T23:02:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375560+0000", 2024-05-11T23:02:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.741e-06 2024-05-11T23:02:52.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420896+0000", 2024-05-11T23:02:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045335435 2024-05-11T23:02:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420987+0000", 2024-05-11T23:02:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1033999999999996e-05 2024-05-11T23:02:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421082+0000", 2024-05-11T23:02:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4896999999999996e-05 2024-05-11T23:02:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.569 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116447 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6316032~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:02:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.189051054, 2024-05-11T23:02:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045745830000000001, 2024-05-11T23:02:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.570 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116447 2024-05-11T23:02:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:02:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:02:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:02:52.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.58e-07 2024-05-11T23:02:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375570+0000", 2024-05-11T23:02:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:02:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375576+0000", 2024-05-11T23:02:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7939999999999999e-06 2024-05-11T23:02:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375581+0000", 2024-05-11T23:02:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9280000000000001e-06 2024-05-11T23:02:52.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421127+0000", 2024-05-11T23:02:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045545785999999998 2024-05-11T23:02:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421214+0000", 2024-05-11T23:02:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7039000000000005e-05 2024-05-11T23:02:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421314+0000", 2024-05-11T23:02:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9772999999999995e-05 2024-05-11T23:02:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.573 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116448 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8314880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:02:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.189031979999999, 2024-05-11T23:02:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045941753000000002, 2024-05-11T23:02:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116448 2024-05-11T23:02:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.574 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:02:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:02:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:02:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.77e-07 2024-05-11T23:02:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375589+0000", 2024-05-11T23:02:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.091e-06 2024-05-11T23:02:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375596+0000", 2024-05-11T23:02:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9179999999999997e-06 2024-05-11T23:02:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375601+0000", 2024-05-11T23:02:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0030000000000002e-06 2024-05-11T23:02:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421363+0000", 2024-05-11T23:02:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045761352999999998 2024-05-11T23:02:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421440+0000", 2024-05-11T23:02:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7585999999999999e-05 2024-05-11T23:02:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421529+0000", 2024-05-11T23:02:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8424999999999994e-05 2024-05-11T23:02:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116449 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6438912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:02:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.189011545, 2024-05-11T23:02:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046082208999999999, 2024-05-11T23:02:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116449 2024-05-11T23:02:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.578 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:02:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:02:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377225+0000", 2024-05-11T23:02:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016177030000000001 2024-05-11T23:02:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377229+0000", 2024-05-11T23:02:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8e-06 2024-05-11T23:02:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377238+0000", 2024-05-11T23:02:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2229999999999996e-06 2024-05-11T23:02:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377244+0000", 2024-05-11T23:02:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4060000000000002e-06 2024-05-11T23:02:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421520+0000", 2024-05-11T23:02:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044275222000000003 2024-05-11T23:02:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421595+0000", 2024-05-11T23:02:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4838000000000004e-05 2024-05-11T23:02:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421690+0000", 2024-05-11T23:02:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5017000000000002e-05 2024-05-11T23:02:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116458 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7094272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:02:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.187197328, 2024-05-11T23:02:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045706877999999999, 2024-05-11T23:02:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116458 2024-05-11T23:02:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:52.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:02:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:02:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:02:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9400000000000005e-07 2024-05-11T23:02:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377424+0000", 2024-05-11T23:02:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1859999999999999e-06 2024-05-11T23:02:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377431+0000", 2024-05-11T23:02:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4049999999999997e-06 2024-05-11T23:02:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377436+0000", 2024-05-11T23:02:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1399999999999999e-06 2024-05-11T23:02:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:52.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.422945+0000", 2024-05-11T23:02:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045509326000000003 2024-05-11T23:02:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423036+0000", 2024-05-11T23:02:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0649999999999994e-05 2024-05-11T23:02:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423128+0000", 2024-05-11T23:02:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2577000000000005e-05 2024-05-11T23:02:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116459 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7258112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:02:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.187176596, 2024-05-11T23:02:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045831945999999998, 2024-05-11T23:02:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116459 2024-05-11T23:02:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:02:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:02:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377443+0000", 2024-05-11T23:02:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999999e-07 2024-05-11T23:02:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:52.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377445+0000", 2024-05-11T23:02:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0209999999999998e-06 2024-05-11T23:02:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377451+0000", 2024-05-11T23:02:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.071e-06 2024-05-11T23:02:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377456+0000", 2024-05-11T23:02:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8289999999999998e-06 2024-05-11T23:02:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:52.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423155+0000", 2024-05-11T23:02:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045699447999999997 2024-05-11T23:02:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423200+0000", 2024-05-11T23:02:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5083000000000002e-05 2024-05-11T23:02:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423274+0000", 2024-05-11T23:02:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4046000000000002e-05 2024-05-11T23:02:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116460 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7299072~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:02:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.187157534000001, 2024-05-11T23:02:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046010987000000003, 2024-05-11T23:02:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116460 2024-05-11T23:02:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:02:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:02:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377462+0000", 2024-05-11T23:02:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:02:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:52.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377464+0000", 2024-05-11T23:02:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1600000000000001e-06 2024-05-11T23:02:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377469+0000", 2024-05-11T23:02:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5430000000000001e-06 2024-05-11T23:02:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377474+0000", 2024-05-11T23:02:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8509999999999998e-06 2024-05-11T23:02:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423312+0000", 2024-05-11T23:02:52.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045838080000000003 2024-05-11T23:02:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423401+0000", 2024-05-11T23:02:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8170000000000005e-05 2024-05-11T23:02:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423472+0000", 2024-05-11T23:02:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1847000000000005e-05 2024-05-11T23:02:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.593 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116461 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7421952~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:02:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.187138764, 2024-05-11T23:02:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046130749999999998, 2024-05-11T23:02:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116461 2024-05-11T23:02:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:02:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:02:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:02:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4700000000000002e-07 2024-05-11T23:02:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:52.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377483+0000", 2024-05-11T23:02:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0219999999999999e-06 2024-05-11T23:02:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377489+0000", 2024-05-11T23:02:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4420000000000003e-06 2024-05-11T23:02:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377494+0000", 2024-05-11T23:02:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9239999999999998e-06 2024-05-11T23:02:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:52.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423500+0000", 2024-05-11T23:02:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046006054999999997 2024-05-11T23:02:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423557+0000", 2024-05-11T23:02:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7169000000000001e-05 2024-05-11T23:02:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423611+0000", 2024-05-11T23:02:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3791e-05 2024-05-11T23:02:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.597 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116462 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7462912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:02:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.187119146000001, 2024-05-11T23:02:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046254666, 2024-05-11T23:02:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116462 2024-05-11T23:02:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:02:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:02:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:02:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:02:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377502+0000", 2024-05-11T23:02:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-05-11T23:02:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377508+0000", 2024-05-11T23:02:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4500000000000003e-06 2024-05-11T23:02:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377512+0000", 2024-05-11T23:02:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7700000000000001e-06 2024-05-11T23:02:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:52.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423649+0000", 2024-05-11T23:02:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046136859000000002 2024-05-11T23:02:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423704+0000", 2024-05-11T23:02:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4938000000000002e-05 2024-05-11T23:02:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423754+0000", 2024-05-11T23:02:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0145999999999997e-05 2024-05-11T23:02:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116463 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7585792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:02:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.18710072, 2024-05-11T23:02:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361763, 2024-05-11T23:02:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116463 2024-05-11T23:02:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:52.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:02:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:02:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377519+0000", 2024-05-11T23:02:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2600000000000002e-07 2024-05-11T23:02:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377521+0000", 2024-05-11T23:02:52.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1710000000000001e-06 2024-05-11T23:02:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377527+0000", 2024-05-11T23:02:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9649999999999996e-06 2024-05-11T23:02:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377532+0000", 2024-05-11T23:02:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.78e-06 2024-05-11T23:02:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423779+0000", 2024-05-11T23:02:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046247706 2024-05-11T23:02:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423829+0000", 2024-05-11T23:02:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9188999999999999e-05 2024-05-11T23:02:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423880+0000", 2024-05-11T23:02:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1425999999999999e-05 2024-05-11T23:02:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.605 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116464 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7749632~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:02:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.187081484, 2024-05-11T23:02:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046491719000000001, 2024-05-11T23:02:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:52.606 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116464 2024-05-11T23:02:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:02:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:02:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377538+0000", 2024-05-11T23:02:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4499999999999998e-07 2024-05-11T23:02:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377540+0000", 2024-05-11T23:02:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2340000000000001e-06 2024-05-11T23:02:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377546+0000", 2024-05-11T23:02:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8289999999999996e-06 2024-05-11T23:02:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377551+0000", 2024-05-11T23:02:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8559999999999998e-06 2024-05-11T23:02:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423913+0000", 2024-05-11T23:02:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361885999999998 2024-05-11T23:02:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423958+0000", 2024-05-11T23:02:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5513999999999998e-05 2024-05-11T23:02:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424029+0000", 2024-05-11T23:02:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1055000000000003e-05 2024-05-11T23:02:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116465 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7626752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:02:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.187062336, 2024-05-11T23:02:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046600057, 2024-05-11T23:02:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:52.610 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116465 2024-05-11T23:02:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:02:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:02:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:02:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3599999999999999e-07 2024-05-11T23:02:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377559+0000", 2024-05-11T23:02:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9640000000000002e-06 2024-05-11T23:02:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377572+0000", 2024-05-11T23:02:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3188e-05 2024-05-11T23:02:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377578+0000", 2024-05-11T23:02:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.942e-06 2024-05-11T23:02:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424058+0000", 2024-05-11T23:02:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046480021000000003 2024-05-11T23:02:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424103+0000", 2024-05-11T23:02:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5021999999999997e-05 2024-05-11T23:02:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:52.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424157+0000", 2024-05-11T23:02:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3483999999999998e-05 2024-05-11T23:02:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116466 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7790592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:02:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.187033754000002, 2024-05-11T23:02:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046743741999999998, 2024-05-11T23:02:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:52.614 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116466 2024-05-11T23:02:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:02:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:02:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377586+0000", 2024-05-11T23:02:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:02:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377588+0000", 2024-05-11T23:02:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3039999999999999e-06 2024-05-11T23:02:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377594+0000", 2024-05-11T23:02:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-05-11T23:02:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377600+0000", 2024-05-11T23:02:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7699999999999998e-06 2024-05-11T23:02:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424185+0000", 2024-05-11T23:02:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046585891999999997 2024-05-11T23:02:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424235+0000", 2024-05-11T23:02:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9356e-05 2024-05-11T23:02:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424329+0000", 2024-05-11T23:02:52.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4036999999999994e-05 2024-05-11T23:02:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116467 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7954432~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:02:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.187013467, 2024-05-11T23:02:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046930134999999998, 2024-05-11T23:02:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:52.618 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116467 2024-05-11T23:02:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:02:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:02:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:52.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379161+0000", 2024-05-11T23:02:52.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015555009999999999 2024-05-11T23:02:52.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:52.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379166+0000", 2024-05-11T23:02:52.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6689999999999997e-06 2024-05-11T23:02:52.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:52.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379176+0000", 2024-05-11T23:02:52.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0549e-05 2024-05-11T23:02:52.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379183+0000", 2024-05-11T23:02:52.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1199999999999996e-06 2024-05-11T23:02:52.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:52.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424359+0000", 2024-05-11T23:02:52.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045175949999999999 2024-05-11T23:02:52.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:52.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424433+0000", 2024-05-11T23:02:52.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3824000000000003e-05 2024-05-11T23:02:52.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:52.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424536+0000", 2024-05-11T23:02:52.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010252200000000001 2024-05-11T23:02:52.621 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.621 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116468 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7913472~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:02:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.185423247999999, 2024-05-11T23:02:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045525364999999998, 2024-05-11T23:02:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116468 2024-05-11T23:02:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.622 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:02:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:02:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:02:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3300000000000002e-07 2024-05-11T23:02:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379198+0000", 2024-05-11T23:02:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2670000000000001e-06 2024-05-11T23:02:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379206+0000", 2024-05-11T23:02:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7149999999999998e-06 2024-05-11T23:02:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379228+0000", 2024-05-11T23:02:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1429000000000001e-05 2024-05-11T23:02:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424574+0000", 2024-05-11T23:02:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045346209999999998 2024-05-11T23:02:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424627+0000", 2024-05-11T23:02:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3352e-05 2024-05-11T23:02:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424721+0000", 2024-05-11T23:02:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3858999999999997e-05 2024-05-11T23:02:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116469 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8077312~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:02:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.185373427999998, 2024-05-11T23:02:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045636687000000002, 2024-05-11T23:02:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116469 2024-05-11T23:02:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:02:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:02:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379246+0000", 2024-05-11T23:02:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.32e-07 2024-05-11T23:02:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:52.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379254+0000", 2024-05-11T23:02:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.807e-06 2024-05-11T23:02:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379261+0000", 2024-05-11T23:02:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.985e-06 2024-05-11T23:02:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379264+0000", 2024-05-11T23:02:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6480000000000001e-06 2024-05-11T23:02:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424744+0000", 2024-05-11T23:02:52.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045479638000000003 2024-05-11T23:02:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424797+0000", 2024-05-11T23:02:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2725000000000001e-05 2024-05-11T23:02:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424882+0000", 2024-05-11T23:02:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5451999999999994e-05 2024-05-11T23:02:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116470 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8118272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:02:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.185348216000001, 2024-05-11T23:02:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045755745, 2024-05-11T23:02:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116470 2024-05-11T23:02:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:52.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:02:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:02:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:02:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3999999999999997e-07 2024-05-11T23:02:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:52.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379273+0000", 2024-05-11T23:02:52.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-05-11T23:02:52.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:52.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379277+0000", 2024-05-11T23:02:52.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4020000000000004e-06 2024-05-11T23:02:52.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379280+0000", 2024-05-11T23:02:52.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-05-11T23:02:52.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:52.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424922+0000", 2024-05-11T23:02:52.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045642598999999999 2024-05-11T23:02:52.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:52.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424973+0000", 2024-05-11T23:02:52.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0674999999999998e-05 2024-05-11T23:02:52.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:52.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425026+0000", 2024-05-11T23:02:52.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3492000000000002e-05 2024-05-11T23:02:52.633 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.633 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.633 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.634 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116471 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8241152~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:52.634 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:02:52.634 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.185335432999999, 2024-05-11T23:02:52.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045867374000000002, 2024-05-11T23:02:52.634 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:52.634 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:52.634 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:52.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:52.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:52.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.634 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116471 2024-05-11T23:02:52.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.634 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:52.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:52.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:02:52.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:52.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:02:52.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:52.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379284+0000", 2024-05-11T23:02:52.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.53e-07 2024-05-11T23:02:52.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:52.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379285+0000", 2024-05-11T23:02:52.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.223e-06 2024-05-11T23:02:52.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:52.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379288+0000", 2024-05-11T23:02:52.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5379999999999998e-06 2024-05-11T23:02:52.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379291+0000", 2024-05-11T23:02:52.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6699999999999998e-06 2024-05-11T23:02:52.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:52.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425055+0000", 2024-05-11T23:02:52.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045763497 2024-05-11T23:02:52.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:52.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425099+0000", 2024-05-11T23:02:52.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3899000000000001e-05 2024-05-11T23:02:52.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:52.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425151+0000", 2024-05-11T23:02:52.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2293999999999998e-05 2024-05-11T23:02:52.637 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.637 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.637 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.637 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116472 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8282112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:52.637 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:02:52.637 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.185324434000002, 2024-05-11T23:02:52.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045973778999999999, 2024-05-11T23:02:52.638 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:52.638 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:52.638 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:52.638 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:52.638 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:52.638 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.638 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116472 2024-05-11T23:02:52.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.638 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:52.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:52.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:02:52.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:52.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:02:52.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:52.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381226+0000", 2024-05-11T23:02:52.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001931627 2024-05-11T23:02:52.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:52.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381231+0000", 2024-05-11T23:02:52.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8679999999999998e-06 2024-05-11T23:02:52.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:52.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381242+0000", 2024-05-11T23:02:52.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0702e-05 2024-05-11T23:02:52.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381249+0000", 2024-05-11T23:02:52.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9800000000000001e-06 2024-05-11T23:02:52.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:52.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425175+0000", 2024-05-11T23:02:52.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043926805999999999 2024-05-11T23:02:52.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:52.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425215+0000", 2024-05-11T23:02:52.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9737999999999998e-05 2024-05-11T23:02:52.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:52.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425268+0000", 2024-05-11T23:02:52.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3057999999999999e-05 2024-05-11T23:02:52.641 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.641 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.641 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.641 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116482 3.1b 3:db6e8843:::rbd_object_map.111e67f27da3.0000000000000028:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:02:52.641 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:02:52.641 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 17.183355891000001, 2024-05-11T23:02:52.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046526076, 2024-05-11T23:02:52.642 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:52.642 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:52.642 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:52.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:52.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:52.642 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:52.642 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116482 2024-05-11T23:02:52.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.642 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:52.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:52.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:02:52.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:52.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:02:52.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:52.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:52.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381264+0000", 2024-05-11T23:02:52.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5300000000000004e-07 2024-05-11T23:02:52.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:52.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381267+0000", 2024-05-11T23:02:52.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1999999999999999e-06 2024-05-11T23:02:52.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:52.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381274+0000", 2024-05-11T23:02:52.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7109999999999995e-06 2024-05-11T23:02:52.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:52.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381285+0000", 2024-05-11T23:02:52.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0856000000000001e-05 2024-05-11T23:02:52.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:52.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425292+0000", 2024-05-11T23:02:52.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044006376 2024-05-11T23:02:52.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:52.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425337+0000", 2024-05-11T23:02:52.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4907999999999998e-05 2024-05-11T23:02:52.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:02:52.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425507+0000", 2024-05-11T23:02:52.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017029999999999999 2024-05-11T23:02:52.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:52.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427668+0000", 2024-05-11T23:02:52.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021612430000000002 2024-05-11T23:02:52.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:52.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427683+0000", 2024-05-11T23:02:52.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5016e-05 2024-05-11T23:02:52.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:52.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427742+0000", 2024-05-11T23:02:52.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8811999999999999e-05 2024-05-11T23:02:52.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:52.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:52.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:52.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427789+0000", 2024-05-11T23:02:52.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7101000000000002e-05 2024-05-11T23:02:52.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.646 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.646 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:52.646 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:52.646 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:52.648 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:02:52.648 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:52.648+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:52.748 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:52.749+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:52.794 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:52.794 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:52.794 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:52.795 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:52.803 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:02:52.849 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:52.850+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:52.909 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:52.909 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:52.909 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:52.909 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:52.909 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:52.917 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:02:52.951 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:52.951+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:53.051 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:53.051+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:53.064 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:53.065 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:53.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:53.065 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:53.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.065 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67817 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:53.065 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:53.065 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.444283333000001, 2024-05-11T23:02:53.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044148845999999999, 2024-05-11T23:02:53.065 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.065 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.065 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.065 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.065 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.065 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67817 2024-05-11T23:02:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620227+0000", 2024-05-11T23:02:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00046792099999999999 2024-05-11T23:02:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620232+0000", 2024-05-11T23:02:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3050000000000003e-06 2024-05-11T23:02:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620243+0000", 2024-05-11T23:02:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1289e-05 2024-05-11T23:02:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620250+0000", 2024-05-11T23:02:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2130000000000002e-06 2024-05-11T23:02:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663745+0000", 2024-05-11T23:02:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043494865000000001 2024-05-11T23:02:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663815+0000", 2024-05-11T23:02:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0284999999999995e-05 2024-05-11T23:02:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663908+0000", 2024-05-11T23:02:53.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2967999999999995e-05 2024-05-11T23:02:53.069 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.069 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.069 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.069 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67820 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4902912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:53.069 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:53.069 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.443733825999999, 2024-05-11T23:02:53.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044096781000000002, 2024-05-11T23:02:53.069 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.069 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.069 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.069 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.070 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.070 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.070 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67820 2024-05-11T23:02:53.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.070 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:53.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:53.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620310+0000", 2024-05-11T23:02:53.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9800000000000004e-07 2024-05-11T23:02:53.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620312+0000", 2024-05-11T23:02:53.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.272e-06 2024-05-11T23:02:53.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620319+0000", 2024-05-11T23:02:53.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9310000000000002e-06 2024-05-11T23:02:53.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620323+0000", 2024-05-11T23:02:53.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:02:53.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664284+0000", 2024-05-11T23:02:53.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043960908999999999 2024-05-11T23:02:53.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664327+0000", 2024-05-11T23:02:53.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2811000000000001e-05 2024-05-11T23:02:53.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664406+0000", 2024-05-11T23:02:53.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8588e-05 2024-05-11T23:02:53.073 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.073 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.073 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.073 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67821 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4927488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:53.073 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:53.073 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.443713416000001, 2024-05-11T23:02:53.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044203340000000001, 2024-05-11T23:02:53.073 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.073 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.073 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.073 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.073 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.074 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.074 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67821 2024-05-11T23:02:53.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.074 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:53.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:53.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620330+0000", 2024-05-11T23:02:53.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:02:53.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620332+0000", 2024-05-11T23:02:53.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:02:53.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620338+0000", 2024-05-11T23:02:53.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6250000000000004e-06 2024-05-11T23:02:53.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620342+0000", 2024-05-11T23:02:53.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7609999999999998e-06 2024-05-11T23:02:53.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664433+0000", 2024-05-11T23:02:53.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044090455000000001 2024-05-11T23:02:53.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664476+0000", 2024-05-11T23:02:53.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2861999999999997e-05 2024-05-11T23:02:53.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664533+0000", 2024-05-11T23:02:53.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7156e-05 2024-05-11T23:02:53.077 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.077 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.077 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.077 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67822 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4976640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:53.077 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:53.077 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.443694378000004, 2024-05-11T23:02:53.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044409416, 2024-05-11T23:02:53.077 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.077 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.077 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.077 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.078 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.078 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.078 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67822 2024-05-11T23:02:53.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.078 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:53.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:53.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620349+0000", 2024-05-11T23:02:53.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.51e-07 2024-05-11T23:02:53.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620351+0000", 2024-05-11T23:02:53.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.181e-06 2024-05-11T23:02:53.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620358+0000", 2024-05-11T23:02:53.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3060000000000003e-06 2024-05-11T23:02:53.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620362+0000", 2024-05-11T23:02:53.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7949999999999998e-06 2024-05-11T23:02:53.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664572+0000", 2024-05-11T23:02:53.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044209318999999997 2024-05-11T23:02:53.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664654+0000", 2024-05-11T23:02:53.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2367999999999995e-05 2024-05-11T23:02:53.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664758+0000", 2024-05-11T23:02:53.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103896 2024-05-11T23:02:53.081 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.081 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.081 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.081 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67823 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4952064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:53.081 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:53.081 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.443674328, 2024-05-11T23:02:53.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044623293000000001, 2024-05-11T23:02:53.081 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.081 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.081 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.082 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67823 2024-05-11T23:02:53.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.082 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:53.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:53.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622228+0000", 2024-05-11T23:02:53.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018592280000000001 2024-05-11T23:02:53.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622232+0000", 2024-05-11T23:02:53.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.403e-06 2024-05-11T23:02:53.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622248+0000", 2024-05-11T23:02:53.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5449000000000001e-05 2024-05-11T23:02:53.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622254+0000", 2024-05-11T23:02:53.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5819999999999998e-06 2024-05-11T23:02:53.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664800+0000", 2024-05-11T23:02:53.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042546048000000003 2024-05-11T23:02:53.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664886+0000", 2024-05-11T23:02:53.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5525000000000005e-05 2024-05-11T23:02:53.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664992+0000", 2024-05-11T23:02:53.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010605800000000001 2024-05-11T23:02:53.085 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.085 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.085 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.085 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67833 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5099520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:53.085 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:53.085 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.441567855000002, 2024-05-11T23:02:53.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044076606999999997, 2024-05-11T23:02:53.085 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.085 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.085 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.085 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.086 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.086 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.086 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67833 2024-05-11T23:02:53.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.086 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:53.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:53.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:53.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8099999999999998e-07 2024-05-11T23:02:53.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622477+0000", 2024-05-11T23:02:53.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0700000000000001e-06 2024-05-11T23:02:53.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622483+0000", 2024-05-11T23:02:53.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0780000000000003e-06 2024-05-11T23:02:53.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622487+0000", 2024-05-11T23:02:53.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.352e-06 2024-05-11T23:02:53.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666460+0000", 2024-05-11T23:02:53.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043973085000000002 2024-05-11T23:02:53.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666498+0000", 2024-05-11T23:02:53.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8441e-05 2024-05-11T23:02:53.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666552+0000", 2024-05-11T23:02:53.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3199999999999999e-05 2024-05-11T23:02:53.089 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.089 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.089 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.089 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67834 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3944448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:53.089 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:53.089 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.441529576999997, 2024-05-11T23:02:53.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044166551999999998, 2024-05-11T23:02:53.090 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.090 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.090 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.090 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.090 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.090 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.090 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67834 2024-05-11T23:02:53.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.090 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:53.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:53.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622514+0000", 2024-05-11T23:02:53.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1100000000000002e-07 2024-05-11T23:02:53.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622515+0000", 2024-05-11T23:02:53.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:02:53.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622519+0000", 2024-05-11T23:02:53.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8260000000000003e-06 2024-05-11T23:02:53.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622522+0000", 2024-05-11T23:02:53.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8899999999999999e-06 2024-05-11T23:02:53.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666580+0000", 2024-05-11T23:02:53.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044058235000000001 2024-05-11T23:02:53.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666624+0000", 2024-05-11T23:02:53.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3501999999999998e-05 2024-05-11T23:02:53.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666680+0000", 2024-05-11T23:02:53.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6042999999999999e-05 2024-05-11T23:02:53.093 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.093 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.093 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.093 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67835 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:53.093 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:53.093 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.441517040999997, 2024-05-11T23:02:53.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044272019000000003, 2024-05-11T23:02:53.094 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.094 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.094 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.094 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.094 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.094 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.094 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67835 2024-05-11T23:02:53.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.094 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:53.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:53.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:53.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1699999999999999e-07 2024-05-11T23:02:53.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622527+0000", 2024-05-11T23:02:53.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2109999999999999e-06 2024-05-11T23:02:53.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622531+0000", 2024-05-11T23:02:53.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4910000000000001e-06 2024-05-11T23:02:53.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622534+0000", 2024-05-11T23:02:53.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7080000000000002e-06 2024-05-11T23:02:53.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666703+0000", 2024-05-11T23:02:53.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044169377000000003 2024-05-11T23:02:53.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666745+0000", 2024-05-11T23:02:53.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1616000000000003e-05 2024-05-11T23:02:53.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666798+0000", 2024-05-11T23:02:53.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3298999999999999e-05 2024-05-11T23:02:53.097 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.097 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.097 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.097 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67836 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:53.097 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:53.097 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.441506381000004, 2024-05-11T23:02:53.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044373692999999999, 2024-05-11T23:02:53.098 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.098 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.098 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.098 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.098 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.098 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.098 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67836 2024-05-11T23:02:53.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.098 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:53.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:53.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622537+0000", 2024-05-11T23:02:53.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:53.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622538+0000", 2024-05-11T23:02:53.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1510000000000001e-06 2024-05-11T23:02:53.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622541+0000", 2024-05-11T23:02:53.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:02:53.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622543+0000", 2024-05-11T23:02:53.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4870000000000001e-06 2024-05-11T23:02:53.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666821+0000", 2024-05-11T23:02:53.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044278038999999998 2024-05-11T23:02:53.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666859+0000", 2024-05-11T23:02:53.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7224000000000001e-05 2024-05-11T23:02:53.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666910+0000", 2024-05-11T23:02:53.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1629999999999999e-05 2024-05-11T23:02:53.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.101 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.101 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67837 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:53.101 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:53.102 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.441496653999998, 2024-05-11T23:02:53.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044477883000000003, 2024-05-11T23:02:53.102 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.102 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.102 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.102 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.102 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.102 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.102 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67837 2024-05-11T23:02:53.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.102 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:53.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:53.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:53.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:53.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622548+0000", 2024-05-11T23:02:53.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:02:53.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622550+0000", 2024-05-11T23:02:53.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-05-11T23:02:53.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622553+0000", 2024-05-11T23:02:53.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.43e-06 2024-05-11T23:02:53.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666932+0000", 2024-05-11T23:02:53.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044379584 2024-05-11T23:02:53.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666971+0000", 2024-05-11T23:02:53.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8427000000000003e-05 2024-05-11T23:02:53.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667024+0000", 2024-05-11T23:02:53.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3226000000000002e-05 2024-05-11T23:02:53.105 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.105 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.105 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.106 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67838 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:53.106 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:53.106 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.441487137000003, 2024-05-11T23:02:53.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044580374999999998, 2024-05-11T23:02:53.106 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.106 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.106 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.106 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.106 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.106 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.106 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67838 2024-05-11T23:02:53.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.106 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:53.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:53.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:53.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:02:53.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622557+0000", 2024-05-11T23:02:53.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.102e-06 2024-05-11T23:02:53.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622560+0000", 2024-05-11T23:02:53.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:02:53.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622562+0000", 2024-05-11T23:02:53.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:02:53.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667045+0000", 2024-05-11T23:02:53.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044482936000000001 2024-05-11T23:02:53.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667083+0000", 2024-05-11T23:02:53.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7443e-05 2024-05-11T23:02:53.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667136+0000", 2024-05-11T23:02:53.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3257000000000002e-05 2024-05-11T23:02:53.109 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.110 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.110 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.110 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67839 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5394432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:53.110 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:53.110 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.441477395, 2024-05-11T23:02:53.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04470263, 2024-05-11T23:02:53.110 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.110 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.110 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.110 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.110 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.110 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.110 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67839 2024-05-11T23:02:53.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.111 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:53.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:53.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622566+0000", 2024-05-11T23:02:53.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-05-11T23:02:53.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622572+0000", 2024-05-11T23:02:53.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5610000000000004e-06 2024-05-11T23:02:53.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622576+0000", 2024-05-11T23:02:53.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3210000000000001e-06 2024-05-11T23:02:53.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622578+0000", 2024-05-11T23:02:53.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4710000000000001e-06 2024-05-11T23:02:53.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667160+0000", 2024-05-11T23:02:53.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044582220999999998 2024-05-11T23:02:53.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667205+0000", 2024-05-11T23:02:53.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5049000000000002e-05 2024-05-11T23:02:53.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667268+0000", 2024-05-11T23:02:53.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2788999999999998e-05 2024-05-11T23:02:53.116 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.116 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.116 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.116 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67840 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5419008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:53.116 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:53.116 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.441461765, 2024-05-11T23:02:53.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044796006999999999, 2024-05-11T23:02:53.116 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.116 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.116 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.116 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.116 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.116 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.117 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67840 2024-05-11T23:02:53.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.117 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:53.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:53.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:53.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:02:53.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622583+0000", 2024-05-11T23:02:53.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4389999999999999e-06 2024-05-11T23:02:53.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622586+0000", 2024-05-11T23:02:53.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:02:53.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622588+0000", 2024-05-11T23:02:53.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4540000000000001e-06 2024-05-11T23:02:53.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667287+0000", 2024-05-11T23:02:53.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044698882000000002 2024-05-11T23:02:53.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667324+0000", 2024-05-11T23:02:53.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7196000000000001e-05 2024-05-11T23:02:53.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667377+0000", 2024-05-11T23:02:53.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2927999999999999e-05 2024-05-11T23:02:53.120 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.120 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.120 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.120 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67841 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5443584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:53.120 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:53.120 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.441451991999998, 2024-05-11T23:02:53.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044907337999999998, 2024-05-11T23:02:53.120 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.120 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.120 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.120 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.120 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.120 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.120 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67841 2024-05-11T23:02:53.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.121 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:53.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:53.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:53.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:53.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622592+0000", 2024-05-11T23:02:53.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-05-11T23:02:53.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622595+0000", 2024-05-11T23:02:53.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:02:53.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622598+0000", 2024-05-11T23:02:53.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5909999999999998e-06 2024-05-11T23:02:53.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667401+0000", 2024-05-11T23:02:53.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044802980999999999 2024-05-11T23:02:53.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667439+0000", 2024-05-11T23:02:53.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8840000000000001e-05 2024-05-11T23:02:53.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667498+0000", 2024-05-11T23:02:53.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8709000000000003e-05 2024-05-11T23:02:53.123 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.124 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.124 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.124 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67842 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5468160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:53.124 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:53.124 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.441442385000002, 2024-05-11T23:02:53.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045005876, 2024-05-11T23:02:53.124 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.124 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.124 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.124 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.124 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.124 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.124 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67842 2024-05-11T23:02:53.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.125 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:53.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:53.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622601+0000", 2024-05-11T23:02:53.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:53.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622602+0000", 2024-05-11T23:02:53.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0780000000000001e-06 2024-05-11T23:02:53.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622605+0000", 2024-05-11T23:02:53.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.914e-06 2024-05-11T23:02:53.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622607+0000", 2024-05-11T23:02:53.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:02:53.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667516+0000", 2024-05-11T23:02:53.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044909156999999998 2024-05-11T23:02:53.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667554+0000", 2024-05-11T23:02:53.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7920000000000003e-05 2024-05-11T23:02:53.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667606+0000", 2024-05-11T23:02:53.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2111000000000003e-05 2024-05-11T23:02:53.127 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.127 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.127 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.128 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67843 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4067328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:53.128 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:53.128 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.441432859000003, 2024-05-11T23:02:53.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045119014999999998, 2024-05-11T23:02:53.128 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.128 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.128 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.128 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.128 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.128 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.128 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67843 2024-05-11T23:02:53.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.128 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:53.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:53.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624228+0000", 2024-05-11T23:02:53.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001617867 2024-05-11T23:02:53.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624232+0000", 2024-05-11T23:02:53.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-05-11T23:02:53.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624240+0000", 2024-05-11T23:02:53.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4589999999999998e-06 2024-05-11T23:02:53.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624247+0000", 2024-05-11T23:02:53.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-05-11T23:02:53.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667629+0000", 2024-05-11T23:02:53.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043382012999999997 2024-05-11T23:02:53.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667674+0000", 2024-05-11T23:02:53.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5172000000000001e-05 2024-05-11T23:02:53.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667729+0000", 2024-05-11T23:02:53.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5086000000000001e-05 2024-05-11T23:02:53.131 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.131 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.131 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.132 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67849 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4190208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:53.132 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:53.132 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.439674738000001, 2024-05-11T23:02:53.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044061433999999997, 2024-05-11T23:02:53.132 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.132 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.132 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.132 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.132 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.132 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.132 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67849 2024-05-11T23:02:53.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.132 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:53.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:53.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624369+0000", 2024-05-11T23:02:53.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5900000000000002e-07 2024-05-11T23:02:53.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624371+0000", 2024-05-11T23:02:53.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0310000000000001e-06 2024-05-11T23:02:53.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624376+0000", 2024-05-11T23:02:53.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4029999999999998e-06 2024-05-11T23:02:53.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624381+0000", 2024-05-11T23:02:53.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4349999999999996e-06 2024-05-11T23:02:53.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668331+0000", 2024-05-11T23:02:53.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043949191999999998 2024-05-11T23:02:53.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668370+0000", 2024-05-11T23:02:53.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9227999999999998e-05 2024-05-11T23:02:53.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668430+0000", 2024-05-11T23:02:53.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9685999999999997e-05 2024-05-11T23:02:53.135 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.135 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.135 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.136 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67850 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:53.136 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:53.136 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.439653505999999, 2024-05-11T23:02:53.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044151953000000001, 2024-05-11T23:02:53.136 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.136 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.136 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.136 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.136 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.136 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.136 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67850 2024-05-11T23:02:53.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.136 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:53.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:53.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624390+0000", 2024-05-11T23:02:53.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:02:53.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624392+0000", 2024-05-11T23:02:53.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1880000000000001e-06 2024-05-11T23:02:53.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624399+0000", 2024-05-11T23:02:53.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7390000000000002e-06 2024-05-11T23:02:53.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624405+0000", 2024-05-11T23:02:53.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1059999999999996e-06 2024-05-11T23:02:53.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668450+0000", 2024-05-11T23:02:53.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044045147999999999 2024-05-11T23:02:53.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668487+0000", 2024-05-11T23:02:53.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7450000000000002e-05 2024-05-11T23:02:53.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668541+0000", 2024-05-11T23:02:53.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3952000000000001e-05 2024-05-11T23:02:53.139 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.139 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.139 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.140 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67851 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5541888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:53.140 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:53.140 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.439626924000002, 2024-05-11T23:02:53.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044264413000000002, 2024-05-11T23:02:53.140 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.140 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.140 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.140 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.140 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.140 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.140 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67851 2024-05-11T23:02:53.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.140 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:53.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:53.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:53.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-05-11T23:02:53.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624419+0000", 2024-05-11T23:02:53.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.21e-06 2024-05-11T23:02:53.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624424+0000", 2024-05-11T23:02:53.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9050000000000002e-06 2024-05-11T23:02:53.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624430+0000", 2024-05-11T23:02:53.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-05-11T23:02:53.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668566+0000", 2024-05-11T23:02:53.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044136208000000003 2024-05-11T23:02:53.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668614+0000", 2024-05-11T23:02:53.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8022000000000002e-05 2024-05-11T23:02:53.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668680+0000", 2024-05-11T23:02:53.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6181999999999998e-05 2024-05-11T23:02:53.143 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.143 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.143 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.143 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67852 3.16 3.1a4a4af6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:53.144 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:53.144 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.439606456999996, 2024-05-11T23:02:53.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044278287999999999, 2024-05-11T23:02:53.144 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.144 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.144 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:53.144 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.144 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.144 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.144 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67852 2024-05-11T23:02:53.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.144 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626226+0000", 2024-05-11T23:02:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017893 2024-05-11T23:02:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626229+0000", 2024-05-11T23:02:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0709999999999999e-06 2024-05-11T23:02:53.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626239+0000", 2024-05-11T23:02:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0507e-05 2024-05-11T23:02:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626244+0000", 2024-05-11T23:02:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4880000000000001e-06 2024-05-11T23:02:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668706+0000", 2024-05-11T23:02:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042462054999999999 2024-05-11T23:02:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668715+0000", 2024-05-11T23:02:53.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.867e-06 2024-05-11T23:02:53.147 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.147 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.147 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.147 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.147 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.147 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:53.147 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-11T23:02:53.149 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:02:53.192 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:53.193+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:53.253 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:53.253+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:53.292 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:53.293 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:53.293 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:53.293 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:53.303 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:02:53.387 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:53.388+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:53.454 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:53.454+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:53.472 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:53.472 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:53.472 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:53.472 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:53.472 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:53.473 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone42 from 0xcf99f9a (overlap 0xfad5b) to 0x4b401 2024-05-11T23:02:53.482 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:02:53.567 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:53.568+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:53.580 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-05-11T23:02:53.602 INFO:teuthology.orchestra.run.smithi005.stdout:4515 write 0xa27a30c thru 0xa2858a3 (0xb598 bytes) 2024-05-11T23:02:53.610 INFO:teuthology.orchestra.run.smithi005.stdout:4516 read 0xb528dba thru 0xb528fef (0x236 bytes) 2024-05-11T23:02:53.610 INFO:teuthology.orchestra.run.smithi005.stdout:4517 write 0x34da5ad thru 0x34df38a (0x4dde bytes) 2024-05-11T23:02:53.615 INFO:teuthology.orchestra.run.smithi005.stdout:4518 write 0x538c09a thru 0x539a5a8 (0xe50f bytes) 2024-05-11T23:02:53.619 INFO:teuthology.orchestra.run.smithi005.stdout:4519 writesame 0x5c1f5ed thru 0x5c24c0c data_size 0x35(0x5620 bytes) 2024-05-11T23:02:53.622 INFO:teuthology.orchestra.run.smithi005.stdout:4520 read 0x32d9795 thru 0x32da3d7 (0xc43 bytes) 2024-05-11T23:02:53.622 INFO:teuthology.orchestra.run.smithi005.stdout:4521 write 0xc2dc487 thru 0xc2e511e (0x8c98 bytes) 2024-05-11T23:02:53.627 INFO:teuthology.orchestra.run.smithi005.stdout:4522 read 0xad07098 thru 0xad0ed85 (0x7cee bytes) 2024-05-11T23:02:53.627 INFO:teuthology.orchestra.run.smithi005.stdout:4523 read 0x920d1a1 thru 0x9218030 (0xae90 bytes) 2024-05-11T23:02:53.627 INFO:teuthology.orchestra.run.smithi005.stdout:4525 read 0x7a47e8d thru 0x7a57d30 (0xfea4 bytes) 2024-05-11T23:02:53.627 INFO:teuthology.orchestra.run.smithi005.stdout:4526 write 0xb82541f thru 0xb83377e (0xe360 bytes) 2024-05-11T23:02:53.635 INFO:teuthology.orchestra.run.smithi005.stdout:4527 writesame 0xd4998bb thru 0xd4a480b data_size 0x1(0xaf51 bytes) 2024-05-11T23:02:53.638 INFO:teuthology.orchestra.run.smithi005.stdout:4528 read 0x17147bc thru 0x171a022 (0x5867 bytes) 2024-05-11T23:02:53.638 INFO:teuthology.orchestra.run.smithi005.stdout:4529 read 0x7664950 thru 0x766f904 (0xafb5 bytes) 2024-05-11T23:02:53.649 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:53.649 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:53.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:53.649 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:53.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.650 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:53.650 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:53.650 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.994812762999999, 2024-05-11T23:02:53.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:02:53.650 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.650 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.650 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.650 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:02:53.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.650 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:53.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:53.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:53.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:02:53.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:02:53.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:02:53.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:02:53.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:02:53.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:02:53.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:02:53.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:02:53.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:02:53.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:02:53.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:02:53.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:02:53.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:02:53.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.653 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.654 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:53.654 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:53.654 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.991431574000003, 2024-05-11T23:02:53.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:02:53.654 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.654 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.654 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.654 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:02:53.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.655 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:53.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:53.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:53.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:02:53.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:02:53.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:02:53.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:02:53.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:02:53.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:02:53.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:02:53.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:02:53.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:02:53.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:02:53.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:02:53.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:02:53.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:02:53.658 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.658 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.658 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.658 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:53.658 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:53.658 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.991412867000001, 2024-05-11T23:02:53.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:02:53.658 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.658 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.658 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.659 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.659 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:02:53.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.659 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:53.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:53.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:02:53.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:02:53.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:02:53.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:02:53.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:02:53.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:02:53.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:02:53.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:02:53.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:02:53.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:02:53.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:02:53.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:02:53.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:02:53.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:02:53.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.662 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.662 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:53.662 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:53.662 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.991394724000003, 2024-05-11T23:02:53.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:02:53.662 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.662 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.662 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.662 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.662 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.663 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.663 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:02:53.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.663 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:53.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:53.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:53.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:02:53.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:02:53.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:02:53.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:02:53.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:02:53.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:02:53.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:02:53.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:02:53.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:02:53.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:02:53.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:02:53.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:02:53.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:02:53.666 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.666 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.666 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.666 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:53.666 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:53.666 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.991367236999999, 2024-05-11T23:02:53.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:02:53.666 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.666 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.666 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.667 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.667 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.667 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.667 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:02:53.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.667 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:53.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:53.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:02:53.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:02:53.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:02:53.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:02:53.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:02:53.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:02:53.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:02:53.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:02:53.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:02:53.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:02:53.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:02:53.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:02:53.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:02:53.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:02:53.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.670 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.670 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.670 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:53.670 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:53.670 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.991346708999998, 2024-05-11T23:02:53.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:02:53.670 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.671 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.671 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.671 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:02:53.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.671 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:53.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:53.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:53.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:02:53.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:02:53.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:02:53.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:02:53.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:02:53.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:02:53.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:02:53.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:02:53.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:02:53.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:02:53.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:02:53.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:02:53.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:02:53.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.674 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.674 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.674 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:53.674 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:53.674 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.991326934, 2024-05-11T23:02:53.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:02:53.675 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.675 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.675 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.675 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:02:53.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.675 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:53.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:53.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:02:53.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:02:53.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:02:53.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:02:53.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:02:53.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:02:53.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:02:53.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:02:53.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:02:53.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:02:53.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:02:53.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:02:53.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:02:53.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:02:53.678 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.678 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.678 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.679 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:53.679 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:53.679 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.991308128, 2024-05-11T23:02:53.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:02:53.679 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.679 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.679 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.679 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:02:53.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.679 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:53.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:53.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:02:53.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:02:53.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:02:53.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:02:53.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:02:53.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:02:53.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:02:53.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:02:53.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:02:53.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:02:53.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:02:53.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:02:53.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:02:53.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:02:53.682 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.683 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.683 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:53.683 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:53.683 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.991289205000001, 2024-05-11T23:02:53.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:02:53.683 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.683 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.683 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.683 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.683 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.683 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.683 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:02:53.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.684 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:53.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:53.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:53.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:02:53.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:02:53.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:02:53.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:02:53.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:02:53.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:02:53.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:02:53.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:02:53.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:02:53.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:02:53.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:02:53.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:02:53.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:02:53.687 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.687 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.687 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.687 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:53.687 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:53.687 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.991270542999999, 2024-05-11T23:02:53.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:02:53.687 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.687 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.687 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.687 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.688 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.688 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:02:53.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.688 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:53.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:53.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:53.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:02:53.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:02:53.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:02:53.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:02:53.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:02:53.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:02:53.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:02:53.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:02:53.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:02:53.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:02:53.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:02:53.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:02:53.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:02:53.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.691 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.691 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:53.691 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:53.691 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.990925380999997, 2024-05-11T23:02:53.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:02:53.691 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.691 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.692 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.692 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:02:53.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.692 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:53.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:53.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:02:53.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:02:53.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:02:53.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:02:53.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:02:53.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:02:53.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:02:53.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:02:53.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:02:53.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:02:53.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:02:53.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:02:53.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:02:53.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:02:53.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.695 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.695 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:53.695 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:53.695 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.990899831, 2024-05-11T23:02:53.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:02:53.696 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.696 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.696 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.696 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:02:53.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.696 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:53.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:53.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:53.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:02:53.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:02:53.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:02:53.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:02:53.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:02:53.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:02:53.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:02:53.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:02:53.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:02:53.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:02:53.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:02:53.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:02:53.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:02:53.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.699 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.699 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.699 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:53.699 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:53.699 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.990817573000001, 2024-05-11T23:02:53.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:02:53.700 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.700 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.700 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.700 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:02:53.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.700 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:53.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:53.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:53.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:02:53.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:02:53.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:02:53.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:02:53.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:02:53.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:02:53.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:02:53.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:02:53.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:02:53.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:02:53.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:02:53.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:02:53.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:02:53.703 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.703 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.703 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.704 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:53.704 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:53.704 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.990795126000002, 2024-05-11T23:02:53.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:02:53.704 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.704 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.704 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.704 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.704 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.704 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.704 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:02:53.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.704 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:53.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:53.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:02:53.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:02:53.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:02:53.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:02:53.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:02:53.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:02:53.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:02:53.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:02:53.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:02:53.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:02:53.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:02:53.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:02:53.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:02:53.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:02:53.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.707 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.707 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.708 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:53.708 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:53.708 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.990775667999998, 2024-05-11T23:02:53.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:02:53.708 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.708 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.708 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.708 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.708 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:02:53.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.708 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:53.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:53.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:53.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:02:53.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:02:53.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:02:53.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:02:53.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:02:53.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:02:53.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:02:53.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:02:53.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:02:53.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:02:53.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:02:53.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:02:53.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:02:53.711 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.711 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.711 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.712 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:53.712 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:53.712 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.990755608000001, 2024-05-11T23:02:53.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:02:53.712 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.712 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.712 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.712 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:02:53.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.712 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:02:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:02:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:02:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:02:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:02:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:02:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:02:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:02:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:02:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:02:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:02:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:02:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:02:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.990736337999998, 2024-05-11T23:02:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:02:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:02:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:02:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:02:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:02:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:02:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:02:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:02:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:02:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:02:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:02:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:02:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:02:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:02:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:02:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.990190272, 2024-05-11T23:02:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:02:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:02:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:02:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:02:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:02:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:02:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:02:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:02:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:02:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:02:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:02:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:02:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:02:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:02:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:02:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:02:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.990158487999999, 2024-05-11T23:02:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:02:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:02:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:02:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:02:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:02:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:02:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:02:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:02:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:02:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:02:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:02:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:02:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:02:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:02:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:02:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:02:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.552913023999999, 2024-05-11T23:02:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:02:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:02:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:02:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:02:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:02:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:02:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:02:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:02:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:02:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:02:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:02:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:02:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:02:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:02:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:02:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:02:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:02:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:02:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:02:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:02:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:02:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:02:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:02:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:02:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:02:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.734 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:53.735 INFO:teuthology.orchestra.run.smithi005.stdout:4530 trunc from 0xd4a480c to 0x32331b4 2024-05-11T23:02:53.735 INFO:teuthology.orchestra.run.smithi005.stdout:4531 write 0xbf7bf3c thru 0xbf83513 (0x75d8 bytes) 2024-05-11T23:02:53.735 INFO:teuthology.orchestra.run.smithi005.stdout:4532 trunc from 0xbf83514 to 0x5918340 2024-05-11T23:02:53.735 INFO:teuthology.orchestra.run.smithi005.stdout:4533 write 0x7d7c8a4 thru 0x7d81e21 (0x557e bytes) 2024-05-11T23:02:53.736 INFO:teuthology.orchestra.run.smithi005.stdout:4534 trunc from 0x7d81e22 to 0x718ff5b 2024-05-11T23:02:53.736 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:02:53.736 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:53.735+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:53.749 INFO:teuthology.orchestra.run.smithi005.stdout:4535 writesame 0xd9fcb85 thru 0xda08034 data_size 0x76(0xb4b0 bytes) 2024-05-11T23:02:53.750 INFO:teuthology.orchestra.run.smithi005.stdout:4536 read 0x8847b51 thru 0x884c223 (0x46d3 bytes) 2024-05-11T23:02:53.751 INFO:teuthology.orchestra.run.smithi005.stdout:4537 trunc from 0xda08035 to 0xed99e7c 2024-05-11T23:02:53.757 INFO:teuthology.orchestra.run.smithi005.stdout:4538 read 0x65d0227 thru 0x65d6f7c (0x6d56 bytes) 2024-05-11T23:02:53.757 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:53.757 INFO:teuthology.orchestra.run.smithi005.stdout:4541 punch from 0x6bb2551 to 0x6bbc01f, (0x9ace bytes) 2024-05-11T23:02:53.757 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:53.769 INFO:teuthology.orchestra.run.smithi005.stdout:4543 trunc from 0xed99e7c to 0x42821f6 2024-05-11T23:02:53.779 INFO:teuthology.orchestra.run.smithi005.stdout:4546 read 0x1c9e2a0 thru 0x1ca19d4 (0x3735 bytes) 2024-05-11T23:02:53.785 INFO:teuthology.orchestra.run.smithi005.stdout:4548 write 0x4b87823 thru 0x4b9615e (0xe93c bytes) 2024-05-11T23:02:53.802 INFO:teuthology.orchestra.run.smithi005.stdout:4549 write 0xc351575 thru 0xc3524bf (0xf4b bytes) 2024-05-11T23:02:53.805 INFO:teuthology.orchestra.run.smithi005.stdout:4550 read 0x73faeeb thru 0x74004da (0x55f0 bytes) 2024-05-11T23:02:53.807 INFO:teuthology.orchestra.run.smithi005.stdout:4553 trunc from 0xc3524c0 to 0xdff2e10 2024-05-11T23:02:53.813 INFO:teuthology.orchestra.run.smithi005.stdout:4555 writesame 0xba9c246 thru 0xba9f596 data_size 0x3(0x3351 bytes) 2024-05-11T23:02:53.818 INFO:teuthology.orchestra.run.smithi005.stdout:4556 trunc from 0xdff2e10 to 0xe0a3665 2024-05-11T23:02:53.834 INFO:teuthology.orchestra.run.smithi005.stdout:4557 trunc from 0xe0a3665 to 0x5508abb 2024-05-11T23:02:53.836 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:53.836+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:53.847 INFO:teuthology.orchestra.run.smithi005.stdout:4558 trunc from 0x5508abb to 0x74639ee 2024-05-11T23:02:53.853 INFO:teuthology.orchestra.run.smithi005.stdout:4559 punch from 0x40640b8 to 0x406553c, (0x1484 bytes) 2024-05-11T23:02:53.853 INFO:teuthology.orchestra.run.smithi005.stdout:4560 read 0x896755 thru 0x8a00aa (0x9956 bytes) 2024-05-11T23:02:53.853 INFO:teuthology.orchestra.run.smithi005.stdout:4561 read 0x3c925d2 thru 0x3c93996 (0x13c5 bytes) 2024-05-11T23:02:53.853 INFO:teuthology.orchestra.run.smithi005.stdout:4562 read 0x368215c thru 0x3690581 (0xe426 bytes) 2024-05-11T23:02:53.854 INFO:teuthology.orchestra.run.smithi005.stdout:4563 punch from 0x9ffda2 to 0xa0544b, (0x56a9 bytes) 2024-05-11T23:02:53.857 INFO:teuthology.orchestra.run.smithi005.stdout:4564 trunc from 0x74639ee to 0x520158c 2024-05-11T23:02:53.863 INFO:teuthology.orchestra.run.smithi005.stdout:4565 punch from 0x297191f to 0x297e608, (0xcce9 bytes) 2024-05-11T23:02:53.863 INFO:teuthology.orchestra.run.smithi005.stdout:4566 punch from 0x25c789d to 0x25d1428, (0x9b8b bytes) 2024-05-11T23:02:53.863 INFO:teuthology.orchestra.run.smithi005.stdout:4568 read 0x886a2f thru 0x88aad9 (0x40ab bytes) 2024-05-11T23:02:53.863 INFO:teuthology.orchestra.run.smithi005.stdout:4570 read 0x35f531e thru 0x3600aa2 (0xb785 bytes) 2024-05-11T23:02:53.864 INFO:teuthology.orchestra.run.smithi005.stdout:4571 writesame 0x14c6ede thru 0x14d6cbf data_size 0x1(0xfde2 bytes) 2024-05-11T23:02:53.898 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop_reply(client.4382.0:167253 3.d e96/19)", 2024-05-11T23:02:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:53.891999+0000", 2024-05-11T23:02:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0064881110000000004, 2024-05-11T23:02:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0065060480000000004, 2024-05-11T23:02:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:53.898 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:53.891999+0000", 2024-05-11T23:02:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:53.891999+0000", 2024-05-11T23:02:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:53.892000+0000", 2024-05-11T23:02:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9299999999999998e-07 2024-05-11T23:02:53.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:53.892003+0000", 2024-05-11T23:02:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2909999999999999e-06 2024-05-11T23:02:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:53.892015+0000", 2024-05-11T23:02:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1805e-05 2024-05-11T23:02:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:53.892021+0000", 2024-05-11T23:02:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4899999999999997e-06 2024-05-11T23:02:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.900 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:167253 3.d 3:b3dc2e39:::rbd_data.111ebf0f641b.0000000000000000:head [call rbd.sparse_copyup in=24402b,set-alloc-hint object_size 8388608 write_size 8388608,write-same in=1b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-05-11T23:02:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:53.870106+0000", 2024-05-11T23:02:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.028381772, 2024-05-11T23:02:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.028457901000000001, 2024-05-11T23:02:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for sub ops", 2024-05-11T23:02:53.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 167253 2024-05-11T23:02:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:53.870106+0000", 2024-05-11T23:02:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:53.870106+0000", 2024-05-11T23:02:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:53.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:53.870106+0000", 2024-05-11T23:02:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5700000000000002e-07 2024-05-11T23:02:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:53.870130+0000", 2024-05-11T23:02:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3414e-05 2024-05-11T23:02:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:53.870140+0000", 2024-05-11T23:02:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0550000000000001e-05 2024-05-11T23:02:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:53.870146+0000", 2024-05-11T23:02:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5040000000000002e-06 2024-05-11T23:02:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:53.870184+0000", 2024-05-11T23:02:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8534000000000001e-05 2024-05-11T23:02:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:53.870277+0000", 2024-05-11T23:02:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2366000000000003e-05 2024-05-11T23:02:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:53.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:53.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 5", 2024-05-11T23:02:53.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:53.876390+0000", 2024-05-11T23:02:53.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0061132840000000001 2024-05-11T23:02:53.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.905 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:53.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:53.905 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-11T23:02:53.905 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 2 2024-05-11T23:02:53.905 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:53.908 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:02:53.910 INFO:teuthology.orchestra.run.smithi005.stdout:4572 trunc from 0x520158c to 0xca58b1f 2024-05-11T23:02:53.915 INFO:teuthology.orchestra.run.smithi005.stdout:4573 write 0x22b3754 thru 0x22b71d2 (0x3a7f bytes) 2024-05-11T23:02:53.921 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:53.921 INFO:teuthology.orchestra.run.smithi005.stdout:4575 punch from 0x714c616 to 0x7154b50, (0x853a bytes) 2024-05-11T23:02:53.921 INFO:teuthology.orchestra.run.smithi005.stdout:4576 write 0xbaae6e thru 0xbae8c8 (0x3a5b bytes) 2024-05-11T23:02:53.927 INFO:teuthology.orchestra.run.smithi005.stdout:4578 read 0xa07d1a0 thru 0xa083b23 (0x6984 bytes) 2024-05-11T23:02:53.927 INFO:teuthology.orchestra.run.smithi005.stdout:4579 writesame 0x99d91c0 thru 0x99e193c data_size 0x7(0x877d bytes) 2024-05-11T23:02:53.931 INFO:teuthology.orchestra.run.smithi005.stdout:4580 writesame 0x2234045 thru 0x2239813 data_size 0x3b(0x57cf bytes) 2024-05-11T23:02:53.933 INFO:teuthology.orchestra.run.smithi005.stdout:4581 read 0x738eb2a thru 0x739877b (0x9c52 bytes) 2024-05-11T23:02:53.933 INFO:teuthology.orchestra.run.smithi005.stdout:4582 write 0x54d4220 thru 0x54e3377 (0xf158 bytes) 2024-05-11T23:02:53.936 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:53.936 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:53.936 INFO:teuthology.orchestra.run.smithi005.stdout:4586 write 0xbb2bef7 thru 0xbb2fd5b (0x3e65 bytes) 2024-05-11T23:02:53.938 INFO:teuthology.orchestra.run.smithi005.stdout:4587 write 0x739ff17 thru 0x73aef7e (0xf068 bytes) 2024-05-11T23:02:53.942 INFO:teuthology.orchestra.run.smithi005.stdout:4588 writesame 0x378de84 thru 0x379a670 data_size 0xd(0xc7ed bytes) 2024-05-11T23:02:53.949 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:53.950+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:53.951 INFO:teuthology.orchestra.run.smithi005.stdout:4589 write 0xe0704a6 thru 0xe078c1e (0x8779 bytes) 2024-05-11T23:02:53.955 INFO:teuthology.orchestra.run.smithi005.stdout:4590 punch from 0xa24c163 to 0xa24d64d, (0x14ea bytes) 2024-05-11T23:02:53.955 INFO:teuthology.orchestra.run.smithi005.stdout:4591 read 0xc9e6ca thru 0xca5821 (0x7158 bytes) 2024-05-11T23:02:53.955 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:53.956 INFO:teuthology.orchestra.run.smithi005.stdout:4594 writesame 0x73ef85e thru 0x73fcc38 data_size 0x1(0xd3db bytes) 2024-05-11T23:02:53.986 INFO:teuthology.orchestra.run.smithi005.stdout:4596 trunc from 0xe078c1f to 0xa86b3ae 2024-05-11T23:02:53.997 INFO:teuthology.orchestra.run.smithi005.stdout:4597 read 0x6beca58 thru 0x6bf1207 (0x47b0 bytes) 2024-05-11T23:02:53.997 INFO:teuthology.orchestra.run.smithi005.stdout:4599 read 0x9d90acb thru 0x9d9712c (0x6662 bytes) 2024-05-11T23:02:54.007 INFO:teuthology.orchestra.run.smithi005.stdout:4600 trunc from 0xa86b3ae to 0x156d364 2024-05-11T23:02:54.027 INFO:teuthology.orchestra.run.smithi005.stdout:4601 read 0x11d6f16 thru 0x11dcc4a (0x5d35 bytes) 2024-05-11T23:02:54.029 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:02:54.039 INFO:teuthology.orchestra.run.smithi005.stdout:4602 write 0x72c73f8 thru 0x72cc6c5 (0x52ce bytes) 2024-05-11T23:02:54.039 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:54.039+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:54.050 INFO:teuthology.orchestra.run.smithi005.stdout:4604 trunc from 0x72cc6c6 to 0xf746e3 2024-05-11T23:02:54.063 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:54.074 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:54.074 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:54.074 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:54.074 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:54.074 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:54.084 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:02:54.086 INFO:teuthology.orchestra.run.smithi005.stdout:4608 writesame 0xe52e671 thru 0xe530f3b data_size 0x3(0x28cb bytes) 2024-05-11T23:02:54.092 INFO:teuthology.orchestra.run.smithi005.stdout:4609 write 0xec08780 thru 0xec169e2 (0xe263 bytes) 2024-05-11T23:02:54.096 INFO:teuthology.orchestra.run.smithi005.stdout:4610 read 0x875a705 thru 0x8765859 (0xb155 bytes) 2024-05-11T23:02:54.096 INFO:teuthology.orchestra.run.smithi005.stdout:4611 read 0xb48076d thru 0xb4896c4 (0x8f58 bytes) 2024-05-11T23:02:54.096 INFO:teuthology.orchestra.run.smithi005.stdout:4613 write 0x9c2863f thru 0x9c31f2e (0x98f0 bytes) 2024-05-11T23:02:54.111 INFO:teuthology.orchestra.run.smithi005.stdout:4614 trunc from 0xec169e3 to 0x60f9802 2024-05-11T23:02:54.117 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-05-11T23:02:54.176 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:54.176+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:54.190 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:54.190 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:54.207 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:54.207 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:54.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:54.207 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:54.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:02:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 97.043283223000003, 2024-05-11T23:02:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:02:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:02:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:02:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:02:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:54.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:02:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:02:54.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:02:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:02:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:02:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:02:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:54.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:02:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:02:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:02:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:02:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:02:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:02:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:02:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:02:54.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:02:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:02:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:02:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:02:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:02:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:02:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:02:54.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:02:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:02:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:02:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:02:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:02:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:54.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:02:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:02:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78429 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:02:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.242013880999998, 2024-05-11T23:02:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044563497000000001, 2024-05-11T23:02:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:54.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78429 2024-05-11T23:02:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:02:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:02:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964872+0000", 2024-05-11T23:02:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035070300000000001 2024-05-11T23:02:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964876+0000", 2024-05-11T23:02:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5009999999999998e-06 2024-05-11T23:02:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964902+0000", 2024-05-11T23:02:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5715999999999999e-05 2024-05-11T23:02:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964910+0000", 2024-05-11T23:02:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3930000000000002e-06 2024-05-11T23:02:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008898+0000", 2024-05-11T23:02:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043988772000000002 2024-05-11T23:02:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008981+0000", 2024-05-11T23:02:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3140999999999996e-05 2024-05-11T23:02:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009085+0000", 2024-05-11T23:02:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010327099999999999 2024-05-11T23:02:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78431 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:02:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.241553572999997, 2024-05-11T23:02:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044542033000000002, 2024-05-11T23:02:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:54.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78431 2024-05-11T23:02:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:02:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:02:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965038+0000", 2024-05-11T23:02:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6993000000000001e-05 2024-05-11T23:02:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965043+0000", 2024-05-11T23:02:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7040000000000002e-06 2024-05-11T23:02:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965053+0000", 2024-05-11T23:02:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8539999999999992e-06 2024-05-11T23:02:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965061+0000", 2024-05-11T23:02:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9810000000000003e-06 2024-05-11T23:02:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009350+0000", 2024-05-11T23:02:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044288739000000001 2024-05-11T23:02:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009429+0000", 2024-05-11T23:02:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9354e-05 2024-05-11T23:02:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009524+0000", 2024-05-11T23:02:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4407999999999995e-05 2024-05-11T23:02:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78432 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:02:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.241462724999998, 2024-05-11T23:02:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044635421000000002, 2024-05-11T23:02:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:54.222 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78432 2024-05-11T23:02:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:02:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:02:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965343+0000", 2024-05-11T23:02:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027056500000000001 2024-05-11T23:02:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965347+0000", 2024-05-11T23:02:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3540000000000002e-06 2024-05-11T23:02:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965356+0000", 2024-05-11T23:02:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2140000000000002e-06 2024-05-11T23:02:54.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965363+0000", 2024-05-11T23:02:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4150000000000004e-06 2024-05-11T23:02:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009551+0000", 2024-05-11T23:02:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044187917 2024-05-11T23:02:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009626+0000", 2024-05-11T23:02:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5024000000000005e-05 2024-05-11T23:02:54.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009708+0000", 2024-05-11T23:02:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1932000000000001e-05 2024-05-11T23:02:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78433 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:02:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.241161837999996, 2024-05-11T23:02:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044564864000000003, 2024-05-11T23:02:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:54.226 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78433 2024-05-11T23:02:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:02:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:54.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:02:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966225+0000", 2024-05-11T23:02:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085220700000000001 2024-05-11T23:02:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966230+0000", 2024-05-11T23:02:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0690000000000003e-06 2024-05-11T23:02:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:54.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966239+0000", 2024-05-11T23:02:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-05-11T23:02:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966245+0000", 2024-05-11T23:02:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5520000000000001e-06 2024-05-11T23:02:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009743+0000", 2024-05-11T23:02:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043498311999999997 2024-05-11T23:02:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:54.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009848+0000", 2024-05-11T23:02:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104885 2024-05-11T23:02:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009938+0000", 2024-05-11T23:02:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9812e-05 2024-05-11T23:02:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78580 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 999424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:02:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.238049154999999, 2024-05-11T23:02:54.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044493076999999999, 2024-05-11T23:02:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78580 2024-05-11T23:02:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:02:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:02:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:02:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:02:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968489+0000", 2024-05-11T23:02:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.52e-06 2024-05-11T23:02:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968496+0000", 2024-05-11T23:02:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9419999999999998e-06 2024-05-11T23:02:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968501+0000", 2024-05-11T23:02:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8720000000000001e-06 2024-05-11T23:02:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012690+0000", 2024-05-11T23:02:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044189655000000001 2024-05-11T23:02:54.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012848+0000", 2024-05-11T23:02:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015758100000000001 2024-05-11T23:02:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012979+0000", 2024-05-11T23:02:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000131066 2024-05-11T23:02:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78581 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:02:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.238028151000002, 2024-05-11T23:02:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044616243, 2024-05-11T23:02:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:54.235 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78581 2024-05-11T23:02:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:02:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:02:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970227+0000", 2024-05-11T23:02:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017200049999999999 2024-05-11T23:02:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970232+0000", 2024-05-11T23:02:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000003e-06 2024-05-11T23:02:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970242+0000", 2024-05-11T23:02:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0192e-05 2024-05-11T23:02:54.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970249+0000", 2024-05-11T23:02:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.888e-06 2024-05-11T23:02:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013008+0000", 2024-05-11T23:02:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042759495000000002 2024-05-11T23:02:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:54.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013063+0000", 2024-05-11T23:02:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5071999999999997e-05 2024-05-11T23:02:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013123+0000", 2024-05-11T23:02:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9930999999999999e-05 2024-05-11T23:02:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78589 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:02:54.239 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.236108907000002, 2024-05-11T23:02:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044587697000000003, 2024-05-11T23:02:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78589 2024-05-11T23:02:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:02:54.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:02:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970427+0000", 2024-05-11T23:02:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:02:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:54.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970429+0000", 2024-05-11T23:02:54.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3800000000000001e-06 2024-05-11T23:02:54.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:54.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970435+0000", 2024-05-11T23:02:54.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2530000000000001e-06 2024-05-11T23:02:54.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:54.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970441+0000", 2024-05-11T23:02:54.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-05-11T23:02:54.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014822+0000", 2024-05-11T23:02:54.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04438168 2024-05-11T23:02:54.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:54.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014911+0000", 2024-05-11T23:02:54.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8882999999999996e-05 2024-05-11T23:02:54.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:54.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015014+0000", 2024-05-11T23:02:54.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010263399999999999 2024-05-11T23:02:54.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.243 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.244 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78590 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:54.244 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:02:54.244 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.236088107999997, 2024-05-11T23:02:54.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044880120000000003, 2024-05-11T23:02:54.244 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:54.244 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:54.244 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:54.244 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:54.244 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:54.244 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:54.244 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78590 2024-05-11T23:02:54.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.245 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:54.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:54.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:02:54.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:54.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:02:54.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:54.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970918+0000", 2024-05-11T23:02:54.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000470834 2024-05-11T23:02:54.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:54.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970925+0000", 2024-05-11T23:02:54.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1690000000000003e-06 2024-05-11T23:02:54.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:54.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970934+0000", 2024-05-11T23:02:54.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9099999999999994e-06 2024-05-11T23:02:54.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:54.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970941+0000", 2024-05-11T23:02:54.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7410000000000003e-06 2024-05-11T23:02:54.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015049+0000", 2024-05-11T23:02:54.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044108392000000003 2024-05-11T23:02:54.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:54.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015133+0000", 2024-05-11T23:02:54.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3568999999999999e-05 2024-05-11T23:02:54.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:54.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015327+0000", 2024-05-11T23:02:54.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019450500000000001 2024-05-11T23:02:54.248 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.248 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.248 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.248 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78591 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:54.248 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:02:54.248 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.235584692000003, 2024-05-11T23:02:54.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044618497, 2024-05-11T23:02:54.248 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:54.248 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:54.248 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:54.248 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:54.249 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:54.249 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:54.249 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78591 2024-05-11T23:02:54.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.249 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:54.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:54.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:02:54.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:54.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:02:54.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:54.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970951+0000", 2024-05-11T23:02:54.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:02:54.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:54.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970953+0000", 2024-05-11T23:02:54.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:02:54.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:54.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970956+0000", 2024-05-11T23:02:54.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-05-11T23:02:54.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:54.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970959+0000", 2024-05-11T23:02:54.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9409999999999999e-06 2024-05-11T23:02:54.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015367+0000", 2024-05-11T23:02:54.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044407621000000001 2024-05-11T23:02:54.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:54.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015453+0000", 2024-05-11T23:02:54.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6685000000000001e-05 2024-05-11T23:02:54.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:54.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015569+0000", 2024-05-11T23:02:54.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115396 2024-05-11T23:02:54.252 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.252 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.252 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.252 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78592 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:54.252 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:02:54.252 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.235571884000002, 2024-05-11T23:02:54.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044831546999999999, 2024-05-11T23:02:54.252 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:54.252 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:54.252 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:54.253 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:54.253 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:54.253 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:54.253 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78592 2024-05-11T23:02:54.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.253 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:54.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:54.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:02:54.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:54.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:02:54.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:54.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972226+0000", 2024-05-11T23:02:54.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00126246 2024-05-11T23:02:54.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:54.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972230+0000", 2024-05-11T23:02:54.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1860000000000002e-06 2024-05-11T23:02:54.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:54.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972237+0000", 2024-05-11T23:02:54.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:02:54.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:54.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972241+0000", 2024-05-11T23:02:54.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.284e-06 2024-05-11T23:02:54.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015611+0000", 2024-05-11T23:02:54.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043369461999999998 2024-05-11T23:02:54.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:54.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015695+0000", 2024-05-11T23:02:54.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4458999999999999e-05 2024-05-11T23:02:54.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.256 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:54.256 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015795+0000", 2024-05-11T23:02:54.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9475999999999994e-05 2024-05-11T23:02:54.256 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.256 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.256 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.256 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78593 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:54.256 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:02:54.256 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.234284965000001, 2024-05-11T23:02:54.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046045214000000001, 2024-05-11T23:02:54.256 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:54.256 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:54.256 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:54.257 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:54.257 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:54.257 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:54.257 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78593 2024-05-11T23:02:54.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.257 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:54.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:54.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:02:54.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:54.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:02:54.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:54.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:02:54.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7599999999999998e-07 2024-05-11T23:02:54.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:54.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972252+0000", 2024-05-11T23:02:54.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7349999999999999e-06 2024-05-11T23:02:54.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:54.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972256+0000", 2024-05-11T23:02:54.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5659999999999998e-06 2024-05-11T23:02:54.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:54.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972259+0000", 2024-05-11T23:02:54.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8040000000000002e-06 2024-05-11T23:02:54.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018088+0000", 2024-05-11T23:02:54.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045829316000000002 2024-05-11T23:02:54.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:54.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018179+0000", 2024-05-11T23:02:54.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1299999999999997e-05 2024-05-11T23:02:54.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:54.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018295+0000", 2024-05-11T23:02:54.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116117 2024-05-11T23:02:54.260 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.260 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.260 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.260 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78594 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:54.260 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:02:54.260 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.234272605000001, 2024-05-11T23:02:54.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046245204999999998, 2024-05-11T23:02:54.261 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:54.261 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:54.261 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:54.261 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:54.261 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:54.261 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:54.261 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78594 2024-05-11T23:02:54.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.261 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:54.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:54.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:02:54.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:54.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:02:54.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:54.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972263+0000", 2024-05-11T23:02:54.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:02:54.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:54.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972264+0000", 2024-05-11T23:02:54.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.206e-06 2024-05-11T23:02:54.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:54.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972267+0000", 2024-05-11T23:02:54.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0690000000000002e-06 2024-05-11T23:02:54.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:54.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972270+0000", 2024-05-11T23:02:54.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5610000000000001e-06 2024-05-11T23:02:54.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018330+0000", 2024-05-11T23:02:54.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04606085 2024-05-11T23:02:54.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:54.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018414+0000", 2024-05-11T23:02:54.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3125000000000001e-05 2024-05-11T23:02:54.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:54.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018508+0000", 2024-05-11T23:02:54.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4153000000000005e-05 2024-05-11T23:02:54.264 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.264 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.264 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.264 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78595 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:54.264 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:02:54.264 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.234259739999999, 2024-05-11T23:02:54.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046396698, 2024-05-11T23:02:54.265 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:54.265 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:54.265 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:54.265 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:54.265 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:54.265 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:54.265 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78595 2024-05-11T23:02:54.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.265 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:54.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:54.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:02:54.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:54.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:02:54.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:54.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972276+0000", 2024-05-11T23:02:54.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5600000000000002e-07 2024-05-11T23:02:54.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:54.266 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972277+0000", 2024-05-11T23:02:54.266 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.576e-06 2024-05-11T23:02:54.266 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.266 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.266 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:54.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972280+0000", 2024-05-11T23:02:54.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-05-11T23:02:54.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:54.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972283+0000", 2024-05-11T23:02:54.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:02:54.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.267 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018538+0000", 2024-05-11T23:02:54.267 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046255255000000002 2024-05-11T23:02:54.267 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.267 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.267 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:54.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018605+0000", 2024-05-11T23:02:54.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6847000000000006e-05 2024-05-11T23:02:54.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.268 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:54.268 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018672+0000", 2024-05-11T23:02:54.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7045000000000006e-05 2024-05-11T23:02:54.268 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.268 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.268 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.268 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.268 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.268 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78596 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:54.268 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:02:54.268 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.234248891, 2024-05-11T23:02:54.268 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046586806000000001, 2024-05-11T23:02:54.269 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:54.269 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:54.269 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:54.269 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:54.269 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:54.269 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:54.269 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78596 2024-05-11T23:02:54.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.269 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:54.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.269 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:54.269 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:02:54.269 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.269 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.269 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:54.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:02:54.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:54.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:02:54.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:02:54.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.270 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.270 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:54.270 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972288+0000", 2024-05-11T23:02:54.270 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2550000000000001e-06 2024-05-11T23:02:54.270 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:54.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972291+0000", 2024-05-11T23:02:54.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9490000000000001e-06 2024-05-11T23:02:54.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:54.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972293+0000", 2024-05-11T23:02:54.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5069999999999999e-06 2024-05-11T23:02:54.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.271 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.271 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018707+0000", 2024-05-11T23:02:54.271 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046413604999999997 2024-05-11T23:02:54.271 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.271 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:54.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018782+0000", 2024-05-11T23:02:54.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5556000000000006e-05 2024-05-11T23:02:54.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.272 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:54.272 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018873+0000", 2024-05-11T23:02:54.272 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0703000000000002e-05 2024-05-11T23:02:54.272 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.272 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.272 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.272 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.272 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.272 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78597 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:54.272 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:02:54.273 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.234239117999998, 2024-05-11T23:02:54.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046825787000000001, 2024-05-11T23:02:54.273 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:54.273 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:54.273 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:54.273 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:54.273 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:54.273 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:54.273 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78597 2024-05-11T23:02:54.273 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.273 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:54.273 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.273 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:54.273 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:02:54.273 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:54.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:02:54.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:54.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:02:54.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-05-11T23:02:54.274 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.274 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.274 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:54.274 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972297+0000", 2024-05-11T23:02:54.274 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3069999999999999e-06 2024-05-11T23:02:54.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:54.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972300+0000", 2024-05-11T23:02:54.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:02:54.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:54.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972303+0000", 2024-05-11T23:02:54.275 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5509999999999998e-06 2024-05-11T23:02:54.275 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.275 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.275 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.275 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018911+0000", 2024-05-11T23:02:54.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046608194999999998 2024-05-11T23:02:54.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:54.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018986+0000", 2024-05-11T23:02:54.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5360999999999999e-05 2024-05-11T23:02:54.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.276 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.276 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:54.276 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019122+0000", 2024-05-11T23:02:54.276 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013526 2024-05-11T23:02:54.276 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.276 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.276 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.276 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.277 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78598 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:54.277 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:02:54.277 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 38.234229218000003, 2024-05-11T23:02:54.277 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046998572000000002, 2024-05-11T23:02:54.277 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:54.277 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:54.277 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:54.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:54.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:54.277 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:54.277 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78598 2024-05-11T23:02:54.277 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.277 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:54.277 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.277 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:54.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:02:54.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:54.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:02:54.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.278 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:54.278 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:02:54.278 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:02:54.278 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.278 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:54.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972307+0000", 2024-05-11T23:02:54.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1799999999999999e-06 2024-05-11T23:02:54.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.279 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:54.279 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972310+0000", 2024-05-11T23:02:54.279 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:02:54.279 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.279 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:54.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972313+0000", 2024-05-11T23:02:54.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4760000000000001e-06 2024-05-11T23:02:54.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019149+0000", 2024-05-11T23:02:54.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046836056000000001 2024-05-11T23:02:54.280 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.280 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.280 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:54.280 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019203+0000", 2024-05-11T23:02:54.280 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-05 2024-05-11T23:02:54.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.281 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:54.281 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019304+0000", 2024-05-11T23:02:54.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101537 2024-05-11T23:02:54.281 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.281 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.281 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.281 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.281 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.281 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:105234 3.15 3:adf885e5:::rbd_object_map.111e913a615f.0000000000000025:head [writefull 0~39] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-11T23:02:54.281 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:02:54.281 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.198913508, 2024-05-11T23:02:54.281 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77797599100000003, 2024-05-11T23:02:54.281 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:54.281 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:54.282 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:54.282 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:54.282 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:54.282 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:54.282 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 105234 2024-05-11T23:02:54.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.282 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:54.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:54.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:02:54.282 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.282 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.282 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.282 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:54.282 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:02:54.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:54.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007623+0000", 2024-05-11T23:02:54.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2240000000000001e-06 2024-05-11T23:02:54.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:54.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007627+0000", 2024-05-11T23:02:54.283 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:02:54.283 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.283 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.283 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:54.283 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007636+0000", 2024-05-11T23:02:54.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2409999999999995e-06 2024-05-11T23:02:54.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:54.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007641+0000", 2024-05-11T23:02:54.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6810000000000003e-06 2024-05-11T23:02:54.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007676+0000", 2024-05-11T23:02:54.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5061999999999997e-05 2024-05-11T23:02:54.284 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.284 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.284 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:02:54.284 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007682+0000", 2024-05-11T23:02:54.284 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0399999999999998e-06 2024-05-11T23:02:54.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783880+0000", 2024-05-11T23:02:54.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77619752900000005 2024-05-11T23:02:54.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:54.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783995+0000", 2024-05-11T23:02:54.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011550800000000001 2024-05-11T23:02:54.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.285 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:02:54.285 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784082+0000", 2024-05-11T23:02:54.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6347999999999994e-05 2024-05-11T23:02:54.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:54.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784682+0000", 2024-05-11T23:02:54.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000600162 2024-05-11T23:02:54.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:54.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785539+0000", 2024-05-11T23:02:54.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085699099999999998 2024-05-11T23:02:54.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:54.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785572+0000", 2024-05-11T23:02:54.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3182999999999997e-05 2024-05-11T23:02:54.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:54.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785598+0000", 2024-05-11T23:02:54.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5522000000000001e-05 2024-05-11T23:02:54.287 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.287 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.287 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.287 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113761 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [call rbd.sparse_copyup in=999496b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:02:54.287 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:02:54.287 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.676226588999999, 2024-05-11T23:02:54.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048572964000000003, 2024-05-11T23:02:54.288 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:54.288 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:54.288 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:54.288 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:54.288 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:54.288 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:54.288 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113761 2024-05-11T23:02:54.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.288 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:54.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:54.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:02:54.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:54.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:02:54.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:54.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530309+0000", 2024-05-11T23:02:54.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:02:54.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.289 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:54.289 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531242+0000", 2024-05-11T23:02:54.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093316899999999997 2024-05-11T23:02:54.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:54.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531249+0000", 2024-05-11T23:02:54.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8140000000000003e-06 2024-05-11T23:02:54.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:54.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531253+0000", 2024-05-11T23:02:54.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4290000000000003e-06 2024-05-11T23:02:54.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570325+0000", 2024-05-11T23:02:54.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039071379000000003 2024-05-11T23:02:54.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:54.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570382+0000", 2024-05-11T23:02:54.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6988999999999999e-05 2024-05-11T23:02:54.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:02:54.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570555+0000", 2024-05-11T23:02:54.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017379799999999999 2024-05-11T23:02:54.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:54.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.575953+0000", 2024-05-11T23:02:54.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053975539999999997 2024-05-11T23:02:54.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:54.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578778+0000", 2024-05-11T23:02:54.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002825616 2024-05-11T23:02:54.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:54.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578837+0000", 2024-05-11T23:02:54.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9005000000000002e-05 2024-05-11T23:02:54.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:54.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578881+0000", 2024-05-11T23:02:54.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3970000000000001e-05 2024-05-11T23:02:54.292 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.292 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.292 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.293 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113762 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [delete] snapc 27=[27] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:02:54.293 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:02:54.293 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 19.675272637999999, 2024-05-11T23:02:54.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047916260000000002, 2024-05-11T23:02:54.293 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:54.293 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:54.293 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:54.293 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:54.293 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:54.293 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:54.293 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113762 2024-05-11T23:02:54.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.293 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:54.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:54.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:02:54.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:54.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:02:54.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:54.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:54.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531263+0000", 2024-05-11T23:02:54.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.85e-07 2024-05-11T23:02:54.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:54.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531265+0000", 2024-05-11T23:02:54.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.827e-06 2024-05-11T23:02:54.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:54.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531268+0000", 2024-05-11T23:02:54.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3780000000000001e-06 2024-05-11T23:02:54.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:54.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531271+0000", 2024-05-11T23:02:54.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8509999999999999e-06 2024-05-11T23:02:54.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:54.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572298+0000", 2024-05-11T23:02:54.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041026990999999999 2024-05-11T23:02:54.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:54.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572345+0000", 2024-05-11T23:02:54.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6878000000000001e-05 2024-05-11T23:02:54.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:02:54.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572443+0000", 2024-05-11T23:02:54.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8184999999999996e-05 2024-05-11T23:02:54.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:54.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578585+0000", 2024-05-11T23:02:54.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006141919 2024-05-11T23:02:54.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:54.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579126+0000", 2024-05-11T23:02:54.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00054063799999999997 2024-05-11T23:02:54.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:54.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579154+0000", 2024-05-11T23:02:54.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8042999999999999e-05 2024-05-11T23:02:54.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:54.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:54.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:54.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579179+0000", 2024-05-11T23:02:54.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4765000000000001e-05 2024-05-11T23:02:54.298 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.298 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.298 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.298 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:54.298 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:54.298 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:54.299 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:54.300 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:54.300 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:54.300 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:54.301 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:02:54.301 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:54.302 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:54.302 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:54.302 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:54.302 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:54.303 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:54.303 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:54.299+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:54.305 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:54.305 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:54.326 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:54.326 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:54.400 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:54.400+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:54.425 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:54.426 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:54.436 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:02:54.500 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:54.501+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:54.592 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:54.592 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:54.602 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:02:54.603 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:54.602+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:54.703 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:54.704+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:54.724 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:54.725 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:54.734 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:02:54.804 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:54.805+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:54.850 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:54.850 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:54.860 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:02:54.905 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:54.905+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:54.980 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:54.980 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:54.989 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:02:55.033 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:55.033+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:55.099 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:55.099 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:55.106 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:55.107+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:55.108 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:02:55.124 INFO:teuthology.orchestra.run.smithi005.stdout:4616 write 0xe7568 thru 0xecf36 (0x59cf bytes) 2024-05-11T23:02:55.128 INFO:teuthology.orchestra.run.smithi005.stdout:4618 punch from 0x385d8b1 to 0x38601a6, (0x28f5 bytes) 2024-05-11T23:02:55.129 INFO:teuthology.orchestra.run.smithi005.stdout:4619 read 0x2689c8 thru 0x26f93a (0x6f73 bytes) 2024-05-11T23:02:55.129 INFO:teuthology.orchestra.run.smithi005.stdout:4620 punch from 0x55d12a6 to 0x55d1c18, (0x972 bytes) 2024-05-11T23:02:55.136 INFO:teuthology.orchestra.run.smithi005.stdout:4622 writesame 0x6645ec7 thru 0x66492bd data_size 0x1(0x33f7 bytes) 2024-05-11T23:02:55.136 INFO:teuthology.orchestra.run.smithi005.stdout:4623 read 0x68d558 thru 0x694fc2 (0x7a6b bytes) 2024-05-11T23:02:55.141 INFO:teuthology.orchestra.run.smithi005.stdout:4624 trunc from 0x66492be to 0xaa8dfea 2024-05-11T23:02:55.147 INFO:teuthology.orchestra.run.smithi005.stdout:4625 read 0x982b4ab thru 0x983768a (0xc1e0 bytes) 2024-05-11T23:02:55.147 INFO:teuthology.orchestra.run.smithi005.stdout:4626 writesame 0x96a87d0 thru 0x96b159e data_size 0x1(0x8dcf bytes) 2024-05-11T23:02:55.166 INFO:teuthology.orchestra.run.smithi005.stdout:4627 read 0x44594e7 thru 0x446460c (0xb126 bytes) 2024-05-11T23:02:55.166 INFO:teuthology.orchestra.run.smithi005.stdout:4628 write 0x593c46a thru 0x59444a1 (0x8038 bytes) 2024-05-11T23:02:55.170 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:55.170 INFO:teuthology.orchestra.run.smithi005.stdout:4631 writesame 0x6b71dd0 thru 0x6b78bcc data_size 0x1(0x6dfd bytes) 2024-05-11T23:02:55.208 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:55.208+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:55.239 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:55.239 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:55.248 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:02:55.308 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:55.309+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:55.398 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:55.398 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:55.406 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:02:55.409 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:55.409+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:55.510 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:55.510+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:55.516 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:55.516 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:55.525 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:02:55.612 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:55.612+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:55.675 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:55.676 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:55.684 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:02:55.712 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:02:55.712+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:02:55.813 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:55.813+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:55.834 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:55.834 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:55.843 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:02:55.928 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:55.928+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:55.994 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:55.995 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:56.004 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:02:56.045 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:56.045+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:56.116 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:56.116+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:56.118 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:56.118 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:56.118 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 5, 7, 3, 6] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:02:56.118 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:02:56.118 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-05-11T23:02:56.119 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-05-11T23:02:56.130 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:02:56.135 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-11T23:02:56.146 INFO:teuthology.orchestra.run.smithi005.stdout:4632 trunc from 0xaa8dfea to 0xec7a308 2024-05-11T23:02:56.160 INFO:teuthology.orchestra.run.smithi005.stdout:4634 trunc from 0xec7a308 to 0x8153f6e 2024-05-11T23:02:56.168 INFO:teuthology.orchestra.run.smithi005.stdout:4635 read 0x6332e0f thru 0x633a3f2 (0x75e4 bytes) 2024-05-11T23:02:56.168 INFO:teuthology.orchestra.run.smithi005.stdout:4636 punch from 0x3bc0343 to 0x3bca62f, (0xa2ec bytes) 2024-05-11T23:02:56.168 INFO:teuthology.orchestra.run.smithi005.stdout:4637 read 0x4e40181 thru 0x4e42fb3 (0x2e33 bytes) 2024-05-11T23:02:56.172 INFO:teuthology.orchestra.run.smithi005.stdout:4638 trunc from 0x8153f6e to 0x513966f 2024-05-11T23:02:56.204 INFO:teuthology.orchestra.run.smithi005.stdout:4639 writesame 0xe5d041e thru 0xe5d9662 data_size 0x1(0x9245 bytes) 2024-05-11T23:02:56.204 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:56.205 INFO:teuthology.orchestra.run.smithi005.stdout:4641 write 0x52b769a thru 0x52c6b04 (0xf46b bytes) 2024-05-11T23:02:56.208 INFO:teuthology.orchestra.run.smithi005.stdout:4642 read 0x515d0e5 thru 0x515f9d8 (0x28f4 bytes) 2024-05-11T23:02:56.208 INFO:teuthology.orchestra.run.smithi005.stdout:4646 write 0xcd61bc1 thru 0xcd64da8 (0x31e8 bytes) 2024-05-11T23:02:56.211 INFO:teuthology.orchestra.run.smithi005.stdout:4647 write 0x69636f thru 0x69fa82 (0x9714 bytes) 2024-05-11T23:02:56.214 INFO:teuthology.orchestra.run.smithi005.stdout:4648 read 0x5f9e14c thru 0x5f9ecfe (0xbb3 bytes) 2024-05-11T23:02:56.217 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:56.217+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:56.218 INFO:teuthology.orchestra.run.smithi005.stdout:4649 trunc from 0xe5d9663 to 0xac7c4d3 2024-05-11T23:02:56.228 INFO:teuthology.orchestra.run.smithi005.stdout:4650 read 0x77485bc thru 0x774cccb (0x4710 bytes) 2024-05-11T23:02:56.228 INFO:teuthology.orchestra.run.smithi005.stdout:4651 punch from 0x47befba to 0x47ccf19, (0xdf5f bytes) 2024-05-11T23:02:56.228 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:56.236 INFO:teuthology.orchestra.run.smithi005.stdout:4653 trunc from 0xac7c4d3 to 0x37e9e8b 2024-05-11T23:02:56.245 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:56.245 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:56.246 INFO:teuthology.orchestra.run.smithi005.stdout:4655 read 0x84f201 thru 0x85e3ee (0xf1ee bytes) 2024-05-11T23:02:56.255 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:02:56.255 INFO:teuthology.orchestra.run.smithi005.stdout:4656 write 0x45c7fb7 thru 0x45d30c3 (0xb10d bytes) 2024-05-11T23:02:56.258 INFO:teuthology.orchestra.run.smithi005.stdout:4657 punch from 0x1340323 to 0x1343844, (0x3521 bytes) 2024-05-11T23:02:56.258 INFO:teuthology.orchestra.run.smithi005.stdout:4658 read 0x3ca69a2 thru 0x3ca9552 (0x2bb1 bytes) 2024-05-11T23:02:56.258 INFO:teuthology.orchestra.run.smithi005.stdout:4660 read 0x22d5d4d thru 0x22dd93b (0x7bef bytes) 2024-05-11T23:02:56.266 INFO:teuthology.orchestra.run.smithi005.stdout:4661 write 0x6b98234 thru 0x6b984da (0x2a7 bytes) 2024-05-11T23:02:56.285 INFO:teuthology.orchestra.run.smithi005.stdout:4662 writesame 0xec65111 thru 0xec6a597 data_size 0x1(0x5487 bytes) 2024-05-11T23:02:56.285 INFO:teuthology.orchestra.run.smithi005.stdout:4663 read 0xf0f460 thru 0xf14715 (0x52b6 bytes) 2024-05-11T23:02:56.286 INFO:teuthology.orchestra.run.smithi005.stdout:4664 punch from 0x6bcc2f5 to 0x6bce2a3, (0x1fae bytes) 2024-05-11T23:02:56.286 INFO:teuthology.orchestra.run.smithi005.stdout:4666 writesame 0xd04d4c thru 0xd14386 data_size 0x7(0xf63b bytes) 2024-05-11T23:02:56.288 INFO:teuthology.orchestra.run.smithi005.stderr:no valid command found; 10 closest matches: 2024-05-11T23:02:56.288 INFO:teuthology.orchestra.run.smithi005.stderr:0 2024-05-11T23:02:56.288 INFO:teuthology.orchestra.run.smithi005.stderr:1 2024-05-11T23:02:56.288 INFO:teuthology.orchestra.run.smithi005.stderr:2 2024-05-11T23:02:56.288 INFO:teuthology.orchestra.run.smithi005.stderr:abort 2024-05-11T23:02:56.288 INFO:teuthology.orchestra.run.smithi005.stderr:assert 2024-05-11T23:02:56.289 INFO:teuthology.orchestra.run.smithi005.stderr:bench [] [] [] [] 2024-05-11T23:02:56.289 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs debug_inject_read_zeros 2024-05-11T23:02:56.289 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs files list 2024-05-11T23:02:56.289 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs stats 2024-05-11T23:02:56.289 INFO:teuthology.orchestra.run.smithi005.stderr:bluestore allocator dump block 2024-05-11T23:02:56.289 INFO:teuthology.orchestra.run.smithi005.stderr:Error EINVAL: invalid command 2024-05-11T23:02:56.290 INFO:teuthology.orchestra.run.smithi005.stdout:4667 punch from 0xa668525 to 0xa66924e, (0xd29 bytes) 2024-05-11T23:02:56.290 INFO:teuthology.orchestra.run.smithi005.stdout:4668 read 0xd48d33c thru 0xd496b5f (0x9824 bytes) 2024-05-11T23:02:56.290 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:56.292 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-11T23:02:56.292 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-11T23:02:56.292 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-05-11T23:02:56.306 INFO:teuthology.orchestra.run.smithi005.stdout:4672 trunc from 0xec6a598 to 0x1679f6c 2024-05-11T23:02:56.318 INFO:teuthology.orchestra.run.smithi005.stdout:4673 read 0xc7ec7b thru 0xc8282b (0x3bb1 bytes) 2024-05-11T23:02:56.318 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:56.318+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:56.339 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:02:56.342 INFO:teuthology.orchestra.run.smithi005.stdout:4674 write 0xd800e1b thru 0xd80a184 (0x936a bytes) 2024-05-11T23:02:56.345 INFO:teuthology.orchestra.run.smithi005.stdout:4676 read 0x1304fd2 thru 0x1313c0d (0xec3c bytes) 2024-05-11T23:02:56.346 INFO:teuthology.orchestra.run.smithi005.stdout:4677 read 0x9908f44 thru 0x99167c1 (0xd87e bytes) 2024-05-11T23:02:56.346 INFO:teuthology.orchestra.run.smithi005.stdout:4679 write 0x14f4a5 thru 0x15ef6b (0xfac7 bytes) 2024-05-11T23:02:56.348 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:56.348 INFO:teuthology.orchestra.run.smithi005.stdout:4682 read 0x346e818 thru 0x3470c39 (0x2422 bytes) 2024-05-11T23:02:56.349 INFO:teuthology.orchestra.run.smithi005.stdout:4683 writesame 0x5dd81d8 thru 0x5de2a74 data_size 0x59(0xa89d bytes) 2024-05-11T23:02:56.353 INFO:teuthology.orchestra.run.smithi005.stdout:4685 read 0x41ad1b0 thru 0x41b7c8d (0xaade bytes) 2024-05-11T23:02:56.353 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:56.353 INFO:teuthology.orchestra.run.smithi005.stdout:4687 writesame 0x3a09f78 thru 0x3a10c8a data_size 0x1(0x6d13 bytes) 2024-05-11T23:02:56.372 INFO:teuthology.orchestra.run.smithi005.stdout:4688 read 0x8f5d5dd thru 0x8f6a28f (0xccb3 bytes) 2024-05-11T23:02:56.373 INFO:teuthology.orchestra.run.smithi005.stdout:4689 write 0x137e044 thru 0x1380922 (0x28df bytes) 2024-05-11T23:02:56.373 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:56.374 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:56.374 INFO:teuthology.orchestra.run.smithi005.stdout:4690 write 0xa25148b thru 0xa258912 (0x7488 bytes) 2024-05-11T23:02:56.378 INFO:teuthology.orchestra.run.smithi005.stdout:4692 write 0xb258a54 thru 0xb25e8f8 (0x5ea5 bytes) 2024-05-11T23:02:56.383 INFO:teuthology.orchestra.run.smithi005.stdout:4694 read 0x60e5e01 thru 0x60f4129 (0xe329 bytes) 2024-05-11T23:02:56.384 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:02:56.394 INFO:teuthology.orchestra.run.smithi005.stdout:4695 trunc from 0xd80a185 to 0x6e1c365 2024-05-11T23:02:56.407 INFO:teuthology.orchestra.run.smithi005.stdout:4696 read 0x250b8b9 thru 0x2519988 (0xe0d0 bytes) 2024-05-11T23:02:56.418 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:56.418+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:56.422 INFO:teuthology.orchestra.run.smithi005.stdout:4697 write 0xbe51128 thru 0xbe53891 (0x276a bytes) 2024-05-11T23:02:56.425 INFO:teuthology.orchestra.run.smithi005.stdout:4698 read 0x122cf7d thru 0x1231da0 (0x4e24 bytes) 2024-05-11T23:02:56.436 INFO:teuthology.orchestra.run.smithi005.stdout:4699 write 0xee5d732 thru 0xee6b27f (0xdb4e bytes) 2024-05-11T23:02:56.440 INFO:teuthology.orchestra.run.smithi005.stdout:4700 read 0x3dfe698 thru 0x3e0738e (0x8cf7 bytes) 2024-05-11T23:02:56.440 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:56.440 INFO:teuthology.orchestra.run.smithi005.stdout:4702 read 0x57043bc thru 0x570569d (0x12e2 bytes) 2024-05-11T23:02:56.440 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:56.440 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:56.440 INFO:teuthology.orchestra.run.smithi005.stdout:4706 punch from 0xecd9c83 to 0xecdf0e8, (0x5465 bytes) 2024-05-11T23:02:56.440 INFO:teuthology.orchestra.run.smithi005.stdout:4707 read 0x31d7ec thru 0x323d45 (0x655a bytes) 2024-05-11T23:02:56.441 INFO:teuthology.orchestra.run.smithi005.stdout:4708 read 0x8460e95 thru 0x846489f (0x3a0b bytes) 2024-05-11T23:02:56.441 INFO:teuthology.orchestra.run.smithi005.stdout:4709 write 0x558e47a thru 0x559c2bf (0xde46 bytes) 2024-05-11T23:02:56.444 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:56.444 INFO:teuthology.orchestra.run.smithi005.stdout:4711 read 0xe5a6f35 thru 0xe5a8344 (0x1410 bytes) 2024-05-11T23:02:56.444 INFO:teuthology.orchestra.run.smithi005.stdout:4712 read 0x2a57807 thru 0x2a63e7d (0xc677 bytes) 2024-05-11T23:02:56.445 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:56.445 INFO:teuthology.orchestra.run.smithi005.stdout:4715 punch from 0x5ad1a3a to 0x5adf0b2, (0xd678 bytes) 2024-05-11T23:02:56.445 INFO:teuthology.orchestra.run.smithi005.stdout:4717 punch from 0x7ea9f83 to 0x7eb891e, (0xe99b bytes) 2024-05-11T23:02:56.445 INFO:teuthology.orchestra.run.smithi005.stdout:4718 write 0x4ffa4f5 thru 0x5001e94 (0x79a0 bytes) 2024-05-11T23:02:56.448 INFO:teuthology.orchestra.run.smithi005.stdout:4720 read 0x43e7ab0 thru 0x43ea70c (0x2c5d bytes) 2024-05-11T23:02:56.448 INFO:teuthology.orchestra.run.smithi005.stdout:4721 write 0xb222a43 thru 0xb22810e (0x56cc bytes) 2024-05-11T23:02:56.451 INFO:teuthology.orchestra.run.smithi005.stdout:4722 punch from 0x811d7b0 to 0x81294b8, (0xbd08 bytes) 2024-05-11T23:02:56.451 INFO:teuthology.orchestra.run.smithi005.stdout:4723 read 0x64482d3 thru 0x64527d1 (0xa4ff bytes) 2024-05-11T23:02:56.451 INFO:teuthology.orchestra.run.smithi005.stdout:4724 write 0x9abbad3 thru 0x9abc9f3 (0xf21 bytes) 2024-05-11T23:02:56.454 INFO:teuthology.orchestra.run.smithi005.stdout:4725 writesame 0x3c1c414 thru 0x3c21567 data_size 0x14(0x5154 bytes) 2024-05-11T23:02:56.458 INFO:teuthology.orchestra.run.smithi005.stdout:4726 punch from 0xae1c1df to 0xae1fec2, (0x3ce3 bytes) 2024-05-11T23:02:56.458 INFO:teuthology.orchestra.run.smithi005.stdout:4727 read 0x140186c thru 0x1410ccd (0xf462 bytes) 2024-05-11T23:02:56.458 INFO:teuthology.orchestra.run.smithi005.stdout:4729 write 0x85fe82e thru 0x8603836 (0x5009 bytes) 2024-05-11T23:02:56.462 INFO:teuthology.orchestra.run.smithi005.stdout:4730 write 0x1461f46 thru 0x146a54e (0x8609 bytes) 2024-05-11T23:02:56.463 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:56.471 INFO:teuthology.orchestra.run.smithi005.stdout:4732 trunc from 0xee6b280 to 0x83792e9 2024-05-11T23:02:56.485 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:56.485 INFO:teuthology.orchestra.run.smithi005.stdout:4734 read 0x2bab8c2 thru 0x2bb4208 (0x8947 bytes) 2024-05-11T23:02:56.486 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:02:56.486 INFO:teuthology.orchestra.run.smithi005.stdout:4736 writesame 0x1691ae2 thru 0x169dc54 data_size 0x1(0xc173 bytes) 2024-05-11T23:02:56.491 INFO:teuthology.orchestra.run.smithi005.stderr:no valid command found; 10 closest matches: 2024-05-11T23:02:56.491 INFO:teuthology.orchestra.run.smithi005.stderr:0 2024-05-11T23:02:56.491 INFO:teuthology.orchestra.run.smithi005.stderr:1 2024-05-11T23:02:56.491 INFO:teuthology.orchestra.run.smithi005.stderr:2 2024-05-11T23:02:56.492 INFO:teuthology.orchestra.run.smithi005.stderr:abort 2024-05-11T23:02:56.492 INFO:teuthology.orchestra.run.smithi005.stderr:assert 2024-05-11T23:02:56.492 INFO:teuthology.orchestra.run.smithi005.stderr:bench [] [] [] [] 2024-05-11T23:02:56.492 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs debug_inject_read_zeros 2024-05-11T23:02:56.492 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs files list 2024-05-11T23:02:56.492 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs stats 2024-05-11T23:02:56.492 INFO:teuthology.orchestra.run.smithi005.stderr:bluestore allocator dump block 2024-05-11T23:02:56.492 INFO:teuthology.orchestra.run.smithi005.stderr:Error EINVAL: invalid command 2024-05-11T23:02:56.494 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:56.494 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:56.496 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-11T23:02:56.496 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-11T23:02:56.496 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-05-11T23:02:56.503 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:02:56.509 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:56.509 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:56.511 INFO:teuthology.orchestra.run.smithi005.stdout:4737 trunc from 0x83792e9 to 0xbfdfca1 2024-05-11T23:02:56.517 INFO:teuthology.orchestra.run.smithi005.stdout:4738 read 0x5f8714 thru 0x5f9f4e (0x183b bytes) 2024-05-11T23:02:56.524 INFO:teuthology.orchestra.run.smithi005.stdout:4739 trunc from 0xbfdfca1 to 0x5cdc533 2024-05-11T23:02:56.531 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:56.531 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:56.537 INFO:teuthology.orchestra.run.smithi005.stdout:4740 read 0x36a9dc8 thru 0x36acc50 (0x2e89 bytes) 2024-05-11T23:02:56.537 INFO:teuthology.orchestra.run.smithi005.stdout:4741 read 0x388c05d thru 0x3890faf (0x4f53 bytes) 2024-05-11T23:02:56.537 INFO:teuthology.orchestra.run.smithi005.stdout:4742 clone 44 order 18 su 1024 sc 10 2024-05-11T23:02:56.546 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:56.547+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:56.550 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:56.550 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:56.569 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:56.569 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:56.588 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:56.588 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:56.606 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:56.606 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:56.611 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:56.611 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:56.611 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:56.620 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:02:56.621 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:56.620+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:56.623 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:56.623 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:56.642 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:56.643 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:56.659 INFO:teuthology.orchestra.run.smithi005.stderr:no valid command found; 10 closest matches: 2024-05-11T23:02:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:0 2024-05-11T23:02:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:1 2024-05-11T23:02:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:2 2024-05-11T23:02:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:abort 2024-05-11T23:02:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:assert 2024-05-11T23:02:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:bench [] [] [] [] 2024-05-11T23:02:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs debug_inject_read_zeros 2024-05-11T23:02:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs files list 2024-05-11T23:02:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs stats 2024-05-11T23:02:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:bluestore allocator dump block 2024-05-11T23:02:56.660 INFO:teuthology.orchestra.run.smithi005.stderr:Error EINVAL: invalid command 2024-05-11T23:02:56.662 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-11T23:02:56.662 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-11T23:02:56.662 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-05-11T23:02:56.721 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:56.721+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:56.767 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:56.767 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:56.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:56.767 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:56.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.767 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67817 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:56.767 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:56.767 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.146379887000002, 2024-05-11T23:02:56.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044148845999999999, 2024-05-11T23:02:56.767 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:56.767 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:56.768 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:56.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:56.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:56.768 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:56.768 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67817 2024-05-11T23:02:56.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.768 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:56.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:56.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:56.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:56.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:02:56.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:56.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620227+0000", 2024-05-11T23:02:56.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00046792099999999999 2024-05-11T23:02:56.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:56.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620232+0000", 2024-05-11T23:02:56.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3050000000000003e-06 2024-05-11T23:02:56.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:56.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620243+0000", 2024-05-11T23:02:56.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1289e-05 2024-05-11T23:02:56.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:56.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620250+0000", 2024-05-11T23:02:56.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2130000000000002e-06 2024-05-11T23:02:56.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:56.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663745+0000", 2024-05-11T23:02:56.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043494865000000001 2024-05-11T23:02:56.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:56.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663815+0000", 2024-05-11T23:02:56.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0284999999999995e-05 2024-05-11T23:02:56.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:56.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663908+0000", 2024-05-11T23:02:56.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2967999999999995e-05 2024-05-11T23:02:56.771 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.771 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.772 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.772 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67820 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4902912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:56.772 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:56.772 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.14583038, 2024-05-11T23:02:56.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044096781000000002, 2024-05-11T23:02:56.772 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:56.772 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:56.772 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:56.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:56.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:56.772 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:56.772 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67820 2024-05-11T23:02:56.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.773 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:56.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:56.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:56.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:56.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:02:56.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:56.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620310+0000", 2024-05-11T23:02:56.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9800000000000004e-07 2024-05-11T23:02:56.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:56.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620312+0000", 2024-05-11T23:02:56.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.272e-06 2024-05-11T23:02:56.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:56.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620319+0000", 2024-05-11T23:02:56.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9310000000000002e-06 2024-05-11T23:02:56.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:56.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620323+0000", 2024-05-11T23:02:56.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:02:56.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:56.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664284+0000", 2024-05-11T23:02:56.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043960908999999999 2024-05-11T23:02:56.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:56.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664327+0000", 2024-05-11T23:02:56.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2811000000000001e-05 2024-05-11T23:02:56.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:56.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664406+0000", 2024-05-11T23:02:56.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8588e-05 2024-05-11T23:02:56.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.776 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.776 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67821 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4927488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:56.776 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:56.776 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.145809970000002, 2024-05-11T23:02:56.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044203340000000001, 2024-05-11T23:02:56.776 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:56.776 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:56.776 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:56.776 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:56.776 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:56.776 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:56.776 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67821 2024-05-11T23:02:56.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.777 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:56.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:56.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:56.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:56.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:02:56.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:56.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620330+0000", 2024-05-11T23:02:56.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:02:56.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:56.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620332+0000", 2024-05-11T23:02:56.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:02:56.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:56.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620338+0000", 2024-05-11T23:02:56.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6250000000000004e-06 2024-05-11T23:02:56.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:56.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620342+0000", 2024-05-11T23:02:56.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7609999999999998e-06 2024-05-11T23:02:56.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:56.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664433+0000", 2024-05-11T23:02:56.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044090455000000001 2024-05-11T23:02:56.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:56.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664476+0000", 2024-05-11T23:02:56.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2861999999999997e-05 2024-05-11T23:02:56.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:56.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664533+0000", 2024-05-11T23:02:56.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7156e-05 2024-05-11T23:02:56.780 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.780 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.780 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.780 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67822 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4976640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:56.780 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:56.780 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.145790931999997, 2024-05-11T23:02:56.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044409416, 2024-05-11T23:02:56.780 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:56.780 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:56.780 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:56.780 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:56.780 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:56.780 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:56.781 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67822 2024-05-11T23:02:56.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.781 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:56.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:56.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:56.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:56.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:02:56.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:56.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620349+0000", 2024-05-11T23:02:56.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.51e-07 2024-05-11T23:02:56.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:56.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620351+0000", 2024-05-11T23:02:56.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.181e-06 2024-05-11T23:02:56.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:56.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620358+0000", 2024-05-11T23:02:56.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3060000000000003e-06 2024-05-11T23:02:56.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:56.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620362+0000", 2024-05-11T23:02:56.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7949999999999998e-06 2024-05-11T23:02:56.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:56.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664572+0000", 2024-05-11T23:02:56.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044209318999999997 2024-05-11T23:02:56.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:56.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664654+0000", 2024-05-11T23:02:56.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2367999999999995e-05 2024-05-11T23:02:56.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:56.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664758+0000", 2024-05-11T23:02:56.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103896 2024-05-11T23:02:56.784 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.784 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.784 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.784 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67823 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4952064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:56.784 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:56.784 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.145770882000001, 2024-05-11T23:02:56.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044623293000000001, 2024-05-11T23:02:56.784 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:56.784 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:56.784 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:56.784 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:56.784 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:56.785 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:56.785 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67823 2024-05-11T23:02:56.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.785 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:56.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:56.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:56.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:56.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:02:56.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:56.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622228+0000", 2024-05-11T23:02:56.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018592280000000001 2024-05-11T23:02:56.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:56.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622232+0000", 2024-05-11T23:02:56.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.403e-06 2024-05-11T23:02:56.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:56.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622248+0000", 2024-05-11T23:02:56.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5449000000000001e-05 2024-05-11T23:02:56.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:56.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622254+0000", 2024-05-11T23:02:56.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5819999999999998e-06 2024-05-11T23:02:56.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:56.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664800+0000", 2024-05-11T23:02:56.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042546048000000003 2024-05-11T23:02:56.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:56.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664886+0000", 2024-05-11T23:02:56.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5525000000000005e-05 2024-05-11T23:02:56.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:56.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664992+0000", 2024-05-11T23:02:56.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010605800000000001 2024-05-11T23:02:56.788 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.788 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.788 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.788 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67833 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5099520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:56.788 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:56.788 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.143664409000003, 2024-05-11T23:02:56.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044076606999999997, 2024-05-11T23:02:56.788 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:56.788 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:56.788 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:56.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67833 2024-05-11T23:02:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:02:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8099999999999998e-07 2024-05-11T23:02:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622477+0000", 2024-05-11T23:02:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0700000000000001e-06 2024-05-11T23:02:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622483+0000", 2024-05-11T23:02:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0780000000000003e-06 2024-05-11T23:02:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622487+0000", 2024-05-11T23:02:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.352e-06 2024-05-11T23:02:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666460+0000", 2024-05-11T23:02:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043973085000000002 2024-05-11T23:02:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666498+0000", 2024-05-11T23:02:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8441e-05 2024-05-11T23:02:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666552+0000", 2024-05-11T23:02:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3199999999999999e-05 2024-05-11T23:02:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67834 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3944448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:56.792 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.143626130999998, 2024-05-11T23:02:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044166551999999998, 2024-05-11T23:02:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67834 2024-05-11T23:02:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:56.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:02:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622514+0000", 2024-05-11T23:02:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1100000000000002e-07 2024-05-11T23:02:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:56.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622515+0000", 2024-05-11T23:02:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:02:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622519+0000", 2024-05-11T23:02:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8260000000000003e-06 2024-05-11T23:02:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622522+0000", 2024-05-11T23:02:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8899999999999999e-06 2024-05-11T23:02:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666580+0000", 2024-05-11T23:02:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044058235000000001 2024-05-11T23:02:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666624+0000", 2024-05-11T23:02:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3501999999999998e-05 2024-05-11T23:02:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666680+0000", 2024-05-11T23:02:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6042999999999999e-05 2024-05-11T23:02:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67835 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.143613594999998, 2024-05-11T23:02:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044272019000000003, 2024-05-11T23:02:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67835 2024-05-11T23:02:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.797 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:02:56.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1699999999999999e-07 2024-05-11T23:02:56.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:56.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622527+0000", 2024-05-11T23:02:56.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2109999999999999e-06 2024-05-11T23:02:56.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622531+0000", 2024-05-11T23:02:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4910000000000001e-06 2024-05-11T23:02:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622534+0000", 2024-05-11T23:02:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7080000000000002e-06 2024-05-11T23:02:56.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:56.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666703+0000", 2024-05-11T23:02:56.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044169377000000003 2024-05-11T23:02:56.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:56.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666745+0000", 2024-05-11T23:02:56.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1616000000000003e-05 2024-05-11T23:02:56.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:56.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666798+0000", 2024-05-11T23:02:56.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3298999999999999e-05 2024-05-11T23:02:56.804 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.804 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.804 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.804 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67836 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:56.804 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:56.804 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.143602934999997, 2024-05-11T23:02:56.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044373692999999999, 2024-05-11T23:02:56.804 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:56.804 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:56.804 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:56.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:56.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:56.805 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:56.805 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67836 2024-05-11T23:02:56.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.805 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:56.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:56.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:56.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:56.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:02:56.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:56.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622537+0000", 2024-05-11T23:02:56.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:56.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:56.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622538+0000", 2024-05-11T23:02:56.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1510000000000001e-06 2024-05-11T23:02:56.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:56.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622541+0000", 2024-05-11T23:02:56.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:02:56.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622543+0000", 2024-05-11T23:02:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4870000000000001e-06 2024-05-11T23:02:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666821+0000", 2024-05-11T23:02:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044278038999999998 2024-05-11T23:02:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666859+0000", 2024-05-11T23:02:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7224000000000001e-05 2024-05-11T23:02:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:56.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666910+0000", 2024-05-11T23:02:56.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1629999999999999e-05 2024-05-11T23:02:56.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.808 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.808 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67837 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:56.808 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:56.808 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.143593207999999, 2024-05-11T23:02:56.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044477883000000003, 2024-05-11T23:02:56.808 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:56.808 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:56.808 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:56.808 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:56.808 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:56.809 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:56.809 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67837 2024-05-11T23:02:56.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.809 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:56.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:56.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:56.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:56.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:56.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:56.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:02:56.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:56.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:56.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622548+0000", 2024-05-11T23:02:56.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:02:56.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:56.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622550+0000", 2024-05-11T23:02:56.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-05-11T23:02:56.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:56.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622553+0000", 2024-05-11T23:02:56.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.43e-06 2024-05-11T23:02:56.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:56.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666932+0000", 2024-05-11T23:02:56.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044379584 2024-05-11T23:02:56.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:56.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666971+0000", 2024-05-11T23:02:56.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8427000000000003e-05 2024-05-11T23:02:56.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:56.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667024+0000", 2024-05-11T23:02:56.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3226000000000002e-05 2024-05-11T23:02:56.812 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.812 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.812 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.812 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67838 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:56.812 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:56.812 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.143583691000003, 2024-05-11T23:02:56.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044580374999999998, 2024-05-11T23:02:56.812 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:56.812 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:56.812 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:56.812 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:56.812 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:56.812 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:56.813 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67838 2024-05-11T23:02:56.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.813 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:56.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:56.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:56.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:56.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:56.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:56.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:02:56.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:02:56.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:56.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622557+0000", 2024-05-11T23:02:56.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.102e-06 2024-05-11T23:02:56.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:56.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622560+0000", 2024-05-11T23:02:56.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:02:56.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:56.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622562+0000", 2024-05-11T23:02:56.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:02:56.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:56.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667045+0000", 2024-05-11T23:02:56.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044482936000000001 2024-05-11T23:02:56.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:56.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667083+0000", 2024-05-11T23:02:56.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7443e-05 2024-05-11T23:02:56.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:56.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667136+0000", 2024-05-11T23:02:56.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3257000000000002e-05 2024-05-11T23:02:56.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.816 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.816 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.816 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67839 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5394432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:56.816 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:56.816 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.143573949, 2024-05-11T23:02:56.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04470263, 2024-05-11T23:02:56.816 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:56.816 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:56.816 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:56.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:56.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:56.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:56.817 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67839 2024-05-11T23:02:56.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.817 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:56.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:56.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:56.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:56.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:02:56.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:56.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622566+0000", 2024-05-11T23:02:56.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-05-11T23:02:56.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:56.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622572+0000", 2024-05-11T23:02:56.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5610000000000004e-06 2024-05-11T23:02:56.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:56.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622576+0000", 2024-05-11T23:02:56.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3210000000000001e-06 2024-05-11T23:02:56.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:56.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622578+0000", 2024-05-11T23:02:56.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4710000000000001e-06 2024-05-11T23:02:56.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:56.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667160+0000", 2024-05-11T23:02:56.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044582220999999998 2024-05-11T23:02:56.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:56.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667205+0000", 2024-05-11T23:02:56.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5049000000000002e-05 2024-05-11T23:02:56.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:56.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667268+0000", 2024-05-11T23:02:56.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2788999999999998e-05 2024-05-11T23:02:56.820 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.820 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.820 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.820 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67840 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5419008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:56.820 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:56.820 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.143558319, 2024-05-11T23:02:56.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044796006999999999, 2024-05-11T23:02:56.820 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:56.820 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:56.821 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:56.821 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:56.821 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:56.821 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:56.821 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67840 2024-05-11T23:02:56.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.821 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:56.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:56.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:56.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:56.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:56.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:56.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:02:56.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:02:56.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:56.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622583+0000", 2024-05-11T23:02:56.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4389999999999999e-06 2024-05-11T23:02:56.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:56.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622586+0000", 2024-05-11T23:02:56.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:02:56.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:56.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622588+0000", 2024-05-11T23:02:56.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4540000000000001e-06 2024-05-11T23:02:56.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:56.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667287+0000", 2024-05-11T23:02:56.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044698882000000002 2024-05-11T23:02:56.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:56.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667324+0000", 2024-05-11T23:02:56.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7196000000000001e-05 2024-05-11T23:02:56.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:56.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667377+0000", 2024-05-11T23:02:56.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2927999999999999e-05 2024-05-11T23:02:56.824 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.824 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.824 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.824 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67841 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5443584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:56.825 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:56.825 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.143548545999998, 2024-05-11T23:02:56.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044907337999999998, 2024-05-11T23:02:56.825 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:56.825 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:56.825 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:56.825 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:56.825 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:56.825 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:56.825 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67841 2024-05-11T23:02:56.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.825 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:56.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:56.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:56.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:56.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:56.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:56.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:02:56.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:56.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:56.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622592+0000", 2024-05-11T23:02:56.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-05-11T23:02:56.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:56.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622595+0000", 2024-05-11T23:02:56.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:02:56.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:56.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622598+0000", 2024-05-11T23:02:56.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5909999999999998e-06 2024-05-11T23:02:56.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:56.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667401+0000", 2024-05-11T23:02:56.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044802980999999999 2024-05-11T23:02:56.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:56.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667439+0000", 2024-05-11T23:02:56.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8840000000000001e-05 2024-05-11T23:02:56.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:56.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667498+0000", 2024-05-11T23:02:56.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8709000000000003e-05 2024-05-11T23:02:56.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.828 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.828 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67842 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5468160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:56.829 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:56.829 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.143538939000003, 2024-05-11T23:02:56.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045005876, 2024-05-11T23:02:56.829 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:56.829 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:56.829 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:56.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:56.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:56.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:56.829 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67842 2024-05-11T23:02:56.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.829 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:56.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:56.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:56.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:56.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:02:56.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:56.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622601+0000", 2024-05-11T23:02:56.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:56.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:56.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622602+0000", 2024-05-11T23:02:56.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0780000000000001e-06 2024-05-11T23:02:56.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:56.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622605+0000", 2024-05-11T23:02:56.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.914e-06 2024-05-11T23:02:56.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:56.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622607+0000", 2024-05-11T23:02:56.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:02:56.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:56.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667516+0000", 2024-05-11T23:02:56.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044909156999999998 2024-05-11T23:02:56.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:56.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667554+0000", 2024-05-11T23:02:56.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7920000000000003e-05 2024-05-11T23:02:56.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:56.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667606+0000", 2024-05-11T23:02:56.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2111000000000003e-05 2024-05-11T23:02:56.832 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.832 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.832 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.832 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67843 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4067328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:56.832 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:56.833 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.143529413000003, 2024-05-11T23:02:56.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045119014999999998, 2024-05-11T23:02:56.833 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:56.833 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:56.833 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:56.833 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:56.833 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:56.833 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:56.833 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67843 2024-05-11T23:02:56.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.833 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:56.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:56.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:56.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:56.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:02:56.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:56.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624228+0000", 2024-05-11T23:02:56.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001617867 2024-05-11T23:02:56.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:56.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624232+0000", 2024-05-11T23:02:56.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-05-11T23:02:56.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:56.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624240+0000", 2024-05-11T23:02:56.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4589999999999998e-06 2024-05-11T23:02:56.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:56.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624247+0000", 2024-05-11T23:02:56.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-05-11T23:02:56.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:56.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667629+0000", 2024-05-11T23:02:56.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043382012999999997 2024-05-11T23:02:56.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:56.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667674+0000", 2024-05-11T23:02:56.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5172000000000001e-05 2024-05-11T23:02:56.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:56.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667729+0000", 2024-05-11T23:02:56.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5086000000000001e-05 2024-05-11T23:02:56.836 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.836 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.836 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.836 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67849 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4190208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:56.837 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:56.837 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.141771292000001, 2024-05-11T23:02:56.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044061433999999997, 2024-05-11T23:02:56.837 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:56.837 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:56.837 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:56.837 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:56.837 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:56.837 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:56.837 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67849 2024-05-11T23:02:56.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.837 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:56.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:56.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:56.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:56.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:02:56.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:56.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624369+0000", 2024-05-11T23:02:56.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5900000000000002e-07 2024-05-11T23:02:56.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:56.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624371+0000", 2024-05-11T23:02:56.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0310000000000001e-06 2024-05-11T23:02:56.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:56.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624376+0000", 2024-05-11T23:02:56.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4029999999999998e-06 2024-05-11T23:02:56.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:56.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624381+0000", 2024-05-11T23:02:56.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4349999999999996e-06 2024-05-11T23:02:56.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:56.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668331+0000", 2024-05-11T23:02:56.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043949191999999998 2024-05-11T23:02:56.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:56.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668370+0000", 2024-05-11T23:02:56.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9227999999999998e-05 2024-05-11T23:02:56.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:56.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668430+0000", 2024-05-11T23:02:56.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9685999999999997e-05 2024-05-11T23:02:56.840 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.840 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.840 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.841 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67850 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:56.841 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:56.841 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.14175006, 2024-05-11T23:02:56.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044151953000000001, 2024-05-11T23:02:56.841 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:56.841 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:56.841 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:56.841 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:56.841 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:56.841 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:56.841 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67850 2024-05-11T23:02:56.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.841 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:56.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:56.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:56.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:56.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:02:56.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:56.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624390+0000", 2024-05-11T23:02:56.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:02:56.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:56.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624392+0000", 2024-05-11T23:02:56.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1880000000000001e-06 2024-05-11T23:02:56.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:56.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624399+0000", 2024-05-11T23:02:56.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7390000000000002e-06 2024-05-11T23:02:56.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:56.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624405+0000", 2024-05-11T23:02:56.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1059999999999996e-06 2024-05-11T23:02:56.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:56.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668450+0000", 2024-05-11T23:02:56.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044045147999999999 2024-05-11T23:02:56.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:56.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668487+0000", 2024-05-11T23:02:56.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7450000000000002e-05 2024-05-11T23:02:56.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:56.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668541+0000", 2024-05-11T23:02:56.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3952000000000001e-05 2024-05-11T23:02:56.844 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.844 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.844 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.845 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67851 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5541888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:56.845 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:56.845 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.141723478000003, 2024-05-11T23:02:56.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044264413000000002, 2024-05-11T23:02:56.845 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:56.845 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:56.845 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:56.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:56.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:56.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:56.845 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67851 2024-05-11T23:02:56.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.845 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:56.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:56.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:56.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:56.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:56.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:56.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:02:56.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-05-11T23:02:56.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:56.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624419+0000", 2024-05-11T23:02:56.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.21e-06 2024-05-11T23:02:56.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:56.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624424+0000", 2024-05-11T23:02:56.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9050000000000002e-06 2024-05-11T23:02:56.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:56.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624430+0000", 2024-05-11T23:02:56.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-05-11T23:02:56.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:56.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668566+0000", 2024-05-11T23:02:56.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044136208000000003 2024-05-11T23:02:56.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:56.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668614+0000", 2024-05-11T23:02:56.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8022000000000002e-05 2024-05-11T23:02:56.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:56.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668680+0000", 2024-05-11T23:02:56.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6181999999999998e-05 2024-05-11T23:02:56.848 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.848 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.849 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.849 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67852 3.16 3.1a4a4af6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:02:56.849 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:56.849 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.141703010999997, 2024-05-11T23:02:56.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044278287999999999, 2024-05-11T23:02:56.849 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:56.849 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:56.849 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:02:56.849 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:56.849 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:56.849 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:56.849 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67852 2024-05-11T23:02:56.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.850 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:56.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:56.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:56.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:56.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:02:56.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:56.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:56.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626226+0000", 2024-05-11T23:02:56.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017893 2024-05-11T23:02:56.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:56.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626229+0000", 2024-05-11T23:02:56.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0709999999999999e-06 2024-05-11T23:02:56.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:56.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626239+0000", 2024-05-11T23:02:56.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0507e-05 2024-05-11T23:02:56.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:56.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626244+0000", 2024-05-11T23:02:56.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4880000000000001e-06 2024-05-11T23:02:56.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:56.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668706+0000", 2024-05-11T23:02:56.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042462054999999999 2024-05-11T23:02:56.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:56.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:56.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:56.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668715+0000", 2024-05-11T23:02:56.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.867e-06 2024-05-11T23:02:56.852 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.852 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.852 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.852 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:56.852 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:56.852 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:56.854 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-11T23:02:56.855 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:02:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:no valid command found; 10 closest matches: 2024-05-11T23:02:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:0 2024-05-11T23:02:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:1 2024-05-11T23:02:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2 2024-05-11T23:02:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:abort 2024-05-11T23:02:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:assert 2024-05-11T23:02:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:bench [] [] [] [] 2024-05-11T23:02:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs debug_inject_read_zeros 2024-05-11T23:02:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs files list 2024-05-11T23:02:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs stats 2024-05-11T23:02:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:bluestore allocator dump block 2024-05-11T23:02:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:Error EINVAL: invalid command 2024-05-11T23:02:56.856 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-11T23:02:56.856 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-05-11T23:02:56.856 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:56.854+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:56.955 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:56.955+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:57.002 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:57.002 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:57.002 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:57.002 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:57.011 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:02:57.034 INFO:teuthology.orchestra.run.smithi005.stderr:no valid command found; 10 closest matches: 2024-05-11T23:02:57.034 INFO:teuthology.orchestra.run.smithi005.stderr:0 2024-05-11T23:02:57.034 INFO:teuthology.orchestra.run.smithi005.stderr:1 2024-05-11T23:02:57.034 INFO:teuthology.orchestra.run.smithi005.stderr:2 2024-05-11T23:02:57.034 INFO:teuthology.orchestra.run.smithi005.stderr:abort 2024-05-11T23:02:57.034 INFO:teuthology.orchestra.run.smithi005.stderr:assert 2024-05-11T23:02:57.034 INFO:teuthology.orchestra.run.smithi005.stderr:bench [] [] [] [] 2024-05-11T23:02:57.034 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs debug_inject_read_zeros 2024-05-11T23:02:57.035 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs files list 2024-05-11T23:02:57.035 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs stats 2024-05-11T23:02:57.035 INFO:teuthology.orchestra.run.smithi005.stderr:bluestore allocator dump block 2024-05-11T23:02:57.035 INFO:teuthology.orchestra.run.smithi005.stderr:Error EINVAL: invalid command 2024-05-11T23:02:57.037 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-11T23:02:57.037 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-11T23:02:57.037 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-05-11T23:02:57.081 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:57.081+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:57.126 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-05-11T23:02:57.160 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:57.160 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:57.160 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:57.160 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:57.160 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:57.170 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:02:57.184 INFO:teuthology.orchestra.run.smithi005.stderr:no valid command found; 10 closest matches: 2024-05-11T23:02:57.184 INFO:teuthology.orchestra.run.smithi005.stderr:0 2024-05-11T23:02:57.184 INFO:teuthology.orchestra.run.smithi005.stderr:1 2024-05-11T23:02:57.184 INFO:teuthology.orchestra.run.smithi005.stderr:2 2024-05-11T23:02:57.184 INFO:teuthology.orchestra.run.smithi005.stderr:abort 2024-05-11T23:02:57.184 INFO:teuthology.orchestra.run.smithi005.stderr:assert 2024-05-11T23:02:57.184 INFO:teuthology.orchestra.run.smithi005.stderr:bench [] [] [] [] 2024-05-11T23:02:57.184 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs debug_inject_read_zeros 2024-05-11T23:02:57.184 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs files list 2024-05-11T23:02:57.185 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs stats 2024-05-11T23:02:57.185 INFO:teuthology.orchestra.run.smithi005.stderr:bluestore allocator dump block 2024-05-11T23:02:57.185 INFO:teuthology.orchestra.run.smithi005.stderr:Error EINVAL: invalid command 2024-05-11T23:02:57.185 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:57.184+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:57.187 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-11T23:02:57.187 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-11T23:02:57.187 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-05-11T23:02:57.205 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone43 from 0x5cdc533 (overlap 0xf746e3) to 0x7ba371 2024-05-11T23:02:57.257 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:57.257+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:57.284 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:57.284 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.629995616000002, 2024-05-11T23:02:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:02:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:02:57.285 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.286 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:02:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:02:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:02:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:02:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:02:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:02:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.287 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:02:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:02:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:02:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:02:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:02:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:02:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.288 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:02:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:02:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.626614427000007, 2024-05-11T23:02:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:02:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.289 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:02:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.290 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:02:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:02:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:02:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:02:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:02:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:02:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.291 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:02:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:02:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:02:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:02:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:02:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:02:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.292 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:02:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:02:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.626595719999997, 2024-05-11T23:02:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:02:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.293 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:02:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:02:57.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:02:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:02:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:02:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:02:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:02:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:02:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:02:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:02:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:02:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:02:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:02:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:02:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:02:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:02:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:57.297 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.626577577000006, 2024-05-11T23:02:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:02:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:02:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:57.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:02:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:02:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:02:57.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:02:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:02:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:02:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:02:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:02:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:02:57.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:02:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:02:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:02:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:02:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:02:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:57.301 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.626550089999995, 2024-05-11T23:02:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:02:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:02:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:57.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:02:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:02:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:02:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:02:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:02:57.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:02:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:02:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:02:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:02:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:02:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:02:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:02:57.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:02:57.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:02:57.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:02:57.305 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.305 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.305 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.305 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:57.305 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:57.305 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.626529562000002, 2024-05-11T23:02:57.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:02:57.306 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.306 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.306 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.306 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:02:57.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.306 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:57.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:57.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:02:57.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:02:57.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:02:57.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:02:57.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:02:57.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:02:57.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:02:57.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:02:57.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:02:57.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:02:57.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:02:57.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:02:57.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:02:57.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:02:57.309 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.309 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.309 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.309 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:57.309 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:57.309 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.626509787000003, 2024-05-11T23:02:57.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:02:57.310 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.310 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.310 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.310 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.310 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.310 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.310 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:02:57.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.310 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.310 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.310 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:57.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:02:57.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:02:57.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:02:57.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:02:57.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:02:57.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:02:57.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:02:57.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:02:57.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:02:57.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:02:57.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:02:57.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:02:57.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:02:57.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:02:57.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:02:57.313 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.313 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.313 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.314 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:57.314 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:57.314 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.626490981000003, 2024-05-11T23:02:57.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:02:57.314 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.314 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.314 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.314 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:02:57.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.314 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:57.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:02:57.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:02:57.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:02:57.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:02:57.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:02:57.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:02:57.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:02:57.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:02:57.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:02:57.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:02:57.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:02:57.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:02:57.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:02:57.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:02:57.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:02:57.317 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.317 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.318 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.318 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:57.318 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:57.318 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.626472058000004, 2024-05-11T23:02:57.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:02:57.318 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.318 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.318 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.318 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:02:57.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.319 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:57.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:57.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:02:57.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:02:57.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:02:57.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:02:57.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:02:57.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:02:57.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:02:57.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:02:57.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:02:57.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:02:57.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:02:57.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:02:57.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:02:57.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:02:57.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.322 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.322 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:57.322 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:57.322 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.626453396000002, 2024-05-11T23:02:57.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:02:57.322 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.322 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.322 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.322 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.322 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.323 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.323 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:02:57.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.323 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:57.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:57.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:02:57.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:02:57.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:02:57.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:02:57.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:02:57.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:02:57.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:02:57.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:02:57.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:02:57.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:02:57.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:02:57.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:02:57.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:02:57.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:02:57.326 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.326 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.326 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.326 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:57.326 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:57.326 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.626108234, 2024-05-11T23:02:57.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:02:57.326 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.326 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.327 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.327 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.327 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.327 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.327 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:02:57.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.327 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:57.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:02:57.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:02:57.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:02:57.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:02:57.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:02:57.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:02:57.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:02:57.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:02:57.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:02:57.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:02:57.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:02:57.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:02:57.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:02:57.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:02:57.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:02:57.330 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.330 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.330 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.330 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:57.330 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:57.330 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.626082683999996, 2024-05-11T23:02:57.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:02:57.331 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.331 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.331 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.331 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.331 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.331 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.331 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:02:57.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.331 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:57.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:57.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:02:57.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:02:57.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:02:57.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:02:57.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:02:57.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:02:57.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:02:57.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:02:57.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:02:57.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:02:57.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:02:57.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:02:57.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:02:57.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:02:57.334 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.334 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.334 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.334 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:57.334 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:57.335 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.626000426000004, 2024-05-11T23:02:57.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:02:57.335 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.335 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.335 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.335 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:02:57.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.335 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:57.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:57.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:02:57.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:02:57.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:02:57.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:02:57.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:02:57.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:02:57.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:02:57.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:02:57.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:02:57.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:02:57.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:02:57.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:02:57.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:02:57.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:02:57.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.338 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.339 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:57.339 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:57.339 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.625977978999998, 2024-05-11T23:02:57.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:02:57.339 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.339 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.339 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.339 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:02:57.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.339 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:57.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:02:57.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:02:57.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:02:57.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:02:57.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:02:57.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:02:57.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:02:57.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:02:57.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:02:57.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:02:57.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:02:57.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:02:57.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:02:57.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:02:57.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:02:57.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.342 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.343 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:57.343 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:57.343 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.625958521000001, 2024-05-11T23:02:57.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:02:57.343 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.343 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.343 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.343 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:02:57.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.343 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:57.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:57.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:02:57.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:02:57.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:02:57.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:02:57.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:02:57.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:02:57.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:02:57.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:02:57.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:02:57.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:02:57.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:02:57.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:02:57.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:02:57.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:02:57.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.346 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.347 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.347 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:57.347 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:57.347 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.625938461000004, 2024-05-11T23:02:57.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:02:57.347 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.347 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.347 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.347 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:02:57.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.348 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:57.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:57.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:02:57.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:02:57.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:02:57.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:02:57.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:02:57.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:02:57.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:02:57.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:02:57.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:02:57.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:02:57.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:02:57.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:02:57.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:02:57.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:02:57.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.351 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.351 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:57.351 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:57.351 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.625919190999994, 2024-05-11T23:02:57.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:02:57.351 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.351 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.351 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.352 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:02:57.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.352 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:57.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:57.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:02:57.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:02:57.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:02:57.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:02:57.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:02:57.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:02:57.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:02:57.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:02:57.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:02:57.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:02:57.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:02:57.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:02:57.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:02:57.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:02:57.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.355 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.355 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:57.355 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:57.355 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.625373124999996, 2024-05-11T23:02:57.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:02:57.355 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.355 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.355 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.355 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.355 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.356 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:02:57.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.356 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:57.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:02:57.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:02:57.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:02:57.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:02:57.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:02:57.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:02:57.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:02:57.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:02:57.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:02:57.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:02:57.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:02:57.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:02:57.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:02:57.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:02:57.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:02:57.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.359 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.359 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:02:57.359 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:57.359 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 64.625341340999995, 2024-05-11T23:02:57.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:02:57.359 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.359 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.359 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.359 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.360 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:02:57.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.360 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:57.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:57.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:02:57.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:02:57.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:02:57.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:02:57.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:02:57.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:02:57.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:02:57.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:02:57.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:02:57.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:02:57.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:02:57.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:02:57.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:02:57.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:02:57.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.363 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.363 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:02:57.363 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:57.363 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.188095877000002, 2024-05-11T23:02:57.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:02:57.364 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.364 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.364 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:57.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.364 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:02:57.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.364 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:57.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:02:57.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:02:57.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:02:57.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:02:57.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:02:57.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:02:57.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:02:57.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:02:57.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:02:57.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:02:57.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:02:57.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:02:57.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:02:57.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:02:57.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:02:57.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:02:57.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:57.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:02:57.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:02:57.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:57.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:02:57.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:02:57.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:57.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:02:57.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:02:57.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:02:57.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:02:57.368 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.369 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.369 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.369 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:57.370 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-05-11T23:02:57.370 INFO:teuthology.orchestra.run.smithi005.stdout:4743 clone 45 order 18 su 16384 sc 6 2024-05-11T23:02:57.372 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:02:57.372 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:57.371+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:57.372 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-11T23:02:57.372 INFO:teuthology.orchestra.run.smithi005.stderr:no valid command found; 10 closest matches: 2024-05-11T23:02:57.372 INFO:teuthology.orchestra.run.smithi005.stderr:0 2024-05-11T23:02:57.373 INFO:teuthology.orchestra.run.smithi005.stderr:1 2024-05-11T23:02:57.373 INFO:teuthology.orchestra.run.smithi005.stderr:2 2024-05-11T23:02:57.373 INFO:teuthology.orchestra.run.smithi005.stderr:abort 2024-05-11T23:02:57.373 INFO:teuthology.orchestra.run.smithi005.stderr:assert 2024-05-11T23:02:57.373 INFO:teuthology.orchestra.run.smithi005.stderr:bench [] [] [] [] 2024-05-11T23:02:57.373 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs debug_inject_read_zeros 2024-05-11T23:02:57.373 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs files list 2024-05-11T23:02:57.373 INFO:teuthology.orchestra.run.smithi005.stderr:bluefs stats 2024-05-11T23:02:57.373 INFO:teuthology.orchestra.run.smithi005.stderr:bluestore allocator dump block 2024-05-11T23:02:57.373 INFO:teuthology.orchestra.run.smithi005.stderr:Error EINVAL: invalid command 2024-05-11T23:02:57.373 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-11T23:02:57.471 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:57.472+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:57.518 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:57.519 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:57.519 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:02:57.519 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:57.527 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:02:57.573 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:57.573+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:57.632 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:57.632 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:02:57.632 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:02:57.632 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:02:57.633 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:57.641 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:02:57.683 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:57.683+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:57.774 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:57.775+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:57.787 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:02:57.788 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:02:57.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:02:57.788 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:02:57.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.788 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:02:57.788 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:57.788 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 100.623681609, 2024-05-11T23:02:57.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:02:57.788 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.788 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.788 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:57.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.788 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:02:57.789 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:02:57.789 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:02:57.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.789 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:57.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:02:57.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:02:57.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:02:57.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:02:57.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:02:57.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:02:57.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:02:57.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:02:57.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:02:57.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:02:57.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:02:57.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:02:57.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:02:57.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:02:57.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:02:57.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:02:57.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:02:57.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:02:57.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:02:57.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:02:57.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:02:57.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:57.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:02:57.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:02:57.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:57.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:02:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:02:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:02:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:02:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:02:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:02:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78429 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:02:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.822412266999997, 2024-05-11T23:02:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044563497000000001, 2024-05-11T23:02:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78429 2024-05-11T23:02:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.795 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:02:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:02:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964872+0000", 2024-05-11T23:02:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035070300000000001 2024-05-11T23:02:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964876+0000", 2024-05-11T23:02:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5009999999999998e-06 2024-05-11T23:02:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964902+0000", 2024-05-11T23:02:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5715999999999999e-05 2024-05-11T23:02:57.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964910+0000", 2024-05-11T23:02:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3930000000000002e-06 2024-05-11T23:02:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008898+0000", 2024-05-11T23:02:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043988772000000002 2024-05-11T23:02:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008981+0000", 2024-05-11T23:02:57.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3140999999999996e-05 2024-05-11T23:02:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009085+0000", 2024-05-11T23:02:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010327099999999999 2024-05-11T23:02:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78431 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:02:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.821951959000003, 2024-05-11T23:02:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044542033000000002, 2024-05-11T23:02:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.799 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78431 2024-05-11T23:02:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:02:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:02:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965038+0000", 2024-05-11T23:02:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6993000000000001e-05 2024-05-11T23:02:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965043+0000", 2024-05-11T23:02:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7040000000000002e-06 2024-05-11T23:02:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965053+0000", 2024-05-11T23:02:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8539999999999992e-06 2024-05-11T23:02:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965061+0000", 2024-05-11T23:02:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9810000000000003e-06 2024-05-11T23:02:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009350+0000", 2024-05-11T23:02:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044288739000000001 2024-05-11T23:02:57.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009429+0000", 2024-05-11T23:02:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9354e-05 2024-05-11T23:02:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009524+0000", 2024-05-11T23:02:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4407999999999995e-05 2024-05-11T23:02:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.803 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78432 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:02:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.821861110999997, 2024-05-11T23:02:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044635421000000002, 2024-05-11T23:02:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78432 2024-05-11T23:02:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.804 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:02:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:02:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965343+0000", 2024-05-11T23:02:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027056500000000001 2024-05-11T23:02:57.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965347+0000", 2024-05-11T23:02:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3540000000000002e-06 2024-05-11T23:02:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965356+0000", 2024-05-11T23:02:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2140000000000002e-06 2024-05-11T23:02:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965363+0000", 2024-05-11T23:02:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4150000000000004e-06 2024-05-11T23:02:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009551+0000", 2024-05-11T23:02:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044187917 2024-05-11T23:02:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009626+0000", 2024-05-11T23:02:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5024000000000005e-05 2024-05-11T23:02:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009708+0000", 2024-05-11T23:02:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1932000000000001e-05 2024-05-11T23:02:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78433 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:02:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.821560224000002, 2024-05-11T23:02:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044564864000000003, 2024-05-11T23:02:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.808 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78433 2024-05-11T23:02:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:02:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:02:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966225+0000", 2024-05-11T23:02:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085220700000000001 2024-05-11T23:02:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966230+0000", 2024-05-11T23:02:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0690000000000003e-06 2024-05-11T23:02:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966239+0000", 2024-05-11T23:02:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-05-11T23:02:57.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966245+0000", 2024-05-11T23:02:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5520000000000001e-06 2024-05-11T23:02:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009743+0000", 2024-05-11T23:02:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043498311999999997 2024-05-11T23:02:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009848+0000", 2024-05-11T23:02:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104885 2024-05-11T23:02:57.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009938+0000", 2024-05-11T23:02:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9812e-05 2024-05-11T23:02:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78580 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 999424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:57.812 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:02:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.818447540999998, 2024-05-11T23:02:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044493076999999999, 2024-05-11T23:02:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78580 2024-05-11T23:02:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.813 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:02:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:02:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:02:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:02:57.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968489+0000", 2024-05-11T23:02:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.52e-06 2024-05-11T23:02:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968496+0000", 2024-05-11T23:02:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9419999999999998e-06 2024-05-11T23:02:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968501+0000", 2024-05-11T23:02:57.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8720000000000001e-06 2024-05-11T23:02:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012690+0000", 2024-05-11T23:02:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044189655000000001 2024-05-11T23:02:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012848+0000", 2024-05-11T23:02:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015758100000000001 2024-05-11T23:02:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012979+0000", 2024-05-11T23:02:57.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000131066 2024-05-11T23:02:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78581 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:02:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.818426537000001, 2024-05-11T23:02:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044616243, 2024-05-11T23:02:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.817 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78581 2024-05-11T23:02:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:02:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:02:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970227+0000", 2024-05-11T23:02:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017200049999999999 2024-05-11T23:02:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970232+0000", 2024-05-11T23:02:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000003e-06 2024-05-11T23:02:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970242+0000", 2024-05-11T23:02:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0192e-05 2024-05-11T23:02:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970249+0000", 2024-05-11T23:02:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.888e-06 2024-05-11T23:02:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013008+0000", 2024-05-11T23:02:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042759495000000002 2024-05-11T23:02:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013063+0000", 2024-05-11T23:02:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5071999999999997e-05 2024-05-11T23:02:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013123+0000", 2024-05-11T23:02:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9930999999999999e-05 2024-05-11T23:02:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78589 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:02:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.816507293000001, 2024-05-11T23:02:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044587697000000003, 2024-05-11T23:02:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.821 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78589 2024-05-11T23:02:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:02:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:02:57.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970427+0000", 2024-05-11T23:02:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:02:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970429+0000", 2024-05-11T23:02:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3800000000000001e-06 2024-05-11T23:02:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970435+0000", 2024-05-11T23:02:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2530000000000001e-06 2024-05-11T23:02:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970441+0000", 2024-05-11T23:02:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-05-11T23:02:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014822+0000", 2024-05-11T23:02:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04438168 2024-05-11T23:02:57.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014911+0000", 2024-05-11T23:02:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8882999999999996e-05 2024-05-11T23:02:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015014+0000", 2024-05-11T23:02:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010263399999999999 2024-05-11T23:02:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.828 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78590 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:02:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.816486494000003, 2024-05-11T23:02:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044880120000000003, 2024-05-11T23:02:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78590 2024-05-11T23:02:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:02:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:02:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970918+0000", 2024-05-11T23:02:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000470834 2024-05-11T23:02:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970925+0000", 2024-05-11T23:02:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1690000000000003e-06 2024-05-11T23:02:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970934+0000", 2024-05-11T23:02:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9099999999999994e-06 2024-05-11T23:02:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970941+0000", 2024-05-11T23:02:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7410000000000003e-06 2024-05-11T23:02:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015049+0000", 2024-05-11T23:02:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044108392000000003 2024-05-11T23:02:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015133+0000", 2024-05-11T23:02:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3568999999999999e-05 2024-05-11T23:02:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015327+0000", 2024-05-11T23:02:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019450500000000001 2024-05-11T23:02:57.832 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78591 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:02:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.815983078000002, 2024-05-11T23:02:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044618497, 2024-05-11T23:02:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.833 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78591 2024-05-11T23:02:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:02:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:02:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970951+0000", 2024-05-11T23:02:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:02:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970953+0000", 2024-05-11T23:02:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:02:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970956+0000", 2024-05-11T23:02:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-05-11T23:02:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970959+0000", 2024-05-11T23:02:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9409999999999999e-06 2024-05-11T23:02:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015367+0000", 2024-05-11T23:02:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044407621000000001 2024-05-11T23:02:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015453+0000", 2024-05-11T23:02:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6685000000000001e-05 2024-05-11T23:02:57.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015569+0000", 2024-05-11T23:02:57.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115396 2024-05-11T23:02:57.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.837 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.837 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78592 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:57.837 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:02:57.837 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.815970270000001, 2024-05-11T23:02:57.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044831546999999999, 2024-05-11T23:02:57.837 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.838 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.838 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.838 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78592 2024-05-11T23:02:57.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.838 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:02:57.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:02:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972226+0000", 2024-05-11T23:02:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00126246 2024-05-11T23:02:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972230+0000", 2024-05-11T23:02:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1860000000000002e-06 2024-05-11T23:02:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972237+0000", 2024-05-11T23:02:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:02:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972241+0000", 2024-05-11T23:02:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.284e-06 2024-05-11T23:02:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015611+0000", 2024-05-11T23:02:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043369461999999998 2024-05-11T23:02:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015695+0000", 2024-05-11T23:02:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4458999999999999e-05 2024-05-11T23:02:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015795+0000", 2024-05-11T23:02:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9475999999999994e-05 2024-05-11T23:02:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78593 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:02:57.841 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.814683350999999, 2024-05-11T23:02:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046045214000000001, 2024-05-11T23:02:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78593 2024-05-11T23:02:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:02:57.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:02:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:02:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7599999999999998e-07 2024-05-11T23:02:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972252+0000", 2024-05-11T23:02:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7349999999999999e-06 2024-05-11T23:02:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972256+0000", 2024-05-11T23:02:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5659999999999998e-06 2024-05-11T23:02:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972259+0000", 2024-05-11T23:02:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8040000000000002e-06 2024-05-11T23:02:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018088+0000", 2024-05-11T23:02:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045829316000000002 2024-05-11T23:02:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018179+0000", 2024-05-11T23:02:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1299999999999997e-05 2024-05-11T23:02:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018295+0000", 2024-05-11T23:02:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116117 2024-05-11T23:02:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.846 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78594 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:57.846 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:02:57.846 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.814670991, 2024-05-11T23:02:57.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046245204999999998, 2024-05-11T23:02:57.846 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.846 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.846 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.846 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78594 2024-05-11T23:02:57.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.846 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:02:57.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:02:57.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972263+0000", 2024-05-11T23:02:57.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:02:57.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972264+0000", 2024-05-11T23:02:57.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.206e-06 2024-05-11T23:02:57.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972267+0000", 2024-05-11T23:02:57.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0690000000000002e-06 2024-05-11T23:02:57.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972270+0000", 2024-05-11T23:02:57.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5610000000000001e-06 2024-05-11T23:02:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018330+0000", 2024-05-11T23:02:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04606085 2024-05-11T23:02:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018414+0000", 2024-05-11T23:02:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3125000000000001e-05 2024-05-11T23:02:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018508+0000", 2024-05-11T23:02:57.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4153000000000005e-05 2024-05-11T23:02:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78595 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:02:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.814658125999998, 2024-05-11T23:02:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046396698, 2024-05-11T23:02:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.850 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78595 2024-05-11T23:02:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:02:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:02:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972276+0000", 2024-05-11T23:02:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5600000000000002e-07 2024-05-11T23:02:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972277+0000", 2024-05-11T23:02:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.576e-06 2024-05-11T23:02:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972280+0000", 2024-05-11T23:02:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-05-11T23:02:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972283+0000", 2024-05-11T23:02:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:02:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018538+0000", 2024-05-11T23:02:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046255255000000002 2024-05-11T23:02:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018605+0000", 2024-05-11T23:02:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6847000000000006e-05 2024-05-11T23:02:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018672+0000", 2024-05-11T23:02:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7045000000000006e-05 2024-05-11T23:02:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78596 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:02:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.814647276999999, 2024-05-11T23:02:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046586806000000001, 2024-05-11T23:02:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.854 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78596 2024-05-11T23:02:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:02:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:02:57.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:02:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:02:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972288+0000", 2024-05-11T23:02:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2550000000000001e-06 2024-05-11T23:02:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972291+0000", 2024-05-11T23:02:57.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9490000000000001e-06 2024-05-11T23:02:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972293+0000", 2024-05-11T23:02:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5069999999999999e-06 2024-05-11T23:02:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018707+0000", 2024-05-11T23:02:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046413604999999997 2024-05-11T23:02:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018782+0000", 2024-05-11T23:02:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5556000000000006e-05 2024-05-11T23:02:57.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018873+0000", 2024-05-11T23:02:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0703000000000002e-05 2024-05-11T23:02:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78597 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:02:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.814637503999997, 2024-05-11T23:02:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046825787000000001, 2024-05-11T23:02:57.858 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78597 2024-05-11T23:02:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:02:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:02:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:02:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-05-11T23:02:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972297+0000", 2024-05-11T23:02:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3069999999999999e-06 2024-05-11T23:02:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972300+0000", 2024-05-11T23:02:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:02:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972303+0000", 2024-05-11T23:02:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5509999999999998e-06 2024-05-11T23:02:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018911+0000", 2024-05-11T23:02:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046608194999999998 2024-05-11T23:02:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018986+0000", 2024-05-11T23:02:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5360999999999999e-05 2024-05-11T23:02:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019122+0000", 2024-05-11T23:02:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013526 2024-05-11T23:02:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.862 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78598 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:02:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:02:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 41.814627604000002, 2024-05-11T23:02:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046998572000000002, 2024-05-11T23:02:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:02:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78598 2024-05-11T23:02:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:02:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:02:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:02:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:02:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972307+0000", 2024-05-11T23:02:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1799999999999999e-06 2024-05-11T23:02:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972310+0000", 2024-05-11T23:02:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:02:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972313+0000", 2024-05-11T23:02:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4760000000000001e-06 2024-05-11T23:02:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019149+0000", 2024-05-11T23:02:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046836056000000001 2024-05-11T23:02:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019203+0000", 2024-05-11T23:02:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-05 2024-05-11T23:02:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019304+0000", 2024-05-11T23:02:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101537 2024-05-11T23:02:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.866 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:105234 3.15 3:adf885e5:::rbd_object_map.111e913a615f.0000000000000025:head [writefull 0~39] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-11T23:02:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:02:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 27.779311893999999, 2024-05-11T23:02:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77797599100000003, 2024-05-11T23:02:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 105234 2024-05-11T23:02:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.867 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:02:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:02:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007623+0000", 2024-05-11T23:02:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2240000000000001e-06 2024-05-11T23:02:57.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007627+0000", 2024-05-11T23:02:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:02:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007636+0000", 2024-05-11T23:02:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2409999999999995e-06 2024-05-11T23:02:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007641+0000", 2024-05-11T23:02:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6810000000000003e-06 2024-05-11T23:02:57.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007676+0000", 2024-05-11T23:02:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5061999999999997e-05 2024-05-11T23:02:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:02:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007682+0000", 2024-05-11T23:02:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0399999999999998e-06 2024-05-11T23:02:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783880+0000", 2024-05-11T23:02:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77619752900000005 2024-05-11T23:02:57.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783995+0000", 2024-05-11T23:02:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011550800000000001 2024-05-11T23:02:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:02:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784082+0000", 2024-05-11T23:02:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6347999999999994e-05 2024-05-11T23:02:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784682+0000", 2024-05-11T23:02:57.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000600162 2024-05-11T23:02:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785539+0000", 2024-05-11T23:02:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085699099999999998 2024-05-11T23:02:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785572+0000", 2024-05-11T23:02:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3182999999999997e-05 2024-05-11T23:02:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785598+0000", 2024-05-11T23:02:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5522000000000001e-05 2024-05-11T23:02:57.872 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113761 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [call rbd.sparse_copyup in=999496b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:02:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:02:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.256624975000001, 2024-05-11T23:02:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048572964000000003, 2024-05-11T23:02:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.873 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113761 2024-05-11T23:02:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:02:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:02:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530309+0000", 2024-05-11T23:02:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:02:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531242+0000", 2024-05-11T23:02:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093316899999999997 2024-05-11T23:02:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531249+0000", 2024-05-11T23:02:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8140000000000003e-06 2024-05-11T23:02:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531253+0000", 2024-05-11T23:02:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4290000000000003e-06 2024-05-11T23:02:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570325+0000", 2024-05-11T23:02:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039071379000000003 2024-05-11T23:02:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570382+0000", 2024-05-11T23:02:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6988999999999999e-05 2024-05-11T23:02:57.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:02:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570555+0000", 2024-05-11T23:02:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017379799999999999 2024-05-11T23:02:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.575953+0000", 2024-05-11T23:02:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053975539999999997 2024-05-11T23:02:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578778+0000", 2024-05-11T23:02:57.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002825616 2024-05-11T23:02:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578837+0000", 2024-05-11T23:02:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9005000000000002e-05 2024-05-11T23:02:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578881+0000", 2024-05-11T23:02:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3970000000000001e-05 2024-05-11T23:02:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113762 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [delete] snapc 27=[27] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:02:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:02:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.255671024000002, 2024-05-11T23:02:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047916260000000002, 2024-05-11T23:02:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:02:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:02:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:02:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:02:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:02:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113762 2024-05-11T23:02:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:02:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:02:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:02:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:02:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:02:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:02:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:02:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531263+0000", 2024-05-11T23:02:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.85e-07 2024-05-11T23:02:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:02:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531265+0000", 2024-05-11T23:02:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.827e-06 2024-05-11T23:02:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:02:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531268+0000", 2024-05-11T23:02:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3780000000000001e-06 2024-05-11T23:02:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:02:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531271+0000", 2024-05-11T23:02:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8509999999999999e-06 2024-05-11T23:02:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:02:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572298+0000", 2024-05-11T23:02:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041026990999999999 2024-05-11T23:02:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:02:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572345+0000", 2024-05-11T23:02:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6878000000000001e-05 2024-05-11T23:02:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:02:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572443+0000", 2024-05-11T23:02:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8184999999999996e-05 2024-05-11T23:02:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:02:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578585+0000", 2024-05-11T23:02:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006141919 2024-05-11T23:02:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:02:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579126+0000", 2024-05-11T23:02:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00054063799999999997 2024-05-11T23:02:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:02:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579154+0000", 2024-05-11T23:02:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8042999999999999e-05 2024-05-11T23:02:57.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:02:57.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:02:57.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:02:57.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579179+0000", 2024-05-11T23:02:57.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4765000000000001e-05 2024-05-11T23:02:57.884 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.884 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.884 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.884 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:02:57.884 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:02:57.884 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:02:57.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:02:57.886 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:57.886+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:57.986 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:57.987+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:58.004 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:58.004 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:58.004 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:02:58.004 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:58.013 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:02:58.088 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:02:58.088+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:02:58.124 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:02:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:02:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:02:58.125 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:58.136 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:02:58.217 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone44 from 0x5cdc533 (overlap 0x5cdc533) to 0x0 2024-05-11T23:02:58.217 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:58.217+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:58.289 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:58.290+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:58.347 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:58.347 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:02:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:02:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:162217 3.1f 3:f9cad7cc:::rbd_data.111ec437b352.0000000000000006:2a [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-05-11T23:02:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:49.201770+0000", 2024-05-11T23:02:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.1439814560000006, 2024-05-11T23:02:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039800218999999998, 2024-05-11T23:02:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 162217 2024-05-11T23:02:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.201770+0000", 2024-05-11T23:02:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.201770+0000", 2024-05-11T23:02:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.201771+0000", 2024-05-11T23:02:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0999999999999999e-07 2024-05-11T23:02:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.201773+0000", 2024-05-11T23:02:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6460000000000002e-06 2024-05-11T23:02:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.201780+0000", 2024-05-11T23:02:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7229999999999998e-06 2024-05-11T23:02:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.201785+0000", 2024-05-11T23:02:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9599999999999999e-06 2024-05-11T23:02:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.238654+0000", 2024-05-11T23:02:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.036868434999999998 2024-05-11T23:02:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.238722+0000", 2024-05-11T23:02:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8435000000000005e-05 2024-05-11T23:02:58.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.241571+0000", 2024-05-11T23:02:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0028486100000000001 2024-05-11T23:02:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:162218 3.1f 3:f9cad7cc:::rbd_data.111ec437b352.0000000000000006:2a [read 223232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-05-11T23:02:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:49.201792+0000", 2024-05-11T23:02:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.1439593820000002, 2024-05-11T23:02:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039951702999999998, 2024-05-11T23:02:58.352 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 162218 2024-05-11T23:02:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.201792+0000", 2024-05-11T23:02:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.201792+0000", 2024-05-11T23:02:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.201793+0000", 2024-05-11T23:02:58.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7300000000000002e-07 2024-05-11T23:02:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.201795+0000", 2024-05-11T23:02:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.357e-06 2024-05-11T23:02:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.201801+0000", 2024-05-11T23:02:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3640000000000004e-06 2024-05-11T23:02:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.201807+0000", 2024-05-11T23:02:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0019999999999997e-06 2024-05-11T23:02:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.241603+0000", 2024-05-11T23:02:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.03979622 2024-05-11T23:02:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.241677+0000", 2024-05-11T23:02:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4614000000000001e-05 2024-05-11T23:02:58.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.241744+0000", 2024-05-11T23:02:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6773000000000006e-05 2024-05-11T23:02:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:162219 3.1f 3:f9cad7cc:::rbd_data.111ec437b352.0000000000000006:2a [read 228352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-05-11T23:02:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:49.202571+0000", 2024-05-11T23:02:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.1431804969999995, 2024-05-11T23:02:58.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039352524, 2024-05-11T23:02:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 162219 2024-05-11T23:02:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202571+0000", 2024-05-11T23:02:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202571+0000", 2024-05-11T23:02:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202572+0000", 2024-05-11T23:02:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3300000000000002e-07 2024-05-11T23:02:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202575+0000", 2024-05-11T23:02:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4709999999999999e-06 2024-05-11T23:02:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202584+0000", 2024-05-11T23:02:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9959999999999999e-06 2024-05-11T23:02:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202593+0000", 2024-05-11T23:02:58.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6999999999999997e-06 2024-05-11T23:02:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.241775+0000", 2024-05-11T23:02:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039181623999999998 2024-05-11T23:02:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.241847+0000", 2024-05-11T23:02:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2593000000000001e-05 2024-05-11T23:02:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.241924+0000", 2024-05-11T23:02:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6507000000000005e-05 2024-05-11T23:02:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:162220 3.1f 3:f9cad7cc:::rbd_data.111ec437b352.0000000000000006:2a [read 233472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-05-11T23:02:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:49.202604+0000", 2024-05-11T23:02:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.1431482230000007, 2024-05-11T23:02:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039487264000000001, 2024-05-11T23:02:58.362 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 162220 2024-05-11T23:02:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202604+0000", 2024-05-11T23:02:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202604+0000", 2024-05-11T23:02:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202604+0000", 2024-05-11T23:02:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9999999999999998e-07 2024-05-11T23:02:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202606+0000", 2024-05-11T23:02:58.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2919999999999998e-06 2024-05-11T23:02:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202614+0000", 2024-05-11T23:02:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4630000000000004e-06 2024-05-11T23:02:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202620+0000", 2024-05-11T23:02:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8510000000000004e-06 2024-05-11T23:02:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.241956+0000", 2024-05-11T23:02:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039336492000000001 2024-05-11T23:02:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.242027+0000", 2024-05-11T23:02:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1113999999999997e-05 2024-05-11T23:02:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.242091+0000", 2024-05-11T23:02:58.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3552000000000004e-05 2024-05-11T23:02:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:162221 3.1f 3:f9cad7cc:::rbd_data.111ec437b352.0000000000000006:2a [read 238592~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-05-11T23:02:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:49.202628+0000", 2024-05-11T23:02:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.1431240339999995, 2024-05-11T23:02:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039633321999999999, 2024-05-11T23:02:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 162221 2024-05-11T23:02:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202628+0000", 2024-05-11T23:02:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202628+0000", 2024-05-11T23:02:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202628+0000", 2024-05-11T23:02:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4799999999999999e-07 2024-05-11T23:02:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202632+0000", 2024-05-11T23:02:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4120000000000001e-06 2024-05-11T23:02:58.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202639+0000", 2024-05-11T23:02:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.187e-06 2024-05-11T23:02:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202644+0000", 2024-05-11T23:02:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3020000000000002e-06 2024-05-11T23:02:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.242120+0000", 2024-05-11T23:02:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039475507999999999 2024-05-11T23:02:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.242191+0000", 2024-05-11T23:02:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0936e-05 2024-05-11T23:02:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.242261+0000", 2024-05-11T23:02:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0529000000000002e-05 2024-05-11T23:02:58.371 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:162222 3.1f 3:f9cad7cc:::rbd_data.111ec437b352.0000000000000006:2a [read 243712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-05-11T23:02:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:49.202652+0000", 2024-05-11T23:02:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.143099823, 2024-05-11T23:02:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039784936999999999, 2024-05-11T23:02:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.372 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 162222 2024-05-11T23:02:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202652+0000", 2024-05-11T23:02:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202652+0000", 2024-05-11T23:02:58.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202652+0000", 2024-05-11T23:02:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:02:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202659+0000", 2024-05-11T23:02:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1069999999999999e-06 2024-05-11T23:02:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202667+0000", 2024-05-11T23:02:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6920000000000002e-06 2024-05-11T23:02:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202672+0000", 2024-05-11T23:02:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.327e-06 2024-05-11T23:02:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.242290+0000", 2024-05-11T23:02:58.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039617336000000003 2024-05-11T23:02:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.242365+0000", 2024-05-11T23:02:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5240999999999993e-05 2024-05-11T23:02:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.242437+0000", 2024-05-11T23:02:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1834000000000004e-05 2024-05-11T23:02:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:162223 3.1f 3:f9cad7cc:::rbd_data.111ec437b352.0000000000000006:2a [read 248832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-05-11T23:02:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:49.202680+0000", 2024-05-11T23:02:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.1430714250000005, 2024-05-11T23:02:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039931376999999997, 2024-05-11T23:02:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.377 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 162223 2024-05-11T23:02:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202680+0000", 2024-05-11T23:02:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202680+0000", 2024-05-11T23:02:58.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202681+0000", 2024-05-11T23:02:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3799999999999998e-07 2024-05-11T23:02:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202683+0000", 2024-05-11T23:02:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2740000000000002e-06 2024-05-11T23:02:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202690+0000", 2024-05-11T23:02:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1269999999999997e-06 2024-05-11T23:02:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202695+0000", 2024-05-11T23:02:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.079e-06 2024-05-11T23:02:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.242467+0000", 2024-05-11T23:02:58.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039771689999999998 2024-05-11T23:02:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.242542+0000", 2024-05-11T23:02:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5123000000000006e-05 2024-05-11T23:02:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.242612+0000", 2024-05-11T23:02:58.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9646000000000004e-05 2024-05-11T23:02:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:162224 3.1f 3:f9cad7cc:::rbd_data.111ec437b352.0000000000000006:2a [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-05-11T23:02:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:49.202703+0000", 2024-05-11T23:02:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.1430489379999997, 2024-05-11T23:02:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040088528999999998, 2024-05-11T23:02:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.382 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 162224 2024-05-11T23:02:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202703+0000", 2024-05-11T23:02:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202703+0000", 2024-05-11T23:02:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202703+0000", 2024-05-11T23:02:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2800000000000002e-07 2024-05-11T23:02:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202705+0000", 2024-05-11T23:02:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.221e-06 2024-05-11T23:02:58.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202713+0000", 2024-05-11T23:02:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1690000000000003e-06 2024-05-11T23:02:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202718+0000", 2024-05-11T23:02:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0139999999999998e-06 2024-05-11T23:02:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.242641+0000", 2024-05-11T23:02:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039923133999999999 2024-05-11T23:02:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.242718+0000", 2024-05-11T23:02:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7503000000000007e-05 2024-05-11T23:02:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.242791+0000", 2024-05-11T23:02:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3059999999999995e-05 2024-05-11T23:02:58.386 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:162225 3.1f 3:f9cad7cc:::rbd_data.111ec437b352.0000000000000006:2a [read 259072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-05-11T23:02:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:49.202725+0000", 2024-05-11T23:02:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.1430263489999994, 2024-05-11T23:02:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040245466000000001, 2024-05-11T23:02:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.387 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 162225 2024-05-11T23:02:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202725+0000", 2024-05-11T23:02:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202725+0000", 2024-05-11T23:02:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202726+0000", 2024-05-11T23:02:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2399999999999998e-07 2024-05-11T23:02:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202728+0000", 2024-05-11T23:02:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.125e-06 2024-05-11T23:02:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202735+0000", 2024-05-11T23:02:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9290000000000001e-06 2024-05-11T23:02:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202740+0000", 2024-05-11T23:02:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0100000000000003e-06 2024-05-11T23:02:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.242820+0000", 2024-05-11T23:02:58.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040080286999999999 2024-05-11T23:02:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.242895+0000", 2024-05-11T23:02:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5180999999999997e-05 2024-05-11T23:02:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.242971+0000", 2024-05-11T23:02:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5409999999999998e-05 2024-05-11T23:02:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.391 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:162238 3.1f 3:fac6da97:::rbd_data.111ec437b352.000000000000000c:2a [read 134144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-05-11T23:02:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:49.202747+0000", 2024-05-11T23:02:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.143004479, 2024-05-11T23:02:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040435480000000003, 2024-05-11T23:02:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 162238 2024-05-11T23:02:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202747+0000", 2024-05-11T23:02:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202747+0000", 2024-05-11T23:02:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202748+0000", 2024-05-11T23:02:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3000000000000001e-07 2024-05-11T23:02:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202750+0000", 2024-05-11T23:02:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2000000000000001e-06 2024-05-11T23:02:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202757+0000", 2024-05-11T23:02:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9190000000000002e-06 2024-05-11T23:02:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202762+0000", 2024-05-11T23:02:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7990000000000001e-06 2024-05-11T23:02:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.243003+0000", 2024-05-11T23:02:58.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040241842 2024-05-11T23:02:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.243092+0000", 2024-05-11T23:02:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8051000000000001e-05 2024-05-11T23:02:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.243183+0000", 2024-05-11T23:02:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1238999999999998e-05 2024-05-11T23:02:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.396 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:162246 3.1f 3:fac6da97:::rbd_data.111ec437b352.000000000000000c:2a [read 139264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-05-11T23:02:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:49.202770+0000", 2024-05-11T23:02:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.1429817609999997, 2024-05-11T23:02:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040602878000000002, 2024-05-11T23:02:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.397 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 162246 2024-05-11T23:02:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202770+0000", 2024-05-11T23:02:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202770+0000", 2024-05-11T23:02:58.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202770+0000", 2024-05-11T23:02:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2599999999999998e-07 2024-05-11T23:02:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202773+0000", 2024-05-11T23:02:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1399999999999998e-06 2024-05-11T23:02:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202780+0000", 2024-05-11T23:02:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0690000000000004e-06 2024-05-11T23:02:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202784+0000", 2024-05-11T23:02:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4399999999999998e-06 2024-05-11T23:02:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.243216+0000", 2024-05-11T23:02:58.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040431505999999999 2024-05-11T23:02:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.243287+0000", 2024-05-11T23:02:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1080000000000004e-05 2024-05-11T23:02:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.243373+0000", 2024-05-11T23:02:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6217000000000005e-05 2024-05-11T23:02:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.401 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:162247 3.1f 3:fac6da97:::rbd_data.111ec437b352.000000000000000c:2a [read 144384~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-05-11T23:02:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:49.202791+0000", 2024-05-11T23:02:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.1429606480000007, 2024-05-11T23:02:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040778321999999999, 2024-05-11T23:02:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.402 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 162247 2024-05-11T23:02:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202791+0000", 2024-05-11T23:02:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202791+0000", 2024-05-11T23:02:58.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202792+0000", 2024-05-11T23:02:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2599999999999998e-07 2024-05-11T23:02:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202794+0000", 2024-05-11T23:02:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0420000000000001e-06 2024-05-11T23:02:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202801+0000", 2024-05-11T23:02:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.041e-06 2024-05-11T23:02:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202805+0000", 2024-05-11T23:02:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6319999999999999e-06 2024-05-11T23:02:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.243402+0000", 2024-05-11T23:02:58.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040596871999999999 2024-05-11T23:02:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.243475+0000", 2024-05-11T23:02:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2799000000000006e-05 2024-05-11T23:02:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.243569+0000", 2024-05-11T23:02:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4510000000000001e-05 2024-05-11T23:02:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.406 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:162248 3.1f 3:fac6da97:::rbd_data.111ec437b352.000000000000000c:2a [read 149504~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-05-11T23:02:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:49.202812+0000", 2024-05-11T23:02:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.1429401800000001, 2024-05-11T23:02:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040949507000000003, 2024-05-11T23:02:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.407 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 162248 2024-05-11T23:02:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202812+0000", 2024-05-11T23:02:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202812+0000", 2024-05-11T23:02:58.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202812+0000", 2024-05-11T23:02:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4200000000000002e-07 2024-05-11T23:02:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202814+0000", 2024-05-11T23:02:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9700000000000002e-06 2024-05-11T23:02:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202821+0000", 2024-05-11T23:02:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8310000000000003e-06 2024-05-11T23:02:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202826+0000", 2024-05-11T23:02:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0300000000000001e-06 2024-05-11T23:02:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.243601+0000", 2024-05-11T23:02:58.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040775046000000002 2024-05-11T23:02:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.243675+0000", 2024-05-11T23:02:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4420999999999998e-05 2024-05-11T23:02:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.243761+0000", 2024-05-11T23:02:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5866999999999996e-05 2024-05-11T23:02:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.411 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:162249 3.1f 3:fac6da97:::rbd_data.111ec437b352.000000000000000c:2a [read 154624~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-05-11T23:02:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:49.202833+0000", 2024-05-11T23:02:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.1429189760000007, 2024-05-11T23:02:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041129415000000003, 2024-05-11T23:02:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.412 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 162249 2024-05-11T23:02:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202833+0000", 2024-05-11T23:02:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202833+0000", 2024-05-11T23:02:58.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202833+0000", 2024-05-11T23:02:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0999999999999999e-07 2024-05-11T23:02:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202835+0000", 2024-05-11T23:02:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.148e-06 2024-05-11T23:02:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202843+0000", 2024-05-11T23:02:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.357e-06 2024-05-11T23:02:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202848+0000", 2024-05-11T23:02:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8010000000000003e-06 2024-05-11T23:02:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.243790+0000", 2024-05-11T23:02:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.040942951999999998 2024-05-11T23:02:58.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.243869+0000", 2024-05-11T23:02:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8123999999999999e-05 2024-05-11T23:02:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.243962+0000", 2024-05-11T23:02:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3622999999999995e-05 2024-05-11T23:02:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.416 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:162257 3.1f 3:fac6da97:::rbd_data.111ec437b352.000000000000000c:2a [read 159744~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-05-11T23:02:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:49.202855+0000", 2024-05-11T23:02:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.1428972129999995, 2024-05-11T23:02:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041296937999999998, 2024-05-11T23:02:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.417 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 162257 2024-05-11T23:02:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202855+0000", 2024-05-11T23:02:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202855+0000", 2024-05-11T23:02:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202855+0000", 2024-05-11T23:02:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8099999999999998e-07 2024-05-11T23:02:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202857+0000", 2024-05-11T23:02:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0540000000000002e-06 2024-05-11T23:02:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202864+0000", 2024-05-11T23:02:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1409999999999999e-06 2024-05-11T23:02:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202869+0000", 2024-05-11T23:02:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:02:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.243993+0000", 2024-05-11T23:02:58.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041124334999999998 2024-05-11T23:02:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.244067+0000", 2024-05-11T23:02:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3640999999999995e-05 2024-05-11T23:02:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.244152+0000", 2024-05-11T23:02:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4571000000000001e-05 2024-05-11T23:02:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.421 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:162305 3.1f 3:fac6da97:::rbd_data.111ec437b352.000000000000000c:2a [read 164864~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-05-11T23:02:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:49.202876+0000", 2024-05-11T23:02:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.1428756500000006, 2024-05-11T23:02:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041468281000000003, 2024-05-11T23:02:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.422 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 162305 2024-05-11T23:02:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202876+0000", 2024-05-11T23:02:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202876+0000", 2024-05-11T23:02:58.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202877+0000", 2024-05-11T23:02:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0200000000000003e-07 2024-05-11T23:02:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202880+0000", 2024-05-11T23:02:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1700000000000001e-06 2024-05-11T23:02:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202887+0000", 2024-05-11T23:02:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0489999999999998e-06 2024-05-11T23:02:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202892+0000", 2024-05-11T23:02:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0699999999999997e-06 2024-05-11T23:02:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.244186+0000", 2024-05-11T23:02:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041293861000000001 2024-05-11T23:02:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.244260+0000", 2024-05-11T23:02:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4709999999999995e-05 2024-05-11T23:02:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.244344+0000", 2024-05-11T23:02:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4018999999999996e-05 2024-05-11T23:02:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:162306 3.1f 3:fac6da97:::rbd_data.111ec437b352.000000000000000c:2a [read 169984~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-05-11T23:02:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:49.202899+0000", 2024-05-11T23:02:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.1428524509999995, 2024-05-11T23:02:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041635533000000002, 2024-05-11T23:02:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 162306 2024-05-11T23:02:58.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202899+0000", 2024-05-11T23:02:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202899+0000", 2024-05-11T23:02:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202900+0000", 2024-05-11T23:02:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3700000000000001e-07 2024-05-11T23:02:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202902+0000", 2024-05-11T23:02:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1239999999999999e-06 2024-05-11T23:02:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202922+0000", 2024-05-11T23:02:58.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0536000000000001e-05 2024-05-11T23:02:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202929+0000", 2024-05-11T23:02:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1449999999999996e-06 2024-05-11T23:02:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.244374+0000", 2024-05-11T23:02:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041445003000000001 2024-05-11T23:02:58.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.244450+0000", 2024-05-11T23:02:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6188999999999998e-05 2024-05-11T23:02:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.244535+0000", 2024-05-11T23:02:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5099000000000006e-05 2024-05-11T23:02:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:162307 3.1f 3:fac6da97:::rbd_data.111ec437b352.000000000000000c:2a [read 175104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-05-11T23:02:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:49.202938+0000", 2024-05-11T23:02:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.1428135019999992, 2024-05-11T23:02:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041790564000000002, 2024-05-11T23:02:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 162307 2024-05-11T23:02:58.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202938+0000", 2024-05-11T23:02:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202938+0000", 2024-05-11T23:02:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202939+0000", 2024-05-11T23:02:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8699999999999995e-07 2024-05-11T23:02:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202945+0000", 2024-05-11T23:02:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4060000000000002e-06 2024-05-11T23:02:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202953+0000", 2024-05-11T23:02:58.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.627e-06 2024-05-11T23:02:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202958+0000", 2024-05-11T23:02:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2410000000000003e-06 2024-05-11T23:02:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.244568+0000", 2024-05-11T23:02:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041609965999999998 2024-05-11T23:02:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.244642+0000", 2024-05-11T23:02:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3938999999999998e-05 2024-05-11T23:02:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.244729+0000", 2024-05-11T23:02:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6897999999999994e-05 2024-05-11T23:02:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:162308 3.1f 3:fac6da97:::rbd_data.111ec437b352.000000000000000c:2a [read 180224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-05-11T23:02:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:49.202966+0000", 2024-05-11T23:02:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.1427859080000005, 2024-05-11T23:02:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041956045999999997, 2024-05-11T23:02:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 162308 2024-05-11T23:02:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.439 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202966+0000", 2024-05-11T23:02:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202966+0000", 2024-05-11T23:02:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202966+0000", 2024-05-11T23:02:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8500000000000002e-07 2024-05-11T23:02:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202969+0000", 2024-05-11T23:02:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.148e-06 2024-05-11T23:02:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202976+0000", 2024-05-11T23:02:58.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2520000000000002e-06 2024-05-11T23:02:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202982+0000", 2024-05-11T23:02:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1480000000000003e-06 2024-05-11T23:02:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.244757+0000", 2024-05-11T23:02:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041774728999999997 2024-05-11T23:02:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.244830+0000", 2024-05-11T23:02:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3603000000000007e-05 2024-05-11T23:02:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.244922+0000", 2024-05-11T23:02:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1681000000000005e-05 2024-05-11T23:02:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:162309 3.1f 3:fac6da97:::rbd_data.111ec437b352.000000000000000c:2a [read 185344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92)", 2024-05-11T23:02:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:49.202990+0000", 2024-05-11T23:02:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.1427615759999998, 2024-05-11T23:02:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.042125285999999998, 2024-05-11T23:02:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 162309 2024-05-11T23:02:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202990+0000", 2024-05-11T23:02:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202990+0000", 2024-05-11T23:02:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202991+0000", 2024-05-11T23:02:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0499999999999999e-07 2024-05-11T23:02:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.202993+0000", 2024-05-11T23:02:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6000000000000001e-06 2024-05-11T23:02:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.203000+0000", 2024-05-11T23:02:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2799999999999998e-06 2024-05-11T23:02:58.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.203006+0000", 2024-05-11T23:02:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0220000000000004e-06 2024-05-11T23:02:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.244956+0000", 2024-05-11T23:02:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.041950369000000001 2024-05-11T23:02:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:58.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.245029+0000", 2024-05-11T23:02:58.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3442000000000006e-05 2024-05-11T23:02:58.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:49.245115+0000", 2024-05-11T23:02:58.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6168000000000006e-05 2024-05-11T23:02:58.448 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.448 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.448 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.448 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.448 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.448 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:58.450 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:02:58.451 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:58.450+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:58.551 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:58.552+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:58.652 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:58.651+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:58.655 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:02:58.667 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:02:58.667 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:02:58.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.668 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172796 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.668 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:02:58.668 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.044547544000000001, 2024-05-11T23:02:58.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044569747999999999, 2024-05-11T23:02:58.668 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.668 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.668 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.668 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.668 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.668 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.668 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172796 2024-05-11T23:02:58.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.668 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:02:58.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:02:58.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:02:58.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:58.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606069+0000", 2024-05-11T23:02:58.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:02:58.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606072+0000", 2024-05-11T23:02:58.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8700000000000001e-06 2024-05-11T23:02:58.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606074+0000", 2024-05-11T23:02:58.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6230000000000001e-06 2024-05-11T23:02:58.671 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.672 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.672 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.672 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172828 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.672 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.612105+0000", 2024-05-11T23:02:58.672 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.038510040000000002, 2024-05-11T23:02:58.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.038592038000000002, 2024-05-11T23:02:58.672 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.672 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.672 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.672 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.672 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.672 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.672 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172828 2024-05-11T23:02:58.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.673 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612105+0000", 2024-05-11T23:02:58.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612105+0000", 2024-05-11T23:02:58.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612106+0000", 2024-05-11T23:02:58.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6400000000000003e-07 2024-05-11T23:02:58.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612107+0000", 2024-05-11T23:02:58.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6700000000000001e-06 2024-05-11T23:02:58.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612116+0000", 2024-05-11T23:02:58.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6170000000000007e-06 2024-05-11T23:02:58.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612120+0000", 2024-05-11T23:02:58.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.817e-06 2024-05-11T23:02:58.675 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.675 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.675 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.675 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172999 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.675 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.620310+0000", 2024-05-11T23:02:58.675 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.030305553999999998, 2024-05-11T23:02:58.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030441196, 2024-05-11T23:02:58.675 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.675 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.675 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.675 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.675 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.676 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.676 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172999 2024-05-11T23:02:58.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.676 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620310+0000", 2024-05-11T23:02:58.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620310+0000", 2024-05-11T23:02:58.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620310+0000", 2024-05-11T23:02:58.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1800000000000001e-07 2024-05-11T23:02:58.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620312+0000", 2024-05-11T23:02:58.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3070000000000001e-06 2024-05-11T23:02:58.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620319+0000", 2024-05-11T23:02:58.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6610000000000003e-06 2024-05-11T23:02:58.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620324+0000", 2024-05-11T23:02:58.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.259e-06 2024-05-11T23:02:58.678 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.678 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.678 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.678 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173031 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.678 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.624135+0000", 2024-05-11T23:02:58.678 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.026479806000000002, 2024-05-11T23:02:58.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.026665132000000001, 2024-05-11T23:02:58.678 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.678 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.678 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.678 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.679 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.679 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.679 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173031 2024-05-11T23:02:58.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.679 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624135+0000", 2024-05-11T23:02:58.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624135+0000", 2024-05-11T23:02:58.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624136+0000", 2024-05-11T23:02:58.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8800000000000003e-07 2024-05-11T23:02:58.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624138+0000", 2024-05-11T23:02:58.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.323e-06 2024-05-11T23:02:58.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624144+0000", 2024-05-11T23:02:58.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0789999999999997e-06 2024-05-11T23:02:58.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624149+0000", 2024-05-11T23:02:58.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9869999999999999e-06 2024-05-11T23:02:58.681 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.681 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.681 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.681 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173063 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.681 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.628043+0000", 2024-05-11T23:02:58.681 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.022572443000000001, 2024-05-11T23:02:58.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022806696000000001, 2024-05-11T23:02:58.681 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.681 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.681 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.682 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.682 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.682 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.682 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173063 2024-05-11T23:02:58.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.682 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628043+0000", 2024-05-11T23:02:58.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628043+0000", 2024-05-11T23:02:58.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628043+0000", 2024-05-11T23:02:58.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.17e-07 2024-05-11T23:02:58.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628044+0000", 2024-05-11T23:02:58.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3239999999999999e-06 2024-05-11T23:02:58.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628048+0000", 2024-05-11T23:02:58.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2389999999999998e-06 2024-05-11T23:02:58.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628050+0000", 2024-05-11T23:02:58.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8310000000000001e-06 2024-05-11T23:02:58.684 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.685 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.685 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.685 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173399 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.685 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.644366+0000", 2024-05-11T23:02:58.685 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0062495850000000002, 2024-05-11T23:02:58.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0065357089999999998, 2024-05-11T23:02:58.685 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.685 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.685 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.685 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.685 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.686 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.686 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173399 2024-05-11T23:02:58.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.686 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644366+0000", 2024-05-11T23:02:58.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644366+0000", 2024-05-11T23:02:58.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644366+0000", 2024-05-11T23:02:58.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9200000000000001e-07 2024-05-11T23:02:58.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644368+0000", 2024-05-11T23:02:58.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9570000000000001e-06 2024-05-11T23:02:58.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644374+0000", 2024-05-11T23:02:58.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6450000000000002e-06 2024-05-11T23:02:58.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644379+0000", 2024-05-11T23:02:58.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7840000000000003e-06 2024-05-11T23:02:58.688 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.688 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.688 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.688 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173432 3.19 3.eec44899 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.688 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648014+0000", 2024-05-11T23:02:58.688 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0026008709999999998, 2024-05-11T23:02:58.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0026487949999999998, 2024-05-11T23:02:58.688 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.688 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.688 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:02:58.689 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.689 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.689 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.689 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173432 2024-05-11T23:02:58.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.689 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648014+0000", 2024-05-11T23:02:58.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648014+0000", 2024-05-11T23:02:58.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648015+0000", 2024-05-11T23:02:58.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7400000000000003e-07 2024-05-11T23:02:58.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648018+0000", 2024-05-11T23:02:58.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6850000000000001e-06 2024-05-11T23:02:58.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648024+0000", 2024-05-11T23:02:58.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5660000000000003e-06 2024-05-11T23:02:58.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648029+0000", 2024-05-11T23:02:58.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.22e-06 2024-05-11T23:02:58.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:58.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650656+0000", 2024-05-11T23:02:58.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.002626663 2024-05-11T23:02:58.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:02:58.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650663+0000", 2024-05-11T23:02:58.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0870000000000001e-06 2024-05-11T23:02:58.691 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.691 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.691 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.692 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172797 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.692 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:02:58.692 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.044537512000000001, 2024-05-11T23:02:58.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044948144000000002, 2024-05-11T23:02:58.693 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.693 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.693 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.693 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.693 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.693 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.693 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172797 2024-05-11T23:02:58.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.693 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:02:58.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:02:58.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607915+0000", 2024-05-11T23:02:58.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018375609999999999 2024-05-11T23:02:58.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607920+0000", 2024-05-11T23:02:58.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8899999999999998e-06 2024-05-11T23:02:58.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607930+0000", 2024-05-11T23:02:58.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4429999999999992e-06 2024-05-11T23:02:58.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607936+0000", 2024-05-11T23:02:58.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7089999999999996e-06 2024-05-11T23:02:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172829 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.612123+0000", 2024-05-11T23:02:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.038491728000000003, 2024-05-11T23:02:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.038952568999999999, 2024-05-11T23:02:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172829 2024-05-11T23:02:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612123+0000", 2024-05-11T23:02:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612123+0000", 2024-05-11T23:02:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612124+0000", 2024-05-11T23:02:58.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.22e-07 2024-05-11T23:02:58.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612125+0000", 2024-05-11T23:02:58.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.175e-06 2024-05-11T23:02:58.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612129+0000", 2024-05-11T23:02:58.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2239999999999997e-06 2024-05-11T23:02:58.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612132+0000", 2024-05-11T23:02:58.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.655e-06 2024-05-11T23:02:58.699 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.699 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.699 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.699 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173000 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.699 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.620334+0000", 2024-05-11T23:02:58.699 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.030281203999999999, 2024-05-11T23:02:58.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030795309, 2024-05-11T23:02:58.700 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.701 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.701 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.701 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.701 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.701 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.701 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173000 2024-05-11T23:02:58.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.701 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620334+0000", 2024-05-11T23:02:58.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620334+0000", 2024-05-11T23:02:58.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620335+0000", 2024-05-11T23:02:58.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.37e-07 2024-05-11T23:02:58.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620337+0000", 2024-05-11T23:02:58.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2819999999999999e-06 2024-05-11T23:02:58.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620344+0000", 2024-05-11T23:02:58.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5719999999999999e-06 2024-05-11T23:02:58.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620349+0000", 2024-05-11T23:02:58.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0989999999999998e-06 2024-05-11T23:02:58.703 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.703 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.703 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.703 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173032 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.703 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.624158+0000", 2024-05-11T23:02:58.704 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.026457155, 2024-05-11T23:02:58.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.027020467999999999, 2024-05-11T23:02:58.704 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.704 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.704 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.704 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.704 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.704 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.704 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173032 2024-05-11T23:02:58.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.704 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.704 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624158+0000", 2024-05-11T23:02:58.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624158+0000", 2024-05-11T23:02:58.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624159+0000", 2024-05-11T23:02:58.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5999999999999999e-07 2024-05-11T23:02:58.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624161+0000", 2024-05-11T23:02:58.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.289e-06 2024-05-11T23:02:58.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624168+0000", 2024-05-11T23:02:58.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4340000000000003e-06 2024-05-11T23:02:58.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624174+0000", 2024-05-11T23:02:58.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5430000000000001e-06 2024-05-11T23:02:58.706 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.706 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.706 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.707 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173064 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.707 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.628054+0000", 2024-05-11T23:02:58.707 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.022561623999999999, 2024-05-11T23:02:58.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023174318999999999, 2024-05-11T23:02:58.707 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.707 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.707 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.707 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.707 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.707 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.707 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173064 2024-05-11T23:02:58.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.707 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628054+0000", 2024-05-11T23:02:58.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628054+0000", 2024-05-11T23:02:58.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628054+0000", 2024-05-11T23:02:58.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:58.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628055+0000", 2024-05-11T23:02:58.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.102e-06 2024-05-11T23:02:58.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628058+0000", 2024-05-11T23:02:58.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.971e-06 2024-05-11T23:02:58.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628061+0000", 2024-05-11T23:02:58.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.621e-06 2024-05-11T23:02:58.709 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.709 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.709 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.710 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173400 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.710 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.644384+0000", 2024-05-11T23:02:58.710 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0062312770000000003, 2024-05-11T23:02:58.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0068937829999999997, 2024-05-11T23:02:58.710 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.710 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.710 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.710 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.710 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.710 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.710 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173400 2024-05-11T23:02:58.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.710 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644384+0000", 2024-05-11T23:02:58.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644384+0000", 2024-05-11T23:02:58.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644384+0000", 2024-05-11T23:02:58.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6499999999999999e-07 2024-05-11T23:02:58.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644386+0000", 2024-05-11T23:02:58.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9549999999999999e-06 2024-05-11T23:02:58.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644393+0000", 2024-05-11T23:02:58.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5180000000000001e-06 2024-05-11T23:02:58.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644397+0000", 2024-05-11T23:02:58.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.566e-06 2024-05-11T23:02:58.712 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.713 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.713 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.713 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172798 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.713 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.607953+0000", 2024-05-11T23:02:58.713 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.042661969000000001, 2024-05-11T23:02:58.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043375591999999998, 2024-05-11T23:02:58.713 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.713 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.713 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.713 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.713 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.713 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.713 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172798 2024-05-11T23:02:58.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.714 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607953+0000", 2024-05-11T23:02:58.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607953+0000", 2024-05-11T23:02:58.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607954+0000", 2024-05-11T23:02:58.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4099999999999999e-07 2024-05-11T23:02:58.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607957+0000", 2024-05-11T23:02:58.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8320000000000002e-06 2024-05-11T23:02:58.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607966+0000", 2024-05-11T23:02:58.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6530000000000006e-06 2024-05-11T23:02:58.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607973+0000", 2024-05-11T23:02:58.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1829999999999997e-06 2024-05-11T23:02:58.716 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.716 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.716 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.716 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172830 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.716 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.612135+0000", 2024-05-11T23:02:58.716 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.038480272000000003, 2024-05-11T23:02:58.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039246626999999999, 2024-05-11T23:02:58.716 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.716 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.716 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.716 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.716 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.716 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.717 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172830 2024-05-11T23:02:58.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.717 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612135+0000", 2024-05-11T23:02:58.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612135+0000", 2024-05-11T23:02:58.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.613913+0000", 2024-05-11T23:02:58.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017783950000000001 2024-05-11T23:02:58.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.613918+0000", 2024-05-11T23:02:58.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3939999999999998e-06 2024-05-11T23:02:58.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.613928+0000", 2024-05-11T23:02:58.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0757000000000001e-05 2024-05-11T23:02:58.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.613936+0000", 2024-05-11T23:02:58.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0650000000000001e-06 2024-05-11T23:02:58.719 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.719 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.719 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.719 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173001 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.719 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.620363+0000", 2024-05-11T23:02:58.719 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.030252555, 2024-05-11T23:02:58.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031068234, 2024-05-11T23:02:58.719 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.719 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.719 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.720 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.720 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.720 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.720 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173001 2024-05-11T23:02:58.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.720 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620363+0000", 2024-05-11T23:02:58.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620363+0000", 2024-05-11T23:02:58.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620363+0000", 2024-05-11T23:02:58.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9699999999999996e-07 2024-05-11T23:02:58.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620366+0000", 2024-05-11T23:02:58.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8100000000000002e-06 2024-05-11T23:02:58.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620373+0000", 2024-05-11T23:02:58.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.888e-06 2024-05-11T23:02:58.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620379+0000", 2024-05-11T23:02:58.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7429999999999999e-06 2024-05-11T23:02:58.722 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.722 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.722 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.722 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173033 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.722 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.624181+0000", 2024-05-11T23:02:58.722 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.026434052999999999, 2024-05-11T23:02:58.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.027299733999999999, 2024-05-11T23:02:58.722 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.723 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.723 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.723 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.723 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.723 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173033 2024-05-11T23:02:58.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.723 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.723 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624181+0000", 2024-05-11T23:02:58.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624181+0000", 2024-05-11T23:02:58.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624182+0000", 2024-05-11T23:02:58.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:02:58.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624184+0000", 2024-05-11T23:02:58.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.266e-06 2024-05-11T23:02:58.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624190+0000", 2024-05-11T23:02:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6460000000000004e-06 2024-05-11T23:02:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624195+0000", 2024-05-11T23:02:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9799999999999998e-06 2024-05-11T23:02:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173065 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.628063+0000", 2024-05-11T23:02:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.022551754, 2024-05-11T23:02:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023471313000000001, 2024-05-11T23:02:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173065 2024-05-11T23:02:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628063+0000", 2024-05-11T23:02:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628063+0000", 2024-05-11T23:02:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629914+0000", 2024-05-11T23:02:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018506620000000001 2024-05-11T23:02:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629920+0000", 2024-05-11T23:02:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2929999999999997e-06 2024-05-11T23:02:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629928+0000", 2024-05-11T23:02:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5820000000000004e-06 2024-05-11T23:02:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629932+0000", 2024-05-11T23:02:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1910000000000002e-06 2024-05-11T23:02:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173436 3.19 3.eec44899 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648059+0000", 2024-05-11T23:02:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0025566120000000002, 2024-05-11T23:02:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0035033379999999999, 2024-05-11T23:02:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173436 2024-05-11T23:02:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648059+0000", 2024-05-11T23:02:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648059+0000", 2024-05-11T23:02:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648059+0000", 2024-05-11T23:02:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2599999999999998e-07 2024-05-11T23:02:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648061+0000", 2024-05-11T23:02:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2589999999999999e-06 2024-05-11T23:02:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648068+0000", 2024-05-11T23:02:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8639999999999998e-06 2024-05-11T23:02:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648074+0000", 2024-05-11T23:02:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7690000000000002e-06 2024-05-11T23:02:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172799 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.607999+0000", 2024-05-11T23:02:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.042616706999999997, 2024-05-11T23:02:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043589917999999998, 2024-05-11T23:02:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172799 2024-05-11T23:02:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607999+0000", 2024-05-11T23:02:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607999+0000", 2024-05-11T23:02:58.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607999+0000", 2024-05-11T23:02:58.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6300000000000001e-07 2024-05-11T23:02:58.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608001+0000", 2024-05-11T23:02:58.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4250000000000001e-06 2024-05-11T23:02:58.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608010+0000", 2024-05-11T23:02:58.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8100000000000004e-06 2024-05-11T23:02:58.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608016+0000", 2024-05-11T23:02:58.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5770000000000001e-06 2024-05-11T23:02:58.734 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.734 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.734 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.735 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173002 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.735 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.620386+0000", 2024-05-11T23:02:58.735 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.030228728, 2024-05-11T23:02:58.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031228717999999999, 2024-05-11T23:02:58.735 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173002 2024-05-11T23:02:58.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.735 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620386+0000", 2024-05-11T23:02:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620386+0000", 2024-05-11T23:02:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620387+0000", 2024-05-11T23:02:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5200000000000002e-07 2024-05-11T23:02:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620389+0000", 2024-05-11T23:02:58.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3980000000000002e-06 2024-05-11T23:02:58.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620396+0000", 2024-05-11T23:02:58.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2179999999999996e-06 2024-05-11T23:02:58.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620401+0000", 2024-05-11T23:02:58.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3940000000000004e-06 2024-05-11T23:02:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173034 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.624202+0000", 2024-05-11T23:02:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.026413531000000001, 2024-05-11T23:02:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.027440493999999999, 2024-05-11T23:02:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173034 2024-05-11T23:02:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624202+0000", 2024-05-11T23:02:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624202+0000", 2024-05-11T23:02:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624202+0000", 2024-05-11T23:02:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8000000000000001e-07 2024-05-11T23:02:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624204+0000", 2024-05-11T23:02:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0899999999999999e-06 2024-05-11T23:02:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624210+0000", 2024-05-11T23:02:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3839999999999996e-06 2024-05-11T23:02:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624214+0000", 2024-05-11T23:02:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1400000000000002e-06 2024-05-11T23:02:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173066 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.629946+0000", 2024-05-11T23:02:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.020669417999999998, 2024-05-11T23:02:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021722801, 2024-05-11T23:02:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.742 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.742 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.742 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.742 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173066 2024-05-11T23:02:58.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.742 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629946+0000", 2024-05-11T23:02:58.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629946+0000", 2024-05-11T23:02:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629946+0000", 2024-05-11T23:02:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4799999999999999e-07 2024-05-11T23:02:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629948+0000", 2024-05-11T23:02:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6619999999999999e-06 2024-05-11T23:02:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629953+0000", 2024-05-11T23:02:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1370000000000001e-06 2024-05-11T23:02:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629956+0000", 2024-05-11T23:02:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.411e-06 2024-05-11T23:02:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173438 3.19 3.eec44899 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648081+0000", 2024-05-11T23:02:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.002533899, 2024-05-11T23:02:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0036134840000000001, 2024-05-11T23:02:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173438 2024-05-11T23:02:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648081+0000", 2024-05-11T23:02:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648081+0000", 2024-05-11T23:02:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648082+0000", 2024-05-11T23:02:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6300000000000005e-07 2024-05-11T23:02:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648084+0000", 2024-05-11T23:02:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1509999999999998e-06 2024-05-11T23:02:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648090+0000", 2024-05-11T23:02:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0179999999999998e-06 2024-05-11T23:02:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648095+0000", 2024-05-11T23:02:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8690000000000003e-06 2024-05-11T23:02:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172800 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.608022+0000", 2024-05-11T23:02:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.042593686999999998, 2024-05-11T23:02:58.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043699504, 2024-05-11T23:02:58.748 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.748 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.748 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.748 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.748 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.748 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.748 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172800 2024-05-11T23:02:58.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.748 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608022+0000", 2024-05-11T23:02:58.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608022+0000", 2024-05-11T23:02:58.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608022+0000", 2024-05-11T23:02:58.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4400000000000001e-07 2024-05-11T23:02:58.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608023+0000", 2024-05-11T23:02:58.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5349999999999999e-06 2024-05-11T23:02:58.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608027+0000", 2024-05-11T23:02:58.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4039999999999999e-06 2024-05-11T23:02:58.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.750 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.750 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608030+0000", 2024-05-11T23:02:58.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.79e-06 2024-05-11T23:02:58.750 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.750 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.750 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.750 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173003 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.750 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.620408+0000", 2024-05-11T23:02:58.751 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.030207464999999999, 2024-05-11T23:02:58.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031339560000000002, 2024-05-11T23:02:58.751 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.751 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.751 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173003 2024-05-11T23:02:58.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.751 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620408+0000", 2024-05-11T23:02:58.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620408+0000", 2024-05-11T23:02:58.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620408+0000", 2024-05-11T23:02:58.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6100000000000002e-07 2024-05-11T23:02:58.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620410+0000", 2024-05-11T23:02:58.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.295e-06 2024-05-11T23:02:58.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620417+0000", 2024-05-11T23:02:58.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.489e-06 2024-05-11T23:02:58.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620422+0000", 2024-05-11T23:02:58.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8720000000000001e-06 2024-05-11T23:02:58.753 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.753 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.753 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.753 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173035 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.754 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.624221+0000", 2024-05-11T23:02:58.754 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.026394609999999999, 2024-05-11T23:02:58.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.027552718, 2024-05-11T23:02:58.754 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.754 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.754 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.754 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.754 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.754 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173035 2024-05-11T23:02:58.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.754 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624221+0000", 2024-05-11T23:02:58.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624221+0000", 2024-05-11T23:02:58.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624221+0000", 2024-05-11T23:02:58.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3300000000000002e-07 2024-05-11T23:02:58.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624223+0000", 2024-05-11T23:02:58.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.057e-06 2024-05-11T23:02:58.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624228+0000", 2024-05-11T23:02:58.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1170000000000003e-06 2024-05-11T23:02:58.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624233+0000", 2024-05-11T23:02:58.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4710000000000001e-06 2024-05-11T23:02:58.756 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.756 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.756 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.757 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173067 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.757 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.629961+0000", 2024-05-11T23:02:58.757 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.02065444, 2024-05-11T23:02:58.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021838740999999998, 2024-05-11T23:02:58.757 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.757 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.757 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.757 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.757 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.757 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.757 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173067 2024-05-11T23:02:58.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.757 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629961+0000", 2024-05-11T23:02:58.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629961+0000", 2024-05-11T23:02:58.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629961+0000", 2024-05-11T23:02:58.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.41e-07 2024-05-11T23:02:58.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629962+0000", 2024-05-11T23:02:58.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2810000000000001e-06 2024-05-11T23:02:58.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629966+0000", 2024-05-11T23:02:58.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.189e-06 2024-05-11T23:02:58.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629970+0000", 2024-05-11T23:02:58.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0709999999999999e-06 2024-05-11T23:02:58.759 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.759 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.759 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.760 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173440 3.19 3.eec44899 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.760 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648101+0000", 2024-05-11T23:02:58.760 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0025141989999999999, 2024-05-11T23:02:58.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003724645, 2024-05-11T23:02:58.760 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.760 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.760 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.760 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.760 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.760 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.760 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173440 2024-05-11T23:02:58.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.760 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648101+0000", 2024-05-11T23:02:58.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648101+0000", 2024-05-11T23:02:58.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648101+0000", 2024-05-11T23:02:58.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.72e-07 2024-05-11T23:02:58.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648104+0000", 2024-05-11T23:02:58.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1349999999999999e-06 2024-05-11T23:02:58.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648110+0000", 2024-05-11T23:02:58.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1859999999999997e-06 2024-05-11T23:02:58.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648115+0000", 2024-05-11T23:02:58.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8019999999999999e-06 2024-05-11T23:02:58.762 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.762 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.762 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.763 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172801 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.763 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.608033+0000", 2024-05-11T23:02:58.763 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.042582043999999999, 2024-05-11T23:02:58.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.043818567000000003, 2024-05-11T23:02:58.763 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.763 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.763 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.763 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.763 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.763 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172801 2024-05-11T23:02:58.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.763 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608033+0000", 2024-05-11T23:02:58.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608033+0000", 2024-05-11T23:02:58.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608033+0000", 2024-05-11T23:02:58.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4400000000000001e-07 2024-05-11T23:02:58.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608038+0000", 2024-05-11T23:02:58.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9799999999999998e-06 2024-05-11T23:02:58.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608042+0000", 2024-05-11T23:02:58.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4290000000000001e-06 2024-05-11T23:02:58.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608047+0000", 2024-05-11T23:02:58.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9660000000000004e-06 2024-05-11T23:02:58.765 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.765 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.766 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.766 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173004 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.766 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.620429+0000", 2024-05-11T23:02:58.766 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.030186470999999999, 2024-05-11T23:02:58.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031456525999999999, 2024-05-11T23:02:58.766 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.766 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.766 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.766 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.766 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.766 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173004 2024-05-11T23:02:58.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620429+0000", 2024-05-11T23:02:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620429+0000", 2024-05-11T23:02:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620429+0000", 2024-05-11T23:02:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6199999999999999e-07 2024-05-11T23:02:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620431+0000", 2024-05-11T23:02:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0379999999999998e-06 2024-05-11T23:02:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620439+0000", 2024-05-11T23:02:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4150000000000002e-06 2024-05-11T23:02:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620444+0000", 2024-05-11T23:02:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3299999999999998e-06 2024-05-11T23:02:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.769 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.769 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.769 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173036 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.769 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.624248+0000", 2024-05-11T23:02:58.769 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.026367380999999999, 2024-05-11T23:02:58.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.027673164, 2024-05-11T23:02:58.769 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.769 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.769 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.769 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.769 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.769 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.769 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173036 2024-05-11T23:02:58.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.770 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624248+0000", 2024-05-11T23:02:58.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624248+0000", 2024-05-11T23:02:58.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624248+0000", 2024-05-11T23:02:58.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1200000000000003e-07 2024-05-11T23:02:58.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624251+0000", 2024-05-11T23:02:58.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6089999999999999e-06 2024-05-11T23:02:58.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624257+0000", 2024-05-11T23:02:58.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4060000000000002e-06 2024-05-11T23:02:58.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624263+0000", 2024-05-11T23:02:58.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2639999999999999e-06 2024-05-11T23:02:58.771 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.772 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.772 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.772 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173068 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.772 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.629973+0000", 2024-05-11T23:02:58.772 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.020642178000000001, 2024-05-11T23:02:58.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022001051000000001, 2024-05-11T23:02:58.772 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.772 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.772 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.772 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.772 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.772 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173068 2024-05-11T23:02:58.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.773 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629973+0000", 2024-05-11T23:02:58.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629973+0000", 2024-05-11T23:02:58.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629973+0000", 2024-05-11T23:02:58.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-05-11T23:02:58.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629975+0000", 2024-05-11T23:02:58.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5239999999999999e-06 2024-05-11T23:02:58.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629978+0000", 2024-05-11T23:02:58.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1e-06 2024-05-11T23:02:58.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629981+0000", 2024-05-11T23:02:58.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8940000000000002e-06 2024-05-11T23:02:58.775 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.775 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.775 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.775 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173442 3.19 3.eec44899 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.775 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648121+0000", 2024-05-11T23:02:58.775 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0024943539999999998, 2024-05-11T23:02:58.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0039060919999999999, 2024-05-11T23:02:58.775 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.775 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.775 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.775 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.775 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.775 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.775 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173442 2024-05-11T23:02:58.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.776 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648121+0000", 2024-05-11T23:02:58.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648121+0000", 2024-05-11T23:02:58.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648121+0000", 2024-05-11T23:02:58.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-05-11T23:02:58.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648123+0000", 2024-05-11T23:02:58.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0140000000000001e-06 2024-05-11T23:02:58.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648129+0000", 2024-05-11T23:02:58.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7209999999999999e-06 2024-05-11T23:02:58.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648134+0000", 2024-05-11T23:02:58.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2839999999999997e-06 2024-05-11T23:02:58.778 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.778 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.778 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.778 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172802 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.778 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.608053+0000", 2024-05-11T23:02:58.778 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.042562054000000002, 2024-05-11T23:02:58.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044071935, 2024-05-11T23:02:58.778 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.778 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.778 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.778 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.778 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.779 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.779 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172802 2024-05-11T23:02:58.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.779 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608053+0000", 2024-05-11T23:02:58.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608053+0000", 2024-05-11T23:02:58.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608053+0000", 2024-05-11T23:02:58.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2099999999999998e-07 2024-05-11T23:02:58.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608056+0000", 2024-05-11T23:02:58.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.21e-06 2024-05-11T23:02:58.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608066+0000", 2024-05-11T23:02:58.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0682e-05 2024-05-11T23:02:58.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608070+0000", 2024-05-11T23:02:58.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3289999999999998e-06 2024-05-11T23:02:58.781 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.781 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.781 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.781 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173005 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.781 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.620450+0000", 2024-05-11T23:02:58.781 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.03016539, 2024-05-11T23:02:58.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031730418000000003, 2024-05-11T23:02:58.781 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.781 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.781 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.781 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.781 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.781 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.782 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173005 2024-05-11T23:02:58.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.782 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620450+0000", 2024-05-11T23:02:58.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620450+0000", 2024-05-11T23:02:58.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621916+0000", 2024-05-11T23:02:58.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001466625 2024-05-11T23:02:58.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621925+0000", 2024-05-11T23:02:58.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.072e-06 2024-05-11T23:02:58.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621936+0000", 2024-05-11T23:02:58.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0987e-05 2024-05-11T23:02:58.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621942+0000", 2024-05-11T23:02:58.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6760000000000001e-06 2024-05-11T23:02:58.784 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.784 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.784 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.784 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173037 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.784 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.624270+0000", 2024-05-11T23:02:58.784 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.026345403999999999, 2024-05-11T23:02:58.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.027963569000000001, 2024-05-11T23:02:58.784 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.784 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.784 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.784 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.784 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.785 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.785 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173037 2024-05-11T23:02:58.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.785 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624270+0000", 2024-05-11T23:02:58.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624270+0000", 2024-05-11T23:02:58.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624270+0000", 2024-05-11T23:02:58.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4200000000000001e-07 2024-05-11T23:02:58.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624272+0000", 2024-05-11T23:02:58.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.092e-06 2024-05-11T23:02:58.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624280+0000", 2024-05-11T23:02:58.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5490000000000001e-06 2024-05-11T23:02:58.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624285+0000", 2024-05-11T23:02:58.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:02:58.787 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.787 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.787 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.787 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173069 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.787 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.629984+0000", 2024-05-11T23:02:58.787 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.020631291, 2024-05-11T23:02:58.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022303666, 2024-05-11T23:02:58.787 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.787 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.787 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.788 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.788 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.788 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.788 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173069 2024-05-11T23:02:58.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.788 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629984+0000", 2024-05-11T23:02:58.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629984+0000", 2024-05-11T23:02:58.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629984+0000", 2024-05-11T23:02:58.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8299999999999998e-07 2024-05-11T23:02:58.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629985+0000", 2024-05-11T23:02:58.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1599999999999999e-06 2024-05-11T23:02:58.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629989+0000", 2024-05-11T23:02:58.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8190000000000002e-06 2024-05-11T23:02:58.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629992+0000", 2024-05-11T23:02:58.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9550000000000001e-06 2024-05-11T23:02:58.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.790 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.790 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173444 3.19 3.eec44899 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.790 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648142+0000", 2024-05-11T23:02:58.790 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.002473547, 2024-05-11T23:02:58.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0042061809999999998, 2024-05-11T23:02:58.790 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.791 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.791 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.791 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.791 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.791 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173444 2024-05-11T23:02:58.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.791 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648142+0000", 2024-05-11T23:02:58.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648142+0000", 2024-05-11T23:02:58.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648142+0000", 2024-05-11T23:02:58.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7e-07 2024-05-11T23:02:58.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648144+0000", 2024-05-11T23:02:58.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1739999999999999e-06 2024-05-11T23:02:58.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648151+0000", 2024-05-11T23:02:58.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5679999999999996e-06 2024-05-11T23:02:58.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648156+0000", 2024-05-11T23:02:58.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9280000000000001e-06 2024-05-11T23:02:58.793 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.793 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.793 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.793 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172803 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.793 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.608074+0000", 2024-05-11T23:02:58.793 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.042540738000000002, 2024-05-11T23:02:58.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.04433235, 2024-05-11T23:02:58.794 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.794 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.794 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.794 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.794 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172803 2024-05-11T23:02:58.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.794 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.794 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.794 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608074+0000", 2024-05-11T23:02:58.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608074+0000", 2024-05-11T23:02:58.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608075+0000", 2024-05-11T23:02:58.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5699999999999999e-07 2024-05-11T23:02:58.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608076+0000", 2024-05-11T23:02:58.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.519e-06 2024-05-11T23:02:58.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608080+0000", 2024-05-11T23:02:58.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0470000000000004e-06 2024-05-11T23:02:58.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608083+0000", 2024-05-11T23:02:58.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.756e-06 2024-05-11T23:02:58.796 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.796 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.796 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.796 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173006 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.796 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.621954+0000", 2024-05-11T23:02:58.796 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.028661096, 2024-05-11T23:02:58.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030495379, 2024-05-11T23:02:58.797 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.797 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.797 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.797 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.797 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.797 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.797 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173006 2024-05-11T23:02:58.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.797 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621954+0000", 2024-05-11T23:02:58.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621954+0000", 2024-05-11T23:02:58.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621955+0000", 2024-05-11T23:02:58.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0100000000000005e-07 2024-05-11T23:02:58.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621958+0000", 2024-05-11T23:02:58.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8959999999999999e-06 2024-05-11T23:02:58.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621964+0000", 2024-05-11T23:02:58.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7800000000000003e-06 2024-05-11T23:02:58.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621969+0000", 2024-05-11T23:02:58.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9099999999999996e-06 2024-05-11T23:02:58.799 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.799 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.799 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.799 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173038 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.799 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.624289+0000", 2024-05-11T23:02:58.800 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.026325899999999999, 2024-05-11T23:02:58.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028193091, 2024-05-11T23:02:58.800 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.800 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.800 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.800 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.800 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.800 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173038 2024-05-11T23:02:58.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.800 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624289+0000", 2024-05-11T23:02:58.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624289+0000", 2024-05-11T23:02:58.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624290+0000", 2024-05-11T23:02:58.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9999999999999999e-07 2024-05-11T23:02:58.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624291+0000", 2024-05-11T23:02:58.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6390000000000001e-06 2024-05-11T23:02:58.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624298+0000", 2024-05-11T23:02:58.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5889999999999999e-06 2024-05-11T23:02:58.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624301+0000", 2024-05-11T23:02:58.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8739999999999999e-06 2024-05-11T23:02:58.802 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.802 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.802 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.802 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173070 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.803 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.629995+0000", 2024-05-11T23:02:58.803 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.020620021999999998, 2024-05-11T23:02:58.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022513650999999999, 2024-05-11T23:02:58.803 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.803 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.803 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.803 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.803 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.803 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.803 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173070 2024-05-11T23:02:58.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.803 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629995+0000", 2024-05-11T23:02:58.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629995+0000", 2024-05-11T23:02:58.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629995+0000", 2024-05-11T23:02:58.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-05-11T23:02:58.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.629997+0000", 2024-05-11T23:02:58.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1880000000000001e-06 2024-05-11T23:02:58.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630000+0000", 2024-05-11T23:02:58.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9500000000000001e-06 2024-05-11T23:02:58.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630002+0000", 2024-05-11T23:02:58.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.593e-06 2024-05-11T23:02:58.805 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.805 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.805 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.806 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173446 3.19 3.eec44899 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.806 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648163+0000", 2024-05-11T23:02:58.806 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0024524690000000001, 2024-05-11T23:02:58.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0043741120000000003, 2024-05-11T23:02:58.806 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.806 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.806 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.806 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.806 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.806 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.806 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173446 2024-05-11T23:02:58.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.806 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648163+0000", 2024-05-11T23:02:58.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648163+0000", 2024-05-11T23:02:58.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648163+0000", 2024-05-11T23:02:58.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5900000000000003e-07 2024-05-11T23:02:58.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648165+0000", 2024-05-11T23:02:58.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0650000000000001e-06 2024-05-11T23:02:58.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648172+0000", 2024-05-11T23:02:58.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6239999999999996e-06 2024-05-11T23:02:58.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648177+0000", 2024-05-11T23:02:58.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8999999999999997e-06 2024-05-11T23:02:58.808 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.808 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.808 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.809 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172773 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.809 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:02:58.809 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.046627567000000002, 2024-05-11T23:02:58.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048578587999999999, 2024-05-11T23:02:58.809 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.809 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.809 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.809 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.809 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.809 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172773 2024-05-11T23:02:58.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.809 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:02:58.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:02:58.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:02:58.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3500000000000002e-07 2024-05-11T23:02:58.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603990+0000", 2024-05-11T23:02:58.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3549999999999999e-06 2024-05-11T23:02:58.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603997+0000", 2024-05-11T23:02:58.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3400000000000003e-06 2024-05-11T23:02:58.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604002+0000", 2024-05-11T23:02:58.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0509999999999996e-06 2024-05-11T23:02:58.811 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.811 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.812 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.812 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172804 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.812 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.608087+0000", 2024-05-11T23:02:58.812 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.042528700000000003, 2024-05-11T23:02:58.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044526417999999998, 2024-05-11T23:02:58.812 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.812 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.812 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.812 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.812 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.812 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.812 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172804 2024-05-11T23:02:58.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.812 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608087+0000", 2024-05-11T23:02:58.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608087+0000", 2024-05-11T23:02:58.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.813 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608087+0000", 2024-05-11T23:02:58.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.17e-07 2024-05-11T23:02:58.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608088+0000", 2024-05-11T23:02:58.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2389999999999999e-06 2024-05-11T23:02:58.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608091+0000", 2024-05-11T23:02:58.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9720000000000001e-06 2024-05-11T23:02:58.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608094+0000", 2024-05-11T23:02:58.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6699999999999998e-06 2024-05-11T23:02:58.814 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.815 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.815 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.815 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173007 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.815 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.621974+0000", 2024-05-11T23:02:58.815 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.028640889999999999, 2024-05-11T23:02:58.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030665998, 2024-05-11T23:02:58.815 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.815 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.815 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.815 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.815 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.815 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.815 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173007 2024-05-11T23:02:58.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.816 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621974+0000", 2024-05-11T23:02:58.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621974+0000", 2024-05-11T23:02:58.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621975+0000", 2024-05-11T23:02:58.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7000000000000001e-07 2024-05-11T23:02:58.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621976+0000", 2024-05-11T23:02:58.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.42e-06 2024-05-11T23:02:58.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621979+0000", 2024-05-11T23:02:58.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.196e-06 2024-05-11T23:02:58.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621982+0000", 2024-05-11T23:02:58.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8959999999999999e-06 2024-05-11T23:02:58.818 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.818 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.818 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.818 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173039 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.818 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.624306+0000", 2024-05-11T23:02:58.818 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.026309471000000001, 2024-05-11T23:02:58.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028365455000000001, 2024-05-11T23:02:58.818 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.818 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.818 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.818 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.819 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.819 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.819 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173039 2024-05-11T23:02:58.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.819 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624306+0000", 2024-05-11T23:02:58.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624306+0000", 2024-05-11T23:02:58.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624306+0000", 2024-05-11T23:02:58.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.28e-07 2024-05-11T23:02:58.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624307+0000", 2024-05-11T23:02:58.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3909999999999999e-06 2024-05-11T23:02:58.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624311+0000", 2024-05-11T23:02:58.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.151e-06 2024-05-11T23:02:58.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624313+0000", 2024-05-11T23:02:58.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5289999999999998e-06 2024-05-11T23:02:58.821 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.821 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.821 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.821 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173071 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.821 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.630005+0000", 2024-05-11T23:02:58.821 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.020610000999999999, 2024-05-11T23:02:58.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022695256, 2024-05-11T23:02:58.821 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.821 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.821 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.822 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.822 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.822 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.822 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173071 2024-05-11T23:02:58.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.822 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630005+0000", 2024-05-11T23:02:58.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630005+0000", 2024-05-11T23:02:58.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630005+0000", 2024-05-11T23:02:58.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.03e-07 2024-05-11T23:02:58.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630007+0000", 2024-05-11T23:02:58.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2359999999999999e-06 2024-05-11T23:02:58.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630010+0000", 2024-05-11T23:02:58.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9330000000000001e-06 2024-05-11T23:02:58.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630012+0000", 2024-05-11T23:02:58.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.537e-06 2024-05-11T23:02:58.824 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.824 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.824 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.824 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173448 3.19 3.eec44899 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.824 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648183+0000", 2024-05-11T23:02:58.824 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.002432146, 2024-05-11T23:02:58.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0045657600000000003, 2024-05-11T23:02:58.824 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.825 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.825 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.825 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.825 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.825 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173448 2024-05-11T23:02:58.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.825 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648183+0000", 2024-05-11T23:02:58.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648183+0000", 2024-05-11T23:02:58.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648183+0000", 2024-05-11T23:02:58.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:02:58.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648186+0000", 2024-05-11T23:02:58.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0549999999999998e-06 2024-05-11T23:02:58.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648191+0000", 2024-05-11T23:02:58.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9220000000000002e-06 2024-05-11T23:02:58.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648196+0000", 2024-05-11T23:02:58.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7960000000000003e-06 2024-05-11T23:02:58.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.827 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.827 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172774 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.827 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:02:58.827 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.046606185000000001, 2024-05-11T23:02:58.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048792869000000003, 2024-05-11T23:02:58.827 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.827 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.828 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.828 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.828 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.828 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172774 2024-05-11T23:02:58.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.828 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:02:58.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:02:58.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:02:58.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:02:58.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604012+0000", 2024-05-11T23:02:58.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2199999999999999e-06 2024-05-11T23:02:58.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604018+0000", 2024-05-11T23:02:58.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1770000000000003e-06 2024-05-11T23:02:58.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604023+0000", 2024-05-11T23:02:58.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0089999999999998e-06 2024-05-11T23:02:58.830 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.830 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.830 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.830 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172805 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.830 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.608097+0000", 2024-05-11T23:02:58.830 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.042518355000000001, 2024-05-11T23:02:58.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044761957999999998, 2024-05-11T23:02:58.831 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.831 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.831 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.831 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.831 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.831 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.831 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172805 2024-05-11T23:02:58.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.831 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608097+0000", 2024-05-11T23:02:58.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608097+0000", 2024-05-11T23:02:58.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608097+0000", 2024-05-11T23:02:58.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-05-11T23:02:58.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608098+0000", 2024-05-11T23:02:58.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.192e-06 2024-05-11T23:02:58.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608101+0000", 2024-05-11T23:02:58.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9469999999999999e-06 2024-05-11T23:02:58.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608104+0000", 2024-05-11T23:02:58.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6089999999999999e-06 2024-05-11T23:02:58.833 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.833 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.833 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.833 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173008 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.833 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.621987+0000", 2024-05-11T23:02:58.833 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.028628269000000001, 2024-05-11T23:02:58.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030912913, 2024-05-11T23:02:58.834 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.834 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.834 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.834 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.834 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.834 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.834 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173008 2024-05-11T23:02:58.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.834 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621987+0000", 2024-05-11T23:02:58.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621987+0000", 2024-05-11T23:02:58.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621987+0000", 2024-05-11T23:02:58.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:02:58.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621988+0000", 2024-05-11T23:02:58.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.266e-06 2024-05-11T23:02:58.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621992+0000", 2024-05-11T23:02:58.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0790000000000001e-06 2024-05-11T23:02:58.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.621995+0000", 2024-05-11T23:02:58.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9400000000000004e-06 2024-05-11T23:02:58.836 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.836 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.836 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.837 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173040 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.837 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.624316+0000", 2024-05-11T23:02:58.837 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.026298924000000001, 2024-05-11T23:02:58.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028641475999999999, 2024-05-11T23:02:58.837 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.837 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.837 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173040 2024-05-11T23:02:58.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.837 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624316+0000", 2024-05-11T23:02:58.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624316+0000", 2024-05-11T23:02:58.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624317+0000", 2024-05-11T23:02:58.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8500000000000002e-07 2024-05-11T23:02:58.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624318+0000", 2024-05-11T23:02:58.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.694e-06 2024-05-11T23:02:58.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624324+0000", 2024-05-11T23:02:58.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5609999999999998e-06 2024-05-11T23:02:58.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624328+0000", 2024-05-11T23:02:58.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5650000000000003e-06 2024-05-11T23:02:58.840 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.840 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.840 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.840 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173072 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.840 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.630017+0000", 2024-05-11T23:02:58.840 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.020598709, 2024-05-11T23:02:58.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022982252000000002, 2024-05-11T23:02:58.840 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.840 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.840 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.840 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.841 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.841 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173072 2024-05-11T23:02:58.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.841 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630017+0000", 2024-05-11T23:02:58.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630017+0000", 2024-05-11T23:02:58.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630017+0000", 2024-05-11T23:02:58.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1e-07 2024-05-11T23:02:58.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630018+0000", 2024-05-11T23:02:58.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3170000000000001e-06 2024-05-11T23:02:58.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630025+0000", 2024-05-11T23:02:58.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9190000000000002e-06 2024-05-11T23:02:58.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630028+0000", 2024-05-11T23:02:58.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.535e-06 2024-05-11T23:02:58.843 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.843 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.843 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.843 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173450 3.19 3.eec44899 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.843 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648224+0000", 2024-05-11T23:02:58.843 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0023909690000000002, 2024-05-11T23:02:58.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0048169600000000003, 2024-05-11T23:02:58.843 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.844 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.844 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.844 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173450 2024-05-11T23:02:58.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.844 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648224+0000", 2024-05-11T23:02:58.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648224+0000", 2024-05-11T23:02:58.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648225+0000", 2024-05-11T23:02:58.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6000000000000004e-07 2024-05-11T23:02:58.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648228+0000", 2024-05-11T23:02:58.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7389999999999999e-06 2024-05-11T23:02:58.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648235+0000", 2024-05-11T23:02:58.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1180000000000003e-06 2024-05-11T23:02:58.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648242+0000", 2024-05-11T23:02:58.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4769999999999997e-06 2024-05-11T23:02:58.846 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.846 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.846 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.846 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172775 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.846 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:02:58.846 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.046585973000000003, 2024-05-11T23:02:58.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049055794, 2024-05-11T23:02:58.846 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.847 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.847 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.847 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.847 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172775 2024-05-11T23:02:58.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.847 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:02:58.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:02:58.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604030+0000", 2024-05-11T23:02:58.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6699999999999999e-07 2024-05-11T23:02:58.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604032+0000", 2024-05-11T23:02:58.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9530000000000002e-06 2024-05-11T23:02:58.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604037+0000", 2024-05-11T23:02:58.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8590000000000001e-06 2024-05-11T23:02:58.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604042+0000", 2024-05-11T23:02:58.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8609999999999997e-06 2024-05-11T23:02:58.849 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.849 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.849 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.849 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172806 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.849 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.608107+0000", 2024-05-11T23:02:58.849 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.042508203000000001, 2024-05-11T23:02:58.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.045021782000000003, 2024-05-11T23:02:58.849 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.850 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.850 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.850 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.850 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.850 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.850 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172806 2024-05-11T23:02:58.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.850 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608107+0000", 2024-05-11T23:02:58.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608107+0000", 2024-05-11T23:02:58.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608107+0000", 2024-05-11T23:02:58.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:02:58.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608109+0000", 2024-05-11T23:02:58.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.618e-06 2024-05-11T23:02:58.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608116+0000", 2024-05-11T23:02:58.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9720000000000003e-06 2024-05-11T23:02:58.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608120+0000", 2024-05-11T23:02:58.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9859999999999996e-06 2024-05-11T23:02:58.855 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.855 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.855 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.855 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173009 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.855 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.622002+0000", 2024-05-11T23:02:58.855 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.028613669000000001, 2024-05-11T23:02:58.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031170578000000001, 2024-05-11T23:02:58.855 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.856 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.856 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.856 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.856 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.856 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.856 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173009 2024-05-11T23:02:58.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.856 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622002+0000", 2024-05-11T23:02:58.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622002+0000", 2024-05-11T23:02:58.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622002+0000", 2024-05-11T23:02:58.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4999999999999998e-07 2024-05-11T23:02:58.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622004+0000", 2024-05-11T23:02:58.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.21e-06 2024-05-11T23:02:58.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622010+0000", 2024-05-11T23:02:58.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.801e-06 2024-05-11T23:02:58.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622014+0000", 2024-05-11T23:02:58.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4410000000000003e-06 2024-05-11T23:02:58.858 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.858 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.858 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.858 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173041 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.858 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.624332+0000", 2024-05-11T23:02:58.858 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.026283092000000001, 2024-05-11T23:02:58.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028879390000000001, 2024-05-11T23:02:58.859 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.859 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.859 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.859 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.859 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.859 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173041 2024-05-11T23:02:58.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.859 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624332+0000", 2024-05-11T23:02:58.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624332+0000", 2024-05-11T23:02:58.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624332+0000", 2024-05-11T23:02:58.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.22e-07 2024-05-11T23:02:58.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624338+0000", 2024-05-11T23:02:58.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4890000000000003e-06 2024-05-11T23:02:58.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624341+0000", 2024-05-11T23:02:58.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.331e-06 2024-05-11T23:02:58.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624344+0000", 2024-05-11T23:02:58.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7259999999999998e-06 2024-05-11T23:02:58.861 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.861 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.861 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.862 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173073 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.862 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.630033+0000", 2024-05-11T23:02:58.862 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.020582613999999999, 2024-05-11T23:02:58.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023221940999999999, 2024-05-11T23:02:58.862 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.862 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.862 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.862 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.862 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.862 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.862 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173073 2024-05-11T23:02:58.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.862 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630033+0000", 2024-05-11T23:02:58.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630033+0000", 2024-05-11T23:02:58.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630033+0000", 2024-05-11T23:02:58.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.03e-07 2024-05-11T23:02:58.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630034+0000", 2024-05-11T23:02:58.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4470000000000001e-06 2024-05-11T23:02:58.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630037+0000", 2024-05-11T23:02:58.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1700000000000001e-06 2024-05-11T23:02:58.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630040+0000", 2024-05-11T23:02:58.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.491e-06 2024-05-11T23:02:58.864 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173452 3.19 3.eec44899 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648250+0000", 2024-05-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0023647189999999999, 2024-05-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.005044008, 2024-05-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.865 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173452 2024-05-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648250+0000", 2024-05-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648250+0000", 2024-05-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648251+0000", 2024-05-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7100000000000002e-07 2024-05-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648253+0000", 2024-05-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4889999999999998e-06 2024-05-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648260+0000", 2024-05-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1020000000000002e-06 2024-05-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648265+0000", 2024-05-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1189999999999996e-06 2024-05-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172776 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.046566682999999998, 2024-05-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049292269, 2024-05-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.868 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172776 2024-05-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2000000000000001e-07 2024-05-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604051+0000", 2024-05-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2879999999999999e-06 2024-05-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604057+0000", 2024-05-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604062+0000", 2024-05-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6800000000000001e-06 2024-05-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172807 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.608135+0000", 2024-05-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.042480089999999998, 2024-05-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.045246201999999999, 2024-05-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.871 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172807 2024-05-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608135+0000", 2024-05-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608135+0000", 2024-05-11T23:02:58.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608136+0000", 2024-05-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5799999999999999e-07 2024-05-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608138+0000", 2024-05-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.368e-06 2024-05-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608145+0000", 2024-05-11T23:02:58.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5180000000000001e-06 2024-05-11T23:02:58.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608150+0000", 2024-05-11T23:02:58.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3000000000000001e-06 2024-05-11T23:02:58.874 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.874 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.874 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.874 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173010 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.874 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.622020+0000", 2024-05-11T23:02:58.874 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.028595635000000001, 2024-05-11T23:02:58.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031403776000000001, 2024-05-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173010 2024-05-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622020+0000", 2024-05-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622020+0000", 2024-05-11T23:02:58.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622020+0000", 2024-05-11T23:02:58.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1800000000000002e-07 2024-05-11T23:02:58.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622022+0000", 2024-05-11T23:02:58.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7290000000000001e-06 2024-05-11T23:02:58.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622026+0000", 2024-05-11T23:02:58.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2459999999999997e-06 2024-05-11T23:02:58.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622030+0000", 2024-05-11T23:02:58.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7910000000000002e-06 2024-05-11T23:02:58.877 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.877 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.877 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.877 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173042 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.878 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.624347+0000", 2024-05-11T23:02:58.878 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.026267775, 2024-05-11T23:02:58.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02911766, 2024-05-11T23:02:58.878 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.878 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.878 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.878 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.878 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.878 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173042 2024-05-11T23:02:58.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.878 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624347+0000", 2024-05-11T23:02:58.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624347+0000", 2024-05-11T23:02:58.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624348+0000", 2024-05-11T23:02:58.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1e-07 2024-05-11T23:02:58.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624349+0000", 2024-05-11T23:02:58.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2610000000000001e-06 2024-05-11T23:02:58.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624352+0000", 2024-05-11T23:02:58.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8250000000000001e-06 2024-05-11T23:02:58.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624356+0000", 2024-05-11T23:02:58.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4950000000000002e-06 2024-05-11T23:02:58.880 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.880 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.880 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173074 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.630043+0000", 2024-05-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.020571951000000002, 2024-05-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023464062000000001, 2024-05-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173074 2024-05-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630043+0000", 2024-05-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630043+0000", 2024-05-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630043+0000", 2024-05-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-05-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630045+0000", 2024-05-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.212e-06 2024-05-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630048+0000", 2024-05-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9440000000000001e-06 2024-05-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630050+0000", 2024-05-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5330000000000001e-06 2024-05-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.883 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173454 3.19 3.eec44899 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648274+0000", 2024-05-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0023415609999999998, 2024-05-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.005273308, 2024-05-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173454 2024-05-11T23:02:58.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648274+0000", 2024-05-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648274+0000", 2024-05-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648274+0000", 2024-05-11T23:02:58.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648276+0000", 2024-05-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.255e-06 2024-05-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648283+0000", 2024-05-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7830000000000001e-06 2024-05-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648288+0000", 2024-05-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0730000000000004e-06 2024-05-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172777 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.046547377000000001, 2024-05-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049630037000000002, 2024-05-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.887 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172777 2024-05-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:02:58.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:02:58.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604075+0000", 2024-05-11T23:02:58.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.109e-06 2024-05-11T23:02:58.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604083+0000", 2024-05-11T23:02:58.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9170000000000006e-06 2024-05-11T23:02:58.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604089+0000", 2024-05-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3399999999999997e-06 2024-05-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172808 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.608159+0000", 2024-05-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.042456125999999997, 2024-05-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.045580262000000003, 2024-05-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.890 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.891 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.891 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.891 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.891 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172808 2024-05-11T23:02:58.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.891 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608159+0000", 2024-05-11T23:02:58.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.608159+0000", 2024-05-11T23:02:58.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.609917+0000", 2024-05-11T23:02:58.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00175819 2024-05-11T23:02:58.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.609922+0000", 2024-05-11T23:02:58.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3760000000000001e-06 2024-05-11T23:02:58.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.609937+0000", 2024-05-11T23:02:58.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5803000000000001e-05 2024-05-11T23:02:58.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.609945+0000", 2024-05-11T23:02:58.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2969999999999998e-06 2024-05-11T23:02:58.893 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.893 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.893 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.893 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173011 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.893 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.622072+0000", 2024-05-11T23:02:58.893 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.028543621000000002, 2024-05-11T23:02:58.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031711493, 2024-05-11T23:02:58.893 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.893 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.893 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.894 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.894 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.894 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.894 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173011 2024-05-11T23:02:58.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.894 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622072+0000", 2024-05-11T23:02:58.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622072+0000", 2024-05-11T23:02:58.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622072+0000", 2024-05-11T23:02:58.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0500000000000004e-07 2024-05-11T23:02:58.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622075+0000", 2024-05-11T23:02:58.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8899999999999999e-06 2024-05-11T23:02:58.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622082+0000", 2024-05-11T23:02:58.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.058e-06 2024-05-11T23:02:58.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622088+0000", 2024-05-11T23:02:58.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6319999999999996e-06 2024-05-11T23:02:58.896 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.896 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.896 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.896 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173043 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.896 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.624359+0000", 2024-05-11T23:02:58.896 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.026255992999999998, 2024-05-11T23:02:58.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029464200999999999, 2024-05-11T23:02:58.897 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.897 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.897 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.897 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.897 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.897 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173043 2024-05-11T23:02:58.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.897 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624359+0000", 2024-05-11T23:02:58.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624359+0000", 2024-05-11T23:02:58.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625914+0000", 2024-05-11T23:02:58.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001554997 2024-05-11T23:02:58.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625919+0000", 2024-05-11T23:02:58.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2860000000000001e-06 2024-05-11T23:02:58.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625927+0000", 2024-05-11T23:02:58.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2770000000000005e-06 2024-05-11T23:02:58.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625931+0000", 2024-05-11T23:02:58.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6210000000000003e-06 2024-05-11T23:02:58.899 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.899 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.899 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.900 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173075 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.900 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.630053+0000", 2024-05-11T23:02:58.900 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.020561958000000002, 2024-05-11T23:02:58.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023813862000000002, 2024-05-11T23:02:58.900 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.900 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.900 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.900 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173075 2024-05-11T23:02:58.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.900 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630053+0000", 2024-05-11T23:02:58.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.630053+0000", 2024-05-11T23:02:58.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631914+0000", 2024-05-11T23:02:58.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018610600000000001 2024-05-11T23:02:58.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631918+0000", 2024-05-11T23:02:58.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9600000000000002e-06 2024-05-11T23:02:58.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631931+0000", 2024-05-11T23:02:58.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2271e-05 2024-05-11T23:02:58.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631935+0000", 2024-05-11T23:02:58.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4819999999999996e-06 2024-05-11T23:02:58.902 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.902 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.902 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.903 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173456 3.19 3.eec44899 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.903 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648295+0000", 2024-05-11T23:02:58.903 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0023203999999999998, 2024-05-11T23:02:58.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0056120629999999996, 2024-05-11T23:02:58.903 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.903 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.903 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.903 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.903 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.903 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.903 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173456 2024-05-11T23:02:58.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.903 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648295+0000", 2024-05-11T23:02:58.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648295+0000", 2024-05-11T23:02:58.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648295+0000", 2024-05-11T23:02:58.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2e-07 2024-05-11T23:02:58.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648297+0000", 2024-05-11T23:02:58.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0600000000000002e-06 2024-05-11T23:02:58.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648304+0000", 2024-05-11T23:02:58.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2310000000000001e-06 2024-05-11T23:02:58.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648308+0000", 2024-05-11T23:02:58.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7759999999999997e-06 2024-05-11T23:02:58.905 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.906 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.906 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.906 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172778 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.906 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:02:58.906 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.046519286999999999, 2024-05-11T23:02:58.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049865013, 2024-05-11T23:02:58.906 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.906 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.906 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.906 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.906 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.906 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172778 2024-05-11T23:02:58.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.907 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:02:58.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:02:58.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:02:58.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5400000000000002e-07 2024-05-11T23:02:58.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604099+0000", 2024-05-11T23:02:58.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4650000000000001e-06 2024-05-11T23:02:58.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604105+0000", 2024-05-11T23:02:58.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.314e-06 2024-05-11T23:02:58.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604110+0000", 2024-05-11T23:02:58.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6310000000000002e-06 2024-05-11T23:02:58.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.909 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.909 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172809 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.909 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.609956+0000", 2024-05-11T23:02:58.909 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.040658804, 2024-05-11T23:02:58.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044048091999999997, 2024-05-11T23:02:58.909 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.909 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.910 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.910 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.910 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.910 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172809 2024-05-11T23:02:58.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.910 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.609956+0000", 2024-05-11T23:02:58.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.609956+0000", 2024-05-11T23:02:58.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.609957+0000", 2024-05-11T23:02:58.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8800000000000003e-07 2024-05-11T23:02:58.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.609960+0000", 2024-05-11T23:02:58.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7690000000000001e-06 2024-05-11T23:02:58.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.609967+0000", 2024-05-11T23:02:58.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.939e-06 2024-05-11T23:02:58.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.609972+0000", 2024-05-11T23:02:58.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2979999999999999e-06 2024-05-11T23:02:58.912 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.912 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.912 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.912 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173012 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.912 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.622096+0000", 2024-05-11T23:02:58.912 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.028519151, 2024-05-11T23:02:58.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031948663000000002, 2024-05-11T23:02:58.912 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.912 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.912 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.912 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.913 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.913 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.913 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173012 2024-05-11T23:02:58.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.913 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622096+0000", 2024-05-11T23:02:58.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622096+0000", 2024-05-11T23:02:58.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622097+0000", 2024-05-11T23:02:58.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7899999999999998e-07 2024-05-11T23:02:58.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622099+0000", 2024-05-11T23:02:58.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2390000000000001e-06 2024-05-11T23:02:58.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622105+0000", 2024-05-11T23:02:58.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1129999999999997e-06 2024-05-11T23:02:58.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622110+0000", 2024-05-11T23:02:58.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0289999999999996e-06 2024-05-11T23:02:58.915 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.915 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.915 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.915 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173044 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.915 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.625942+0000", 2024-05-11T23:02:58.915 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.024672968999999999, 2024-05-11T23:02:58.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028145238999999999, 2024-05-11T23:02:58.915 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.916 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.916 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.916 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.916 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.916 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.916 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173044 2024-05-11T23:02:58.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.916 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625942+0000", 2024-05-11T23:02:58.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625942+0000", 2024-05-11T23:02:58.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625943+0000", 2024-05-11T23:02:58.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.46e-07 2024-05-11T23:02:58.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625944+0000", 2024-05-11T23:02:58.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.618e-06 2024-05-11T23:02:58.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625948+0000", 2024-05-11T23:02:58.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7699999999999999e-06 2024-05-11T23:02:58.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625951+0000", 2024-05-11T23:02:58.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9390000000000002e-06 2024-05-11T23:02:58.918 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.918 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.918 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.918 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173076 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.918 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.631943+0000", 2024-05-11T23:02:58.918 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.018672148999999999, 2024-05-11T23:02:58.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022185038000000001, 2024-05-11T23:02:58.919 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.919 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.919 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.919 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.919 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.919 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.919 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173076 2024-05-11T23:02:58.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.919 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631943+0000", 2024-05-11T23:02:58.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631943+0000", 2024-05-11T23:02:58.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631943+0000", 2024-05-11T23:02:58.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:02:58.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631945+0000", 2024-05-11T23:02:58.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.601e-06 2024-05-11T23:02:58.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631949+0000", 2024-05-11T23:02:58.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4960000000000001e-06 2024-05-11T23:02:58.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631952+0000", 2024-05-11T23:02:58.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9360000000000001e-06 2024-05-11T23:02:58.921 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.921 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.921 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.921 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173458 3.19 3.eec44899 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.921 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648317+0000", 2024-05-11T23:02:58.922 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0022986629999999998, 2024-05-11T23:02:58.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0058537509999999999, 2024-05-11T23:02:58.922 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.922 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.922 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.922 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.922 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.922 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.922 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173458 2024-05-11T23:02:58.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.922 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648317+0000", 2024-05-11T23:02:58.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648317+0000", 2024-05-11T23:02:58.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648317+0000", 2024-05-11T23:02:58.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7399999999999998e-07 2024-05-11T23:02:58.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648319+0000", 2024-05-11T23:02:58.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1519999999999999e-06 2024-05-11T23:02:58.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648325+0000", 2024-05-11T23:02:58.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1750000000000002e-06 2024-05-11T23:02:58.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648330+0000", 2024-05-11T23:02:58.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.113e-06 2024-05-11T23:02:58.924 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.924 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.924 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.924 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172779 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.924 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:02:58.925 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.046499127000000001, 2024-05-11T23:02:58.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050097315000000003, 2024-05-11T23:02:58.925 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.925 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.925 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.925 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.925 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.925 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172779 2024-05-11T23:02:58.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.925 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:02:58.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:02:58.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:02:58.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:02:58.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604119+0000", 2024-05-11T23:02:58.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2529999999999999e-06 2024-05-11T23:02:58.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604125+0000", 2024-05-11T23:02:58.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1670000000000004e-06 2024-05-11T23:02:58.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604130+0000", 2024-05-11T23:02:58.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.702e-06 2024-05-11T23:02:58.927 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.927 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.927 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.928 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172810 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.928 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.609979+0000", 2024-05-11T23:02:58.928 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.040636026999999998, 2024-05-11T23:02:58.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044276612, 2024-05-11T23:02:58.928 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.928 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.928 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.928 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.928 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.928 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.928 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172810 2024-05-11T23:02:58.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.928 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.609979+0000", 2024-05-11T23:02:58.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.609979+0000", 2024-05-11T23:02:58.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.609980+0000", 2024-05-11T23:02:58.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.65e-07 2024-05-11T23:02:58.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.609982+0000", 2024-05-11T23:02:58.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6199999999999999e-06 2024-05-11T23:02:58.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.609989+0000", 2024-05-11T23:02:58.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2269999999999996e-06 2024-05-11T23:02:58.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.609994+0000", 2024-05-11T23:02:58.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0100000000000003e-06 2024-05-11T23:02:58.930 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.930 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.930 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.931 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173013 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.931 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.622116+0000", 2024-05-11T23:02:58.931 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.028499132, 2024-05-11T23:02:58.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032180984000000003, 2024-05-11T23:02:58.931 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.931 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.931 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.931 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.931 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.931 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.931 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173013 2024-05-11T23:02:58.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.931 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622116+0000", 2024-05-11T23:02:58.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622116+0000", 2024-05-11T23:02:58.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622116+0000", 2024-05-11T23:02:58.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5900000000000003e-07 2024-05-11T23:02:58.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622119+0000", 2024-05-11T23:02:58.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.243e-06 2024-05-11T23:02:58.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622125+0000", 2024-05-11T23:02:58.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8889999999999999e-06 2024-05-11T23:02:58.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622129+0000", 2024-05-11T23:02:58.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8579999999999999e-06 2024-05-11T23:02:58.933 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.933 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.933 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.934 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173045 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.934 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.625955+0000", 2024-05-11T23:02:58.934 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.024660257000000001, 2024-05-11T23:02:58.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028384711999999999, 2024-05-11T23:02:58.934 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.934 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.934 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.934 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.934 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.934 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.934 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173045 2024-05-11T23:02:58.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.934 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625955+0000", 2024-05-11T23:02:58.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625955+0000", 2024-05-11T23:02:58.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625955+0000", 2024-05-11T23:02:58.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:02:58.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625957+0000", 2024-05-11T23:02:58.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.579e-06 2024-05-11T23:02:58.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625960+0000", 2024-05-11T23:02:58.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2830000000000002e-06 2024-05-11T23:02:58.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625963+0000", 2024-05-11T23:02:58.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6620000000000001e-06 2024-05-11T23:02:58.936 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.936 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.936 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.937 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173077 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.937 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.631962+0000", 2024-05-11T23:02:58.937 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.018653286000000002, 2024-05-11T23:02:58.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022419649999999999, 2024-05-11T23:02:58.937 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.937 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.937 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.937 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.937 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.937 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.937 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173077 2024-05-11T23:02:58.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.937 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631962+0000", 2024-05-11T23:02:58.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631962+0000", 2024-05-11T23:02:58.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631962+0000", 2024-05-11T23:02:58.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8700000000000002e-07 2024-05-11T23:02:58.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631967+0000", 2024-05-11T23:02:58.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.989e-06 2024-05-11T23:02:58.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631971+0000", 2024-05-11T23:02:58.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1740000000000002e-06 2024-05-11T23:02:58.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631975+0000", 2024-05-11T23:02:58.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1559999999999999e-06 2024-05-11T23:02:58.940 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.940 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.940 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.940 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173460 3.19 3.eec44899 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.940 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648337+0000", 2024-05-11T23:02:58.940 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0022784699999999999, 2024-05-11T23:02:58.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0060876669999999997, 2024-05-11T23:02:58.940 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.940 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.940 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.940 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173460 2024-05-11T23:02:58.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.941 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648337+0000", 2024-05-11T23:02:58.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648337+0000", 2024-05-11T23:02:58.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648337+0000", 2024-05-11T23:02:58.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8699999999999995e-07 2024-05-11T23:02:58.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648340+0000", 2024-05-11T23:02:58.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2910000000000002e-06 2024-05-11T23:02:58.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648345+0000", 2024-05-11T23:02:58.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9340000000000003e-06 2024-05-11T23:02:58.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648350+0000", 2024-05-11T23:02:58.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5839999999999996e-06 2024-05-11T23:02:58.943 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.943 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.943 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.943 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172780 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.943 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:02:58.943 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.046478976999999998, 2024-05-11T23:02:58.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050330990999999999, 2024-05-11T23:02:58.943 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.943 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.943 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.943 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.943 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.944 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.944 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172780 2024-05-11T23:02:58.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.944 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:02:58.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:02:58.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604137+0000", 2024-05-11T23:02:58.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4400000000000001e-07 2024-05-11T23:02:58.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604139+0000", 2024-05-11T23:02:58.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3429999999999998e-06 2024-05-11T23:02:58.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604145+0000", 2024-05-11T23:02:58.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1940000000000003e-06 2024-05-11T23:02:58.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604150+0000", 2024-05-11T23:02:58.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:02:58.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.946 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.946 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172811 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.946 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.610001+0000", 2024-05-11T23:02:58.946 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.040613799999999999, 2024-05-11T23:02:58.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044508180000000001, 2024-05-11T23:02:58.946 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.946 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.946 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.947 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.947 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.947 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172811 2024-05-11T23:02:58.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.947 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610001+0000", 2024-05-11T23:02:58.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610001+0000", 2024-05-11T23:02:58.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610002+0000", 2024-05-11T23:02:58.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3999999999999997e-07 2024-05-11T23:02:58.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610004+0000", 2024-05-11T23:02:58.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1380000000000001e-06 2024-05-11T23:02:58.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610012+0000", 2024-05-11T23:02:58.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.96e-06 2024-05-11T23:02:58.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610017+0000", 2024-05-11T23:02:58.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1699999999999996e-06 2024-05-11T23:02:58.949 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.949 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.949 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.949 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173014 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.949 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.622136+0000", 2024-05-11T23:02:58.949 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.028479480000000001, 2024-05-11T23:02:58.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032413654, 2024-05-11T23:02:58.949 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.949 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.950 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.950 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.950 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.950 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173014 2024-05-11T23:02:58.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.950 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622136+0000", 2024-05-11T23:02:58.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622136+0000", 2024-05-11T23:02:58.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622136+0000", 2024-05-11T23:02:58.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5900000000000003e-07 2024-05-11T23:02:58.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622138+0000", 2024-05-11T23:02:58.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0320000000000002e-06 2024-05-11T23:02:58.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622145+0000", 2024-05-11T23:02:58.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9890000000000003e-06 2024-05-11T23:02:58.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622151+0000", 2024-05-11T23:02:58.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.93e-06 2024-05-11T23:02:58.952 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.952 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.952 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.952 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173046 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.952 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.625967+0000", 2024-05-11T23:02:58.953 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.024648473000000001, 2024-05-11T23:02:58.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028625654, 2024-05-11T23:02:58.953 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.953 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.953 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.953 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.953 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.953 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.953 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173046 2024-05-11T23:02:58.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.953 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625967+0000", 2024-05-11T23:02:58.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625967+0000", 2024-05-11T23:02:58.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625967+0000", 2024-05-11T23:02:58.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:58.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625968+0000", 2024-05-11T23:02:58.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:02:58.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625971+0000", 2024-05-11T23:02:58.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9859999999999999e-06 2024-05-11T23:02:58.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625974+0000", 2024-05-11T23:02:58.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.559e-06 2024-05-11T23:02:58.955 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.955 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.955 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.956 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173078 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.956 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.631980+0000", 2024-05-11T23:02:58.956 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.018634835999999998, 2024-05-11T23:02:58.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022652657, 2024-05-11T23:02:58.956 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.956 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.956 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.956 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.956 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.956 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173078 2024-05-11T23:02:58.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.956 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631980+0000", 2024-05-11T23:02:58.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631980+0000", 2024-05-11T23:02:58.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631981+0000", 2024-05-11T23:02:58.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9799999999999999e-07 2024-05-11T23:02:58.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631983+0000", 2024-05-11T23:02:58.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0159999999999998e-06 2024-05-11T23:02:58.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.631995+0000", 2024-05-11T23:02:58.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2614e-05 2024-05-11T23:02:58.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632001+0000", 2024-05-11T23:02:58.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4569999999999996e-06 2024-05-11T23:02:58.958 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.958 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.958 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.959 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173462 3.19 3.eec44899 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.959 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648356+0000", 2024-05-11T23:02:58.959 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.002259011, 2024-05-11T23:02:58.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.006316192, 2024-05-11T23:02:58.959 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.959 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.959 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.959 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.959 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.959 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.959 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173462 2024-05-11T23:02:58.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.959 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648356+0000", 2024-05-11T23:02:58.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648356+0000", 2024-05-11T23:02:58.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648357+0000", 2024-05-11T23:02:58.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.08e-07 2024-05-11T23:02:58.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648359+0000", 2024-05-11T23:02:58.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0609999999999998e-06 2024-05-11T23:02:58.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648365+0000", 2024-05-11T23:02:58.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1399999999999997e-06 2024-05-11T23:02:58.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648370+0000", 2024-05-11T23:02:58.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7230000000000004e-06 2024-05-11T23:02:58.961 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.961 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.961 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.962 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172781 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.962 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:02:58.962 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.046447136999999999, 2024-05-11T23:02:58.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050549898000000003, 2024-05-11T23:02:58.962 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.962 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.962 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.962 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.962 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.962 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.962 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172781 2024-05-11T23:02:58.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.963 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:02:58.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:02:58.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:02:58.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:02:58.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604171+0000", 2024-05-11T23:02:58.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.21e-06 2024-05-11T23:02:58.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604176+0000", 2024-05-11T23:02:58.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2970000000000003e-06 2024-05-11T23:02:58.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604179+0000", 2024-05-11T23:02:58.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8459999999999999e-06 2024-05-11T23:02:58.964 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.965 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.965 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.965 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172812 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.965 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.610024+0000", 2024-05-11T23:02:58.965 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.040591439, 2024-05-11T23:02:58.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044733878999999997, 2024-05-11T23:02:58.965 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.965 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.965 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.965 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.965 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.965 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.965 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172812 2024-05-11T23:02:58.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.966 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610024+0000", 2024-05-11T23:02:58.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610024+0000", 2024-05-11T23:02:58.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.966 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.966 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610024+0000", 2024-05-11T23:02:58.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9200000000000002e-07 2024-05-11T23:02:58.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610027+0000", 2024-05-11T23:02:58.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.48e-06 2024-05-11T23:02:58.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610033+0000", 2024-05-11T23:02:58.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2689999999999996e-06 2024-05-11T23:02:58.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610039+0000", 2024-05-11T23:02:58.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2689999999999996e-06 2024-05-11T23:02:58.968 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.968 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.968 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.968 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173015 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.968 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.622158+0000", 2024-05-11T23:02:58.968 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.028457040999999999, 2024-05-11T23:02:58.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032641736999999997, 2024-05-11T23:02:58.968 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.968 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.968 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.968 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.968 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.969 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.969 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173015 2024-05-11T23:02:58.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.969 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622158+0000", 2024-05-11T23:02:58.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622158+0000", 2024-05-11T23:02:58.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622159+0000", 2024-05-11T23:02:58.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4900000000000001e-07 2024-05-11T23:02:58.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622161+0000", 2024-05-11T23:02:58.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0430000000000002e-06 2024-05-11T23:02:58.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622182+0000", 2024-05-11T23:02:58.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0891e-05 2024-05-11T23:02:58.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622189+0000", 2024-05-11T23:02:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6350000000000006e-06 2024-05-11T23:02:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173047 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.625977+0000", 2024-05-11T23:02:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.024638134999999999, 2024-05-11T23:02:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.028863436999999999, 2024-05-11T23:02:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173047 2024-05-11T23:02:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625977+0000", 2024-05-11T23:02:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625977+0000", 2024-05-11T23:02:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625977+0000", 2024-05-11T23:02:58.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.04e-07 2024-05-11T23:02:58.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625978+0000", 2024-05-11T23:02:58.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0780000000000001e-06 2024-05-11T23:02:58.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625983+0000", 2024-05-11T23:02:58.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3309999999999997e-06 2024-05-11T23:02:58.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625986+0000", 2024-05-11T23:02:58.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.365e-06 2024-05-11T23:02:58.974 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.974 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.974 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.974 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173079 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.974 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.632009+0000", 2024-05-11T23:02:58.974 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.018605857999999999, 2024-05-11T23:02:58.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022874782999999999, 2024-05-11T23:02:58.974 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.974 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.974 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.975 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.975 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.975 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173079 2024-05-11T23:02:58.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.975 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632009+0000", 2024-05-11T23:02:58.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632009+0000", 2024-05-11T23:02:58.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632010+0000", 2024-05-11T23:02:58.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0299999999999999e-07 2024-05-11T23:02:58.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632012+0000", 2024-05-11T23:02:58.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0549999999999998e-06 2024-05-11T23:02:58.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632016+0000", 2024-05-11T23:02:58.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3290000000000004e-06 2024-05-11T23:02:58.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632019+0000", 2024-05-11T23:02:58.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9129999999999999e-06 2024-05-11T23:02:58.977 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.977 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.977 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.977 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173464 3.19 3.eec44899 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.977 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648376+0000", 2024-05-11T23:02:58.977 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0022392839999999998, 2024-05-11T23:02:58.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.006552759, 2024-05-11T23:02:58.977 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.977 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.977 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.977 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.978 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.978 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.978 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173464 2024-05-11T23:02:58.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.978 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648376+0000", 2024-05-11T23:02:58.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648376+0000", 2024-05-11T23:02:58.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648376+0000", 2024-05-11T23:02:58.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.01e-07 2024-05-11T23:02:58.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648378+0000", 2024-05-11T23:02:58.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9250000000000002e-06 2024-05-11T23:02:58.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648384+0000", 2024-05-11T23:02:58.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9460000000000003e-06 2024-05-11T23:02:58.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648389+0000", 2024-05-11T23:02:58.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6890000000000003e-06 2024-05-11T23:02:58.980 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.980 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.980 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.980 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172782 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.980 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:02:58.980 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.046432388999999998, 2024-05-11T23:02:58.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050792099, 2024-05-11T23:02:58.980 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.980 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.981 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.981 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.981 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.981 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.981 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172782 2024-05-11T23:02:58.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.981 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:02:58.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:02:58.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:02:58.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-05-11T23:02:58.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604184+0000", 2024-05-11T23:02:58.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:02:58.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604187+0000", 2024-05-11T23:02:58.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.044e-06 2024-05-11T23:02:58.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604190+0000", 2024-05-11T23:02:58.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6460000000000002e-06 2024-05-11T23:02:58.983 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.983 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.983 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.983 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172813 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.983 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.610051+0000", 2024-05-11T23:02:58.983 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.040563762000000003, 2024-05-11T23:02:58.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.044963754000000002, 2024-05-11T23:02:58.983 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.984 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.984 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.984 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.984 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.984 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.984 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172813 2024-05-11T23:02:58.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.984 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610051+0000", 2024-05-11T23:02:58.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610051+0000", 2024-05-11T23:02:58.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610052+0000", 2024-05-11T23:02:58.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.32e-07 2024-05-11T23:02:58.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610055+0000", 2024-05-11T23:02:58.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7010000000000001e-06 2024-05-11T23:02:58.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610063+0000", 2024-05-11T23:02:58.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.106e-06 2024-05-11T23:02:58.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610068+0000", 2024-05-11T23:02:58.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0340000000000004e-06 2024-05-11T23:02:58.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.986 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.986 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173016 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.986 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.622197+0000", 2024-05-11T23:02:58.986 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.028418035000000001, 2024-05-11T23:02:58.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.032867504999999998, 2024-05-11T23:02:58.986 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.987 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.987 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.987 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.987 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.987 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.987 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173016 2024-05-11T23:02:58.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.987 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622197+0000", 2024-05-11T23:02:58.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622197+0000", 2024-05-11T23:02:58.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622198+0000", 2024-05-11T23:02:58.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-05-11T23:02:58.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622200+0000", 2024-05-11T23:02:58.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7020000000000001e-06 2024-05-11T23:02:58.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622207+0000", 2024-05-11T23:02:58.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5370000000000003e-06 2024-05-11T23:02:58.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622212+0000", 2024-05-11T23:02:58.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9230000000000001e-06 2024-05-11T23:02:58.989 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.989 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.989 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.989 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173048 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.989 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.625989+0000", 2024-05-11T23:02:58.989 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.024625859999999999, 2024-05-11T23:02:58.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029115711999999998, 2024-05-11T23:02:58.990 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.990 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.990 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.990 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.990 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.990 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.990 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173048 2024-05-11T23:02:58.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.990 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625989+0000", 2024-05-11T23:02:58.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625989+0000", 2024-05-11T23:02:58.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625990+0000", 2024-05-11T23:02:58.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:58.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625991+0000", 2024-05-11T23:02:58.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.238e-06 2024-05-11T23:02:58.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625994+0000", 2024-05-11T23:02:58.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.05e-06 2024-05-11T23:02:58.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.625996+0000", 2024-05-11T23:02:58.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5959999999999998e-06 2024-05-11T23:02:58.992 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.992 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.992 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.993 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173080 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.993 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.632023+0000", 2024-05-11T23:02:58.993 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.018591995, 2024-05-11T23:02:58.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023124671999999999, 2024-05-11T23:02:58.993 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.993 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.993 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.993 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.993 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.993 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.993 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173080 2024-05-11T23:02:58.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.993 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632023+0000", 2024-05-11T23:02:58.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632023+0000", 2024-05-11T23:02:58.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632023+0000", 2024-05-11T23:02:58.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-05-11T23:02:58.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632025+0000", 2024-05-11T23:02:58.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2449999999999999e-06 2024-05-11T23:02:58.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632032+0000", 2024-05-11T23:02:58.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6979999999999998e-06 2024-05-11T23:02:58.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632036+0000", 2024-05-11T23:02:58.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1920000000000001e-06 2024-05-11T23:02:58.995 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.996 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.996 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.996 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173466 3.19 3.eec44899 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.996 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648395+0000", 2024-05-11T23:02:58.996 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0022206790000000001, 2024-05-11T23:02:58.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0067931709999999998, 2024-05-11T23:02:58.996 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.996 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:58.996 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:58.997 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:58.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173466 2024-05-11T23:02:58.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.997 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:58.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:58.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648395+0000", 2024-05-11T23:02:58.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:58.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648395+0000", 2024-05-11T23:02:58.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:58.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:58.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648395+0000", 2024-05-11T23:02:58.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3799999999999997e-07 2024-05-11T23:02:58.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:58.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648397+0000", 2024-05-11T23:02:58.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1789999999999998e-06 2024-05-11T23:02:58.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:58.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648407+0000", 2024-05-11T23:02:58.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7780000000000002e-06 2024-05-11T23:02:58.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:58.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648412+0000", 2024-05-11T23:02:58.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1950000000000002e-06 2024-05-11T23:02:58.999 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.999 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:58.999 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:58.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:58.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:58.999 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172783 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:58.999 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:02:58.999 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.046422070000000003, 2024-05-11T23:02:58.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051041696999999997, 2024-05-11T23:02:58.999 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:58.999 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:58.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:58.999 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.000 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.000 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.000 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172783 2024-05-11T23:02:59.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.000 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:02:59.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:02:59.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:02:59.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-05-11T23:02:59.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604195+0000", 2024-05-11T23:02:59.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.19e-06 2024-05-11T23:02:59.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604198+0000", 2024-05-11T23:02:59.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:02:59.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604200+0000", 2024-05-11T23:02:59.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4719999999999998e-06 2024-05-11T23:02:59.002 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.002 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.002 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.002 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172814 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.002 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.610077+0000", 2024-05-11T23:02:59.002 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.040538654, 2024-05-11T23:02:59.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.045198493999999999, 2024-05-11T23:02:59.002 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.002 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.003 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.003 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.003 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.003 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.003 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172814 2024-05-11T23:02:59.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.003 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610077+0000", 2024-05-11T23:02:59.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610077+0000", 2024-05-11T23:02:59.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610077+0000", 2024-05-11T23:02:59.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5400000000000002e-07 2024-05-11T23:02:59.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610079+0000", 2024-05-11T23:02:59.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4320000000000002e-06 2024-05-11T23:02:59.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610086+0000", 2024-05-11T23:02:59.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5259999999999999e-06 2024-05-11T23:02:59.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610092+0000", 2024-05-11T23:02:59.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3400000000000003e-06 2024-05-11T23:02:59.005 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.005 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.005 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.005 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173017 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.005 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.622219+0000", 2024-05-11T23:02:59.006 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.028396603999999999, 2024-05-11T23:02:59.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033100408999999997, 2024-05-11T23:02:59.006 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.006 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.006 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.006 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.006 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.006 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.006 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173017 2024-05-11T23:02:59.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.006 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622219+0000", 2024-05-11T23:02:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622219+0000", 2024-05-11T23:02:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622219+0000", 2024-05-11T23:02:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-05-11T23:02:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.007 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622226+0000", 2024-05-11T23:02:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8399999999999997e-06 2024-05-11T23:02:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622236+0000", 2024-05-11T23:02:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0241000000000001e-05 2024-05-11T23:02:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622242+0000", 2024-05-11T23:02:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:02:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173049 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.626001+0000", 2024-05-11T23:02:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.024614680999999999, 2024-05-11T23:02:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029358737999999999, 2024-05-11T23:02:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173049 2024-05-11T23:02:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626001+0000", 2024-05-11T23:02:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626001+0000", 2024-05-11T23:02:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626001+0000", 2024-05-11T23:02:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9999999999999999e-07 2024-05-11T23:02:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626003+0000", 2024-05-11T23:02:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7549999999999999e-06 2024-05-11T23:02:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626008+0000", 2024-05-11T23:02:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7620000000000001e-06 2024-05-11T23:02:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626013+0000", 2024-05-11T23:02:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2910000000000001e-06 2024-05-11T23:02:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.011 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173081 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.632039+0000", 2024-05-11T23:02:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.018575716999999999, 2024-05-11T23:02:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023360322999999999, 2024-05-11T23:02:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173081 2024-05-11T23:02:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.012 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632039+0000", 2024-05-11T23:02:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632039+0000", 2024-05-11T23:02:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632040+0000", 2024-05-11T23:02:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1e-07 2024-05-11T23:02:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632041+0000", 2024-05-11T23:02:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.42e-06 2024-05-11T23:02:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632054+0000", 2024-05-11T23:02:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2994e-05 2024-05-11T23:02:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632058+0000", 2024-05-11T23:02:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.822e-06 2024-05-11T23:02:59.014 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173385 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.644068+0000", 2024-05-11T23:02:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0065469059999999999, 2024-05-11T23:02:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011374260000000001, 2024-05-11T23:02:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173385 2024-05-11T23:02:59.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644068+0000", 2024-05-11T23:02:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644068+0000", 2024-05-11T23:02:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644069+0000", 2024-05-11T23:02:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5799999999999999e-07 2024-05-11T23:02:59.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644071+0000", 2024-05-11T23:02:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.356e-06 2024-05-11T23:02:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644077+0000", 2024-05-11T23:02:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1240000000000002e-06 2024-05-11T23:02:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644082+0000", 2024-05-11T23:02:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7400000000000004e-06 2024-05-11T23:02:59.017 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173468 3.19 3.eec44899 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648419+0000", 2024-05-11T23:02:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0021963040000000001, 2024-05-11T23:02:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0070634649999999997, 2024-05-11T23:02:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.018 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173468 2024-05-11T23:02:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648419+0000", 2024-05-11T23:02:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648419+0000", 2024-05-11T23:02:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648419+0000", 2024-05-11T23:02:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1100000000000001e-07 2024-05-11T23:02:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648421+0000", 2024-05-11T23:02:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1639999999999999e-06 2024-05-11T23:02:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648427+0000", 2024-05-11T23:02:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8440000000000003e-06 2024-05-11T23:02:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648433+0000", 2024-05-11T23:02:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9349999999999999e-06 2024-05-11T23:02:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172784 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:02:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.046412051000000003, 2024-05-11T23:02:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051325255, 2024-05-11T23:02:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.021 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172784 2024-05-11T23:02:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:02:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:02:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:02:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.03e-07 2024-05-11T23:02:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604205+0000", 2024-05-11T23:02:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2300000000000001e-06 2024-05-11T23:02:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604207+0000", 2024-05-11T23:02:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8729999999999999e-06 2024-05-11T23:02:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604210+0000", 2024-05-11T23:02:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.858e-06 2024-05-11T23:02:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172815 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.610098+0000", 2024-05-11T23:02:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.040516871000000003, 2024-05-11T23:02:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.045469974000000003, 2024-05-11T23:02:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172815 2024-05-11T23:02:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610098+0000", 2024-05-11T23:02:59.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610098+0000", 2024-05-11T23:02:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610099+0000", 2024-05-11T23:02:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4999999999999999e-07 2024-05-11T23:02:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610100+0000", 2024-05-11T23:02:59.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.629e-06 2024-05-11T23:02:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610104+0000", 2024-05-11T23:02:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4809999999999998e-06 2024-05-11T23:02:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610106+0000", 2024-05-11T23:02:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.729e-06 2024-05-11T23:02:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173018 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.622260+0000", 2024-05-11T23:02:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.028355086000000002, 2024-05-11T23:02:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033352103000000001, 2024-05-11T23:02:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173018 2024-05-11T23:02:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622260+0000", 2024-05-11T23:02:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622260+0000", 2024-05-11T23:02:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622260+0000", 2024-05-11T23:02:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.53e-07 2024-05-11T23:02:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622263+0000", 2024-05-11T23:02:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.103e-06 2024-05-11T23:02:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622267+0000", 2024-05-11T23:02:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5159999999999996e-06 2024-05-11T23:02:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622270+0000", 2024-05-11T23:02:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.095e-06 2024-05-11T23:02:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.030 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173050 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.626020+0000", 2024-05-11T23:02:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.024595342999999999, 2024-05-11T23:02:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029632316999999998, 2024-05-11T23:02:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173050 2024-05-11T23:02:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.031 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626020+0000", 2024-05-11T23:02:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626020+0000", 2024-05-11T23:02:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626020+0000", 2024-05-11T23:02:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8799999999999998e-07 2024-05-11T23:02:59.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626023+0000", 2024-05-11T23:02:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5390000000000002e-06 2024-05-11T23:02:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626029+0000", 2024-05-11T23:02:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.015e-06 2024-05-11T23:02:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626033+0000", 2024-05-11T23:02:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5589999999999999e-06 2024-05-11T23:02:59.033 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173082 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.632062+0000", 2024-05-11T23:02:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.018553198999999999, 2024-05-11T23:02:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023633491, 2024-05-11T23:02:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.034 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173082 2024-05-11T23:02:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632062+0000", 2024-05-11T23:02:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632062+0000", 2024-05-11T23:02:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632062+0000", 2024-05-11T23:02:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:02:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632064+0000", 2024-05-11T23:02:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4360000000000001e-06 2024-05-11T23:02:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632071+0000", 2024-05-11T23:02:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8700000000000003e-06 2024-05-11T23:02:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632075+0000", 2024-05-11T23:02:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7400000000000004e-06 2024-05-11T23:02:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173386 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.644095+0000", 2024-05-11T23:02:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0065202089999999999, 2024-05-11T23:02:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011640686000000001, 2024-05-11T23:02:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.037 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173386 2024-05-11T23:02:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644095+0000", 2024-05-11T23:02:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644095+0000", 2024-05-11T23:02:59.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644096+0000", 2024-05-11T23:02:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1799999999999995e-07 2024-05-11T23:02:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644098+0000", 2024-05-11T23:02:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3120000000000001e-06 2024-05-11T23:02:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644105+0000", 2024-05-11T23:02:59.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1999999999999997e-06 2024-05-11T23:02:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644110+0000", 2024-05-11T23:02:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1880000000000001e-06 2024-05-11T23:02:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173470 3.19 3.eec44899 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648440+0000", 2024-05-11T23:02:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.002175616, 2024-05-11T23:02:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0073380149999999998, 2024-05-11T23:02:59.040 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173470 2024-05-11T23:02:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648440+0000", 2024-05-11T23:02:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648440+0000", 2024-05-11T23:02:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648440+0000", 2024-05-11T23:02:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.27e-07 2024-05-11T23:02:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648442+0000", 2024-05-11T23:02:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.784e-06 2024-05-11T23:02:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648447+0000", 2024-05-11T23:02:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3539999999999999e-06 2024-05-11T23:02:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648453+0000", 2024-05-11T23:02:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7960000000000001e-06 2024-05-11T23:02:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172785 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:02:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.046399781000000001, 2024-05-11T23:02:59.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051605067999999997, 2024-05-11T23:02:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172785 2024-05-11T23:02:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:02:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:02:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604216+0000", 2024-05-11T23:02:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:02:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604218+0000", 2024-05-11T23:02:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7549999999999999e-06 2024-05-11T23:02:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604222+0000", 2024-05-11T23:02:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6700000000000002e-06 2024-05-11T23:02:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604227+0000", 2024-05-11T23:02:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4399999999999998e-06 2024-05-11T23:02:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172816 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.610110+0000", 2024-05-11T23:02:59.046 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.040505337000000002, 2024-05-11T23:02:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.045753083, 2024-05-11T23:02:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172816 2024-05-11T23:02:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610110+0000", 2024-05-11T23:02:59.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610110+0000", 2024-05-11T23:02:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610110+0000", 2024-05-11T23:02:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610111+0000", 2024-05-11T23:02:59.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.204e-06 2024-05-11T23:02:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610116+0000", 2024-05-11T23:02:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8849999999999998e-06 2024-05-11T23:02:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610119+0000", 2024-05-11T23:02:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.576e-06 2024-05-11T23:02:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173019 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.622274+0000", 2024-05-11T23:02:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.028340995000000001, 2024-05-11T23:02:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033627972999999999, 2024-05-11T23:02:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.050 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173019 2024-05-11T23:02:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622274+0000", 2024-05-11T23:02:59.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622274+0000", 2024-05-11T23:02:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622274+0000", 2024-05-11T23:02:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.16e-07 2024-05-11T23:02:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622276+0000", 2024-05-11T23:02:59.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.356e-06 2024-05-11T23:02:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622279+0000", 2024-05-11T23:02:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0920000000000002e-06 2024-05-11T23:02:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622282+0000", 2024-05-11T23:02:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3790000000000002e-06 2024-05-11T23:02:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.056 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173051 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.626038+0000", 2024-05-11T23:02:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.024577076999999999, 2024-05-11T23:02:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029917909999999999, 2024-05-11T23:02:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173051 2024-05-11T23:02:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.057 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626038+0000", 2024-05-11T23:02:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626038+0000", 2024-05-11T23:02:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626038+0000", 2024-05-11T23:02:59.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:02:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626040+0000", 2024-05-11T23:02:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.522e-06 2024-05-11T23:02:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626044+0000", 2024-05-11T23:02:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2030000000000002e-06 2024-05-11T23:02:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626047+0000", 2024-05-11T23:02:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7580000000000001e-06 2024-05-11T23:02:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173083 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.632079+0000", 2024-05-11T23:02:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.018536450999999999, 2024-05-11T23:02:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023918198000000002, 2024-05-11T23:02:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.060 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173083 2024-05-11T23:02:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632079+0000", 2024-05-11T23:02:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632079+0000", 2024-05-11T23:02:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632079+0000", 2024-05-11T23:02:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:02:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632080+0000", 2024-05-11T23:02:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2020000000000001e-06 2024-05-11T23:02:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632084+0000", 2024-05-11T23:02:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0670000000000002e-06 2024-05-11T23:02:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632087+0000", 2024-05-11T23:02:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8219999999999998e-06 2024-05-11T23:02:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173387 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.644118+0000", 2024-05-11T23:02:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0064975800000000002, 2024-05-11T23:02:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011923362999999999, 2024-05-11T23:02:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.063 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173387 2024-05-11T23:02:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644118+0000", 2024-05-11T23:02:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644118+0000", 2024-05-11T23:02:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644118+0000", 2024-05-11T23:02:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0299999999999999e-07 2024-05-11T23:02:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644120+0000", 2024-05-11T23:02:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1749999999999999e-06 2024-05-11T23:02:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644126+0000", 2024-05-11T23:02:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7690000000000002e-06 2024-05-11T23:02:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644131+0000", 2024-05-11T23:02:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5700000000000003e-06 2024-05-11T23:02:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173472 3.19 3.eec44899 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648459+0000", 2024-05-11T23:02:59.066 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.002156439, 2024-05-11T23:02:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0076224420000000001, 2024-05-11T23:02:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173472 2024-05-11T23:02:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.067 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648459+0000", 2024-05-11T23:02:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648459+0000", 2024-05-11T23:02:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648459+0000", 2024-05-11T23:02:59.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6e-07 2024-05-11T23:02:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648464+0000", 2024-05-11T23:02:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8860000000000003e-06 2024-05-11T23:02:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648470+0000", 2024-05-11T23:02:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1530000000000002e-06 2024-05-11T23:02:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648476+0000", 2024-05-11T23:02:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8089999999999998e-06 2024-05-11T23:02:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172786 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:02:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.046382726999999999, 2024-05-11T23:02:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051894435000000003, 2024-05-11T23:02:59.070 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172786 2024-05-11T23:02:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:02:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:02:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604233+0000", 2024-05-11T23:02:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6899999999999999e-07 2024-05-11T23:02:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604235+0000", 2024-05-11T23:02:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8369999999999999e-06 2024-05-11T23:02:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604240+0000", 2024-05-11T23:02:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0329999999999999e-06 2024-05-11T23:02:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604244+0000", 2024-05-11T23:02:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9859999999999996e-06 2024-05-11T23:02:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.073 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172817 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.610122+0000", 2024-05-11T23:02:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.040493128000000003, 2024-05-11T23:02:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.046047829999999998, 2024-05-11T23:02:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.074 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172817 2024-05-11T23:02:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610122+0000", 2024-05-11T23:02:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610122+0000", 2024-05-11T23:02:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610122+0000", 2024-05-11T23:02:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:02:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610124+0000", 2024-05-11T23:02:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2139999999999999e-06 2024-05-11T23:02:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610127+0000", 2024-05-11T23:02:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0469999999999998e-06 2024-05-11T23:02:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610131+0000", 2024-05-11T23:02:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5519999999999998e-06 2024-05-11T23:02:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173020 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.622286+0000", 2024-05-11T23:02:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.028329300000000002, 2024-05-11T23:02:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033924019, 2024-05-11T23:02:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.077 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173020 2024-05-11T23:02:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622286+0000", 2024-05-11T23:02:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622286+0000", 2024-05-11T23:02:59.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622286+0000", 2024-05-11T23:02:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1399999999999998e-07 2024-05-11T23:02:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622288+0000", 2024-05-11T23:02:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.282e-06 2024-05-11T23:02:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622291+0000", 2024-05-11T23:02:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.994e-06 2024-05-11T23:02:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622293+0000", 2024-05-11T23:02:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6529999999999998e-06 2024-05-11T23:02:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173052 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.626050+0000", 2024-05-11T23:02:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.024565268000000001, 2024-05-11T23:02:59.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030202320000000001, 2024-05-11T23:02:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173052 2024-05-11T23:02:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626050+0000", 2024-05-11T23:02:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626050+0000", 2024-05-11T23:02:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626050+0000", 2024-05-11T23:02:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:02:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626052+0000", 2024-05-11T23:02:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3799999999999999e-06 2024-05-11T23:02:59.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626057+0000", 2024-05-11T23:02:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4480000000000002e-06 2024-05-11T23:02:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626060+0000", 2024-05-11T23:02:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3189999999999999e-06 2024-05-11T23:02:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.083 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173084 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.632090+0000", 2024-05-11T23:02:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.01852496, 2024-05-11T23:02:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024201848000000001, 2024-05-11T23:02:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173084 2024-05-11T23:02:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.084 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632090+0000", 2024-05-11T23:02:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632090+0000", 2024-05-11T23:02:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632090+0000", 2024-05-11T23:02:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:02:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632092+0000", 2024-05-11T23:02:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.189e-06 2024-05-11T23:02:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632095+0000", 2024-05-11T23:02:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.932e-06 2024-05-11T23:02:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632097+0000", 2024-05-11T23:02:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.678e-06 2024-05-11T23:02:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173388 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.644137+0000", 2024-05-11T23:02:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0064786109999999996, 2024-05-11T23:02:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012198881, 2024-05-11T23:02:59.087 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173388 2024-05-11T23:02:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644137+0000", 2024-05-11T23:02:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644137+0000", 2024-05-11T23:02:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644137+0000", 2024-05-11T23:02:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2300000000000001e-07 2024-05-11T23:02:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644139+0000", 2024-05-11T23:02:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3700000000000002e-06 2024-05-11T23:02:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644146+0000", 2024-05-11T23:02:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0680000000000002e-06 2024-05-11T23:02:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644150+0000", 2024-05-11T23:02:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8579999999999999e-06 2024-05-11T23:02:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173560 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648484+0000", 2024-05-11T23:02:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.002130846, 2024-05-11T23:02:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0078910309999999997, 2024-05-11T23:02:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173560 2024-05-11T23:02:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.091 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648484+0000", 2024-05-11T23:02:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648484+0000", 2024-05-11T23:02:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648485+0000", 2024-05-11T23:02:59.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8599999999999999e-07 2024-05-11T23:02:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648487+0000", 2024-05-11T23:02:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3659999999999999e-06 2024-05-11T23:02:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648493+0000", 2024-05-11T23:02:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3269999999999997e-06 2024-05-11T23:02:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648500+0000", 2024-05-11T23:02:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1940000000000003e-06 2024-05-11T23:02:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172818 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.610135+0000", 2024-05-11T23:02:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.040479742999999999, 2024-05-11T23:02:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.046291598000000003, 2024-05-11T23:02:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.094 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172818 2024-05-11T23:02:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610135+0000", 2024-05-11T23:02:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610135+0000", 2024-05-11T23:02:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610136+0000", 2024-05-11T23:02:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-05-11T23:02:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610137+0000", 2024-05-11T23:02:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1650000000000001e-06 2024-05-11T23:02:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610140+0000", 2024-05-11T23:02:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9409999999999999e-06 2024-05-11T23:02:59.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610142+0000", 2024-05-11T23:02:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.672e-06 2024-05-11T23:02:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172989 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.620078+0000", 2024-05-11T23:02:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.030536964, 2024-05-11T23:02:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.036388986999999998, 2024-05-11T23:02:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.097 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172989 2024-05-11T23:02:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620078+0000", 2024-05-11T23:02:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620078+0000", 2024-05-11T23:02:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620079+0000", 2024-05-11T23:02:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8999999999999997e-07 2024-05-11T23:02:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620081+0000", 2024-05-11T23:02:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2460000000000002e-06 2024-05-11T23:02:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620087+0000", 2024-05-11T23:02:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:02:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620093+0000", 2024-05-11T23:02:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1329999999999998e-06 2024-05-11T23:02:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173021 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.622296+0000", 2024-05-11T23:02:59.100 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.028318962, 2024-05-11T23:02:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034214082, 2024-05-11T23:02:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173021 2024-05-11T23:02:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622296+0000", 2024-05-11T23:02:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622296+0000", 2024-05-11T23:02:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622296+0000", 2024-05-11T23:02:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1799999999999999e-07 2024-05-11T23:02:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622298+0000", 2024-05-11T23:02:59.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.212e-06 2024-05-11T23:02:59.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622301+0000", 2024-05-11T23:02:59.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9579999999999999e-06 2024-05-11T23:02:59.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622303+0000", 2024-05-11T23:02:59.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6769999999999999e-06 2024-05-11T23:02:59.103 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.103 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.103 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173053 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.626064+0000", 2024-05-11T23:02:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.024551679, 2024-05-11T23:02:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030486571000000001, 2024-05-11T23:02:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173053 2024-05-11T23:02:59.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626064+0000", 2024-05-11T23:02:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626064+0000", 2024-05-11T23:02:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626064+0000", 2024-05-11T23:02:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.05e-07 2024-05-11T23:02:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626068+0000", 2024-05-11T23:02:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6650000000000002e-06 2024-05-11T23:02:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626073+0000", 2024-05-11T23:02:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.391e-06 2024-05-11T23:02:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626076+0000", 2024-05-11T23:02:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1439999999999999e-06 2024-05-11T23:02:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173085 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.632101+0000", 2024-05-11T23:02:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.018514626999999999, 2024-05-11T23:02:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024519419000000001, 2024-05-11T23:02:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.107 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173085 2024-05-11T23:02:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632101+0000", 2024-05-11T23:02:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.632101+0000", 2024-05-11T23:02:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633914+0000", 2024-05-11T23:02:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018132739999999999 2024-05-11T23:02:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633917+0000", 2024-05-11T23:02:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.32e-06 2024-05-11T23:02:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633924+0000", 2024-05-11T23:02:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0790000000000003e-06 2024-05-11T23:02:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633928+0000", 2024-05-11T23:02:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0409999999999999e-06 2024-05-11T23:02:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.110 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173389 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.644157+0000", 2024-05-11T23:02:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0064585069999999996, 2024-05-11T23:02:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012504235000000001, 2024-05-11T23:02:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173389 2024-05-11T23:02:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644157+0000", 2024-05-11T23:02:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644157+0000", 2024-05-11T23:02:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644157+0000", 2024-05-11T23:02:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6300000000000001e-07 2024-05-11T23:02:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644159+0000", 2024-05-11T23:02:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8819999999999999e-06 2024-05-11T23:02:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644165+0000", 2024-05-11T23:02:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6509999999999998e-06 2024-05-11T23:02:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644169+0000", 2024-05-11T23:02:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5199999999999999e-06 2024-05-11T23:02:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173562 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648506+0000", 2024-05-11T23:02:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0021091069999999998, 2024-05-11T23:02:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0081986000000000003, 2024-05-11T23:02:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173562 2024-05-11T23:02:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648506+0000", 2024-05-11T23:02:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648506+0000", 2024-05-11T23:02:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648506+0000", 2024-05-11T23:02:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7e-07 2024-05-11T23:02:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648509+0000", 2024-05-11T23:02:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0530000000000001e-06 2024-05-11T23:02:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648515+0000", 2024-05-11T23:02:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9730000000000002e-06 2024-05-11T23:02:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648521+0000", 2024-05-11T23:02:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5590000000000002e-06 2024-05-11T23:02:59.116 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172787 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:02:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.046366380999999998, 2024-05-11T23:02:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.052499399000000002, 2024-05-11T23:02:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.117 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172787 2024-05-11T23:02:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:02:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:02:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605916+0000", 2024-05-11T23:02:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00166752 2024-05-11T23:02:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605921+0000", 2024-05-11T23:02:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0669999999999999e-06 2024-05-11T23:02:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605934+0000", 2024-05-11T23:02:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2271e-05 2024-05-11T23:02:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605940+0000", 2024-05-11T23:02:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6880000000000002e-06 2024-05-11T23:02:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172819 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.610146+0000", 2024-05-11T23:02:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.040469567999999997, 2024-05-11T23:02:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.046642555000000002, 2024-05-11T23:02:59.120 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172819 2024-05-11T23:02:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610146+0000", 2024-05-11T23:02:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.610146+0000", 2024-05-11T23:02:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611914+0000", 2024-05-11T23:02:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00176838 2024-05-11T23:02:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611918+0000", 2024-05-11T23:02:59.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4790000000000001e-06 2024-05-11T23:02:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611931+0000", 2024-05-11T23:02:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3137e-05 2024-05-11T23:02:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611935+0000", 2024-05-11T23:02:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7180000000000004e-06 2024-05-11T23:02:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172990 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.620100+0000", 2024-05-11T23:02:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.030515054999999999, 2024-05-11T23:02:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.036729143999999998, 2024-05-11T23:02:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172990 2024-05-11T23:02:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620100+0000", 2024-05-11T23:02:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620100+0000", 2024-05-11T23:02:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620101+0000", 2024-05-11T23:02:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.82e-07 2024-05-11T23:02:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620103+0000", 2024-05-11T23:02:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2280000000000001e-06 2024-05-11T23:02:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620109+0000", 2024-05-11T23:02:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3520000000000003e-06 2024-05-11T23:02:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620114+0000", 2024-05-11T23:02:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8799999999999999e-06 2024-05-11T23:02:59.126 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173022 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.622307+0000", 2024-05-11T23:02:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.028308691, 2024-05-11T23:02:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034562320000000001, 2024-05-11T23:02:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.127 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173022 2024-05-11T23:02:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622307+0000", 2024-05-11T23:02:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622307+0000", 2024-05-11T23:02:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622307+0000", 2024-05-11T23:02:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:02:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622308+0000", 2024-05-11T23:02:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3480000000000001e-06 2024-05-11T23:02:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622311+0000", 2024-05-11T23:02:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:02:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622315+0000", 2024-05-11T23:02:59.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.788e-06 2024-05-11T23:02:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173054 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.626081+0000", 2024-05-11T23:02:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.024534046, 2024-05-11T23:02:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030833335, 2024-05-11T23:02:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.130 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173054 2024-05-11T23:02:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626081+0000", 2024-05-11T23:02:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626081+0000", 2024-05-11T23:02:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626081+0000", 2024-05-11T23:02:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-05-11T23:02:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626083+0000", 2024-05-11T23:02:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3659999999999999e-06 2024-05-11T23:02:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626086+0000", 2024-05-11T23:02:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0089999999999999e-06 2024-05-11T23:02:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626088+0000", 2024-05-11T23:02:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6060000000000001e-06 2024-05-11T23:02:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173086 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.633943+0000", 2024-05-11T23:02:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.016672196, 2024-05-11T23:02:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023012268999999998, 2024-05-11T23:02:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.133 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173086 2024-05-11T23:02:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633943+0000", 2024-05-11T23:02:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633943+0000", 2024-05-11T23:02:59.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633943+0000", 2024-05-11T23:02:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8999999999999998e-07 2024-05-11T23:02:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633945+0000", 2024-05-11T23:02:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.691e-06 2024-05-11T23:02:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633949+0000", 2024-05-11T23:02:59.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7290000000000002e-06 2024-05-11T23:02:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633959+0000", 2024-05-11T23:02:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0319000000000001e-05 2024-05-11T23:02:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173390 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.644175+0000", 2024-05-11T23:02:59.136 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.006440192, 2024-05-11T23:02:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01281991, 2024-05-11T23:02:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173390 2024-05-11T23:02:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644175+0000", 2024-05-11T23:02:59.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644175+0000", 2024-05-11T23:02:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644175+0000", 2024-05-11T23:02:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0999999999999999e-07 2024-05-11T23:02:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644178+0000", 2024-05-11T23:02:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.159e-06 2024-05-11T23:02:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644184+0000", 2024-05-11T23:02:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5799999999999997e-06 2024-05-11T23:02:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644189+0000", 2024-05-11T23:02:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9080000000000003e-06 2024-05-11T23:02:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173564 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648528+0000", 2024-05-11T23:02:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.002086755, 2024-05-11T23:02:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0085069069999999993, 2024-05-11T23:02:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173564 2024-05-11T23:02:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648528+0000", 2024-05-11T23:02:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648528+0000", 2024-05-11T23:02:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648530+0000", 2024-05-11T23:02:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.344e-06 2024-05-11T23:02:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648532+0000", 2024-05-11T23:02:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2759999999999999e-06 2024-05-11T23:02:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648538+0000", 2024-05-11T23:02:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9630000000000003e-06 2024-05-11T23:02:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648543+0000", 2024-05-11T23:02:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9860000000000002e-06 2024-05-11T23:02:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172788 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.605950+0000", 2024-05-11T23:02:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.044664879999999997, 2024-05-11T23:02:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051129584999999998, 2024-05-11T23:02:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172788 2024-05-11T23:02:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605950+0000", 2024-05-11T23:02:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605950+0000", 2024-05-11T23:02:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605951+0000", 2024-05-11T23:02:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7700000000000005e-07 2024-05-11T23:02:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605953+0000", 2024-05-11T23:02:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0700000000000001e-06 2024-05-11T23:02:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605958+0000", 2024-05-11T23:02:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4739999999999996e-06 2024-05-11T23:02:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605962+0000", 2024-05-11T23:02:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1750000000000001e-06 2024-05-11T23:02:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172820 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.611944+0000", 2024-05-11T23:02:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.038671059000000001, 2024-05-11T23:02:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.045176204999999997, 2024-05-11T23:02:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172820 2024-05-11T23:02:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.146 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611944+0000", 2024-05-11T23:02:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611944+0000", 2024-05-11T23:02:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611944+0000", 2024-05-11T23:02:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:02:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611946+0000", 2024-05-11T23:02:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7320000000000001e-06 2024-05-11T23:02:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611950+0000", 2024-05-11T23:02:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5810000000000001e-06 2024-05-11T23:02:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611953+0000", 2024-05-11T23:02:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.63e-06 2024-05-11T23:02:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172991 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.620123+0000", 2024-05-11T23:02:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.030492103999999999, 2024-05-11T23:02:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.037037833999999999, 2024-05-11T23:02:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172991 2024-05-11T23:02:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620123+0000", 2024-05-11T23:02:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620123+0000", 2024-05-11T23:02:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620124+0000", 2024-05-11T23:02:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9499999999999998e-07 2024-05-11T23:02:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620126+0000", 2024-05-11T23:02:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3879999999999998e-06 2024-05-11T23:02:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620132+0000", 2024-05-11T23:02:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6059999999999999e-06 2024-05-11T23:02:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620137+0000", 2024-05-11T23:02:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9830000000000004e-06 2024-05-11T23:02:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173023 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.622321+0000", 2024-05-11T23:02:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.028294646999999999, 2024-05-11T23:02:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034893696000000002, 2024-05-11T23:02:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173023 2024-05-11T23:02:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622321+0000", 2024-05-11T23:02:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.622321+0000", 2024-05-11T23:02:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.623916+0000", 2024-05-11T23:02:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015950420000000001 2024-05-11T23:02:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.623921+0000", 2024-05-11T23:02:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1819999999999997e-06 2024-05-11T23:02:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.623933+0000", 2024-05-11T23:02:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2086000000000001e-05 2024-05-11T23:02:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.623941+0000", 2024-05-11T23:02:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6930000000000007e-06 2024-05-11T23:02:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173055 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.626092+0000", 2024-05-11T23:02:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.024523692999999999, 2024-05-11T23:02:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031177742000000001, 2024-05-11T23:02:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173055 2024-05-11T23:02:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626092+0000", 2024-05-11T23:02:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.626092+0000", 2024-05-11T23:02:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627916+0000", 2024-05-11T23:02:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001824696 2024-05-11T23:02:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627920+0000", 2024-05-11T23:02:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1060000000000002e-06 2024-05-11T23:02:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627931+0000", 2024-05-11T23:02:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.044e-05 2024-05-11T23:02:59.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627937+0000", 2024-05-11T23:02:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7290000000000003e-06 2024-05-11T23:02:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173087 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.633969+0000", 2024-05-11T23:02:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.016646186, 2024-05-11T23:02:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023342372, 2024-05-11T23:02:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.159 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173087 2024-05-11T23:02:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633969+0000", 2024-05-11T23:02:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633969+0000", 2024-05-11T23:02:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633969+0000", 2024-05-11T23:02:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3200000000000001e-07 2024-05-11T23:02:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633971+0000", 2024-05-11T23:02:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4750000000000001e-06 2024-05-11T23:02:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633975+0000", 2024-05-11T23:02:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0409999999999999e-06 2024-05-11T23:02:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633978+0000", 2024-05-11T23:02:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.853e-06 2024-05-11T23:02:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173391 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.162 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.644197+0000", 2024-05-11T23:02:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0064186099999999999, 2024-05-11T23:02:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013155234, 2024-05-11T23:02:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173391 2024-05-11T23:02:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644197+0000", 2024-05-11T23:02:59.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644197+0000", 2024-05-11T23:02:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644197+0000", 2024-05-11T23:02:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.46e-07 2024-05-11T23:02:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644199+0000", 2024-05-11T23:02:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1710000000000001e-06 2024-05-11T23:02:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644205+0000", 2024-05-11T23:02:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7289999999999997e-06 2024-05-11T23:02:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644210+0000", 2024-05-11T23:02:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6129999999999997e-06 2024-05-11T23:02:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173566 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648549+0000", 2024-05-11T23:02:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0020659139999999999, 2024-05-11T23:02:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0088430090000000006, 2024-05-11T23:02:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173566 2024-05-11T23:02:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648549+0000", 2024-05-11T23:02:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648549+0000", 2024-05-11T23:02:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648550+0000", 2024-05-11T23:02:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4700000000000002e-07 2024-05-11T23:02:59.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648552+0000", 2024-05-11T23:02:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1110000000000002e-06 2024-05-11T23:02:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648558+0000", 2024-05-11T23:02:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7819999999999999e-06 2024-05-11T23:02:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648563+0000", 2024-05-11T23:02:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0610000000000003e-06 2024-05-11T23:02:59.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172789 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.605966+0000", 2024-05-11T23:02:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.044649174, 2024-05-11T23:02:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051470498000000003, 2024-05-11T23:02:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172789 2024-05-11T23:02:59.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605966+0000", 2024-05-11T23:02:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605966+0000", 2024-05-11T23:02:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605966+0000", 2024-05-11T23:02:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5600000000000002e-07 2024-05-11T23:02:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605971+0000", 2024-05-11T23:02:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9169999999999997e-06 2024-05-11T23:02:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605975+0000", 2024-05-11T23:02:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6179999999999999e-06 2024-05-11T23:02:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605978+0000", 2024-05-11T23:02:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.14e-06 2024-05-11T23:02:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172821 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.611977+0000", 2024-05-11T23:02:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.03863813, 2024-05-11T23:02:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.045500211999999998, 2024-05-11T23:02:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.172 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172821 2024-05-11T23:02:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611977+0000", 2024-05-11T23:02:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611977+0000", 2024-05-11T23:02:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611977+0000", 2024-05-11T23:02:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0699999999999998e-07 2024-05-11T23:02:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611979+0000", 2024-05-11T23:02:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7230000000000001e-06 2024-05-11T23:02:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611983+0000", 2024-05-11T23:02:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7529999999999999e-06 2024-05-11T23:02:59.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611986+0000", 2024-05-11T23:02:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1530000000000001e-06 2024-05-11T23:02:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.175 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172992 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.620144+0000", 2024-05-11T23:02:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.030470995000000001, 2024-05-11T23:02:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.037373551999999997, 2024-05-11T23:02:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172992 2024-05-11T23:02:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620144+0000", 2024-05-11T23:02:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620144+0000", 2024-05-11T23:02:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620145+0000", 2024-05-11T23:02:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1100000000000001e-07 2024-05-11T23:02:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620147+0000", 2024-05-11T23:02:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0789999999999999e-06 2024-05-11T23:02:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620152+0000", 2024-05-11T23:02:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6890000000000001e-06 2024-05-11T23:02:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620157+0000", 2024-05-11T23:02:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.87e-06 2024-05-11T23:02:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173024 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.623954+0000", 2024-05-11T23:02:59.179 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.026661067, 2024-05-11T23:02:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033603347999999998, 2024-05-11T23:02:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173024 2024-05-11T23:02:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.623954+0000", 2024-05-11T23:02:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.623954+0000", 2024-05-11T23:02:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.623955+0000", 2024-05-11T23:02:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0900000000000001e-07 2024-05-11T23:02:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.623958+0000", 2024-05-11T23:02:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0149999999999999e-06 2024-05-11T23:02:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.623965+0000", 2024-05-11T23:02:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4669999999999998e-06 2024-05-11T23:02:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.623971+0000", 2024-05-11T23:02:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1840000000000004e-06 2024-05-11T23:02:59.182 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173056 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.627948+0000", 2024-05-11T23:02:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.022667322, 2024-05-11T23:02:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029649325000000001, 2024-05-11T23:02:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.183 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173056 2024-05-11T23:02:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627948+0000", 2024-05-11T23:02:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627948+0000", 2024-05-11T23:02:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627948+0000", 2024-05-11T23:02:59.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.89e-07 2024-05-11T23:02:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627950+0000", 2024-05-11T23:02:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9719999999999999e-06 2024-05-11T23:02:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627954+0000", 2024-05-11T23:02:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-06 2024-05-11T23:02:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627958+0000", 2024-05-11T23:02:59.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2370000000000001e-06 2024-05-11T23:02:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173088 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.633981+0000", 2024-05-11T23:02:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.016633954999999999, 2024-05-11T23:02:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023655787000000001, 2024-05-11T23:02:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.186 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173088 2024-05-11T23:02:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633981+0000", 2024-05-11T23:02:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633981+0000", 2024-05-11T23:02:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633981+0000", 2024-05-11T23:02:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:02:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633983+0000", 2024-05-11T23:02:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.235e-06 2024-05-11T23:02:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633991+0000", 2024-05-11T23:02:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2160000000000006e-06 2024-05-11T23:02:59.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633995+0000", 2024-05-11T23:02:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0160000000000002e-06 2024-05-11T23:02:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173392 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.644217+0000", 2024-05-11T23:02:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0063983310000000002, 2024-05-11T23:02:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013459113, 2024-05-11T23:02:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173392 2024-05-11T23:02:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644217+0000", 2024-05-11T23:02:59.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644217+0000", 2024-05-11T23:02:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644217+0000", 2024-05-11T23:02:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5900000000000003e-07 2024-05-11T23:02:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644223+0000", 2024-05-11T23:02:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7300000000000002e-06 2024-05-11T23:02:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644230+0000", 2024-05-11T23:02:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8430000000000004e-06 2024-05-11T23:02:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644235+0000", 2024-05-11T23:02:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1270000000000002e-06 2024-05-11T23:02:59.192 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:174737 3.a e101/97 3:5083f432:::rbd_data.111ebabb4604.0000000000000010:head v 101'462, mlcod=101'460)", 2024-05-11T23:02:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.649348+0000", 2024-05-11T23:02:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0012671690000000001, 2024-05-11T23:02:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0083692999999999997, 2024-05-11T23:02:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:02:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.649348+0000", 2024-05-11T23:02:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.649348+0000", 2024-05-11T23:02:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.649349+0000", 2024-05-11T23:02:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3300000000000001e-07 2024-05-11T23:02:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.649356+0000", 2024-05-11T23:02:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6930000000000001e-06 2024-05-11T23:02:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.649365+0000", 2024-05-11T23:02:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4800000000000007e-06 2024-05-11T23:02:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.649371+0000", 2024-05-11T23:02:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2840000000000003e-06 2024-05-11T23:02:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:02:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.649579+0000", 2024-05-11T23:02:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00020779500000000001 2024-05-11T23:02:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:02:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.649602+0000", 2024-05-11T23:02:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2442999999999998e-05 2024-05-11T23:02:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172790 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.196 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.605983+0000", 2024-05-11T23:02:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.044632234999999999, 2024-05-11T23:02:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051780842000000001, 2024-05-11T23:02:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172790 2024-05-11T23:02:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605983+0000", 2024-05-11T23:02:59.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605983+0000", 2024-05-11T23:02:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605983+0000", 2024-05-11T23:02:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4499999999999998e-07 2024-05-11T23:02:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605984+0000", 2024-05-11T23:02:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.257e-06 2024-05-11T23:02:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605988+0000", 2024-05-11T23:02:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1240000000000001e-06 2024-05-11T23:02:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605992+0000", 2024-05-11T23:02:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0790000000000002e-06 2024-05-11T23:02:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172822 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.611990+0000", 2024-05-11T23:02:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.038625242999999997, 2024-05-11T23:02:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.045813996000000003, 2024-05-11T23:02:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172822 2024-05-11T23:02:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611990+0000", 2024-05-11T23:02:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611990+0000", 2024-05-11T23:02:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611990+0000", 2024-05-11T23:02:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.34e-07 2024-05-11T23:02:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611992+0000", 2024-05-11T23:02:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3030000000000001e-06 2024-05-11T23:02:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611995+0000", 2024-05-11T23:02:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7390000000000001e-06 2024-05-11T23:02:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.611998+0000", 2024-05-11T23:02:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6900000000000001e-06 2024-05-11T23:02:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.202 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172993 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.620163+0000", 2024-05-11T23:02:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.030452112999999999, 2024-05-11T23:02:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.037680861000000003, 2024-05-11T23:02:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172993 2024-05-11T23:02:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.203 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620163+0000", 2024-05-11T23:02:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620163+0000", 2024-05-11T23:02:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620163+0000", 2024-05-11T23:02:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:02:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620170+0000", 2024-05-11T23:02:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8759999999999999e-06 2024-05-11T23:02:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620177+0000", 2024-05-11T23:02:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4529999999999999e-06 2024-05-11T23:02:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620182+0000", 2024-05-11T23:02:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.451e-06 2024-05-11T23:02:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173025 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.623981+0000", 2024-05-11T23:02:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.026634233, 2024-05-11T23:02:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.033903020999999998, 2024-05-11T23:02:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.206 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173025 2024-05-11T23:02:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.623981+0000", 2024-05-11T23:02:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.623981+0000", 2024-05-11T23:02:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.623982+0000", 2024-05-11T23:02:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7599999999999997e-07 2024-05-11T23:02:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.623984+0000", 2024-05-11T23:02:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.582e-06 2024-05-11T23:02:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.623991+0000", 2024-05-11T23:02:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7830000000000001e-06 2024-05-11T23:02:59.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.623996+0000", 2024-05-11T23:02:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4269999999999999e-06 2024-05-11T23:02:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173057 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.627961+0000", 2024-05-11T23:02:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.022653741000000002, 2024-05-11T23:02:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.029973110000000001, 2024-05-11T23:02:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173057 2024-05-11T23:02:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627961+0000", 2024-05-11T23:02:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627961+0000", 2024-05-11T23:02:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627962+0000", 2024-05-11T23:02:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.03e-07 2024-05-11T23:02:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627963+0000", 2024-05-11T23:02:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.299e-06 2024-05-11T23:02:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627967+0000", 2024-05-11T23:02:59.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4000000000000002e-06 2024-05-11T23:02:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627970+0000", 2024-05-11T23:02:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.649e-06 2024-05-11T23:02:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173089 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.633999+0000", 2024-05-11T23:02:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.016616638, 2024-05-11T23:02:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023976776000000002, 2024-05-11T23:02:59.212 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.213 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.213 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173089 2024-05-11T23:02:59.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.213 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633999+0000", 2024-05-11T23:02:59.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633999+0000", 2024-05-11T23:02:59.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.633999+0000", 2024-05-11T23:02:59.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:02:59.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.634003+0000", 2024-05-11T23:02:59.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4789999999999998e-06 2024-05-11T23:02:59.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.634008+0000", 2024-05-11T23:02:59.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7129999999999996e-06 2024-05-11T23:02:59.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.634011+0000", 2024-05-11T23:02:59.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7709999999999998e-06 2024-05-11T23:02:59.215 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.215 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.215 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.215 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173393 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.215 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.644242+0000", 2024-05-11T23:02:59.215 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0063735249999999997, 2024-05-11T23:02:59.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013773427, 2024-05-11T23:02:59.216 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.216 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.216 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.216 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.216 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.216 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.216 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173393 2024-05-11T23:02:59.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.216 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644242+0000", 2024-05-11T23:02:59.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644242+0000", 2024-05-11T23:02:59.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644242+0000", 2024-05-11T23:02:59.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4199999999999996e-07 2024-05-11T23:02:59.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644244+0000", 2024-05-11T23:02:59.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2620000000000001e-06 2024-05-11T23:02:59.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644250+0000", 2024-05-11T23:02:59.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7359999999999998e-06 2024-05-11T23:02:59.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644255+0000", 2024-05-11T23:02:59.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7010000000000004e-06 2024-05-11T23:02:59.218 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.218 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.218 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.218 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173568 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.219 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.648569+0000", 2024-05-11T23:02:59.219 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0020461569999999998, 2024-05-11T23:02:59.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0094876200000000004, 2024-05-11T23:02:59.219 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.219 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.219 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.219 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.219 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.219 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173568 2024-05-11T23:02:59.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.219 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648569+0000", 2024-05-11T23:02:59.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.648569+0000", 2024-05-11T23:02:59.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.649932+0000", 2024-05-11T23:02:59.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001362703 2024-05-11T23:02:59.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.649936+0000", 2024-05-11T23:02:59.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3830000000000002e-06 2024-05-11T23:02:59.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.649947+0000", 2024-05-11T23:02:59.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0448999999999999e-05 2024-05-11T23:02:59.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.649953+0000", 2024-05-11T23:02:59.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8279999999999997e-06 2024-05-11T23:02:59.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.221 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.221 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172791 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.221 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.605995+0000", 2024-05-11T23:02:59.222 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.044619890000000002, 2024-05-11T23:02:59.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.052104954000000002, 2024-05-11T23:02:59.222 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.222 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172791 2024-05-11T23:02:59.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.222 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605995+0000", 2024-05-11T23:02:59.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605995+0000", 2024-05-11T23:02:59.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605996+0000", 2024-05-11T23:02:59.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1500000000000001e-07 2024-05-11T23:02:59.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605997+0000", 2024-05-11T23:02:59.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2550000000000001e-06 2024-05-11T23:02:59.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606000+0000", 2024-05-11T23:02:59.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9280000000000002e-06 2024-05-11T23:02:59.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606002+0000", 2024-05-11T23:02:59.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6819999999999999e-06 2024-05-11T23:02:59.224 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.224 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.224 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.224 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172823 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.225 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.612001+0000", 2024-05-11T23:02:59.225 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.038614018999999999, 2024-05-11T23:02:59.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.046138889000000002, 2024-05-11T23:02:59.225 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.225 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.225 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.225 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.225 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172823 2024-05-11T23:02:59.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.225 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612001+0000", 2024-05-11T23:02:59.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612001+0000", 2024-05-11T23:02:59.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612001+0000", 2024-05-11T23:02:59.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:02:59.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612003+0000", 2024-05-11T23:02:59.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2020000000000001e-06 2024-05-11T23:02:59.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612006+0000", 2024-05-11T23:02:59.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1389999999999999e-06 2024-05-11T23:02:59.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612008+0000", 2024-05-11T23:02:59.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:02:59.227 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.227 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.227 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.228 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172994 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.228 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.620190+0000", 2024-05-11T23:02:59.228 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.030425522999999999, 2024-05-11T23:02:59.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.037989948000000003, 2024-05-11T23:02:59.228 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.228 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.228 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.228 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.228 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.228 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172994 2024-05-11T23:02:59.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.229 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620190+0000", 2024-05-11T23:02:59.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620190+0000", 2024-05-11T23:02:59.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620190+0000", 2024-05-11T23:02:59.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9100000000000004e-07 2024-05-11T23:02:59.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620193+0000", 2024-05-11T23:02:59.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3659999999999999e-06 2024-05-11T23:02:59.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620199+0000", 2024-05-11T23:02:59.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9499999999999998e-06 2024-05-11T23:02:59.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620204+0000", 2024-05-11T23:02:59.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0139999999999998e-06 2024-05-11T23:02:59.231 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.231 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.231 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.231 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173026 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.231 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.624004+0000", 2024-05-11T23:02:59.232 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.026610984000000001, 2024-05-11T23:02:59.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034215923000000002, 2024-05-11T23:02:59.232 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.232 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.232 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.232 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.232 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.232 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173026 2024-05-11T23:02:59.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.232 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624004+0000", 2024-05-11T23:02:59.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624004+0000", 2024-05-11T23:02:59.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624005+0000", 2024-05-11T23:02:59.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3600000000000004e-07 2024-05-11T23:02:59.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624007+0000", 2024-05-11T23:02:59.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3120000000000001e-06 2024-05-11T23:02:59.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624013+0000", 2024-05-11T23:02:59.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.117e-06 2024-05-11T23:02:59.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624018+0000", 2024-05-11T23:02:59.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8019999999999999e-06 2024-05-11T23:02:59.235 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.235 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.235 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.235 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173058 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.235 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.627973+0000", 2024-05-11T23:02:59.235 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.022641720000000001, 2024-05-11T23:02:59.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030286363, 2024-05-11T23:02:59.235 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.235 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.235 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.235 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.236 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.236 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173058 2024-05-11T23:02:59.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.236 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627973+0000", 2024-05-11T23:02:59.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627973+0000", 2024-05-11T23:02:59.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627974+0000", 2024-05-11T23:02:59.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.29e-07 2024-05-11T23:02:59.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627975+0000", 2024-05-11T23:02:59.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2669999999999999e-06 2024-05-11T23:02:59.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627982+0000", 2024-05-11T23:02:59.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9399999999999996e-06 2024-05-11T23:02:59.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627990+0000", 2024-05-11T23:02:59.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1329999999999999e-06 2024-05-11T23:02:59.238 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.238 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.238 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.238 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173090 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.238 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.634014+0000", 2024-05-11T23:02:59.238 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.016601004999999999, 2024-05-11T23:02:59.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024286208, 2024-05-11T23:02:59.238 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.238 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.238 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.239 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.239 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.239 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173090 2024-05-11T23:02:59.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.239 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.634014+0000", 2024-05-11T23:02:59.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.634014+0000", 2024-05-11T23:02:59.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.634014+0000", 2024-05-11T23:02:59.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3699999999999999e-07 2024-05-11T23:02:59.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.634016+0000", 2024-05-11T23:02:59.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:02:59.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.634019+0000", 2024-05-11T23:02:59.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9620000000000002e-06 2024-05-11T23:02:59.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.634022+0000", 2024-05-11T23:02:59.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.118e-06 2024-05-11T23:02:59.241 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.241 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.241 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.241 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173394 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.241 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.644261+0000", 2024-05-11T23:02:59.241 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.006354696, 2024-05-11T23:02:59.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014079992, 2024-05-11T23:02:59.241 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.242 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.242 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.242 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.242 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173394 2024-05-11T23:02:59.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.242 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644261+0000", 2024-05-11T23:02:59.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644261+0000", 2024-05-11T23:02:59.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644261+0000", 2024-05-11T23:02:59.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5600000000000001e-07 2024-05-11T23:02:59.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644263+0000", 2024-05-11T23:02:59.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0470000000000001e-06 2024-05-11T23:02:59.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644268+0000", 2024-05-11T23:02:59.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.259e-06 2024-05-11T23:02:59.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644273+0000", 2024-05-11T23:02:59.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4989999999999996e-06 2024-05-11T23:02:59.244 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.244 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.244 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.244 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173570 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.244 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.650053+0000", 2024-05-11T23:02:59.244 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.00056200300000000005, 2024-05-11T23:02:59.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0083265430000000005, 2024-05-11T23:02:59.245 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.245 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.245 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.245 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.245 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.245 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173570 2024-05-11T23:02:59.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.245 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650053+0000", 2024-05-11T23:02:59.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650053+0000", 2024-05-11T23:02:59.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650054+0000", 2024-05-11T23:02:59.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5700000000000002e-07 2024-05-11T23:02:59.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650057+0000", 2024-05-11T23:02:59.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.943e-06 2024-05-11T23:02:59.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650064+0000", 2024-05-11T23:02:59.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8870000000000003e-06 2024-05-11T23:02:59.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650069+0000", 2024-05-11T23:02:59.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7060000000000001e-06 2024-05-11T23:02:59.247 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.247 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.247 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.247 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172792 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.247 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606006+0000", 2024-05-11T23:02:59.248 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.044609543000000002, 2024-05-11T23:02:59.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.052417759000000001, 2024-05-11T23:02:59.248 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.248 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.248 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.248 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.248 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.248 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.248 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172792 2024-05-11T23:02:59.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.248 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606006+0000", 2024-05-11T23:02:59.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606006+0000", 2024-05-11T23:02:59.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606006+0000", 2024-05-11T23:02:59.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0100000000000001e-07 2024-05-11T23:02:59.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606007+0000", 2024-05-11T23:02:59.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.203e-06 2024-05-11T23:02:59.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606010+0000", 2024-05-11T23:02:59.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9170000000000002e-06 2024-05-11T23:02:59.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606013+0000", 2024-05-11T23:02:59.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5830000000000001e-06 2024-05-11T23:02:59.250 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.250 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.250 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.251 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172824 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.251 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.612011+0000", 2024-05-11T23:02:59.251 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.038603833999999997, 2024-05-11T23:02:59.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.046451830999999999, 2024-05-11T23:02:59.251 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.251 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.251 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.251 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.251 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.251 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.251 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172824 2024-05-11T23:02:59.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.251 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612011+0000", 2024-05-11T23:02:59.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612011+0000", 2024-05-11T23:02:59.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612012+0000", 2024-05-11T23:02:59.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.05e-07 2024-05-11T23:02:59.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612033+0000", 2024-05-11T23:02:59.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1900999999999999e-05 2024-05-11T23:02:59.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612038+0000", 2024-05-11T23:02:59.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5369999999999999e-06 2024-05-11T23:02:59.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612041+0000", 2024-05-11T23:02:59.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.162e-06 2024-05-11T23:02:59.253 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.253 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.254 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.254 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172995 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.254 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.620224+0000", 2024-05-11T23:02:59.254 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.030391560000000001, 2024-05-11T23:02:59.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.038280481999999998, 2024-05-11T23:02:59.254 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.254 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.254 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.254 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.254 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.254 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.254 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172995 2024-05-11T23:02:59.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.255 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620224+0000", 2024-05-11T23:02:59.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620224+0000", 2024-05-11T23:02:59.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620224+0000", 2024-05-11T23:02:59.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7299999999999996e-07 2024-05-11T23:02:59.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620227+0000", 2024-05-11T23:02:59.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6819999999999999e-06 2024-05-11T23:02:59.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620234+0000", 2024-05-11T23:02:59.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8299999999999998e-06 2024-05-11T23:02:59.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620239+0000", 2024-05-11T23:02:59.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2310000000000004e-06 2024-05-11T23:02:59.257 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.257 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.257 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.257 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173027 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.257 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.624024+0000", 2024-05-11T23:02:59.257 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.026590747000000001, 2024-05-11T23:02:59.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034519543, 2024-05-11T23:02:59.257 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.257 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.257 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.257 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.257 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.257 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.258 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173027 2024-05-11T23:02:59.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.258 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624024+0000", 2024-05-11T23:02:59.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624024+0000", 2024-05-11T23:02:59.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624025+0000", 2024-05-11T23:02:59.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7300000000000002e-07 2024-05-11T23:02:59.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624027+0000", 2024-05-11T23:02:59.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2170000000000001e-06 2024-05-11T23:02:59.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624033+0000", 2024-05-11T23:02:59.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3289999999999999e-06 2024-05-11T23:02:59.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624038+0000", 2024-05-11T23:02:59.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6689999999999997e-06 2024-05-11T23:02:59.260 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.260 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.260 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.260 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173059 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.260 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.627994+0000", 2024-05-11T23:02:59.260 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.022620814999999999, 2024-05-11T23:02:59.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030590111, 2024-05-11T23:02:59.261 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.261 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.261 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.261 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.261 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.261 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.261 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173059 2024-05-11T23:02:59.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.261 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627994+0000", 2024-05-11T23:02:59.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627994+0000", 2024-05-11T23:02:59.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627995+0000", 2024-05-11T23:02:59.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.16e-07 2024-05-11T23:02:59.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.627996+0000", 2024-05-11T23:02:59.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4789999999999999e-06 2024-05-11T23:02:59.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628000+0000", 2024-05-11T23:02:59.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3889999999999998e-06 2024-05-11T23:02:59.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628003+0000", 2024-05-11T23:02:59.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8880000000000001e-06 2024-05-11T23:02:59.264 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.264 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.264 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.264 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173395 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.264 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.644281+0000", 2024-05-11T23:02:59.264 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.006334152, 2024-05-11T23:02:59.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014343257999999999, 2024-05-11T23:02:59.264 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.264 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.265 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.265 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.265 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.265 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.265 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173395 2024-05-11T23:02:59.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.265 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644281+0000", 2024-05-11T23:02:59.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644281+0000", 2024-05-11T23:02:59.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644282+0000", 2024-05-11T23:02:59.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.39e-07 2024-05-11T23:02:59.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644284+0000", 2024-05-11T23:02:59.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1689999999999999e-06 2024-05-11T23:02:59.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644290+0000", 2024-05-11T23:02:59.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2920000000000001e-06 2024-05-11T23:02:59.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644295+0000", 2024-05-11T23:02:59.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.916e-06 2024-05-11T23:02:59.267 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.267 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.267 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.268 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173572 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.268 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.650136+0000", 2024-05-11T23:02:59.268 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.00047946099999999998, 2024-05-11T23:02:59.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0085287769999999995, 2024-05-11T23:02:59.268 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.268 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.268 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.268 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.268 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.268 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173572 2024-05-11T23:02:59.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.268 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650136+0000", 2024-05-11T23:02:59.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650136+0000", 2024-05-11T23:02:59.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650136+0000", 2024-05-11T23:02:59.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3900000000000005e-07 2024-05-11T23:02:59.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650140+0000", 2024-05-11T23:02:59.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2260000000000001e-06 2024-05-11T23:02:59.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650148+0000", 2024-05-11T23:02:59.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5730000000000008e-06 2024-05-11T23:02:59.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650273+0000", 2024-05-11T23:02:59.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000124664 2024-05-11T23:02:59.270 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.270 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.270 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.271 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172793 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.271 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:02:59.271 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.044599534000000003, 2024-05-11T23:02:59.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.052691577000000003, 2024-05-11T23:02:59.271 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.271 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.271 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.271 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.271 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.271 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.271 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172793 2024-05-11T23:02:59.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.271 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:02:59.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:02:59.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:02:59.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3300000000000001e-07 2024-05-11T23:02:59.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606017+0000", 2024-05-11T23:02:59.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4920000000000001e-06 2024-05-11T23:02:59.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606020+0000", 2024-05-11T23:02:59.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:02:59.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606023+0000", 2024-05-11T23:02:59.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6680000000000001e-06 2024-05-11T23:02:59.273 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.274 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.274 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.274 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172825 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.274 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.612046+0000", 2024-05-11T23:02:59.274 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.038569099000000003, 2024-05-11T23:02:59.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.046701113000000002, 2024-05-11T23:02:59.274 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.274 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.274 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.274 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.274 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.274 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.275 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172825 2024-05-11T23:02:59.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.275 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612046+0000", 2024-05-11T23:02:59.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612046+0000", 2024-05-11T23:02:59.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612046+0000", 2024-05-11T23:02:59.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0100000000000001e-07 2024-05-11T23:02:59.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612048+0000", 2024-05-11T23:02:59.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.48e-06 2024-05-11T23:02:59.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612052+0000", 2024-05-11T23:02:59.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8310000000000003e-06 2024-05-11T23:02:59.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612054+0000", 2024-05-11T23:02:59.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5500000000000001e-06 2024-05-11T23:02:59.277 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.277 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.277 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.277 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172996 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.277 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.620247+0000", 2024-05-11T23:02:59.277 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.030368437000000002, 2024-05-11T23:02:59.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.038540360000000003, 2024-05-11T23:02:59.277 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.277 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.277 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.278 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.278 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.278 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172996 2024-05-11T23:02:59.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.278 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620247+0000", 2024-05-11T23:02:59.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.278 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.278 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620247+0000", 2024-05-11T23:02:59.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620247+0000", 2024-05-11T23:02:59.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6000000000000004e-07 2024-05-11T23:02:59.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620250+0000", 2024-05-11T23:02:59.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.734e-06 2024-05-11T23:02:59.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620256+0000", 2024-05-11T23:02:59.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3160000000000002e-06 2024-05-11T23:02:59.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620261+0000", 2024-05-11T23:02:59.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0300000000000001e-06 2024-05-11T23:02:59.280 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.280 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.280 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.280 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173028 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.281 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.624045+0000", 2024-05-11T23:02:59.281 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.026570548999999999, 2024-05-11T23:02:59.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.034782800000000003, 2024-05-11T23:02:59.281 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.281 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.281 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.281 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.281 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173028 2024-05-11T23:02:59.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.281 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624045+0000", 2024-05-11T23:02:59.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.282 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624045+0000", 2024-05-11T23:02:59.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.282 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624045+0000", 2024-05-11T23:02:59.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6100000000000002e-07 2024-05-11T23:02:59.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624048+0000", 2024-05-11T23:02:59.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5210000000000001e-06 2024-05-11T23:02:59.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624054+0000", 2024-05-11T23:02:59.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:02:59.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624059+0000", 2024-05-11T23:02:59.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.135e-06 2024-05-11T23:02:59.283 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.283 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.284 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.284 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.284 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173060 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.284 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.628007+0000", 2024-05-11T23:02:59.284 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.022608568999999999, 2024-05-11T23:02:59.284 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.030860821, 2024-05-11T23:02:59.284 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.284 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.284 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.284 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.284 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.284 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173060 2024-05-11T23:02:59.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.285 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628007+0000", 2024-05-11T23:02:59.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628007+0000", 2024-05-11T23:02:59.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.285 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.285 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628007+0000", 2024-05-11T23:02:59.285 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0599999999999999e-07 2024-05-11T23:02:59.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.286 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.286 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628008+0000", 2024-05-11T23:02:59.286 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.424e-06 2024-05-11T23:02:59.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.286 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.286 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628012+0000", 2024-05-11T23:02:59.286 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0280000000000002e-06 2024-05-11T23:02:59.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.286 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.286 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.286 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628015+0000", 2024-05-11T23:02:59.286 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6970000000000002e-06 2024-05-11T23:02:59.287 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.287 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.287 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.287 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173396 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.287 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.644301+0000", 2024-05-11T23:02:59.287 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0063143660000000001, 2024-05-11T23:02:59.287 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014606513, 2024-05-11T23:02:59.287 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.287 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.287 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.287 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.287 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.287 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.287 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173396 2024-05-11T23:02:59.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.288 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644301+0000", 2024-05-11T23:02:59.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644301+0000", 2024-05-11T23:02:59.288 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.288 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.288 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644301+0000", 2024-05-11T23:02:59.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-05-11T23:02:59.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644303+0000", 2024-05-11T23:02:59.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.165e-06 2024-05-11T23:02:59.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644311+0000", 2024-05-11T23:02:59.289 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4229999999999999e-06 2024-05-11T23:02:59.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.289 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.289 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644316+0000", 2024-05-11T23:02:59.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8709999999999996e-06 2024-05-11T23:02:59.290 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.290 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.290 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.290 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.290 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173574 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.290 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.650284+0000", 2024-05-11T23:02:59.290 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.00033155700000000001, 2024-05-11T23:02:59.290 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0086688129999999992, 2024-05-11T23:02:59.290 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.290 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.290 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.290 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.290 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.291 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.291 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173574 2024-05-11T23:02:59.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.291 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.291 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.291 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650284+0000", 2024-05-11T23:02:59.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.291 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.291 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650284+0000", 2024-05-11T23:02:59.291 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.291 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.291 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650284+0000", 2024-05-11T23:02:59.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4000000000000001e-07 2024-05-11T23:02:59.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650317+0000", 2024-05-11T23:02:59.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2787000000000003e-05 2024-05-11T23:02:59.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650327+0000", 2024-05-11T23:02:59.292 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0373999999999999e-05 2024-05-11T23:02:59.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.292 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.292 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.292 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650403+0000", 2024-05-11T23:02:59.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6037000000000004e-05 2024-05-11T23:02:59.293 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.293 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.293 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.293 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.293 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172794 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.293 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606044+0000", 2024-05-11T23:02:59.293 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.044570734000000001, 2024-05-11T23:02:59.293 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.052951443000000001, 2024-05-11T23:02:59.293 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.293 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.293 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.293 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.293 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.294 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.294 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172794 2024-05-11T23:02:59.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.294 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606044+0000", 2024-05-11T23:02:59.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.294 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.294 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606044+0000", 2024-05-11T23:02:59.294 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.295 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.295 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606045+0000", 2024-05-11T23:02:59.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1899999999999998e-07 2024-05-11T23:02:59.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.295 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.295 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606047+0000", 2024-05-11T23:02:59.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.812e-06 2024-05-11T23:02:59.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.295 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.295 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606051+0000", 2024-05-11T23:02:59.295 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8410000000000002e-06 2024-05-11T23:02:59.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.296 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.296 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606053+0000", 2024-05-11T23:02:59.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8260000000000001e-06 2024-05-11T23:02:59.296 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.296 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.296 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.296 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172826 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.296 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.612058+0000", 2024-05-11T23:02:59.296 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.038557670000000002, 2024-05-11T23:02:59.296 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.046980964, 2024-05-11T23:02:59.296 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.297 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.297 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.297 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.297 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.297 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172826 2024-05-11T23:02:59.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.297 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.297 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612058+0000", 2024-05-11T23:02:59.297 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.297 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.297 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612058+0000", 2024-05-11T23:02:59.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612058+0000", 2024-05-11T23:02:59.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1e-07 2024-05-11T23:02:59.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.298 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.298 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612059+0000", 2024-05-11T23:02:59.298 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1799999999999999e-06 2024-05-11T23:02:59.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.298 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.299 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.299 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612067+0000", 2024-05-11T23:02:59.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9149999999999996e-06 2024-05-11T23:02:59.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.299 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.299 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612070+0000", 2024-05-11T23:02:59.299 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.988e-06 2024-05-11T23:02:59.299 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.299 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.299 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.300 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172997 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.300 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.620268+0000", 2024-05-11T23:02:59.300 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.030347004, 2024-05-11T23:02:59.300 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.038811677000000003, 2024-05-11T23:02:59.300 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.300 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.300 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.300 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.300 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.300 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.300 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172997 2024-05-11T23:02:59.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.300 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620268+0000", 2024-05-11T23:02:59.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.301 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620268+0000", 2024-05-11T23:02:59.301 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.301 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.301 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.302 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620269+0000", 2024-05-11T23:02:59.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:02:59.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.302 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.302 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620271+0000", 2024-05-11T23:02:59.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2879999999999999e-06 2024-05-11T23:02:59.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.302 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.302 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620277+0000", 2024-05-11T23:02:59.302 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7660000000000004e-06 2024-05-11T23:02:59.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.302 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.303 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.303 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620282+0000", 2024-05-11T23:02:59.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0300000000000001e-06 2024-05-11T23:02:59.303 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.303 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.303 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.303 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173029 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.303 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.624080+0000", 2024-05-11T23:02:59.303 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.026534767000000001, 2024-05-11T23:02:59.303 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.035041931999999998, 2024-05-11T23:02:59.303 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.303 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.303 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.304 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.304 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.304 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173029 2024-05-11T23:02:59.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.304 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.304 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624080+0000", 2024-05-11T23:02:59.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.304 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.304 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.304 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624080+0000", 2024-05-11T23:02:59.304 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624081+0000", 2024-05-11T23:02:59.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1799999999999995e-07 2024-05-11T23:02:59.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624089+0000", 2024-05-11T23:02:59.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0050000000000004e-06 2024-05-11T23:02:59.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.305 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.305 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.305 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624097+0000", 2024-05-11T23:02:59.305 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8660000000000006e-06 2024-05-11T23:02:59.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.306 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.306 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624104+0000", 2024-05-11T23:02:59.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.883e-06 2024-05-11T23:02:59.306 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.306 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.306 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.306 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173061 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.306 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.628018+0000", 2024-05-11T23:02:59.306 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.022597065999999999, 2024-05-11T23:02:59.306 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031144272000000001, 2024-05-11T23:02:59.306 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.307 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.307 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.307 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.307 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173061 2024-05-11T23:02:59.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.307 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.307 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.307 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628018+0000", 2024-05-11T23:02:59.307 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.307 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.307 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.308 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628018+0000", 2024-05-11T23:02:59.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.308 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.308 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628018+0000", 2024-05-11T23:02:59.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1500000000000001e-07 2024-05-11T23:02:59.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.308 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.308 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628020+0000", 2024-05-11T23:02:59.308 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2589999999999999e-06 2024-05-11T23:02:59.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.308 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.308 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.309 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628023+0000", 2024-05-11T23:02:59.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9909999999999998e-06 2024-05-11T23:02:59.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.309 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.309 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628025+0000", 2024-05-11T23:02:59.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6879999999999999e-06 2024-05-11T23:02:59.309 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.309 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.309 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.309 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.309 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173397 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.309 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.644321+0000", 2024-05-11T23:02:59.309 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0062937699999999997, 2024-05-11T23:02:59.309 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014881910999999999, 2024-05-11T23:02:59.310 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.310 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.310 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.310 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.310 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.310 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.310 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173397 2024-05-11T23:02:59.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.310 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.310 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.310 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644321+0000", 2024-05-11T23:02:59.310 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.310 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.311 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644321+0000", 2024-05-11T23:02:59.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.311 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644322+0000", 2024-05-11T23:02:59.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.27e-07 2024-05-11T23:02:59.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.311 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644324+0000", 2024-05-11T23:02:59.311 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0229999999999999e-06 2024-05-11T23:02:59.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.311 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.311 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644330+0000", 2024-05-11T23:02:59.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2489999999999998e-06 2024-05-11T23:02:59.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.312 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.312 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644337+0000", 2024-05-11T23:02:59.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0770000000000002e-06 2024-05-11T23:02:59.312 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.312 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.312 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.312 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.312 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172759 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.312 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.650646+0000", 2024-05-11T23:02:59.312 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4294967295.9999695, 2024-05-11T23:02:59.312 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0085975689999999994, 2024-05-11T23:02:59.313 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.313 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.313 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.313 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.313 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.313 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.313 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172759 2024-05-11T23:02:59.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.313 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.313 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.313 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650646+0000", 2024-05-11T23:02:59.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650646+0000", 2024-05-11T23:02:59.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650647+0000", 2024-05-11T23:02:59.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0559999999999999e-06 2024-05-11T23:02:59.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650652+0000", 2024-05-11T23:02:59.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6319999999999996e-06 2024-05-11T23:02:59.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650665+0000", 2024-05-11T23:02:59.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2785e-05 2024-05-11T23:02:59.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650674+0000", 2024-05-11T23:02:59.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2840000000000006e-06 2024-05-11T23:02:59.315 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.315 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.315 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.315 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172795 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.315 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:02:59.316 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.044558107, 2024-05-11T23:02:59.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.053229679000000002, 2024-05-11T23:02:59.316 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.316 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.316 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.316 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.316 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.316 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.316 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172795 2024-05-11T23:02:59.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.316 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:02:59.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:02:59.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:02:59.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9499999999999998e-07 2024-05-11T23:02:59.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606059+0000", 2024-05-11T23:02:59.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-05-11T23:02:59.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606062+0000", 2024-05-11T23:02:59.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.977e-06 2024-05-11T23:02:59.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606064+0000", 2024-05-11T23:02:59.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7410000000000001e-06 2024-05-11T23:02:59.318 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.318 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.318 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.318 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172827 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.319 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.612073+0000", 2024-05-11T23:02:59.319 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.038541771000000002, 2024-05-11T23:02:59.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.047252909000000003, 2024-05-11T23:02:59.319 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.319 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.319 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.319 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.319 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.319 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.319 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172827 2024-05-11T23:02:59.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.319 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612073+0000", 2024-05-11T23:02:59.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.320 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612073+0000", 2024-05-11T23:02:59.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.320 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612074+0000", 2024-05-11T23:02:59.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3900000000000001e-07 2024-05-11T23:02:59.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.320 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612085+0000", 2024-05-11T23:02:59.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1484999999999999e-05 2024-05-11T23:02:59.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612089+0000", 2024-05-11T23:02:59.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1690000000000002e-06 2024-05-11T23:02:59.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.612094+0000", 2024-05-11T23:02:59.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2869999999999998e-06 2024-05-11T23:02:59.321 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.321 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.321 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.321 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172998 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.322 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.620289+0000", 2024-05-11T23:02:59.322 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.030326394999999999, 2024-05-11T23:02:59.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.039079882000000003, 2024-05-11T23:02:59.322 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.322 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.322 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.322 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.322 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.322 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.322 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172998 2024-05-11T23:02:59.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.322 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620289+0000", 2024-05-11T23:02:59.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620289+0000", 2024-05-11T23:02:59.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620289+0000", 2024-05-11T23:02:59.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.27e-07 2024-05-11T23:02:59.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620292+0000", 2024-05-11T23:02:59.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.435e-06 2024-05-11T23:02:59.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620298+0000", 2024-05-11T23:02:59.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3600000000000001e-06 2024-05-11T23:02:59.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.620303+0000", 2024-05-11T23:02:59.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:02:59.324 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.324 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.324 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.325 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173030 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.325 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.624113+0000", 2024-05-11T23:02:59.325 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.026502466999999998, 2024-05-11T23:02:59.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.035296108, 2024-05-11T23:02:59.325 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.325 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.325 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.325 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.325 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173030 2024-05-11T23:02:59.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.325 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624113+0000", 2024-05-11T23:02:59.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624113+0000", 2024-05-11T23:02:59.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624113+0000", 2024-05-11T23:02:59.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7700000000000004e-07 2024-05-11T23:02:59.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624116+0000", 2024-05-11T23:02:59.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4830000000000002e-06 2024-05-11T23:02:59.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624123+0000", 2024-05-11T23:02:59.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7599999999999997e-06 2024-05-11T23:02:59.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.624128+0000", 2024-05-11T23:02:59.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3600000000000004e-06 2024-05-11T23:02:59.327 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.328 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.328 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.328 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173062 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.328 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.628028+0000", 2024-05-11T23:02:59.328 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.022586736, 2024-05-11T23:02:59.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.031420807000000002, 2024-05-11T23:02:59.328 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.328 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.328 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.328 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.328 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.328 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.328 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173062 2024-05-11T23:02:59.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.329 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628028+0000", 2024-05-11T23:02:59.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628028+0000", 2024-05-11T23:02:59.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628029+0000", 2024-05-11T23:02:59.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.03e-07 2024-05-11T23:02:59.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628030+0000", 2024-05-11T23:02:59.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.161e-06 2024-05-11T23:02:59.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628033+0000", 2024-05-11T23:02:59.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9780000000000001e-06 2024-05-11T23:02:59.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.628037+0000", 2024-05-11T23:02:59.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5009999999999998e-06 2024-05-11T23:02:59.331 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.331 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.331 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.331 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:173398 3.1a 3.5ed7497a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.331 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.644344+0000", 2024-05-11T23:02:59.331 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.006270945, 2024-05-11T23:02:59.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015145662000000001, 2024-05-11T23:02:59.331 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.331 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.331 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.331 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.331 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.331 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.332 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 173398 2024-05-11T23:02:59.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.332 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644344+0000", 2024-05-11T23:02:59.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644344+0000", 2024-05-11T23:02:59.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644345+0000", 2024-05-11T23:02:59.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7100000000000002e-07 2024-05-11T23:02:59.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644347+0000", 2024-05-11T23:02:59.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3329999999999999e-06 2024-05-11T23:02:59.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644353+0000", 2024-05-11T23:02:59.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0140000000000004e-06 2024-05-11T23:02:59.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.644360+0000", 2024-05-11T23:02:59.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4949999999999996e-06 2024-05-11T23:02:59.334 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.334 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.334 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.334 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172760 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:02:59.334 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.650688+0000", 2024-05-11T23:02:59.334 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4294967295.9999275, 2024-05-11T23:02:59.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0088438389999999992, 2024-05-11T23:02:59.334 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:02:59.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:02:59.334 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:02:59.334 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:02:59.334 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:02:59.334 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:02:59.335 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172760 2024-05-11T23:02:59.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.335 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:02:59.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:02:59.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650688+0000", 2024-05-11T23:02:59.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:02:59.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650688+0000", 2024-05-11T23:02:59.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:02:59.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:02:59.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650688+0000", 2024-05-11T23:02:59.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5300000000000004e-07 2024-05-11T23:02:59.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:02:59.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650691+0000", 2024-05-11T23:02:59.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1039999999999998e-06 2024-05-11T23:02:59.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:02:59.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650699+0000", 2024-05-11T23:02:59.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3969999999999997e-06 2024-05-11T23:02:59.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:02:59.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:02:59.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:02:59.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.650705+0000", 2024-05-11T23:02:59.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0990000000000004e-06 2024-05-11T23:02:59.337 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.337 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:02:59.337 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.337 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:02:59.337 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:02:59.337 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 207 2024-05-11T23:02:59.337 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:02:59.338 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-11T23:02:59.338 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:02:59.339+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:02:59.339 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:02:59.340 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:02:59.340 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:02:59.340 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:02:59.341 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:02:59.341 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:02:59.341 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:02:59.342 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:02:59.342 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:02:59.342 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:02:59.342 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:02:59.343 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:02:59.343 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:02:59.343 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:02:59.343 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:02:59.344 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:02:59.345 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:02:59.439 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:02:59.439+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:02:59.541 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:02:59.540+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:02:59.562 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:59.562 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:59.576 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:02:59.640 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:02:59.639+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:02:59.741 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:02:59.740+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:02:59.782 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:02:59.782 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:02:59.795 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:02:59.881 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:59.880+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:02:59.943 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:02:59.943+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:00.012 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:00.012 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:00.025 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:03:00.044 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:00.044+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:00.137 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-05-11T23:03:00.146 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:00.146+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:00.233 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:00.233 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:00.242 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:03:00.247 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:00.247+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:00.347 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:00.347+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:00.351 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:00.351 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:00.360 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:03:00.449 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:00.449+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:00.550 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:00.550+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:00.573 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:00.573 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:00.586 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:03:00.650 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:00.650+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:00.751 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:00.751+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:00.803 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:00.803 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:00.816 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:03:00.852 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:00.851+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:00.953 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:00.952+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:00.981 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:00.981 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:00.992 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:03:01.054 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:01.054+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:01.110 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:01.110 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:01.119 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:03:01.163 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:01.163+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:01.246 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:01.246 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:01.255 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:01.255+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:01.257 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:03:01.346 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:03:01.356 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:01.356+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:01.413 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:01.413 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:01.422 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:03:01.460 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:01.460+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:01.498 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:01.499 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:01.518 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:01.518 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:01.534 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:01.534 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:01.552 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:03:01.552 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:03:01.559 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:01.559+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:01.571 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:01.571 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:01.585 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:01.585 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:01.591 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:01.596 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:03:01.613 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:01.632 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:01.633 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:01.687 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:01.686+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:01.761 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:01.761+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:01.781 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:01.781 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:01.781 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:01.781 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:01.781 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:01.793 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:03:01.862 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:01.862+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:01.947 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:01.947 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:01.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:01.947 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:01.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.948 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67817 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:01.948 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:03:01.948 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.326091957000003, 2024-05-11T23:03:01.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044148845999999999, 2024-05-11T23:03:01.948 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:01.948 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:01.948 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:01.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:01.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:01.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:01.948 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67817 2024-05-11T23:03:01.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.948 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:01.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:01.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:03:01.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:01.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:03:01.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:01.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620227+0000", 2024-05-11T23:03:01.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00046792099999999999 2024-05-11T23:03:01.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:01.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620232+0000", 2024-05-11T23:03:01.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3050000000000003e-06 2024-05-11T23:03:01.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:01.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620243+0000", 2024-05-11T23:03:01.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1289e-05 2024-05-11T23:03:01.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:01.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620250+0000", 2024-05-11T23:03:01.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2130000000000002e-06 2024-05-11T23:03:01.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:01.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663745+0000", 2024-05-11T23:03:01.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043494865000000001 2024-05-11T23:03:01.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:01.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663815+0000", 2024-05-11T23:03:01.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0284999999999995e-05 2024-05-11T23:03:01.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:01.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663908+0000", 2024-05-11T23:03:01.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2967999999999995e-05 2024-05-11T23:03:01.951 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.951 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:01.951 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.952 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67820 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4902912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:01.952 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:03:01.952 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.32554245, 2024-05-11T23:03:01.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044096781000000002, 2024-05-11T23:03:01.952 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:01.952 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:01.952 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:01.952 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:01.952 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:01.952 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:01.952 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67820 2024-05-11T23:03:01.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.952 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:01.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:01.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:03:01.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:01.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:03:01.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:01.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620310+0000", 2024-05-11T23:03:01.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9800000000000004e-07 2024-05-11T23:03:01.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:01.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620312+0000", 2024-05-11T23:03:01.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.272e-06 2024-05-11T23:03:01.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:01.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620319+0000", 2024-05-11T23:03:01.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9310000000000002e-06 2024-05-11T23:03:01.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:01.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620323+0000", 2024-05-11T23:03:01.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:03:01.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:01.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664284+0000", 2024-05-11T23:03:01.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043960908999999999 2024-05-11T23:03:01.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:01.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664327+0000", 2024-05-11T23:03:01.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2811000000000001e-05 2024-05-11T23:03:01.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:01.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664406+0000", 2024-05-11T23:03:01.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8588e-05 2024-05-11T23:03:01.955 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.955 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:01.955 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.956 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67821 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4927488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:01.956 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:03:01.956 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.325522040000003, 2024-05-11T23:03:01.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044203340000000001, 2024-05-11T23:03:01.956 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:01.956 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:01.956 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:01.956 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:01.956 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:01.956 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:01.956 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67821 2024-05-11T23:03:01.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.956 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:01.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:01.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:03:01.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:01.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:03:01.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:01.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620330+0000", 2024-05-11T23:03:01.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:03:01.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:01.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620332+0000", 2024-05-11T23:03:01.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:03:01.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:01.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620338+0000", 2024-05-11T23:03:01.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6250000000000004e-06 2024-05-11T23:03:01.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:01.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620342+0000", 2024-05-11T23:03:01.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7609999999999998e-06 2024-05-11T23:03:01.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:01.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664433+0000", 2024-05-11T23:03:01.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044090455000000001 2024-05-11T23:03:01.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:01.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664476+0000", 2024-05-11T23:03:01.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2861999999999997e-05 2024-05-11T23:03:01.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:01.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664533+0000", 2024-05-11T23:03:01.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7156e-05 2024-05-11T23:03:01.959 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.959 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:01.959 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.960 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67822 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4976640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:01.960 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:03:01.960 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.325503001999998, 2024-05-11T23:03:01.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044409416, 2024-05-11T23:03:01.960 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:01.960 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:01.960 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:01.960 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:01.960 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:01.960 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:01.960 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67822 2024-05-11T23:03:01.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.960 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:01.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:01.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:03:01.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:01.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:03:01.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:01.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620349+0000", 2024-05-11T23:03:01.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.51e-07 2024-05-11T23:03:01.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:01.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620351+0000", 2024-05-11T23:03:01.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.181e-06 2024-05-11T23:03:01.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:01.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620358+0000", 2024-05-11T23:03:01.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3060000000000003e-06 2024-05-11T23:03:01.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:01.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620362+0000", 2024-05-11T23:03:01.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7949999999999998e-06 2024-05-11T23:03:01.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:01.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664572+0000", 2024-05-11T23:03:01.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044209318999999997 2024-05-11T23:03:01.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:01.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664654+0000", 2024-05-11T23:03:01.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2367999999999995e-05 2024-05-11T23:03:01.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:01.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664758+0000", 2024-05-11T23:03:01.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103896 2024-05-11T23:03:01.963 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.964 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:01.964 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.964 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67823 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4952064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:01.964 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:03:01.964 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.325482952000002, 2024-05-11T23:03:01.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044623293000000001, 2024-05-11T23:03:01.964 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:01.964 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:01.964 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:01.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:01.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:01.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:01.964 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67823 2024-05-11T23:03:01.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.964 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:01.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:01.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:03:01.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:01.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:03:01.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:01.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622228+0000", 2024-05-11T23:03:01.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018592280000000001 2024-05-11T23:03:01.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:01.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622232+0000", 2024-05-11T23:03:01.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.403e-06 2024-05-11T23:03:01.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:01.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622248+0000", 2024-05-11T23:03:01.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5449000000000001e-05 2024-05-11T23:03:01.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:01.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622254+0000", 2024-05-11T23:03:01.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5819999999999998e-06 2024-05-11T23:03:01.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:01.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664800+0000", 2024-05-11T23:03:01.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042546048000000003 2024-05-11T23:03:01.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:01.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664886+0000", 2024-05-11T23:03:01.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5525000000000005e-05 2024-05-11T23:03:01.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:01.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664992+0000", 2024-05-11T23:03:01.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010605800000000001 2024-05-11T23:03:01.967 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.967 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:01.968 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.968 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67833 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5099520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:01.968 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:03:01.968 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.323376478999997, 2024-05-11T23:03:01.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044076606999999997, 2024-05-11T23:03:01.968 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:01.968 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:01.968 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:01.968 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:01.968 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:01.968 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:01.968 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67833 2024-05-11T23:03:01.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.969 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:01.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:01.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:03:01.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:01.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:03:01.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:01.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:03:01.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8099999999999998e-07 2024-05-11T23:03:01.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:01.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622477+0000", 2024-05-11T23:03:01.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0700000000000001e-06 2024-05-11T23:03:01.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:01.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622483+0000", 2024-05-11T23:03:01.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0780000000000003e-06 2024-05-11T23:03:01.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:01.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622487+0000", 2024-05-11T23:03:01.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.352e-06 2024-05-11T23:03:01.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:01.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666460+0000", 2024-05-11T23:03:01.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043973085000000002 2024-05-11T23:03:01.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:01.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666498+0000", 2024-05-11T23:03:01.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8441e-05 2024-05-11T23:03:01.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:01.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666552+0000", 2024-05-11T23:03:01.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3199999999999999e-05 2024-05-11T23:03:01.972 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.972 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:01.972 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.972 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67834 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3944448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:01.972 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:03:01.972 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.323338200999999, 2024-05-11T23:03:01.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044166551999999998, 2024-05-11T23:03:01.972 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:01.972 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:01.972 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:01.972 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:01.972 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:01.972 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:01.972 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67834 2024-05-11T23:03:01.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.973 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:01.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:01.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:03:01.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:01.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:03:01.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:01.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622514+0000", 2024-05-11T23:03:01.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1100000000000002e-07 2024-05-11T23:03:01.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:01.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622515+0000", 2024-05-11T23:03:01.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:03:01.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:01.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622519+0000", 2024-05-11T23:03:01.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8260000000000003e-06 2024-05-11T23:03:01.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:01.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622522+0000", 2024-05-11T23:03:01.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8899999999999999e-06 2024-05-11T23:03:01.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:01.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666580+0000", 2024-05-11T23:03:01.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044058235000000001 2024-05-11T23:03:01.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:01.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666624+0000", 2024-05-11T23:03:01.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3501999999999998e-05 2024-05-11T23:03:01.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:01.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666680+0000", 2024-05-11T23:03:01.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6042999999999999e-05 2024-05-11T23:03:01.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.976 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:01.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.976 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67835 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:01.976 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:03:01.976 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.323325664999999, 2024-05-11T23:03:01.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044272019000000003, 2024-05-11T23:03:01.976 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:01.976 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:01.976 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:01.976 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:01.976 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:01.976 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:01.977 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67835 2024-05-11T23:03:01.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.977 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:01.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:01.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:03:01.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:01.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:03:01.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:01.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:03:01.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1699999999999999e-07 2024-05-11T23:03:01.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:01.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622527+0000", 2024-05-11T23:03:01.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2109999999999999e-06 2024-05-11T23:03:01.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:01.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622531+0000", 2024-05-11T23:03:01.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4910000000000001e-06 2024-05-11T23:03:01.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:01.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622534+0000", 2024-05-11T23:03:01.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7080000000000002e-06 2024-05-11T23:03:01.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:01.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666703+0000", 2024-05-11T23:03:01.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044169377000000003 2024-05-11T23:03:01.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:01.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666745+0000", 2024-05-11T23:03:01.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1616000000000003e-05 2024-05-11T23:03:01.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:01.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666798+0000", 2024-05-11T23:03:01.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3298999999999999e-05 2024-05-11T23:03:01.980 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.980 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:01.980 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.980 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67836 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:01.980 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:03:01.980 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.323315004999998, 2024-05-11T23:03:01.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044373692999999999, 2024-05-11T23:03:01.980 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:01.980 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:01.980 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:01.981 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:01.981 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:01.981 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:01.981 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67836 2024-05-11T23:03:01.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.981 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:01.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:01.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:03:01.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:01.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:03:01.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:01.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622537+0000", 2024-05-11T23:03:01.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:03:01.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:01.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622538+0000", 2024-05-11T23:03:01.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1510000000000001e-06 2024-05-11T23:03:01.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:01.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622541+0000", 2024-05-11T23:03:01.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:03:01.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:01.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622543+0000", 2024-05-11T23:03:01.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4870000000000001e-06 2024-05-11T23:03:01.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:01.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666821+0000", 2024-05-11T23:03:01.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044278038999999998 2024-05-11T23:03:01.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:01.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666859+0000", 2024-05-11T23:03:01.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7224000000000001e-05 2024-05-11T23:03:01.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:01.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666910+0000", 2024-05-11T23:03:01.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1629999999999999e-05 2024-05-11T23:03:01.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.984 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:01.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.984 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67837 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:01.984 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:03:01.984 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.323305277999999, 2024-05-11T23:03:01.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044477883000000003, 2024-05-11T23:03:01.984 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:01.985 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:01.985 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:01.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:01.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:01.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:01.985 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67837 2024-05-11T23:03:01.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.985 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:01.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:01.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:03:01.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:01.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:03:01.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:01.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:03:01.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:01.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:01.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622548+0000", 2024-05-11T23:03:01.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:03:01.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:01.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622550+0000", 2024-05-11T23:03:01.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-05-11T23:03:01.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:01.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622553+0000", 2024-05-11T23:03:01.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.43e-06 2024-05-11T23:03:01.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:01.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666932+0000", 2024-05-11T23:03:01.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044379584 2024-05-11T23:03:01.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:01.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666971+0000", 2024-05-11T23:03:01.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8427000000000003e-05 2024-05-11T23:03:01.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:01.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667024+0000", 2024-05-11T23:03:01.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3226000000000002e-05 2024-05-11T23:03:01.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.988 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:01.988 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.988 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67838 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:01.988 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:03:01.988 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.323295760999997, 2024-05-11T23:03:01.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044580374999999998, 2024-05-11T23:03:01.988 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:01.988 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:01.988 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:01.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:01.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:01.989 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:01.989 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67838 2024-05-11T23:03:01.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.989 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:01.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:01.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:03:01.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:01.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:03:01.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:01.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:03:01.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:03:01.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:01.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622557+0000", 2024-05-11T23:03:01.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.102e-06 2024-05-11T23:03:01.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:01.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622560+0000", 2024-05-11T23:03:01.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:03:01.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:01.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622562+0000", 2024-05-11T23:03:01.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:03:01.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:01.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667045+0000", 2024-05-11T23:03:01.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044482936000000001 2024-05-11T23:03:01.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:01.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667083+0000", 2024-05-11T23:03:01.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7443e-05 2024-05-11T23:03:01.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:01.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667136+0000", 2024-05-11T23:03:01.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3257000000000002e-05 2024-05-11T23:03:01.992 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.992 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:01.992 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.992 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67839 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5394432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:01.993 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:03:01.993 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.323286019000001, 2024-05-11T23:03:01.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04470263, 2024-05-11T23:03:01.993 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:01.993 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:01.993 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:01.993 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:01.993 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:01.993 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:01.993 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67839 2024-05-11T23:03:01.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.993 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:01.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:01.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:03:01.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:01.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:03:01.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:01.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622566+0000", 2024-05-11T23:03:01.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-05-11T23:03:01.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:01.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622572+0000", 2024-05-11T23:03:01.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5610000000000004e-06 2024-05-11T23:03:01.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:01.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622576+0000", 2024-05-11T23:03:01.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3210000000000001e-06 2024-05-11T23:03:01.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:01.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622578+0000", 2024-05-11T23:03:01.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4710000000000001e-06 2024-05-11T23:03:01.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:01.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667160+0000", 2024-05-11T23:03:01.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044582220999999998 2024-05-11T23:03:01.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:01.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667205+0000", 2024-05-11T23:03:01.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5049000000000002e-05 2024-05-11T23:03:01.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:01.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667268+0000", 2024-05-11T23:03:01.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2788999999999998e-05 2024-05-11T23:03:01.996 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.996 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:01.996 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:01.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.996 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67840 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5419008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:01.996 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:03:01.996 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.323270389000001, 2024-05-11T23:03:01.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044796006999999999, 2024-05-11T23:03:01.997 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:01.997 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:01.997 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:01.997 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:01.997 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:01.997 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:01.997 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67840 2024-05-11T23:03:01.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.997 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:01.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:01.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:03:01.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:01.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:03:01.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:01.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:01.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:03:01.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:03:01.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:01.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622583+0000", 2024-05-11T23:03:01.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4389999999999999e-06 2024-05-11T23:03:01.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:01.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622586+0000", 2024-05-11T23:03:01.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:03:01.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:01.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622588+0000", 2024-05-11T23:03:01.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4540000000000001e-06 2024-05-11T23:03:01.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:01.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667287+0000", 2024-05-11T23:03:01.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044698882000000002 2024-05-11T23:03:01.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:01.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:01.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:01.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667324+0000", 2024-05-11T23:03:02.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7196000000000001e-05 2024-05-11T23:03:02.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667377+0000", 2024-05-11T23:03:02.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2927999999999999e-05 2024-05-11T23:03:02.000 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.000 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.000 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.001 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67841 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5443584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:02.001 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:03:02.001 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.323260615999999, 2024-05-11T23:03:02.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044907337999999998, 2024-05-11T23:03:02.001 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.001 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.001 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.001 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67841 2024-05-11T23:03:02.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.001 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:03:02.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:03:02.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:03:02.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:02.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622592+0000", 2024-05-11T23:03:02.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-05-11T23:03:02.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622595+0000", 2024-05-11T23:03:02.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:03:02.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622598+0000", 2024-05-11T23:03:02.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5909999999999998e-06 2024-05-11T23:03:02.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667401+0000", 2024-05-11T23:03:02.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044802980999999999 2024-05-11T23:03:02.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667439+0000", 2024-05-11T23:03:02.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8840000000000001e-05 2024-05-11T23:03:02.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667498+0000", 2024-05-11T23:03:02.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8709000000000003e-05 2024-05-11T23:03:02.004 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.004 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.004 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.004 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67842 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5468160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:03:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.323251009000003, 2024-05-11T23:03:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045005876, 2024-05-11T23:03:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67842 2024-05-11T23:03:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:03:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:03:02.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622601+0000", 2024-05-11T23:03:02.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:03:02.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622602+0000", 2024-05-11T23:03:02.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0780000000000001e-06 2024-05-11T23:03:02.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622605+0000", 2024-05-11T23:03:02.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.914e-06 2024-05-11T23:03:02.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622607+0000", 2024-05-11T23:03:02.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:03:02.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667516+0000", 2024-05-11T23:03:02.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044909156999999998 2024-05-11T23:03:02.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667554+0000", 2024-05-11T23:03:02.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7920000000000003e-05 2024-05-11T23:03:02.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667606+0000", 2024-05-11T23:03:02.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2111000000000003e-05 2024-05-11T23:03:02.008 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.008 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.008 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.008 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67843 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4067328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:02.009 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:03:02.009 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.323241482999997, 2024-05-11T23:03:02.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045119014999999998, 2024-05-11T23:03:02.009 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.009 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.009 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.009 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.009 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.009 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.009 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67843 2024-05-11T23:03:02.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.009 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:03:02.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:03:02.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624228+0000", 2024-05-11T23:03:02.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001617867 2024-05-11T23:03:02.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624232+0000", 2024-05-11T23:03:02.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-05-11T23:03:02.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624240+0000", 2024-05-11T23:03:02.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4589999999999998e-06 2024-05-11T23:03:02.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624247+0000", 2024-05-11T23:03:02.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-05-11T23:03:02.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667629+0000", 2024-05-11T23:03:02.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043382012999999997 2024-05-11T23:03:02.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667674+0000", 2024-05-11T23:03:02.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5172000000000001e-05 2024-05-11T23:03:02.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667729+0000", 2024-05-11T23:03:02.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5086000000000001e-05 2024-05-11T23:03:02.012 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.012 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.012 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.013 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67849 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4190208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:02.013 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:03:02.013 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.321483362000002, 2024-05-11T23:03:02.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044061433999999997, 2024-05-11T23:03:02.013 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.013 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.013 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.013 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67849 2024-05-11T23:03:02.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.013 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:03:02.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:03:02.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624369+0000", 2024-05-11T23:03:02.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5900000000000002e-07 2024-05-11T23:03:02.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624371+0000", 2024-05-11T23:03:02.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0310000000000001e-06 2024-05-11T23:03:02.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624376+0000", 2024-05-11T23:03:02.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4029999999999998e-06 2024-05-11T23:03:02.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624381+0000", 2024-05-11T23:03:02.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4349999999999996e-06 2024-05-11T23:03:02.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668331+0000", 2024-05-11T23:03:02.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043949191999999998 2024-05-11T23:03:02.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668370+0000", 2024-05-11T23:03:02.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9227999999999998e-05 2024-05-11T23:03:02.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668430+0000", 2024-05-11T23:03:02.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9685999999999997e-05 2024-05-11T23:03:02.016 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.016 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.016 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.017 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67850 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:02.017 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:03:02.017 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.32146213, 2024-05-11T23:03:02.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044151953000000001, 2024-05-11T23:03:02.017 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.017 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.017 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.017 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.017 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.017 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.017 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67850 2024-05-11T23:03:02.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.017 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:03:02.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:03:02.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624390+0000", 2024-05-11T23:03:02.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:03:02.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624392+0000", 2024-05-11T23:03:02.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1880000000000001e-06 2024-05-11T23:03:02.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624399+0000", 2024-05-11T23:03:02.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7390000000000002e-06 2024-05-11T23:03:02.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624405+0000", 2024-05-11T23:03:02.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1059999999999996e-06 2024-05-11T23:03:02.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668450+0000", 2024-05-11T23:03:02.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044045147999999999 2024-05-11T23:03:02.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668487+0000", 2024-05-11T23:03:02.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7450000000000002e-05 2024-05-11T23:03:02.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668541+0000", 2024-05-11T23:03:02.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3952000000000001e-05 2024-05-11T23:03:02.020 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.020 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.020 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.021 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67851 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5541888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:02.021 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:03:02.021 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.321435547999997, 2024-05-11T23:03:02.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044264413000000002, 2024-05-11T23:03:02.021 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.021 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.021 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.021 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67851 2024-05-11T23:03:02.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.021 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:03:02.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:03:02.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:03:02.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-05-11T23:03:02.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624419+0000", 2024-05-11T23:03:02.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.21e-06 2024-05-11T23:03:02.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624424+0000", 2024-05-11T23:03:02.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9050000000000002e-06 2024-05-11T23:03:02.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624430+0000", 2024-05-11T23:03:02.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-05-11T23:03:02.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668566+0000", 2024-05-11T23:03:02.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044136208000000003 2024-05-11T23:03:02.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668614+0000", 2024-05-11T23:03:02.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8022000000000002e-05 2024-05-11T23:03:02.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668680+0000", 2024-05-11T23:03:02.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6181999999999998e-05 2024-05-11T23:03:02.025 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.025 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.025 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.025 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67852 3.16 3.1a4a4af6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:02.025 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:03:02.025 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.321415080999998, 2024-05-11T23:03:02.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044278287999999999, 2024-05-11T23:03:02.025 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.025 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.025 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:03:02.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.026 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67852 2024-05-11T23:03:02.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.026 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:03:02.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:03:02.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626226+0000", 2024-05-11T23:03:02.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017893 2024-05-11T23:03:02.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626229+0000", 2024-05-11T23:03:02.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0709999999999999e-06 2024-05-11T23:03:02.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626239+0000", 2024-05-11T23:03:02.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0507e-05 2024-05-11T23:03:02.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626244+0000", 2024-05-11T23:03:02.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4880000000000001e-06 2024-05-11T23:03:02.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668706+0000", 2024-05-11T23:03:02.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042462054999999999 2024-05-11T23:03:02.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668715+0000", 2024-05-11T23:03:02.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.867e-06 2024-05-11T23:03:02.028 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.028 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.028 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.028 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.028 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.029 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:02.030 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:03:02.030 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:02.030+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:02.131 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:02.131+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:02.156 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-11T23:03:02.226 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:02.226 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:02.226 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:02.226 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:02.232 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:02.232+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:02.238 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:03:02.333 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:02.333+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:02.374 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 5, 7, 3, 6] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:03:02.375 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:03:02.375 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-11T23:03:02.375 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.935317 2024-05-11T23:03:02.375 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0.9353173976768921 2024-05-11T23:03:02.403 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:02.403 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:02.403 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:02.403 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:02.403 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:02.415 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:03:02.433 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:02.433+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:02.534 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:02.534+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:02.558 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.903454577000005, 2024-05-11T23:03:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:03:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:03:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:03:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:03:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:03:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:03:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:03:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:03:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:03:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:03:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:03:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:03:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:03:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:03:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:03:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.900073387999996, 2024-05-11T23:03:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:03:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:03:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:03:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:03:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:03:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:03:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:03:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:03:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:03:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:03:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:03:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:03:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:03:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:03:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.900054681, 2024-05-11T23:03:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:03:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:03:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:03:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:03:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:03:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:03:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:03:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:03:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:03:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:03:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:03:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:03:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:03:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:03:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:03:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:03:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.900036537999995, 2024-05-11T23:03:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:03:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:03:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:03:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:03:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:03:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:03:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:03:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:03:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:03:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:03:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:03:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:03:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:03:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:03:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:03:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.900009050999998, 2024-05-11T23:03:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:03:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:03:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:03:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:03:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:03:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:03:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:03:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:03:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:03:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:03:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:03:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:03:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:03:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:03:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:03:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:03:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.899988523000005, 2024-05-11T23:03:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:03:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:03:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:03:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:03:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:03:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:03:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:03:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:03:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:03:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:03:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:03:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:03:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:03:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:03:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:03:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.899968748000006, 2024-05-11T23:03:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:03:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:03:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:03:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:03:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:03:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:03:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:03:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:03:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:03:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:03:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:03:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:03:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:03:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:03:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:03:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:03:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.899949942000006, 2024-05-11T23:03:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:03:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:03:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:03:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:03:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:03:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:03:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:03:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:03:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:03:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:03:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:03:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:03:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:03:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:03:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:03:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:03:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.899931018999993, 2024-05-11T23:03:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:03:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:03:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:03:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:03:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:03:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:03:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:03:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:03:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:03:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:03:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:03:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:03:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:03:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:03:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:03:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.899912357000005, 2024-05-11T23:03:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:03:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:03:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:02.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:03:02.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:03:02.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:03:02.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:03:02.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:03:02.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:03:02.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:03:02.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:03:02.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:03:02.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:03:02.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:03:02.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:03:02.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:03:02.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.600 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.600 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:02.600 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:02.600 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.899567195000003, 2024-05-11T23:03:02.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:03:02.600 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.601 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.601 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.601 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:03:02.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.601 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:02.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:02.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:03:02.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:03:02.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:03:02.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:03:02.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:03:02.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:03:02.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:03:02.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:03:02.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:03:02.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:03:02.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:03:02.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:03:02.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:03:02.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:03:02.604 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.604 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.604 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.604 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:02.604 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:02.605 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.899541644999999, 2024-05-11T23:03:02.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:03:02.605 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.605 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.605 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.605 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.605 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.605 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.605 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:03:02.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.605 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:02.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:02.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:02.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:03:02.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:03:02.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:03:02.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:03:02.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:03:02.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:03:02.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:03:02.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:03:02.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:03:02.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:03:02.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:03:02.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:03:02.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:03:02.608 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.608 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.608 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.608 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:02.609 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:02.609 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.899459386999993, 2024-05-11T23:03:02.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:03:02.609 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.609 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.609 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.609 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.609 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.609 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.609 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:03:02.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.609 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:02.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:02.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:02.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:03:02.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:03:02.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:03:02.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:03:02.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:03:02.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:03:02.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:03:02.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:03:02.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:03:02.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:03:02.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:03:02.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:03:02.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:03:02.612 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.612 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.612 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.613 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:02.613 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:02.613 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.899436940000001, 2024-05-11T23:03:02.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:03:02.613 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.613 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.613 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.613 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.613 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.613 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.613 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:03:02.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.613 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:02.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:02.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:03:02.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:03:02.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:03:02.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:03:02.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:03:02.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:03:02.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:03:02.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:03:02.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:03:02.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:03:02.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:03:02.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:03:02.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:03:02.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:03:02.616 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.616 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.616 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.617 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:02.617 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:02.617 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.899417482000004, 2024-05-11T23:03:02.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:03:02.617 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.617 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.617 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.617 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.617 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.617 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.617 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:03:02.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.617 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:02.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:02.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:02.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:03:02.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:03:02.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:03:02.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:03:02.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:03:02.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:03:02.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:03:02.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:03:02.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:03:02.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:03:02.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:03:02.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:03:02.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:03:02.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.620 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.620 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.621 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:02.621 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:02.621 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.899397422000007, 2024-05-11T23:03:02.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:03:02.621 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.621 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.621 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.621 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.621 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:03:02.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.621 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:02.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:02.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:02.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:02.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:03:02.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:03:02.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:03:02.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:03:02.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:03:02.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:03:02.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:03:02.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:03:02.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:03:02.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:03:02.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:03:02.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:03:02.624 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.624 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.624 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.625 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:02.625 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:02.625 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.899378151999997, 2024-05-11T23:03:02.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:03:02.625 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.625 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.625 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.625 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.625 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.625 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.625 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:03:02.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.625 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:02.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:02.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:02.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:03:02.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:03:02.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:03:02.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:03:02.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:03:02.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:03:02.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:03:02.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:03:02.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:03:02.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:03:02.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:03:02.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:03:02.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:03:02.628 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.629 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.629 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.629 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:02.629 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:02.629 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.898832085999999, 2024-05-11T23:03:02.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:03:02.629 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.629 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.629 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.629 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.629 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.630 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.630 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:03:02.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.630 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:02.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:02.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:03:02.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:03:02.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:03:02.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:03:02.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:03:02.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:03:02.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:03:02.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:03:02.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:03:02.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:03:02.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:03:02.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:03:02.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:03:02.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:03:02.633 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.633 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.633 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.633 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:02.633 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:02.633 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 69.898800301999998, 2024-05-11T23:03:02.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:03:02.633 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.633 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.634 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.634 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:03:02.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.634 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:02.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:02.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:02.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:03:02.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:03:02.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:03:02.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:03:02.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:03:02.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:03:02.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:03:02.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:03:02.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:03:02.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:03:02.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:03:02.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:03:02.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:03:02.638 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.638 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.638 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.638 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:03:02.638 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:02.638 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.461554837999998, 2024-05-11T23:03:02.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:03:02.638 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.638 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.638 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:02.638 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.639 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.639 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.639 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:03:02.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.639 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:02.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:02.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:03:02.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:03:02.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:03:02.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:03:02.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.640 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.640 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:03:02.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:03:02.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:03:02.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:03:02.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:03:02.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:03:02.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:03:02.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:03:02.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:03:02.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:03:02.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:03:02.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:02.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:03:02.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:03:02.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:02.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:03:02.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:03:02.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:02.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:03:02.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:03:02.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:02.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:03:02.643 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:03:02.643 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.643 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.643 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.643 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.643 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.643 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:02.645 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:03:02.645 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:02.644+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:02.746 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:02.746+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:02.819 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:02.819 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:02.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.819 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:233792 3.15 3:aed7ee2e:::rbd_data.111ebabb4604.0000000000000136:head [call rbd.sparse_copyup in=104488b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:02.819 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:02.816623+0000", 2024-05-11T23:03:02.819 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0025590639999999998, 2024-05-11T23:03:02.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0025891590000000002, 2024-05-11T23:03:02.819 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.820 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.820 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for sub ops", 2024-05-11T23:03:02.820 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.820 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.820 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.820 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 233792 2024-05-11T23:03:02.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.820 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816623+0000", 2024-05-11T23:03:02.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816623+0000", 2024-05-11T23:03:02.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816625+0000", 2024-05-11T23:03:02.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5910000000000001e-06 2024-05-11T23:03:02.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816742+0000", 2024-05-11T23:03:02.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011675099999999999 2024-05-11T23:03:02.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816758+0000", 2024-05-11T23:03:02.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6424000000000001e-05 2024-05-11T23:03:02.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816768+0000", 2024-05-11T23:03:02.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6639999999999993e-06 2024-05-11T23:03:02.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816830+0000", 2024-05-11T23:03:02.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1866999999999996e-05 2024-05-11T23:03:02.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816930+0000", 2024-05-11T23:03:02.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100356 2024-05-11T23:03:02.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:03:02.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.817113+0000", 2024-05-11T23:03:02.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018276999999999999 2024-05-11T23:03:02.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:02.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.819009+0000", 2024-05-11T23:03:02.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018957679999999999 2024-05-11T23:03:02.823 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.824 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.824 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.824 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:233795 3.15 3:aed7ee2e:::rbd_data.111ebabb4604.0000000000000136:head [delete] snapc 2f=[2f] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:02.824 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:02.816790+0000", 2024-05-11T23:03:02.824 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.002392127, 2024-05-11T23:03:02.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0025250170000000001, 2024-05-11T23:03:02.824 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.824 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.824 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for sub ops", 2024-05-11T23:03:02.824 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.824 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.824 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.825 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 233795 2024-05-11T23:03:02.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.825 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816790+0000", 2024-05-11T23:03:02.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816790+0000", 2024-05-11T23:03:02.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816791+0000", 2024-05-11T23:03:02.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-07 2024-05-11T23:03:02.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816794+0000", 2024-05-11T23:03:02.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.455e-06 2024-05-11T23:03:02.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816803+0000", 2024-05-11T23:03:02.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5359999999999992e-06 2024-05-11T23:03:02.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816809+0000", 2024-05-11T23:03:02.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0950000000000001e-06 2024-05-11T23:03:02.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.817999+0000", 2024-05-11T23:03:02.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011900249999999999 2024-05-11T23:03:02.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.818079+0000", 2024-05-11T23:03:02.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9567999999999994e-05 2024-05-11T23:03:02.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:03:02.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.818263+0000", 2024-05-11T23:03:02.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00018476300000000001 2024-05-11T23:03:02.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.828 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.828 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:233816 3.e 3:7359118f:::rbd_data.111ebabb4604.0000000000000139:head [call rbd.sparse_copyup in=104488b,set-alloc-hint object_size 262144 write_size 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:02.828 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:02.816853+0000", 2024-05-11T23:03:02.828 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0023291359999999999, 2024-05-11T23:03:02.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0025435369999999998, 2024-05-11T23:03:02.828 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.828 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.828 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for sub ops", 2024-05-11T23:03:02.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.829 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 233816 2024-05-11T23:03:02.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.829 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816853+0000", 2024-05-11T23:03:02.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816853+0000", 2024-05-11T23:03:02.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816854+0000", 2024-05-11T23:03:02.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0800000000000004e-07 2024-05-11T23:03:02.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816927+0000", 2024-05-11T23:03:02.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3392999999999993e-05 2024-05-11T23:03:02.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816936+0000", 2024-05-11T23:03:02.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.098e-06 2024-05-11T23:03:02.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816943+0000", 2024-05-11T23:03:02.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1970000000000001e-06 2024-05-11T23:03:02.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.817013+0000", 2024-05-11T23:03:02.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0821000000000005e-05 2024-05-11T23:03:02.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.817090+0000", 2024-05-11T23:03:02.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6582000000000006e-05 2024-05-11T23:03:02.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-05-11T23:03:02.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.817288+0000", 2024-05-11T23:03:02.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019786000000000001 2024-05-11T23:03:02.832 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.832 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.832 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.832 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:233818 3.e 3:7359118f:::rbd_data.111ebabb4604.0000000000000139:head [delete] snapc 2f=[2f] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:02.832 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:02.816969+0000", 2024-05-11T23:03:02.832 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.0022138150000000001, 2024-05-11T23:03:02.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0025064449999999999, 2024-05-11T23:03:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for sub ops", 2024-05-11T23:03:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 233818 2024-05-11T23:03:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816969+0000", 2024-05-11T23:03:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816969+0000", 2024-05-11T23:03:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816969+0000", 2024-05-11T23:03:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:03:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816977+0000", 2024-05-11T23:03:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8280000000000003e-06 2024-05-11T23:03:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816986+0000", 2024-05-11T23:03:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5639999999999997e-06 2024-05-11T23:03:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.816995+0000", 2024-05-11T23:03:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9250000000000007e-06 2024-05-11T23:03:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.819035+0000", 2024-05-11T23:03:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002039384 2024-05-11T23:03:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.819160+0000", 2024-05-11T23:03:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000124969 2024-05-11T23:03:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 1", 2024-05-11T23:03:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.819324+0000", 2024-05-11T23:03:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016396499999999999 2024-05-11T23:03:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_repop(client.4382.0:233811 3.9 e102/97 3:975fb030:::rbd_data.111ebabb4604.0000000000000137:head v 102'457, mlcod=102'453)", 2024-05-11T23:03:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:02.818167+0000", 2024-05-11T23:03:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.001015766, 2024-05-11T23:03:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0013823609999999999, 2024-05-11T23:03:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:02.836 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.818167+0000", 2024-05-11T23:03:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.818167+0000", 2024-05-11T23:03:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.818167+0000", 2024-05-11T23:03:02.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.23e-07 2024-05-11T23:03:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.818173+0000", 2024-05-11T23:03:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9630000000000003e-06 2024-05-11T23:03:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.818183+0000", 2024-05-11T23:03:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8840000000000006e-06 2024-05-11T23:03:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:02.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.818189+0000", 2024-05-11T23:03:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1430000000000003e-06 2024-05-11T23:03:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.818368+0000", 2024-05-11T23:03:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000179078 2024-05-11T23:03:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:02.818388+0000", 2024-05-11T23:03:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9752e-05 2024-05-11T23:03:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:02.839 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-11T23:03:02.840 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 5 2024-05-11T23:03:02.840 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:02.841 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:03:02.847 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:02.847+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:02.948 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:02.947+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:02.962 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:02.962 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:02.962 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:02.962 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:02.962 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:02.972 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:03:03.048 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:03.048+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:03.122 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:03.122 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:03.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:03.122 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:03.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.122 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:03:03.122 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:03.122 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 105.957953159, 2024-05-11T23:03:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:03:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:03:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:03:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:03:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:03:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:03:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:03:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:03:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:03:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:03:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:03:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:03:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:03:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:03:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:03:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:03:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:03:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:03:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:03:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:03:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:03:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:03:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:03:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:03:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:03:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:03:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:03:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:03:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:03:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:03:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:03:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:03:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78429 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.156683817000001, 2024-05-11T23:03:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044563497000000001, 2024-05-11T23:03:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78429 2024-05-11T23:03:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964872+0000", 2024-05-11T23:03:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035070300000000001 2024-05-11T23:03:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964876+0000", 2024-05-11T23:03:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5009999999999998e-06 2024-05-11T23:03:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964902+0000", 2024-05-11T23:03:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5715999999999999e-05 2024-05-11T23:03:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964910+0000", 2024-05-11T23:03:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3930000000000002e-06 2024-05-11T23:03:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008898+0000", 2024-05-11T23:03:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043988772000000002 2024-05-11T23:03:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008981+0000", 2024-05-11T23:03:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3140999999999996e-05 2024-05-11T23:03:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009085+0000", 2024-05-11T23:03:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010327099999999999 2024-05-11T23:03:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78431 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.156223509, 2024-05-11T23:03:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044542033000000002, 2024-05-11T23:03:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78431 2024-05-11T23:03:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965038+0000", 2024-05-11T23:03:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6993000000000001e-05 2024-05-11T23:03:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965043+0000", 2024-05-11T23:03:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7040000000000002e-06 2024-05-11T23:03:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965053+0000", 2024-05-11T23:03:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8539999999999992e-06 2024-05-11T23:03:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965061+0000", 2024-05-11T23:03:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9810000000000003e-06 2024-05-11T23:03:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009350+0000", 2024-05-11T23:03:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044288739000000001 2024-05-11T23:03:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009429+0000", 2024-05-11T23:03:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9354e-05 2024-05-11T23:03:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009524+0000", 2024-05-11T23:03:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4407999999999995e-05 2024-05-11T23:03:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78432 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.156132661000001, 2024-05-11T23:03:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044635421000000002, 2024-05-11T23:03:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78432 2024-05-11T23:03:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965343+0000", 2024-05-11T23:03:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027056500000000001 2024-05-11T23:03:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965347+0000", 2024-05-11T23:03:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3540000000000002e-06 2024-05-11T23:03:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965356+0000", 2024-05-11T23:03:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2140000000000002e-06 2024-05-11T23:03:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965363+0000", 2024-05-11T23:03:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4150000000000004e-06 2024-05-11T23:03:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009551+0000", 2024-05-11T23:03:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044187917 2024-05-11T23:03:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009626+0000", 2024-05-11T23:03:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5024000000000005e-05 2024-05-11T23:03:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009708+0000", 2024-05-11T23:03:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1932000000000001e-05 2024-05-11T23:03:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78433 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.155831773999999, 2024-05-11T23:03:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044564864000000003, 2024-05-11T23:03:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78433 2024-05-11T23:03:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966225+0000", 2024-05-11T23:03:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085220700000000001 2024-05-11T23:03:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966230+0000", 2024-05-11T23:03:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0690000000000003e-06 2024-05-11T23:03:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966239+0000", 2024-05-11T23:03:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-05-11T23:03:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966245+0000", 2024-05-11T23:03:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5520000000000001e-06 2024-05-11T23:03:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009743+0000", 2024-05-11T23:03:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043498311999999997 2024-05-11T23:03:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009848+0000", 2024-05-11T23:03:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104885 2024-05-11T23:03:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009938+0000", 2024-05-11T23:03:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9812e-05 2024-05-11T23:03:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78580 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 999424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.152719091000002, 2024-05-11T23:03:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044493076999999999, 2024-05-11T23:03:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78580 2024-05-11T23:03:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:03:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968489+0000", 2024-05-11T23:03:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.52e-06 2024-05-11T23:03:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968496+0000", 2024-05-11T23:03:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9419999999999998e-06 2024-05-11T23:03:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968501+0000", 2024-05-11T23:03:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8720000000000001e-06 2024-05-11T23:03:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012690+0000", 2024-05-11T23:03:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044189655000000001 2024-05-11T23:03:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012848+0000", 2024-05-11T23:03:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015758100000000001 2024-05-11T23:03:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012979+0000", 2024-05-11T23:03:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000131066 2024-05-11T23:03:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78581 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.152698086999997, 2024-05-11T23:03:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044616243, 2024-05-11T23:03:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78581 2024-05-11T23:03:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970227+0000", 2024-05-11T23:03:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017200049999999999 2024-05-11T23:03:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970232+0000", 2024-05-11T23:03:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000003e-06 2024-05-11T23:03:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970242+0000", 2024-05-11T23:03:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0192e-05 2024-05-11T23:03:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970249+0000", 2024-05-11T23:03:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.888e-06 2024-05-11T23:03:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013008+0000", 2024-05-11T23:03:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042759495000000002 2024-05-11T23:03:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013063+0000", 2024-05-11T23:03:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5071999999999997e-05 2024-05-11T23:03:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013123+0000", 2024-05-11T23:03:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9930999999999999e-05 2024-05-11T23:03:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78589 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.150778842999998, 2024-05-11T23:03:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044587697000000003, 2024-05-11T23:03:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78589 2024-05-11T23:03:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970427+0000", 2024-05-11T23:03:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:03:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970429+0000", 2024-05-11T23:03:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3800000000000001e-06 2024-05-11T23:03:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970435+0000", 2024-05-11T23:03:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2530000000000001e-06 2024-05-11T23:03:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970441+0000", 2024-05-11T23:03:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-05-11T23:03:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014822+0000", 2024-05-11T23:03:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04438168 2024-05-11T23:03:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014911+0000", 2024-05-11T23:03:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8882999999999996e-05 2024-05-11T23:03:03.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:03.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015014+0000", 2024-05-11T23:03:03.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010263399999999999 2024-05-11T23:03:03.157 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.157 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.157 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.157 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78590 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:03.157 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:03.157 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.150758044, 2024-05-11T23:03:03.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044880120000000003, 2024-05-11T23:03:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78590 2024-05-11T23:03:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970918+0000", 2024-05-11T23:03:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000470834 2024-05-11T23:03:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970925+0000", 2024-05-11T23:03:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1690000000000003e-06 2024-05-11T23:03:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970934+0000", 2024-05-11T23:03:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9099999999999994e-06 2024-05-11T23:03:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970941+0000", 2024-05-11T23:03:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7410000000000003e-06 2024-05-11T23:03:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015049+0000", 2024-05-11T23:03:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044108392000000003 2024-05-11T23:03:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015133+0000", 2024-05-11T23:03:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3568999999999999e-05 2024-05-11T23:03:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015327+0000", 2024-05-11T23:03:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019450500000000001 2024-05-11T23:03:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78591 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.150254627999999, 2024-05-11T23:03:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044618497, 2024-05-11T23:03:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78591 2024-05-11T23:03:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970951+0000", 2024-05-11T23:03:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:03:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970953+0000", 2024-05-11T23:03:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:03:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970956+0000", 2024-05-11T23:03:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-05-11T23:03:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970959+0000", 2024-05-11T23:03:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9409999999999999e-06 2024-05-11T23:03:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015367+0000", 2024-05-11T23:03:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044407621000000001 2024-05-11T23:03:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015453+0000", 2024-05-11T23:03:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6685000000000001e-05 2024-05-11T23:03:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015569+0000", 2024-05-11T23:03:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115396 2024-05-11T23:03:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78592 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.150241819999998, 2024-05-11T23:03:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044831546999999999, 2024-05-11T23:03:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78592 2024-05-11T23:03:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972226+0000", 2024-05-11T23:03:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00126246 2024-05-11T23:03:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972230+0000", 2024-05-11T23:03:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1860000000000002e-06 2024-05-11T23:03:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972237+0000", 2024-05-11T23:03:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:03:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972241+0000", 2024-05-11T23:03:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.284e-06 2024-05-11T23:03:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015611+0000", 2024-05-11T23:03:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043369461999999998 2024-05-11T23:03:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015695+0000", 2024-05-11T23:03:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4458999999999999e-05 2024-05-11T23:03:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015795+0000", 2024-05-11T23:03:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9475999999999994e-05 2024-05-11T23:03:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78593 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.148954901000003, 2024-05-11T23:03:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046045214000000001, 2024-05-11T23:03:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78593 2024-05-11T23:03:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7599999999999998e-07 2024-05-11T23:03:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972252+0000", 2024-05-11T23:03:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7349999999999999e-06 2024-05-11T23:03:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972256+0000", 2024-05-11T23:03:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5659999999999998e-06 2024-05-11T23:03:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972259+0000", 2024-05-11T23:03:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8040000000000002e-06 2024-05-11T23:03:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018088+0000", 2024-05-11T23:03:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045829316000000002 2024-05-11T23:03:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018179+0000", 2024-05-11T23:03:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1299999999999997e-05 2024-05-11T23:03:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018295+0000", 2024-05-11T23:03:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116117 2024-05-11T23:03:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78594 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.148942540999997, 2024-05-11T23:03:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046245204999999998, 2024-05-11T23:03:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78594 2024-05-11T23:03:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972263+0000", 2024-05-11T23:03:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:03:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:03.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972264+0000", 2024-05-11T23:03:03.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.206e-06 2024-05-11T23:03:03.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:03.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972267+0000", 2024-05-11T23:03:03.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0690000000000002e-06 2024-05-11T23:03:03.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972270+0000", 2024-05-11T23:03:03.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5610000000000001e-06 2024-05-11T23:03:03.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018330+0000", 2024-05-11T23:03:03.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04606085 2024-05-11T23:03:03.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:03.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018414+0000", 2024-05-11T23:03:03.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3125000000000001e-05 2024-05-11T23:03:03.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:03.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018508+0000", 2024-05-11T23:03:03.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4153000000000005e-05 2024-05-11T23:03:03.177 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.177 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.177 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.178 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78595 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:03.178 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:03.178 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.148929676000002, 2024-05-11T23:03:03.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046396698, 2024-05-11T23:03:03.178 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:03.178 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:03.178 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:03.178 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:03.178 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:03.178 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.178 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78595 2024-05-11T23:03:03.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.178 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:03.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:03.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:03.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:03.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:03.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:03.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972276+0000", 2024-05-11T23:03:03.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5600000000000002e-07 2024-05-11T23:03:03.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:03.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972277+0000", 2024-05-11T23:03:03.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.576e-06 2024-05-11T23:03:03.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:03.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972280+0000", 2024-05-11T23:03:03.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-05-11T23:03:03.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972283+0000", 2024-05-11T23:03:03.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:03:03.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018538+0000", 2024-05-11T23:03:03.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046255255000000002 2024-05-11T23:03:03.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:03.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018605+0000", 2024-05-11T23:03:03.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6847000000000006e-05 2024-05-11T23:03:03.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:03.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018672+0000", 2024-05-11T23:03:03.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7045000000000006e-05 2024-05-11T23:03:03.181 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.182 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.182 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.182 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78596 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:03.182 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:03.182 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.148918827000003, 2024-05-11T23:03:03.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046586806000000001, 2024-05-11T23:03:03.182 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:03.182 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:03.182 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:03.182 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:03.182 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:03.182 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.182 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78596 2024-05-11T23:03:03.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.183 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:03.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:03.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:03.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:03.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:03.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:03.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:03.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:03:03.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972288+0000", 2024-05-11T23:03:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2550000000000001e-06 2024-05-11T23:03:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972291+0000", 2024-05-11T23:03:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9490000000000001e-06 2024-05-11T23:03:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972293+0000", 2024-05-11T23:03:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5069999999999999e-06 2024-05-11T23:03:03.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018707+0000", 2024-05-11T23:03:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046413604999999997 2024-05-11T23:03:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018782+0000", 2024-05-11T23:03:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5556000000000006e-05 2024-05-11T23:03:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018873+0000", 2024-05-11T23:03:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0703000000000002e-05 2024-05-11T23:03:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.185 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78597 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.148909054000001, 2024-05-11T23:03:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046825787000000001, 2024-05-11T23:03:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.186 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78597 2024-05-11T23:03:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:03.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-05-11T23:03:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972297+0000", 2024-05-11T23:03:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3069999999999999e-06 2024-05-11T23:03:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972300+0000", 2024-05-11T23:03:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:03:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972303+0000", 2024-05-11T23:03:03.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5509999999999998e-06 2024-05-11T23:03:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018911+0000", 2024-05-11T23:03:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046608194999999998 2024-05-11T23:03:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018986+0000", 2024-05-11T23:03:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5360999999999999e-05 2024-05-11T23:03:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019122+0000", 2024-05-11T23:03:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013526 2024-05-11T23:03:03.189 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78598 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 47.148899153999999, 2024-05-11T23:03:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046998572000000002, 2024-05-11T23:03:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.190 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78598 2024-05-11T23:03:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:03:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972307+0000", 2024-05-11T23:03:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1799999999999999e-06 2024-05-11T23:03:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972310+0000", 2024-05-11T23:03:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:03:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972313+0000", 2024-05-11T23:03:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4760000000000001e-06 2024-05-11T23:03:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019149+0000", 2024-05-11T23:03:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046836056000000001 2024-05-11T23:03:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019203+0000", 2024-05-11T23:03:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-05 2024-05-11T23:03:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019304+0000", 2024-05-11T23:03:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101537 2024-05-11T23:03:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:105234 3.15 3:adf885e5:::rbd_object_map.111e913a615f.0000000000000025:head [writefull 0~39] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-11T23:03:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.113583444, 2024-05-11T23:03:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77797599100000003, 2024-05-11T23:03:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:03.194 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 105234 2024-05-11T23:03:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007623+0000", 2024-05-11T23:03:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2240000000000001e-06 2024-05-11T23:03:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007627+0000", 2024-05-11T23:03:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:03:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007636+0000", 2024-05-11T23:03:03.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2409999999999995e-06 2024-05-11T23:03:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007641+0000", 2024-05-11T23:03:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6810000000000003e-06 2024-05-11T23:03:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007676+0000", 2024-05-11T23:03:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5061999999999997e-05 2024-05-11T23:03:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:03:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007682+0000", 2024-05-11T23:03:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0399999999999998e-06 2024-05-11T23:03:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783880+0000", 2024-05-11T23:03:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77619752900000005 2024-05-11T23:03:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783995+0000", 2024-05-11T23:03:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011550800000000001 2024-05-11T23:03:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:03:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784082+0000", 2024-05-11T23:03:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6347999999999994e-05 2024-05-11T23:03:03.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784682+0000", 2024-05-11T23:03:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000600162 2024-05-11T23:03:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785539+0000", 2024-05-11T23:03:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085699099999999998 2024-05-11T23:03:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:03.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785572+0000", 2024-05-11T23:03:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3182999999999997e-05 2024-05-11T23:03:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785598+0000", 2024-05-11T23:03:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5522000000000001e-05 2024-05-11T23:03:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113761 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [call rbd.sparse_copyup in=999496b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:03:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.590896525000002, 2024-05-11T23:03:03.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048572964000000003, 2024-05-11T23:03:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113761 2024-05-11T23:03:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:03.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530309+0000", 2024-05-11T23:03:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:03:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531242+0000", 2024-05-11T23:03:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093316899999999997 2024-05-11T23:03:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:03.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531249+0000", 2024-05-11T23:03:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8140000000000003e-06 2024-05-11T23:03:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531253+0000", 2024-05-11T23:03:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4290000000000003e-06 2024-05-11T23:03:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570325+0000", 2024-05-11T23:03:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039071379000000003 2024-05-11T23:03:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:03.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570382+0000", 2024-05-11T23:03:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6988999999999999e-05 2024-05-11T23:03:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:03:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570555+0000", 2024-05-11T23:03:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017379799999999999 2024-05-11T23:03:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.575953+0000", 2024-05-11T23:03:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053975539999999997 2024-05-11T23:03:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578778+0000", 2024-05-11T23:03:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002825616 2024-05-11T23:03:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578837+0000", 2024-05-11T23:03:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9005000000000002e-05 2024-05-11T23:03:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578881+0000", 2024-05-11T23:03:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3970000000000001e-05 2024-05-11T23:03:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.205 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113762 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [delete] snapc 27=[27] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:03:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.589942573999998, 2024-05-11T23:03:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047916260000000002, 2024-05-11T23:03:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.206 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113762 2024-05-11T23:03:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531263+0000", 2024-05-11T23:03:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.85e-07 2024-05-11T23:03:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531265+0000", 2024-05-11T23:03:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.827e-06 2024-05-11T23:03:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531268+0000", 2024-05-11T23:03:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3780000000000001e-06 2024-05-11T23:03:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531271+0000", 2024-05-11T23:03:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8509999999999999e-06 2024-05-11T23:03:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572298+0000", 2024-05-11T23:03:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041026990999999999 2024-05-11T23:03:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572345+0000", 2024-05-11T23:03:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6878000000000001e-05 2024-05-11T23:03:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:03:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572443+0000", 2024-05-11T23:03:03.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8184999999999996e-05 2024-05-11T23:03:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578585+0000", 2024-05-11T23:03:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006141919 2024-05-11T23:03:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579126+0000", 2024-05-11T23:03:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00054063799999999997 2024-05-11T23:03:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:03.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579154+0000", 2024-05-11T23:03:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8042999999999999e-05 2024-05-11T23:03:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579179+0000", 2024-05-11T23:03:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4765000000000001e-05 2024-05-11T23:03:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:03.211 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:03.211 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:03.212 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.1 primary-affinity to 0.935317 (8612962) 2024-05-11T23:03:03.213 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-05-11T23:03:03.214 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:03:03.215 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:03.212+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:03.314 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:03.313+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:03.415 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:03.415+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:03.426 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:03.426 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:03.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.426 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238479 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.426 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.390754+0000", 2024-05-11T23:03:03.426 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.015672393, 2024-05-11T23:03:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015693819000000001, 2024-05-11T23:03:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238479 2024-05-11T23:03:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390754+0000", 2024-05-11T23:03:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390754+0000", 2024-05-11T23:03:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390754+0000", 2024-05-11T23:03:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6800000000000001e-07 2024-05-11T23:03:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390757+0000", 2024-05-11T23:03:03.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.508e-06 2024-05-11T23:03:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390770+0000", 2024-05-11T23:03:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3623e-05 2024-05-11T23:03:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390777+0000", 2024-05-11T23:03:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.438e-06 2024-05-11T23:03:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238511 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.391992+0000", 2024-05-11T23:03:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.014434327, 2024-05-11T23:03:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014523582, 2024-05-11T23:03:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238511 2024-05-11T23:03:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.391992+0000", 2024-05-11T23:03:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.391992+0000", 2024-05-11T23:03:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.391992+0000", 2024-05-11T23:03:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8599999999999998e-07 2024-05-11T23:03:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.391995+0000", 2024-05-11T23:03:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4430000000000002e-06 2024-05-11T23:03:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392003+0000", 2024-05-11T23:03:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0420000000000003e-06 2024-05-11T23:03:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392008+0000", 2024-05-11T23:03:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1939999999999997e-06 2024-05-11T23:03:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.432 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238543 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.392879+0000", 2024-05-11T23:03:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013547569000000001, 2024-05-11T23:03:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013695690999999999, 2024-05-11T23:03:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238543 2024-05-11T23:03:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.433 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392879+0000", 2024-05-11T23:03:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392879+0000", 2024-05-11T23:03:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392879+0000", 2024-05-11T23:03:03.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-05-11T23:03:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392880+0000", 2024-05-11T23:03:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.13e-06 2024-05-11T23:03:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392883+0000", 2024-05-11T23:03:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2830000000000002e-06 2024-05-11T23:03:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392886+0000", 2024-05-11T23:03:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4499999999999998e-06 2024-05-11T23:03:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238575 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393593+0000", 2024-05-11T23:03:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012833091, 2024-05-11T23:03:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01304494, 2024-05-11T23:03:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238575 2024-05-11T23:03:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393593+0000", 2024-05-11T23:03:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393593+0000", 2024-05-11T23:03:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393593+0000", 2024-05-11T23:03:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4999999999999999e-07 2024-05-11T23:03:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393594+0000", 2024-05-11T23:03:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.102e-06 2024-05-11T23:03:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393598+0000", 2024-05-11T23:03:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.247e-06 2024-05-11T23:03:03.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393600+0000", 2024-05-11T23:03:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5830000000000001e-06 2024-05-11T23:03:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238609 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394257+0000", 2024-05-11T23:03:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012169139000000001, 2024-05-11T23:03:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012440052, 2024-05-11T23:03:03.439 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238609 2024-05-11T23:03:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394257+0000", 2024-05-11T23:03:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394257+0000", 2024-05-11T23:03:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394257+0000", 2024-05-11T23:03:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3599999999999999e-07 2024-05-11T23:03:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394260+0000", 2024-05-11T23:03:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1909999999999999e-06 2024-05-11T23:03:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394270+0000", 2024-05-11T23:03:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0268000000000001e-05 2024-05-11T23:03:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394280+0000", 2024-05-11T23:03:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0329999999999999e-05 2024-05-11T23:03:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.442 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238641 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394954+0000", 2024-05-11T23:03:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011471985000000001, 2024-05-11T23:03:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011804181, 2024-05-11T23:03:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238641 2024-05-11T23:03:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.443 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394954+0000", 2024-05-11T23:03:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394954+0000", 2024-05-11T23:03:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394955+0000", 2024-05-11T23:03:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4899999999999995e-07 2024-05-11T23:03:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394957+0000", 2024-05-11T23:03:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4590000000000001e-06 2024-05-11T23:03:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394964+0000", 2024-05-11T23:03:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3250000000000002e-06 2024-05-11T23:03:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394970+0000", 2024-05-11T23:03:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3489999999999999e-06 2024-05-11T23:03:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238673 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.397129+0000", 2024-05-11T23:03:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0092972660000000002, 2024-05-11T23:03:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0096916659999999998, 2024-05-11T23:03:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238673 2024-05-11T23:03:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397129+0000", 2024-05-11T23:03:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397129+0000", 2024-05-11T23:03:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.398916+0000", 2024-05-11T23:03:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017871860000000001 2024-05-11T23:03:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.398921+0000", 2024-05-11T23:03:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6009999999999997e-06 2024-05-11T23:03:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.398930+0000", 2024-05-11T23:03:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4800000000000007e-06 2024-05-11T23:03:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.398937+0000", 2024-05-11T23:03:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9280000000000004e-06 2024-05-11T23:03:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.448 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238709 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.403117+0000", 2024-05-11T23:03:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0033093609999999998, 2024-05-11T23:03:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00376589, 2024-05-11T23:03:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238709 2024-05-11T23:03:03.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403117+0000", 2024-05-11T23:03:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403117+0000", 2024-05-11T23:03:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403117+0000", 2024-05-11T23:03:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4999999999999998e-07 2024-05-11T23:03:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403119+0000", 2024-05-11T23:03:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1679999999999998e-06 2024-05-11T23:03:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403126+0000", 2024-05-11T23:03:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2319999999999998e-06 2024-05-11T23:03:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403131+0000", 2024-05-11T23:03:03.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1630000000000004e-06 2024-05-11T23:03:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238480 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.390790+0000", 2024-05-11T23:03:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.015635609000000002, 2024-05-11T23:03:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016157279, 2024-05-11T23:03:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.452 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238480 2024-05-11T23:03:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390790+0000", 2024-05-11T23:03:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390790+0000", 2024-05-11T23:03:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390791+0000", 2024-05-11T23:03:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6199999999999998e-07 2024-05-11T23:03:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390794+0000", 2024-05-11T23:03:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.57e-06 2024-05-11T23:03:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390803+0000", 2024-05-11T23:03:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4129999999999995e-06 2024-05-11T23:03:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390808+0000", 2024-05-11T23:03:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1549999999999998e-06 2024-05-11T23:03:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238512 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.392038+0000", 2024-05-11T23:03:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.014388211999999999, 2024-05-11T23:03:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014971948000000001, 2024-05-11T23:03:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.455 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238512 2024-05-11T23:03:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392038+0000", 2024-05-11T23:03:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392038+0000", 2024-05-11T23:03:03.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392038+0000", 2024-05-11T23:03:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7599999999999997e-07 2024-05-11T23:03:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392041+0000", 2024-05-11T23:03:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.768e-06 2024-05-11T23:03:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392049+0000", 2024-05-11T23:03:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3370000000000002e-06 2024-05-11T23:03:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392058+0000", 2024-05-11T23:03:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6220000000000004e-06 2024-05-11T23:03:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238544 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.392889+0000", 2024-05-11T23:03:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013537543000000001, 2024-05-11T23:03:03.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014180679, 2024-05-11T23:03:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238544 2024-05-11T23:03:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392889+0000", 2024-05-11T23:03:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392889+0000", 2024-05-11T23:03:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392889+0000", 2024-05-11T23:03:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2999999999999999e-07 2024-05-11T23:03:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392890+0000", 2024-05-11T23:03:03.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0860000000000001e-06 2024-05-11T23:03:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392893+0000", 2024-05-11T23:03:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1709999999999998e-06 2024-05-11T23:03:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392895+0000", 2024-05-11T23:03:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3939999999999999e-06 2024-05-11T23:03:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238576 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393604+0000", 2024-05-11T23:03:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012822205999999999, 2024-05-11T23:03:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013526247, 2024-05-11T23:03:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238576 2024-05-11T23:03:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393604+0000", 2024-05-11T23:03:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393604+0000", 2024-05-11T23:03:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393604+0000", 2024-05-11T23:03:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.16e-07 2024-05-11T23:03:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393605+0000", 2024-05-11T23:03:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.083e-06 2024-05-11T23:03:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393609+0000", 2024-05-11T23:03:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9539999999999998e-06 2024-05-11T23:03:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393612+0000", 2024-05-11T23:03:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5270000000000001e-06 2024-05-11T23:03:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.464 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238610 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394296+0000", 2024-05-11T23:03:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012129891, 2024-05-11T23:03:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012892073, 2024-05-11T23:03:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238610 2024-05-11T23:03:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.465 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394296+0000", 2024-05-11T23:03:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394296+0000", 2024-05-11T23:03:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394297+0000", 2024-05-11T23:03:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7800000000000002e-07 2024-05-11T23:03:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394299+0000", 2024-05-11T23:03:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.322e-06 2024-05-11T23:03:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394312+0000", 2024-05-11T23:03:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3458999999999999e-05 2024-05-11T23:03:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394322+0000", 2024-05-11T23:03:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4180000000000003e-06 2024-05-11T23:03:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238642 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394979+0000", 2024-05-11T23:03:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011446959, 2024-05-11T23:03:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012268036, 2024-05-11T23:03:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.468 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238642 2024-05-11T23:03:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394979+0000", 2024-05-11T23:03:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394979+0000", 2024-05-11T23:03:03.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394980+0000", 2024-05-11T23:03:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:03:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394982+0000", 2024-05-11T23:03:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2510000000000001e-06 2024-05-11T23:03:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394988+0000", 2024-05-11T23:03:03.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6640000000000001e-06 2024-05-11T23:03:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394995+0000", 2024-05-11T23:03:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3069999999999999e-06 2024-05-11T23:03:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238674 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.398949+0000", 2024-05-11T23:03:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0074770440000000004, 2024-05-11T23:03:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0083581929999999999, 2024-05-11T23:03:03.471 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238674 2024-05-11T23:03:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.398949+0000", 2024-05-11T23:03:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.398949+0000", 2024-05-11T23:03:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.398949+0000", 2024-05-11T23:03:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2399999999999999e-07 2024-05-11T23:03:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.398952+0000", 2024-05-11T23:03:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7209999999999999e-06 2024-05-11T23:03:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.398959+0000", 2024-05-11T23:03:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1849999999999998e-06 2024-05-11T23:03:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.398965+0000", 2024-05-11T23:03:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3539999999999999e-06 2024-05-11T23:03:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.474 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238710 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.403137+0000", 2024-05-11T23:03:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.003288713, 2024-05-11T23:03:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0042285600000000001, 2024-05-11T23:03:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238710 2024-05-11T23:03:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403137+0000", 2024-05-11T23:03:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403137+0000", 2024-05-11T23:03:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403138+0000", 2024-05-11T23:03:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.84e-07 2024-05-11T23:03:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403140+0000", 2024-05-11T23:03:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:03:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403147+0000", 2024-05-11T23:03:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.396e-06 2024-05-11T23:03:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403152+0000", 2024-05-11T23:03:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1019999999999996e-06 2024-05-11T23:03:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.477 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238481 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.390825+0000", 2024-05-11T23:03:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.015600686000000001, 2024-05-11T23:03:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016598061000000001, 2024-05-11T23:03:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238481 2024-05-11T23:03:03.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390825+0000", 2024-05-11T23:03:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390825+0000", 2024-05-11T23:03:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390826+0000", 2024-05-11T23:03:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:03:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390828+0000", 2024-05-11T23:03:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4099999999999998e-06 2024-05-11T23:03:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390836+0000", 2024-05-11T23:03:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2989999999999999e-06 2024-05-11T23:03:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390841+0000", 2024-05-11T23:03:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1950000000000002e-06 2024-05-11T23:03:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238513 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.392085+0000", 2024-05-11T23:03:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.014341372999999999, 2024-05-11T23:03:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015397605999999999, 2024-05-11T23:03:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.481 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238513 2024-05-11T23:03:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392085+0000", 2024-05-11T23:03:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392085+0000", 2024-05-11T23:03:03.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392085+0000", 2024-05-11T23:03:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6400000000000002e-07 2024-05-11T23:03:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392088+0000", 2024-05-11T23:03:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7089999999999998e-06 2024-05-11T23:03:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392095+0000", 2024-05-11T23:03:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3089999999999998e-06 2024-05-11T23:03:03.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392100+0000", 2024-05-11T23:03:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.989e-06 2024-05-11T23:03:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238545 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.392898+0000", 2024-05-11T23:03:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013527796, 2024-05-11T23:03:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014644012, 2024-05-11T23:03:03.484 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238545 2024-05-11T23:03:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392898+0000", 2024-05-11T23:03:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392898+0000", 2024-05-11T23:03:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392899+0000", 2024-05-11T23:03:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:03:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392903+0000", 2024-05-11T23:03:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1540000000000004e-06 2024-05-11T23:03:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392906+0000", 2024-05-11T23:03:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.557e-06 2024-05-11T23:03:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392918+0000", 2024-05-11T23:03:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1778e-05 2024-05-11T23:03:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238577 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393616+0000", 2024-05-11T23:03:03.487 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012810178, 2024-05-11T23:03:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013986521999999999, 2024-05-11T23:03:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238577 2024-05-11T23:03:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393616+0000", 2024-05-11T23:03:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393616+0000", 2024-05-11T23:03:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393616+0000", 2024-05-11T23:03:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3200000000000001e-07 2024-05-11T23:03:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393617+0000", 2024-05-11T23:03:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.079e-06 2024-05-11T23:03:03.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393621+0000", 2024-05-11T23:03:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3340000000000002e-06 2024-05-11T23:03:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393623+0000", 2024-05-11T23:03:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:03:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.490 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238611 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394369+0000", 2024-05-11T23:03:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012057385, 2024-05-11T23:03:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013294934, 2024-05-11T23:03:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238611 2024-05-11T23:03:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394369+0000", 2024-05-11T23:03:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394369+0000", 2024-05-11T23:03:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394369+0000", 2024-05-11T23:03:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6300000000000001e-07 2024-05-11T23:03:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394371+0000", 2024-05-11T23:03:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0269999999999998e-06 2024-05-11T23:03:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394381+0000", 2024-05-11T23:03:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.6579999999999997e-06 2024-05-11T23:03:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394387+0000", 2024-05-11T23:03:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7320000000000003e-06 2024-05-11T23:03:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238643 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395005+0000", 2024-05-11T23:03:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011420616999999999, 2024-05-11T23:03:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012719950000000001, 2024-05-11T23:03:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238643 2024-05-11T23:03:03.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395005+0000", 2024-05-11T23:03:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395005+0000", 2024-05-11T23:03:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395006+0000", 2024-05-11T23:03:03.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7e-07 2024-05-11T23:03:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395008+0000", 2024-05-11T23:03:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.26e-06 2024-05-11T23:03:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395017+0000", 2024-05-11T23:03:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.388e-06 2024-05-11T23:03:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395022+0000", 2024-05-11T23:03:03.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1120000000000004e-06 2024-05-11T23:03:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238675 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.398972+0000", 2024-05-11T23:03:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0074538800000000004, 2024-05-11T23:03:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0088135590000000003, 2024-05-11T23:03:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.497 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238675 2024-05-11T23:03:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.398972+0000", 2024-05-11T23:03:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.398972+0000", 2024-05-11T23:03:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.398973+0000", 2024-05-11T23:03:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1800000000000001e-07 2024-05-11T23:03:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.398975+0000", 2024-05-11T23:03:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6520000000000002e-06 2024-05-11T23:03:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.398982+0000", 2024-05-11T23:03:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.516e-06 2024-05-11T23:03:03.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.398987+0000", 2024-05-11T23:03:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9110000000000001e-06 2024-05-11T23:03:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238711 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.403159+0000", 2024-05-11T23:03:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0032670149999999998, 2024-05-11T23:03:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0046869060000000002, 2024-05-11T23:03:03.500 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238711 2024-05-11T23:03:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403159+0000", 2024-05-11T23:03:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403159+0000", 2024-05-11T23:03:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403160+0000", 2024-05-11T23:03:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3099999999999998e-07 2024-05-11T23:03:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403162+0000", 2024-05-11T23:03:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2199999999999999e-06 2024-05-11T23:03:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403168+0000", 2024-05-11T23:03:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4509999999999997e-06 2024-05-11T23:03:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403173+0000", 2024-05-11T23:03:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8049999999999997e-06 2024-05-11T23:03:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238482 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.390851+0000", 2024-05-11T23:03:03.503 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.015575209, 2024-05-11T23:03:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017055270000000001, 2024-05-11T23:03:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238482 2024-05-11T23:03:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390851+0000", 2024-05-11T23:03:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390851+0000", 2024-05-11T23:03:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390851+0000", 2024-05-11T23:03:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8200000000000001e-07 2024-05-11T23:03:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390854+0000", 2024-05-11T23:03:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.948e-06 2024-05-11T23:03:03.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390863+0000", 2024-05-11T23:03:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0010000000000007e-06 2024-05-11T23:03:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.390869+0000", 2024-05-11T23:03:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.908e-06 2024-05-11T23:03:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.506 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238514 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.392129+0000", 2024-05-11T23:03:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.014296612, 2024-05-11T23:03:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015840776000000001, 2024-05-11T23:03:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238514 2024-05-11T23:03:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392129+0000", 2024-05-11T23:03:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392129+0000", 2024-05-11T23:03:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392130+0000", 2024-05-11T23:03:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5799999999999999e-07 2024-05-11T23:03:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392133+0000", 2024-05-11T23:03:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.706e-06 2024-05-11T23:03:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392140+0000", 2024-05-11T23:03:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1040000000000001e-06 2024-05-11T23:03:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392147+0000", 2024-05-11T23:03:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7939999999999997e-06 2024-05-11T23:03:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238546 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.392929+0000", 2024-05-11T23:03:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013497454000000001, 2024-05-11T23:03:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015103836000000001, 2024-05-11T23:03:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238546 2024-05-11T23:03:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392929+0000", 2024-05-11T23:03:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392929+0000", 2024-05-11T23:03:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392929+0000", 2024-05-11T23:03:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7099999999999998e-07 2024-05-11T23:03:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392931+0000", 2024-05-11T23:03:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.728e-06 2024-05-11T23:03:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392938+0000", 2024-05-11T23:03:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0620000000000003e-06 2024-05-11T23:03:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392942+0000", 2024-05-11T23:03:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2679999999999996e-06 2024-05-11T23:03:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.512 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238578 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393626+0000", 2024-05-11T23:03:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012799975999999999, 2024-05-11T23:03:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01446621, 2024-05-11T23:03:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238578 2024-05-11T23:03:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.513 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393626+0000", 2024-05-11T23:03:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393626+0000", 2024-05-11T23:03:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393626+0000", 2024-05-11T23:03:03.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-05-11T23:03:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393627+0000", 2024-05-11T23:03:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1009999999999999e-06 2024-05-11T23:03:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393631+0000", 2024-05-11T23:03:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.207e-06 2024-05-11T23:03:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393633+0000", 2024-05-11T23:03:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5409999999999999e-06 2024-05-11T23:03:03.515 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238612 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394396+0000", 2024-05-11T23:03:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012030323000000001, 2024-05-11T23:03:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013755392999999999, 2024-05-11T23:03:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.516 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238612 2024-05-11T23:03:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394396+0000", 2024-05-11T23:03:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394396+0000", 2024-05-11T23:03:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394396+0000", 2024-05-11T23:03:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.22e-07 2024-05-11T23:03:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394397+0000", 2024-05-11T23:03:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3459999999999999e-06 2024-05-11T23:03:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394403+0000", 2024-05-11T23:03:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6729999999999997e-06 2024-05-11T23:03:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394407+0000", 2024-05-11T23:03:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9949999999999999e-06 2024-05-11T23:03:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238644 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395029+0000", 2024-05-11T23:03:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011397575, 2024-05-11T23:03:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013181365, 2024-05-11T23:03:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.519 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238644 2024-05-11T23:03:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395029+0000", 2024-05-11T23:03:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395029+0000", 2024-05-11T23:03:03.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395029+0000", 2024-05-11T23:03:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:03:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395031+0000", 2024-05-11T23:03:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3020000000000002e-06 2024-05-11T23:03:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395039+0000", 2024-05-11T23:03:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2379999999999997e-06 2024-05-11T23:03:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395045+0000", 2024-05-11T23:03:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1259999999999997e-06 2024-05-11T23:03:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238676 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.398995+0000", 2024-05-11T23:03:03.522 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0074306290000000002, 2024-05-11T23:03:03.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0092756690000000003, 2024-05-11T23:03:03.523 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.523 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.523 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.523 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.523 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.523 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.523 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238676 2024-05-11T23:03:03.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.523 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.398995+0000", 2024-05-11T23:03:03.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.398995+0000", 2024-05-11T23:03:03.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.398996+0000", 2024-05-11T23:03:03.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:03:03.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.398998+0000", 2024-05-11T23:03:03.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.396e-06 2024-05-11T23:03:03.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399006+0000", 2024-05-11T23:03:03.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.345e-06 2024-05-11T23:03:03.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399012+0000", 2024-05-11T23:03:03.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1380000000000004e-06 2024-05-11T23:03:03.525 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.525 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.525 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.525 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238712 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.526 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.403179+0000", 2024-05-11T23:03:03.526 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.003246751, 2024-05-11T23:03:03.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0051529779999999999, 2024-05-11T23:03:03.526 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.526 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.526 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.526 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.526 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.526 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.526 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238712 2024-05-11T23:03:03.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.526 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403179+0000", 2024-05-11T23:03:03.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403179+0000", 2024-05-11T23:03:03.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403180+0000", 2024-05-11T23:03:03.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9299999999999998e-07 2024-05-11T23:03:03.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403182+0000", 2024-05-11T23:03:03.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.018e-06 2024-05-11T23:03:03.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403188+0000", 2024-05-11T23:03:03.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5060000000000001e-06 2024-05-11T23:03:03.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403194+0000", 2024-05-11T23:03:03.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9440000000000002e-06 2024-05-11T23:03:03.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.528 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.529 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238515 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.529 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.392156+0000", 2024-05-11T23:03:03.529 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.014269693, 2024-05-11T23:03:03.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016237221999999999, 2024-05-11T23:03:03.529 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.529 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.529 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238515 2024-05-11T23:03:03.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.529 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392156+0000", 2024-05-11T23:03:03.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392156+0000", 2024-05-11T23:03:03.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392157+0000", 2024-05-11T23:03:03.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4000000000000002e-07 2024-05-11T23:03:03.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392159+0000", 2024-05-11T23:03:03.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3489999999999999e-06 2024-05-11T23:03:03.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392167+0000", 2024-05-11T23:03:03.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0509999999999997e-06 2024-05-11T23:03:03.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392173+0000", 2024-05-11T23:03:03.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5119999999999999e-06 2024-05-11T23:03:03.531 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.532 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.532 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.532 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238547 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.532 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.392972+0000", 2024-05-11T23:03:03.532 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013454107999999999, 2024-05-11T23:03:03.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015482624, 2024-05-11T23:03:03.532 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.532 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.532 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.532 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.532 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.532 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.532 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238547 2024-05-11T23:03:03.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.533 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392972+0000", 2024-05-11T23:03:03.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392972+0000", 2024-05-11T23:03:03.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392973+0000", 2024-05-11T23:03:03.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3799999999999997e-07 2024-05-11T23:03:03.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392975+0000", 2024-05-11T23:03:03.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5160000000000001e-06 2024-05-11T23:03:03.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392984+0000", 2024-05-11T23:03:03.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8429999999999999e-06 2024-05-11T23:03:03.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392989+0000", 2024-05-11T23:03:03.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2009999999999998e-06 2024-05-11T23:03:03.535 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.535 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.535 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.535 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238579 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.535 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393636+0000", 2024-05-11T23:03:03.535 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012789821, 2024-05-11T23:03:03.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014920556999999999, 2024-05-11T23:03:03.535 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.535 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.535 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.536 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.536 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.536 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.536 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238579 2024-05-11T23:03:03.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.536 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393636+0000", 2024-05-11T23:03:03.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393636+0000", 2024-05-11T23:03:03.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393636+0000", 2024-05-11T23:03:03.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0800000000000001e-07 2024-05-11T23:03:03.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393638+0000", 2024-05-11T23:03:03.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0300000000000001e-06 2024-05-11T23:03:03.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393641+0000", 2024-05-11T23:03:03.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1769999999999998e-06 2024-05-11T23:03:03.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393644+0000", 2024-05-11T23:03:03.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6799999999999999e-06 2024-05-11T23:03:03.538 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.538 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.538 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.538 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238613 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.538 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394410+0000", 2024-05-11T23:03:03.538 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012015705999999999, 2024-05-11T23:03:03.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014208604, 2024-05-11T23:03:03.539 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.539 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.539 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.539 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.539 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.539 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.539 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238613 2024-05-11T23:03:03.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.539 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394410+0000", 2024-05-11T23:03:03.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394410+0000", 2024-05-11T23:03:03.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394411+0000", 2024-05-11T23:03:03.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7099999999999998e-07 2024-05-11T23:03:03.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394412+0000", 2024-05-11T23:03:03.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1310000000000001e-06 2024-05-11T23:03:03.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394421+0000", 2024-05-11T23:03:03.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3579999999999992e-06 2024-05-11T23:03:03.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394425+0000", 2024-05-11T23:03:03.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9519999999999996e-06 2024-05-11T23:03:03.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.541 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.541 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238645 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.542 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395051+0000", 2024-05-11T23:03:03.542 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011375053, 2024-05-11T23:03:03.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013627617, 2024-05-11T23:03:03.542 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.542 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.542 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.542 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.542 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.542 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238645 2024-05-11T23:03:03.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.542 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395051+0000", 2024-05-11T23:03:03.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395051+0000", 2024-05-11T23:03:03.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395051+0000", 2024-05-11T23:03:03.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.65e-07 2024-05-11T23:03:03.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395054+0000", 2024-05-11T23:03:03.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.21e-06 2024-05-11T23:03:03.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395061+0000", 2024-05-11T23:03:03.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3839999999999999e-06 2024-05-11T23:03:03.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395066+0000", 2024-05-11T23:03:03.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9280000000000001e-06 2024-05-11T23:03:03.544 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.544 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.544 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.545 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238677 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.545 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.399021+0000", 2024-05-11T23:03:03.545 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0074053360000000002, 2024-05-11T23:03:03.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0097233370000000003, 2024-05-11T23:03:03.545 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.545 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.545 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.545 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.545 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.545 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.545 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238677 2024-05-11T23:03:03.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.546 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399021+0000", 2024-05-11T23:03:03.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399021+0000", 2024-05-11T23:03:03.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.546 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.546 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399021+0000", 2024-05-11T23:03:03.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.15e-07 2024-05-11T23:03:03.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399024+0000", 2024-05-11T23:03:03.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.368e-06 2024-05-11T23:03:03.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399030+0000", 2024-05-11T23:03:03.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7599999999999997e-06 2024-05-11T23:03:03.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399035+0000", 2024-05-11T23:03:03.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0690000000000001e-06 2024-05-11T23:03:03.548 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.548 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.548 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.548 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238713 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.548 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.403216+0000", 2024-05-11T23:03:03.548 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.003209721, 2024-05-11T23:03:03.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0055870729999999997, 2024-05-11T23:03:03.548 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.548 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.548 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.548 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.549 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.549 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238713 2024-05-11T23:03:03.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.549 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403216+0000", 2024-05-11T23:03:03.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403216+0000", 2024-05-11T23:03:03.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403217+0000", 2024-05-11T23:03:03.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:03:03.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.550 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.550 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403219+0000", 2024-05-11T23:03:03.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5339999999999998e-06 2024-05-11T23:03:03.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403227+0000", 2024-05-11T23:03:03.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5449999999999998e-06 2024-05-11T23:03:03.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403232+0000", 2024-05-11T23:03:03.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2719999999999997e-06 2024-05-11T23:03:03.551 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.551 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.551 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.552 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238516 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.552 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.392181+0000", 2024-05-11T23:03:03.552 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.01424517, 2024-05-11T23:03:03.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016687058000000001, 2024-05-11T23:03:03.552 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.552 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.552 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.552 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.552 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238516 2024-05-11T23:03:03.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.552 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392181+0000", 2024-05-11T23:03:03.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392181+0000", 2024-05-11T23:03:03.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392181+0000", 2024-05-11T23:03:03.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8099999999999998e-07 2024-05-11T23:03:03.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392184+0000", 2024-05-11T23:03:03.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2840000000000001e-06 2024-05-11T23:03:03.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392192+0000", 2024-05-11T23:03:03.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1720000000000007e-06 2024-05-11T23:03:03.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.554 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.554 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392197+0000", 2024-05-11T23:03:03.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1680000000000003e-06 2024-05-11T23:03:03.554 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.554 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.555 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.555 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238548 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.555 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393000+0000", 2024-05-11T23:03:03.555 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013425764, 2024-05-11T23:03:03.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015931424999999999, 2024-05-11T23:03:03.555 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.555 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.555 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.555 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.555 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238548 2024-05-11T23:03:03.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.556 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393000+0000", 2024-05-11T23:03:03.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393000+0000", 2024-05-11T23:03:03.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393001+0000", 2024-05-11T23:03:03.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8899999999999999e-07 2024-05-11T23:03:03.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393003+0000", 2024-05-11T23:03:03.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4080000000000001e-06 2024-05-11T23:03:03.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393010+0000", 2024-05-11T23:03:03.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8959999999999997e-06 2024-05-11T23:03:03.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393015+0000", 2024-05-11T23:03:03.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2610000000000001e-06 2024-05-11T23:03:03.558 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.558 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.558 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.558 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238580 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.558 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393647+0000", 2024-05-11T23:03:03.558 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012778606, 2024-05-11T23:03:03.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015350582999999999, 2024-05-11T23:03:03.558 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.558 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.558 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.558 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.558 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238580 2024-05-11T23:03:03.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.559 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393647+0000", 2024-05-11T23:03:03.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393647+0000", 2024-05-11T23:03:03.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393648+0000", 2024-05-11T23:03:03.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.17e-07 2024-05-11T23:03:03.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393649+0000", 2024-05-11T23:03:03.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1519999999999999e-06 2024-05-11T23:03:03.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393653+0000", 2024-05-11T23:03:03.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1579999999999998e-06 2024-05-11T23:03:03.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393657+0000", 2024-05-11T23:03:03.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6720000000000002e-06 2024-05-11T23:03:03.561 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.561 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.561 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.561 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238614 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.561 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394430+0000", 2024-05-11T23:03:03.561 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011996556, 2024-05-11T23:03:03.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014630539, 2024-05-11T23:03:03.561 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.561 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.562 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.562 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.562 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.562 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.562 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238614 2024-05-11T23:03:03.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.562 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394430+0000", 2024-05-11T23:03:03.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394430+0000", 2024-05-11T23:03:03.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394430+0000", 2024-05-11T23:03:03.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:03.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394432+0000", 2024-05-11T23:03:03.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.717e-06 2024-05-11T23:03:03.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394446+0000", 2024-05-11T23:03:03.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3465999999999999e-05 2024-05-11T23:03:03.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394452+0000", 2024-05-11T23:03:03.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8549999999999998e-06 2024-05-11T23:03:03.564 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.564 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.564 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.564 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238646 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.564 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395072+0000", 2024-05-11T23:03:03.564 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011353845, 2024-05-11T23:03:03.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014054283000000001, 2024-05-11T23:03:03.565 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.565 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.565 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.565 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.565 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.565 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238646 2024-05-11T23:03:03.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.565 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395072+0000", 2024-05-11T23:03:03.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395072+0000", 2024-05-11T23:03:03.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395073+0000", 2024-05-11T23:03:03.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2899999999999999e-07 2024-05-11T23:03:03.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395075+0000", 2024-05-11T23:03:03.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1660000000000001e-06 2024-05-11T23:03:03.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395085+0000", 2024-05-11T23:03:03.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0061000000000001e-05 2024-05-11T23:03:03.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395093+0000", 2024-05-11T23:03:03.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0350000000000001e-06 2024-05-11T23:03:03.568 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.568 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.568 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.568 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238678 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.568 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.399043+0000", 2024-05-11T23:03:03.568 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0073826550000000001, 2024-05-11T23:03:03.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.010143945999999999, 2024-05-11T23:03:03.569 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.569 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.569 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.569 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.569 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.569 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238678 2024-05-11T23:03:03.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.569 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399043+0000", 2024-05-11T23:03:03.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399043+0000", 2024-05-11T23:03:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399044+0000", 2024-05-11T23:03:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9400000000000001e-07 2024-05-11T23:03:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399050+0000", 2024-05-11T23:03:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2480000000000001e-06 2024-05-11T23:03:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399057+0000", 2024-05-11T23:03:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.216e-06 2024-05-11T23:03:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399062+0000", 2024-05-11T23:03:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1669999999999998e-06 2024-05-11T23:03:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238714 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.403239+0000", 2024-05-11T23:03:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0031867110000000001, 2024-05-11T23:03:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0060117369999999996, 2024-05-11T23:03:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238714 2024-05-11T23:03:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403239+0000", 2024-05-11T23:03:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403239+0000", 2024-05-11T23:03:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403240+0000", 2024-05-11T23:03:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6599999999999996e-07 2024-05-11T23:03:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403242+0000", 2024-05-11T23:03:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2029999999999999e-06 2024-05-11T23:03:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403248+0000", 2024-05-11T23:03:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1750000000000002e-06 2024-05-11T23:03:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403253+0000", 2024-05-11T23:03:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1019999999999996e-06 2024-05-11T23:03:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238517 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.392204+0000", 2024-05-11T23:03:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.01422238, 2024-05-11T23:03:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017109488999999999, 2024-05-11T23:03:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238517 2024-05-11T23:03:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392204+0000", 2024-05-11T23:03:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392204+0000", 2024-05-11T23:03:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392204+0000", 2024-05-11T23:03:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1900000000000003e-07 2024-05-11T23:03:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392206+0000", 2024-05-11T23:03:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1390000000000002e-06 2024-05-11T23:03:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392213+0000", 2024-05-11T23:03:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8469999999999998e-06 2024-05-11T23:03:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392218+0000", 2024-05-11T23:03:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0509999999999996e-06 2024-05-11T23:03:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238549 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393023+0000", 2024-05-11T23:03:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013403459, 2024-05-11T23:03:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016356655000000001, 2024-05-11T23:03:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238549 2024-05-11T23:03:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393023+0000", 2024-05-11T23:03:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393023+0000", 2024-05-11T23:03:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393023+0000", 2024-05-11T23:03:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0200000000000003e-07 2024-05-11T23:03:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393025+0000", 2024-05-11T23:03:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9719999999999999e-06 2024-05-11T23:03:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393034+0000", 2024-05-11T23:03:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2380000000000003e-06 2024-05-11T23:03:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393040+0000", 2024-05-11T23:03:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6550000000000001e-06 2024-05-11T23:03:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238581 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393664+0000", 2024-05-11T23:03:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012762351, 2024-05-11T23:03:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015775896000000001, 2024-05-11T23:03:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238581 2024-05-11T23:03:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393664+0000", 2024-05-11T23:03:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393664+0000", 2024-05-11T23:03:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393664+0000", 2024-05-11T23:03:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9299999999999998e-07 2024-05-11T23:03:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393670+0000", 2024-05-11T23:03:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8649999999999998e-06 2024-05-11T23:03:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393677+0000", 2024-05-11T23:03:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5729999999999996e-06 2024-05-11T23:03:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393683+0000", 2024-05-11T23:03:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3799999999999999e-06 2024-05-11T23:03:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238615 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394461+0000", 2024-05-11T23:03:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011965584, 2024-05-11T23:03:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015043608, 2024-05-11T23:03:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238615 2024-05-11T23:03:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394461+0000", 2024-05-11T23:03:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394461+0000", 2024-05-11T23:03:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394461+0000", 2024-05-11T23:03:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.72e-07 2024-05-11T23:03:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394462+0000", 2024-05-11T23:03:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4839999999999999e-06 2024-05-11T23:03:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394467+0000", 2024-05-11T23:03:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9339999999999997e-06 2024-05-11T23:03:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394470+0000", 2024-05-11T23:03:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.762e-06 2024-05-11T23:03:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238647 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395102+0000", 2024-05-11T23:03:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011324338999999999, 2024-05-11T23:03:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014462414, 2024-05-11T23:03:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238647 2024-05-11T23:03:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395102+0000", 2024-05-11T23:03:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395102+0000", 2024-05-11T23:03:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395102+0000", 2024-05-11T23:03:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:03:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395105+0000", 2024-05-11T23:03:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.447e-06 2024-05-11T23:03:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395118+0000", 2024-05-11T23:03:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3604999999999999e-05 2024-05-11T23:03:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395128+0000", 2024-05-11T23:03:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0047999999999999e-05 2024-05-11T23:03:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238679 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.399070+0000", 2024-05-11T23:03:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0073559350000000001, 2024-05-11T23:03:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.010557283000000001, 2024-05-11T23:03:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238679 2024-05-11T23:03:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399070+0000", 2024-05-11T23:03:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399070+0000", 2024-05-11T23:03:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399071+0000", 2024-05-11T23:03:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8200000000000001e-07 2024-05-11T23:03:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399073+0000", 2024-05-11T23:03:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1959999999999998e-06 2024-05-11T23:03:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399078+0000", 2024-05-11T23:03:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4070000000000001e-06 2024-05-11T23:03:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399081+0000", 2024-05-11T23:03:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.999e-06 2024-05-11T23:03:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238715 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.403260+0000", 2024-05-11T23:03:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.003166061, 2024-05-11T23:03:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0064288119999999999, 2024-05-11T23:03:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238715 2024-05-11T23:03:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403260+0000", 2024-05-11T23:03:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403260+0000", 2024-05-11T23:03:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403260+0000", 2024-05-11T23:03:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.96e-07 2024-05-11T23:03:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403263+0000", 2024-05-11T23:03:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.142e-06 2024-05-11T23:03:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403269+0000", 2024-05-11T23:03:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9769999999999997e-06 2024-05-11T23:03:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403275+0000", 2024-05-11T23:03:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4860000000000002e-06 2024-05-11T23:03:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238518 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.392244+0000", 2024-05-11T23:03:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.014182563, 2024-05-11T23:03:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017507814999999999, 2024-05-11T23:03:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238518 2024-05-11T23:03:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392244+0000", 2024-05-11T23:03:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392244+0000", 2024-05-11T23:03:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392244+0000", 2024-05-11T23:03:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9100000000000004e-07 2024-05-11T23:03:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392247+0000", 2024-05-11T23:03:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5979999999999999e-06 2024-05-11T23:03:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392256+0000", 2024-05-11T23:03:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0180000000000007e-06 2024-05-11T23:03:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392261+0000", 2024-05-11T23:03:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.994e-06 2024-05-11T23:03:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238550 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393048+0000", 2024-05-11T23:03:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013378065999999999, 2024-05-11T23:03:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016764935000000002, 2024-05-11T23:03:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238550 2024-05-11T23:03:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393048+0000", 2024-05-11T23:03:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393048+0000", 2024-05-11T23:03:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393048+0000", 2024-05-11T23:03:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1800000000000001e-07 2024-05-11T23:03:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393051+0000", 2024-05-11T23:03:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2730000000000001e-06 2024-05-11T23:03:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393059+0000", 2024-05-11T23:03:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7350000000000002e-06 2024-05-11T23:03:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393065+0000", 2024-05-11T23:03:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4199999999999998e-06 2024-05-11T23:03:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238582 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393690+0000", 2024-05-11T23:03:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.01273607, 2024-05-11T23:03:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016183199999999998, 2024-05-11T23:03:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238582 2024-05-11T23:03:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393690+0000", 2024-05-11T23:03:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393690+0000", 2024-05-11T23:03:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393690+0000", 2024-05-11T23:03:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8000000000000001e-07 2024-05-11T23:03:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393693+0000", 2024-05-11T23:03:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1299999999999999e-06 2024-05-11T23:03:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393699+0000", 2024-05-11T23:03:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2670000000000003e-06 2024-05-11T23:03:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393706+0000", 2024-05-11T23:03:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2300000000000002e-06 2024-05-11T23:03:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238616 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394473+0000", 2024-05-11T23:03:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011952838, 2024-05-11T23:03:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015469070999999999, 2024-05-11T23:03:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238616 2024-05-11T23:03:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394473+0000", 2024-05-11T23:03:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394473+0000", 2024-05-11T23:03:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394473+0000", 2024-05-11T23:03:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0599999999999999e-07 2024-05-11T23:03:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394475+0000", 2024-05-11T23:03:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1289999999999999e-06 2024-05-11T23:03:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394478+0000", 2024-05-11T23:03:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2049999999999998e-06 2024-05-11T23:03:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394480+0000", 2024-05-11T23:03:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6359999999999998e-06 2024-05-11T23:03:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238648 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395136+0000", 2024-05-11T23:03:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011289888999999999, 2024-05-11T23:03:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014866927, 2024-05-11T23:03:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238648 2024-05-11T23:03:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395136+0000", 2024-05-11T23:03:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395136+0000", 2024-05-11T23:03:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395137+0000", 2024-05-11T23:03:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:03:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395139+0000", 2024-05-11T23:03:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5170000000000002e-06 2024-05-11T23:03:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395151+0000", 2024-05-11T23:03:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1990999999999999e-05 2024-05-11T23:03:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395156+0000", 2024-05-11T23:03:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2580000000000003e-06 2024-05-11T23:03:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238680 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.399085+0000", 2024-05-11T23:03:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0073408370000000002, 2024-05-11T23:03:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.010983781999999999, 2024-05-11T23:03:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238680 2024-05-11T23:03:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399085+0000", 2024-05-11T23:03:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399085+0000", 2024-05-11T23:03:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399086+0000", 2024-05-11T23:03:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4999999999999999e-07 2024-05-11T23:03:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399087+0000", 2024-05-11T23:03:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.393e-06 2024-05-11T23:03:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399090+0000", 2024-05-11T23:03:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3280000000000002e-06 2024-05-11T23:03:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399093+0000", 2024-05-11T23:03:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7199999999999998e-06 2024-05-11T23:03:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238716 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.403282+0000", 2024-05-11T23:03:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.003144296, 2024-05-11T23:03:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0068484820000000004, 2024-05-11T23:03:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238716 2024-05-11T23:03:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403282+0000", 2024-05-11T23:03:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403282+0000", 2024-05-11T23:03:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403282+0000", 2024-05-11T23:03:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1100000000000001e-07 2024-05-11T23:03:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403284+0000", 2024-05-11T23:03:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2230000000000001e-06 2024-05-11T23:03:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403291+0000", 2024-05-11T23:03:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0569999999999998e-06 2024-05-11T23:03:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403295+0000", 2024-05-11T23:03:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8579999999999999e-06 2024-05-11T23:03:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238519 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.392290+0000", 2024-05-11T23:03:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.014136381999999999, 2024-05-11T23:03:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0179052, 2024-05-11T23:03:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238519 2024-05-11T23:03:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392290+0000", 2024-05-11T23:03:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392290+0000", 2024-05-11T23:03:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392290+0000", 2024-05-11T23:03:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.75e-07 2024-05-11T23:03:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392293+0000", 2024-05-11T23:03:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5500000000000001e-06 2024-05-11T23:03:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392300+0000", 2024-05-11T23:03:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4200000000000001e-06 2024-05-11T23:03:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392306+0000", 2024-05-11T23:03:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1090000000000003e-06 2024-05-11T23:03:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238551 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393072+0000", 2024-05-11T23:03:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013354517999999999, 2024-05-11T23:03:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017183428000000001, 2024-05-11T23:03:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238551 2024-05-11T23:03:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393072+0000", 2024-05-11T23:03:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393072+0000", 2024-05-11T23:03:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393072+0000", 2024-05-11T23:03:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4200000000000001e-07 2024-05-11T23:03:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393074+0000", 2024-05-11T23:03:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4200000000000001e-06 2024-05-11T23:03:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393081+0000", 2024-05-11T23:03:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6270000000000002e-06 2024-05-11T23:03:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393086+0000", 2024-05-11T23:03:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.203e-06 2024-05-11T23:03:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238583 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393713+0000", 2024-05-11T23:03:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012713051, 2024-05-11T23:03:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016604029999999999, 2024-05-11T23:03:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238583 2024-05-11T23:03:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393713+0000", 2024-05-11T23:03:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393713+0000", 2024-05-11T23:03:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393713+0000", 2024-05-11T23:03:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.72e-07 2024-05-11T23:03:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393716+0000", 2024-05-11T23:03:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1459999999999999e-06 2024-05-11T23:03:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393724+0000", 2024-05-11T23:03:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1100000000000003e-06 2024-05-11T23:03:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393730+0000", 2024-05-11T23:03:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5189999999999998e-06 2024-05-11T23:03:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238617 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394484+0000", 2024-05-11T23:03:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011942565, 2024-05-11T23:03:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015892377999999999, 2024-05-11T23:03:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238617 2024-05-11T23:03:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394484+0000", 2024-05-11T23:03:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394484+0000", 2024-05-11T23:03:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394484+0000", 2024-05-11T23:03:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394485+0000", 2024-05-11T23:03:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.114e-06 2024-05-11T23:03:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394488+0000", 2024-05-11T23:03:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2339999999999999e-06 2024-05-11T23:03:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394491+0000", 2024-05-11T23:03:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5490000000000001e-06 2024-05-11T23:03:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238649 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395182+0000", 2024-05-11T23:03:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011244545999999999, 2024-05-11T23:03:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015257315, 2024-05-11T23:03:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238649 2024-05-11T23:03:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395182+0000", 2024-05-11T23:03:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395182+0000", 2024-05-11T23:03:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395182+0000", 2024-05-11T23:03:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4400000000000001e-07 2024-05-11T23:03:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395184+0000", 2024-05-11T23:03:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1560000000000002e-06 2024-05-11T23:03:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395189+0000", 2024-05-11T23:03:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3240000000000002e-06 2024-05-11T23:03:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395193+0000", 2024-05-11T23:03:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1410000000000001e-06 2024-05-11T23:03:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238681 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.399096+0000", 2024-05-11T23:03:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.007330095, 2024-05-11T23:03:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011402045, 2024-05-11T23:03:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238681 2024-05-11T23:03:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399096+0000", 2024-05-11T23:03:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399096+0000", 2024-05-11T23:03:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399096+0000", 2024-05-11T23:03:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8799999999999998e-07 2024-05-11T23:03:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399098+0000", 2024-05-11T23:03:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.251e-06 2024-05-11T23:03:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399101+0000", 2024-05-11T23:03:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2839999999999998e-06 2024-05-11T23:03:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399104+0000", 2024-05-11T23:03:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.456e-06 2024-05-11T23:03:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238717 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.403302+0000", 2024-05-11T23:03:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.003124281, 2024-05-11T23:03:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0072583150000000004, 2024-05-11T23:03:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238717 2024-05-11T23:03:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403302+0000", 2024-05-11T23:03:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403302+0000", 2024-05-11T23:03:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403302+0000", 2024-05-11T23:03:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3599999999999999e-07 2024-05-11T23:03:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403304+0000", 2024-05-11T23:03:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.148e-06 2024-05-11T23:03:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403311+0000", 2024-05-11T23:03:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2260000000000002e-06 2024-05-11T23:03:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403315+0000", 2024-05-11T23:03:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8470000000000003e-06 2024-05-11T23:03:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238520 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.392329+0000", 2024-05-11T23:03:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.014097421000000001, 2024-05-11T23:03:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018291836999999998, 2024-05-11T23:03:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238520 2024-05-11T23:03:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392329+0000", 2024-05-11T23:03:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392329+0000", 2024-05-11T23:03:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392329+0000", 2024-05-11T23:03:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6499999999999998e-07 2024-05-11T23:03:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392332+0000", 2024-05-11T23:03:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.492e-06 2024-05-11T23:03:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392344+0000", 2024-05-11T23:03:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2055999999999999e-05 2024-05-11T23:03:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392350+0000", 2024-05-11T23:03:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3149999999999997e-06 2024-05-11T23:03:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238552 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393094+0000", 2024-05-11T23:03:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013331658, 2024-05-11T23:03:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017676030999999998, 2024-05-11T23:03:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238552 2024-05-11T23:03:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393094+0000", 2024-05-11T23:03:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393094+0000", 2024-05-11T23:03:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393095+0000", 2024-05-11T23:03:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2199999999999999e-07 2024-05-11T23:03:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393097+0000", 2024-05-11T23:03:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1569999999999998e-06 2024-05-11T23:03:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393104+0000", 2024-05-11T23:03:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2810000000000003e-06 2024-05-11T23:03:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393111+0000", 2024-05-11T23:03:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5259999999999999e-06 2024-05-11T23:03:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238584 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393739+0000", 2024-05-11T23:03:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.01268672, 2024-05-11T23:03:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017092401, 2024-05-11T23:03:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.653 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.653 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238584 2024-05-11T23:03:03.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.653 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393739+0000", 2024-05-11T23:03:03.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393739+0000", 2024-05-11T23:03:03.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393740+0000", 2024-05-11T23:03:03.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:03:03.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393742+0000", 2024-05-11T23:03:03.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:03:03.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393755+0000", 2024-05-11T23:03:03.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3421000000000001e-05 2024-05-11T23:03:03.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393762+0000", 2024-05-11T23:03:03.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0280000000000003e-06 2024-05-11T23:03:03.655 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.655 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.655 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.655 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238618 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.655 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394494+0000", 2024-05-11T23:03:03.655 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011932467, 2024-05-11T23:03:03.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016397689, 2024-05-11T23:03:03.655 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.655 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.655 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.656 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.656 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.656 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.656 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238618 2024-05-11T23:03:03.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.656 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394494+0000", 2024-05-11T23:03:03.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394494+0000", 2024-05-11T23:03:03.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394494+0000", 2024-05-11T23:03:03.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.22e-07 2024-05-11T23:03:03.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394495+0000", 2024-05-11T23:03:03.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0470000000000001e-06 2024-05-11T23:03:03.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394498+0000", 2024-05-11T23:03:03.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.27e-06 2024-05-11T23:03:03.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394502+0000", 2024-05-11T23:03:03.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0219999999999998e-06 2024-05-11T23:03:03.658 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.658 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.658 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.658 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238650 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.658 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395197+0000", 2024-05-11T23:03:03.658 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011228871, 2024-05-11T23:03:03.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015761013000000001, 2024-05-11T23:03:03.658 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.659 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.659 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.659 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.659 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.659 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238650 2024-05-11T23:03:03.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.659 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395197+0000", 2024-05-11T23:03:03.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395197+0000", 2024-05-11T23:03:03.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395198+0000", 2024-05-11T23:03:03.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1800000000000002e-07 2024-05-11T23:03:03.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395199+0000", 2024-05-11T23:03:03.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2240000000000001e-06 2024-05-11T23:03:03.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395203+0000", 2024-05-11T23:03:03.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1699999999999999e-06 2024-05-11T23:03:03.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395205+0000", 2024-05-11T23:03:03.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.509e-06 2024-05-11T23:03:03.661 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.661 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.661 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.661 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238682 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.661 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.399107+0000", 2024-05-11T23:03:03.661 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0073186329999999997, 2024-05-11T23:03:03.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011912562999999999, 2024-05-11T23:03:03.661 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.661 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.662 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.662 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.662 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.662 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.662 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238682 2024-05-11T23:03:03.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.662 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.662 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399107+0000", 2024-05-11T23:03:03.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399107+0000", 2024-05-11T23:03:03.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399108+0000", 2024-05-11T23:03:03.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.22e-07 2024-05-11T23:03:03.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399109+0000", 2024-05-11T23:03:03.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1990000000000001e-06 2024-05-11T23:03:03.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399112+0000", 2024-05-11T23:03:03.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1709999999999998e-06 2024-05-11T23:03:03.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399115+0000", 2024-05-11T23:03:03.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5629999999999999e-06 2024-05-11T23:03:03.664 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.664 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.664 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.665 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238718 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.665 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.403322+0000", 2024-05-11T23:03:03.665 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0031042349999999999, 2024-05-11T23:03:03.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0077625710000000002, 2024-05-11T23:03:03.665 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.665 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.665 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.665 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.665 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.665 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.665 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238718 2024-05-11T23:03:03.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.665 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403322+0000", 2024-05-11T23:03:03.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403322+0000", 2024-05-11T23:03:03.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403322+0000", 2024-05-11T23:03:03.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.77e-07 2024-05-11T23:03:03.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403328+0000", 2024-05-11T23:03:03.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9059999999999999e-06 2024-05-11T23:03:03.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403335+0000", 2024-05-11T23:03:03.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.041e-06 2024-05-11T23:03:03.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403340+0000", 2024-05-11T23:03:03.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9590000000000003e-06 2024-05-11T23:03:03.668 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.668 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.668 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.668 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238521 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.668 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.392359+0000", 2024-05-11T23:03:03.668 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.014066652000000001, 2024-05-11T23:03:03.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018786516, 2024-05-11T23:03:03.668 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.668 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.669 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.669 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.669 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.669 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.669 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238521 2024-05-11T23:03:03.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.669 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392359+0000", 2024-05-11T23:03:03.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392359+0000", 2024-05-11T23:03:03.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392360+0000", 2024-05-11T23:03:03.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2799999999999996e-07 2024-05-11T23:03:03.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392366+0000", 2024-05-11T23:03:03.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9270000000000002e-06 2024-05-11T23:03:03.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392373+0000", 2024-05-11T23:03:03.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.075e-06 2024-05-11T23:03:03.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392378+0000", 2024-05-11T23:03:03.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5029999999999997e-06 2024-05-11T23:03:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238553 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393121+0000", 2024-05-11T23:03:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013305052, 2024-05-11T23:03:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018089065000000001, 2024-05-11T23:03:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.672 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238553 2024-05-11T23:03:03.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.673 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393121+0000", 2024-05-11T23:03:03.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393121+0000", 2024-05-11T23:03:03.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393122+0000", 2024-05-11T23:03:03.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6000000000000004e-07 2024-05-11T23:03:03.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393124+0000", 2024-05-11T23:03:03.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3700000000000002e-06 2024-05-11T23:03:03.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393131+0000", 2024-05-11T23:03:03.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6810000000000001e-06 2024-05-11T23:03:03.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393136+0000", 2024-05-11T23:03:03.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1309999999999997e-06 2024-05-11T23:03:03.675 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.675 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.675 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.675 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238585 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.675 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393772+0000", 2024-05-11T23:03:03.675 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012654021, 2024-05-11T23:03:03.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017497867, 2024-05-11T23:03:03.675 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.675 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.675 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.675 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.675 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.675 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.675 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238585 2024-05-11T23:03:03.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.676 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393772+0000", 2024-05-11T23:03:03.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393772+0000", 2024-05-11T23:03:03.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393773+0000", 2024-05-11T23:03:03.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0800000000000004e-07 2024-05-11T23:03:03.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393775+0000", 2024-05-11T23:03:03.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.407e-06 2024-05-11T23:03:03.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393787+0000", 2024-05-11T23:03:03.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1727e-05 2024-05-11T23:03:03.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393794+0000", 2024-05-11T23:03:03.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7780000000000002e-06 2024-05-11T23:03:03.678 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.678 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.678 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.678 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238619 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.678 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394506+0000", 2024-05-11T23:03:03.678 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011920490000000001, 2024-05-11T23:03:03.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016826355000000001, 2024-05-11T23:03:03.678 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.678 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.678 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.679 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.679 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.679 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.679 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238619 2024-05-11T23:03:03.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.679 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394506+0000", 2024-05-11T23:03:03.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394506+0000", 2024-05-11T23:03:03.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394506+0000", 2024-05-11T23:03:03.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-05-11T23:03:03.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394510+0000", 2024-05-11T23:03:03.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4129999999999999e-06 2024-05-11T23:03:03.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394516+0000", 2024-05-11T23:03:03.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7180000000000001e-06 2024-05-11T23:03:03.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394519+0000", 2024-05-11T23:03:03.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.021e-06 2024-05-11T23:03:03.681 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.681 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.681 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.681 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238651 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.681 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395209+0000", 2024-05-11T23:03:03.681 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011217118999999999, 2024-05-11T23:03:03.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016182350000000002, 2024-05-11T23:03:03.682 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.682 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.682 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.682 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.682 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.682 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.682 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238651 2024-05-11T23:03:03.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.682 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395209+0000", 2024-05-11T23:03:03.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395209+0000", 2024-05-11T23:03:03.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395209+0000", 2024-05-11T23:03:03.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-05-11T23:03:03.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395210+0000", 2024-05-11T23:03:03.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2750000000000001e-06 2024-05-11T23:03:03.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395214+0000", 2024-05-11T23:03:03.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3359999999999999e-06 2024-05-11T23:03:03.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395216+0000", 2024-05-11T23:03:03.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5749999999999999e-06 2024-05-11T23:03:03.684 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.684 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.684 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.685 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238683 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.685 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.399118+0000", 2024-05-11T23:03:03.685 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0073083009999999997, 2024-05-11T23:03:03.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012335595, 2024-05-11T23:03:03.685 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.685 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.685 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.685 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.685 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.685 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.685 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238683 2024-05-11T23:03:03.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.686 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399118+0000", 2024-05-11T23:03:03.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399118+0000", 2024-05-11T23:03:03.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399118+0000", 2024-05-11T23:03:03.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0699999999999999e-07 2024-05-11T23:03:03.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399119+0000", 2024-05-11T23:03:03.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.172e-06 2024-05-11T23:03:03.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399122+0000", 2024-05-11T23:03:03.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1700000000000001e-06 2024-05-11T23:03:03.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399125+0000", 2024-05-11T23:03:03.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4980000000000001e-06 2024-05-11T23:03:03.688 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.688 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.688 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.688 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238522 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.688 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.392406+0000", 2024-05-11T23:03:03.688 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.01402051, 2024-05-11T23:03:03.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019108557000000002, 2024-05-11T23:03:03.688 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.688 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.688 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.688 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.689 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.689 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.689 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238522 2024-05-11T23:03:03.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.689 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392406+0000", 2024-05-11T23:03:03.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392406+0000", 2024-05-11T23:03:03.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392406+0000", 2024-05-11T23:03:03.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9100000000000004e-07 2024-05-11T23:03:03.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392409+0000", 2024-05-11T23:03:03.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6639999999999998e-06 2024-05-11T23:03:03.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392416+0000", 2024-05-11T23:03:03.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3200000000000002e-06 2024-05-11T23:03:03.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392421+0000", 2024-05-11T23:03:03.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9189999999999998e-06 2024-05-11T23:03:03.691 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.691 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.691 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.691 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238554 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.691 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393142+0000", 2024-05-11T23:03:03.691 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013283649, 2024-05-11T23:03:03.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018433966999999999, 2024-05-11T23:03:03.692 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.692 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.692 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.692 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.692 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.692 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.692 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238554 2024-05-11T23:03:03.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.692 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393142+0000", 2024-05-11T23:03:03.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393142+0000", 2024-05-11T23:03:03.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393143+0000", 2024-05-11T23:03:03.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8000000000000001e-07 2024-05-11T23:03:03.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393145+0000", 2024-05-11T23:03:03.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0710000000000002e-06 2024-05-11T23:03:03.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393151+0000", 2024-05-11T23:03:03.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2149999999999998e-06 2024-05-11T23:03:03.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393156+0000", 2024-05-11T23:03:03.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7119999999999999e-06 2024-05-11T23:03:03.694 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.694 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.694 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.694 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238586 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.694 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393801+0000", 2024-05-11T23:03:03.695 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.01262463, 2024-05-11T23:03:03.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017834570000000001, 2024-05-11T23:03:03.695 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.695 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.695 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.695 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.695 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.695 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238586 2024-05-11T23:03:03.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.695 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393801+0000", 2024-05-11T23:03:03.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393801+0000", 2024-05-11T23:03:03.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393802+0000", 2024-05-11T23:03:03.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.27e-07 2024-05-11T23:03:03.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393804+0000", 2024-05-11T23:03:03.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3120000000000001e-06 2024-05-11T23:03:03.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393811+0000", 2024-05-11T23:03:03.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7150000000000001e-06 2024-05-11T23:03:03.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393816+0000", 2024-05-11T23:03:03.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9559999999999996e-06 2024-05-11T23:03:03.697 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.697 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.697 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.697 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238620 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.697 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394522+0000", 2024-05-11T23:03:03.697 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011903687, 2024-05-11T23:03:03.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017177911000000001, 2024-05-11T23:03:03.698 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.698 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.698 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.698 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.698 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.698 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.698 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238620 2024-05-11T23:03:03.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.698 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394522+0000", 2024-05-11T23:03:03.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394522+0000", 2024-05-11T23:03:03.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394523+0000", 2024-05-11T23:03:03.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:03.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394524+0000", 2024-05-11T23:03:03.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.13e-06 2024-05-11T23:03:03.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394527+0000", 2024-05-11T23:03:03.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4580000000000002e-06 2024-05-11T23:03:03.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394530+0000", 2024-05-11T23:03:03.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5830000000000001e-06 2024-05-11T23:03:03.700 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.700 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.700 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.700 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238652 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.700 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395220+0000", 2024-05-11T23:03:03.700 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011206071999999999, 2024-05-11T23:03:03.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016544132999999999, 2024-05-11T23:03:03.701 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.701 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.701 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.701 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.701 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.701 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.701 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238652 2024-05-11T23:03:03.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.701 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395220+0000", 2024-05-11T23:03:03.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395220+0000", 2024-05-11T23:03:03.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395220+0000", 2024-05-11T23:03:03.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:03.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395222+0000", 2024-05-11T23:03:03.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3289999999999999e-06 2024-05-11T23:03:03.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395225+0000", 2024-05-11T23:03:03.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3239999999999999e-06 2024-05-11T23:03:03.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395230+0000", 2024-05-11T23:03:03.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4550000000000003e-06 2024-05-11T23:03:03.703 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.703 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.703 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.704 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238684 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.704 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.399128+0000", 2024-05-11T23:03:03.704 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0072980060000000001, 2024-05-11T23:03:03.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012698909, 2024-05-11T23:03:03.704 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.704 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.704 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.704 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.704 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.704 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.704 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238684 2024-05-11T23:03:03.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.705 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399128+0000", 2024-05-11T23:03:03.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399128+0000", 2024-05-11T23:03:03.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399128+0000", 2024-05-11T23:03:03.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:03:03.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399129+0000", 2024-05-11T23:03:03.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1829999999999999e-06 2024-05-11T23:03:03.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399133+0000", 2024-05-11T23:03:03.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.146e-06 2024-05-11T23:03:03.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399135+0000", 2024-05-11T23:03:03.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4889999999999998e-06 2024-05-11T23:03:03.707 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.707 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.707 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.707 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238719 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.707 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.403347+0000", 2024-05-11T23:03:03.708 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0030790150000000001, 2024-05-11T23:03:03.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0085398990000000001, 2024-05-11T23:03:03.708 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.708 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.708 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.708 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.708 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.708 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.708 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238719 2024-05-11T23:03:03.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.708 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403347+0000", 2024-05-11T23:03:03.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403347+0000", 2024-05-11T23:03:03.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.404914+0000", 2024-05-11T23:03:03.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001567019 2024-05-11T23:03:03.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.404918+0000", 2024-05-11T23:03:03.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2679999999999996e-06 2024-05-11T23:03:03.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.404931+0000", 2024-05-11T23:03:03.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2169e-05 2024-05-11T23:03:03.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.404939+0000", 2024-05-11T23:03:03.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3299999999999999e-06 2024-05-11T23:03:03.710 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.710 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.710 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.711 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238523 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.711 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.392436+0000", 2024-05-11T23:03:03.711 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013990109000000001, 2024-05-11T23:03:03.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019519491, 2024-05-11T23:03:03.711 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.711 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.711 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.711 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.711 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.711 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.711 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238523 2024-05-11T23:03:03.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.711 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392436+0000", 2024-05-11T23:03:03.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392436+0000", 2024-05-11T23:03:03.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392436+0000", 2024-05-11T23:03:03.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:03:03.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392439+0000", 2024-05-11T23:03:03.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3039999999999999e-06 2024-05-11T23:03:03.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392446+0000", 2024-05-11T23:03:03.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9090000000000003e-06 2024-05-11T23:03:03.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392451+0000", 2024-05-11T23:03:03.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9930000000000003e-06 2024-05-11T23:03:03.713 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.713 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.713 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.714 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238555 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.714 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393164+0000", 2024-05-11T23:03:03.714 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013262592, 2024-05-11T23:03:03.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018854600999999999, 2024-05-11T23:03:03.714 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.714 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.714 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.714 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.714 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.714 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.714 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238555 2024-05-11T23:03:03.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.714 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393164+0000", 2024-05-11T23:03:03.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393164+0000", 2024-05-11T23:03:03.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393164+0000", 2024-05-11T23:03:03.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3500000000000002e-07 2024-05-11T23:03:03.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393166+0000", 2024-05-11T23:03:03.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1849999999999998e-06 2024-05-11T23:03:03.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393174+0000", 2024-05-11T23:03:03.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1110000000000008e-06 2024-05-11T23:03:03.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393179+0000", 2024-05-11T23:03:03.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0119999999999996e-06 2024-05-11T23:03:03.716 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.716 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.716 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.717 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238587 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.717 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393823+0000", 2024-05-11T23:03:03.717 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012603217, 2024-05-11T23:03:03.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018255077000000001, 2024-05-11T23:03:03.717 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.717 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.717 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.717 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.717 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238587 2024-05-11T23:03:03.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.717 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393823+0000", 2024-05-11T23:03:03.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393823+0000", 2024-05-11T23:03:03.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393823+0000", 2024-05-11T23:03:03.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:03:03.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393825+0000", 2024-05-11T23:03:03.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.148e-06 2024-05-11T23:03:03.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393832+0000", 2024-05-11T23:03:03.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4529999999999999e-06 2024-05-11T23:03:03.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393837+0000", 2024-05-11T23:03:03.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0669999999999999e-06 2024-05-11T23:03:03.720 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.720 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.720 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.720 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238621 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.720 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394533+0000", 2024-05-11T23:03:03.720 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011893236, 2024-05-11T23:03:03.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017606710000000001, 2024-05-11T23:03:03.720 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.720 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.720 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.720 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.721 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.721 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.721 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238621 2024-05-11T23:03:03.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.721 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394533+0000", 2024-05-11T23:03:03.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394533+0000", 2024-05-11T23:03:03.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394533+0000", 2024-05-11T23:03:03.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1199999999999999e-07 2024-05-11T23:03:03.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394534+0000", 2024-05-11T23:03:03.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.136e-06 2024-05-11T23:03:03.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394539+0000", 2024-05-11T23:03:03.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.425e-06 2024-05-11T23:03:03.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.723 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.723 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394541+0000", 2024-05-11T23:03:03.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6299999999999998e-06 2024-05-11T23:03:03.723 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.723 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.723 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.723 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238653 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.723 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395237+0000", 2024-05-11T23:03:03.724 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011189358, 2024-05-11T23:03:03.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016962090999999999, 2024-05-11T23:03:03.724 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.724 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.724 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.724 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.724 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.724 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.724 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238653 2024-05-11T23:03:03.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.724 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395237+0000", 2024-05-11T23:03:03.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395237+0000", 2024-05-11T23:03:03.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395237+0000", 2024-05-11T23:03:03.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9700000000000003e-07 2024-05-11T23:03:03.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395239+0000", 2024-05-11T23:03:03.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9800000000000001e-06 2024-05-11T23:03:03.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395245+0000", 2024-05-11T23:03:03.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6609999999999997e-06 2024-05-11T23:03:03.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395249+0000", 2024-05-11T23:03:03.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7899999999999999e-06 2024-05-11T23:03:03.726 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.726 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.726 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.727 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238685 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.727 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.399138+0000", 2024-05-11T23:03:03.727 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0072879169999999997, 2024-05-11T23:03:03.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013124274999999999, 2024-05-11T23:03:03.727 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.727 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.727 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.727 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.728 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.728 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238685 2024-05-11T23:03:03.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.728 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399138+0000", 2024-05-11T23:03:03.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.399138+0000", 2024-05-11T23:03:03.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.400917+0000", 2024-05-11T23:03:03.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001778871 2024-05-11T23:03:03.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.400922+0000", 2024-05-11T23:03:03.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7269999999999998e-06 2024-05-11T23:03:03.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.400934+0000", 2024-05-11T23:03:03.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2299e-05 2024-05-11T23:03:03.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.400943+0000", 2024-05-11T23:03:03.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3840000000000003e-06 2024-05-11T23:03:03.730 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.730 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.730 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.730 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238720 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.730 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.404951+0000", 2024-05-11T23:03:03.730 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.001475526, 2024-05-11T23:03:03.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0073721710000000003, 2024-05-11T23:03:03.730 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.730 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.731 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.731 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.731 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.731 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238720 2024-05-11T23:03:03.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.731 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.731 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.731 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.404951+0000", 2024-05-11T23:03:03.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.731 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.731 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.404951+0000", 2024-05-11T23:03:03.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.404951+0000", 2024-05-11T23:03:03.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0900000000000001e-07 2024-05-11T23:03:03.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.404954+0000", 2024-05-11T23:03:03.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.774e-06 2024-05-11T23:03:03.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.404962+0000", 2024-05-11T23:03:03.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7400000000000004e-06 2024-05-11T23:03:03.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.404967+0000", 2024-05-11T23:03:03.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6799999999999998e-06 2024-05-11T23:03:03.733 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.733 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.733 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.733 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238524 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.733 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.392457+0000", 2024-05-11T23:03:03.733 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013969145000000001, 2024-05-11T23:03:03.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019929278000000002, 2024-05-11T23:03:03.733 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.734 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.734 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.734 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.734 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.734 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.734 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238524 2024-05-11T23:03:03.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.734 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392457+0000", 2024-05-11T23:03:03.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.735 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392457+0000", 2024-05-11T23:03:03.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.735 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.735 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392457+0000", 2024-05-11T23:03:03.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.08e-07 2024-05-11T23:03:03.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.735 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.735 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392459+0000", 2024-05-11T23:03:03.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9089999999999998e-06 2024-05-11T23:03:03.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.735 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.735 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392465+0000", 2024-05-11T23:03:03.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8089999999999998e-06 2024-05-11T23:03:03.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392474+0000", 2024-05-11T23:03:03.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9020000000000005e-06 2024-05-11T23:03:03.736 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.736 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.736 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.736 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238556 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.736 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393187+0000", 2024-05-11T23:03:03.736 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013239419000000001, 2024-05-11T23:03:03.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019258181999999999, 2024-05-11T23:03:03.737 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.737 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.737 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.737 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.737 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.737 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.737 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238556 2024-05-11T23:03:03.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.737 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393187+0000", 2024-05-11T23:03:03.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393187+0000", 2024-05-11T23:03:03.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393187+0000", 2024-05-11T23:03:03.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.58e-07 2024-05-11T23:03:03.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393189+0000", 2024-05-11T23:03:03.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2730000000000001e-06 2024-05-11T23:03:03.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.739 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393197+0000", 2024-05-11T23:03:03.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1679999999999998e-06 2024-05-11T23:03:03.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.739 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.739 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393202+0000", 2024-05-11T23:03:03.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.916e-06 2024-05-11T23:03:03.739 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.739 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.739 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.740 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238588 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.740 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393843+0000", 2024-05-11T23:03:03.740 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012582766, 2024-05-11T23:03:03.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018664027, 2024-05-11T23:03:03.740 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.740 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.740 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.740 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.740 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.740 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.740 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238588 2024-05-11T23:03:03.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.740 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393843+0000", 2024-05-11T23:03:03.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393843+0000", 2024-05-11T23:03:03.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393844+0000", 2024-05-11T23:03:03.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:03:03.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393846+0000", 2024-05-11T23:03:03.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.906e-06 2024-05-11T23:03:03.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393854+0000", 2024-05-11T23:03:03.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3699999999999995e-06 2024-05-11T23:03:03.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393859+0000", 2024-05-11T23:03:03.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2320000000000001e-06 2024-05-11T23:03:03.743 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.743 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.743 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.743 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238622 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.743 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394544+0000", 2024-05-11T23:03:03.743 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.01188177, 2024-05-11T23:03:03.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018021725999999998, 2024-05-11T23:03:03.743 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.743 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.744 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.744 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.744 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.744 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.744 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238622 2024-05-11T23:03:03.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.744 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394544+0000", 2024-05-11T23:03:03.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394544+0000", 2024-05-11T23:03:03.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394545+0000", 2024-05-11T23:03:03.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1500000000000001e-07 2024-05-11T23:03:03.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394546+0000", 2024-05-11T23:03:03.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.226e-06 2024-05-11T23:03:03.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394549+0000", 2024-05-11T23:03:03.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3790000000000002e-06 2024-05-11T23:03:03.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.746 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.746 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394552+0000", 2024-05-11T23:03:03.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6029999999999999e-06 2024-05-11T23:03:03.747 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.747 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.747 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.747 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238654 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.747 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395256+0000", 2024-05-11T23:03:03.747 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011170582, 2024-05-11T23:03:03.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017372612999999999, 2024-05-11T23:03:03.747 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.747 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.747 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.747 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.747 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.747 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.748 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238654 2024-05-11T23:03:03.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.748 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395256+0000", 2024-05-11T23:03:03.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395256+0000", 2024-05-11T23:03:03.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395256+0000", 2024-05-11T23:03:03.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.13e-07 2024-05-11T23:03:03.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395262+0000", 2024-05-11T23:03:03.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.28e-06 2024-05-11T23:03:03.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395269+0000", 2024-05-11T23:03:03.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8090000000000004e-06 2024-05-11T23:03:03.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395274+0000", 2024-05-11T23:03:03.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8069999999999999e-06 2024-05-11T23:03:03.750 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.750 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.750 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.750 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238686 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.750 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.400957+0000", 2024-05-11T23:03:03.750 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0054691879999999998, 2024-05-11T23:03:03.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011732381, 2024-05-11T23:03:03.750 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.750 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.751 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238686 2024-05-11T23:03:03.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.751 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.400957+0000", 2024-05-11T23:03:03.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.400957+0000", 2024-05-11T23:03:03.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.400958+0000", 2024-05-11T23:03:03.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:03:03.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.400961+0000", 2024-05-11T23:03:03.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9450000000000002e-06 2024-05-11T23:03:03.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.400968+0000", 2024-05-11T23:03:03.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7980000000000006e-06 2024-05-11T23:03:03.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.400974+0000", 2024-05-11T23:03:03.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9780000000000002e-06 2024-05-11T23:03:03.753 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.753 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.753 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.753 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238721 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.753 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.404976+0000", 2024-05-11T23:03:03.753 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.001449887, 2024-05-11T23:03:03.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.007775222, 2024-05-11T23:03:03.754 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.754 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.754 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.754 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.754 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.754 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238721 2024-05-11T23:03:03.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.754 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.404976+0000", 2024-05-11T23:03:03.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.404976+0000", 2024-05-11T23:03:03.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.404977+0000", 2024-05-11T23:03:03.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7300000000000001e-07 2024-05-11T23:03:03.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.404979+0000", 2024-05-11T23:03:03.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4509999999999999e-06 2024-05-11T23:03:03.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.404986+0000", 2024-05-11T23:03:03.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9280000000000004e-06 2024-05-11T23:03:03.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.404991+0000", 2024-05-11T23:03:03.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2769999999999996e-06 2024-05-11T23:03:03.756 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.756 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.756 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.757 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238525 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.757 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.392480+0000", 2024-05-11T23:03:03.757 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013946079, 2024-05-11T23:03:03.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020333002999999999, 2024-05-11T23:03:03.757 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.757 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.757 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.757 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.757 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.757 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.757 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238525 2024-05-11T23:03:03.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.758 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392480+0000", 2024-05-11T23:03:03.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392480+0000", 2024-05-11T23:03:03.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392481+0000", 2024-05-11T23:03:03.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.124e-06 2024-05-11T23:03:03.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392483+0000", 2024-05-11T23:03:03.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.979e-06 2024-05-11T23:03:03.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392489+0000", 2024-05-11T23:03:03.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9159999999999998e-06 2024-05-11T23:03:03.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392494+0000", 2024-05-11T23:03:03.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2809999999999999e-06 2024-05-11T23:03:03.760 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.760 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.760 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.760 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238557 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.760 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393206+0000", 2024-05-11T23:03:03.760 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.01322006, 2024-05-11T23:03:03.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019667882000000001, 2024-05-11T23:03:03.760 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.760 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.760 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.761 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.761 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.761 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.761 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238557 2024-05-11T23:03:03.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.761 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393206+0000", 2024-05-11T23:03:03.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393206+0000", 2024-05-11T23:03:03.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393206+0000", 2024-05-11T23:03:03.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0699999999999998e-07 2024-05-11T23:03:03.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393211+0000", 2024-05-11T23:03:03.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8779999999999997e-06 2024-05-11T23:03:03.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393217+0000", 2024-05-11T23:03:03.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4739999999999996e-06 2024-05-11T23:03:03.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393220+0000", 2024-05-11T23:03:03.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1829999999999998e-06 2024-05-11T23:03:03.763 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.763 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.763 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.764 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238589 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.764 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393866+0000", 2024-05-11T23:03:03.764 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012560134000000001, 2024-05-11T23:03:03.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019077597000000002, 2024-05-11T23:03:03.764 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.764 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.764 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.764 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.764 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.764 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.764 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238589 2024-05-11T23:03:03.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.764 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393866+0000", 2024-05-11T23:03:03.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393866+0000", 2024-05-11T23:03:03.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393866+0000", 2024-05-11T23:03:03.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.27e-07 2024-05-11T23:03:03.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393869+0000", 2024-05-11T23:03:03.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2529999999999999e-06 2024-05-11T23:03:03.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393879+0000", 2024-05-11T23:03:03.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9189999999999994e-06 2024-05-11T23:03:03.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393885+0000", 2024-05-11T23:03:03.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9869999999999996e-06 2024-05-11T23:03:03.766 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.766 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.766 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.767 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238623 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.767 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394565+0000", 2024-05-11T23:03:03.767 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.01186102, 2024-05-11T23:03:03.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018437546999999999, 2024-05-11T23:03:03.767 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.767 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.767 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.767 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.767 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.767 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.767 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238623 2024-05-11T23:03:03.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.767 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394565+0000", 2024-05-11T23:03:03.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394565+0000", 2024-05-11T23:03:03.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394566+0000", 2024-05-11T23:03:03.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0100000000000005e-07 2024-05-11T23:03:03.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394568+0000", 2024-05-11T23:03:03.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3580000000000001e-06 2024-05-11T23:03:03.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394580+0000", 2024-05-11T23:03:03.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1667e-05 2024-05-11T23:03:03.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394587+0000", 2024-05-11T23:03:03.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7780000000000007e-06 2024-05-11T23:03:03.769 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.769 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.770 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.770 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238655 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.770 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395281+0000", 2024-05-11T23:03:03.770 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011145113999999999, 2024-05-11T23:03:03.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017785043, 2024-05-11T23:03:03.770 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.770 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.770 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.770 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.770 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.770 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.770 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238655 2024-05-11T23:03:03.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.771 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395281+0000", 2024-05-11T23:03:03.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395281+0000", 2024-05-11T23:03:03.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.771 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.771 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395282+0000", 2024-05-11T23:03:03.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2500000000000005e-07 2024-05-11T23:03:03.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395284+0000", 2024-05-11T23:03:03.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1600000000000001e-06 2024-05-11T23:03:03.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395292+0000", 2024-05-11T23:03:03.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1659999999999994e-06 2024-05-11T23:03:03.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395297+0000", 2024-05-11T23:03:03.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1030000000000001e-06 2024-05-11T23:03:03.773 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.773 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.773 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.773 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238687 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.773 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.400982+0000", 2024-05-11T23:03:03.773 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0054440019999999999, 2024-05-11T23:03:03.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012142941000000001, 2024-05-11T23:03:03.773 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.774 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.774 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.774 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.774 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.774 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238687 2024-05-11T23:03:03.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.774 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.400982+0000", 2024-05-11T23:03:03.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.400982+0000", 2024-05-11T23:03:03.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.400983+0000", 2024-05-11T23:03:03.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.15e-07 2024-05-11T23:03:03.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.400985+0000", 2024-05-11T23:03:03.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4779999999999998e-06 2024-05-11T23:03:03.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.400992+0000", 2024-05-11T23:03:03.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2799999999999998e-06 2024-05-11T23:03:03.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.400999+0000", 2024-05-11T23:03:03.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7309999999999996e-06 2024-05-11T23:03:03.777 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.777 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.777 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.777 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238722 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.777 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.404998+0000", 2024-05-11T23:03:03.777 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.001427956, 2024-05-11T23:03:03.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0081908150000000006, 2024-05-11T23:03:03.777 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.777 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.777 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.778 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.778 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.778 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238722 2024-05-11T23:03:03.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.778 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.404998+0000", 2024-05-11T23:03:03.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.404998+0000", 2024-05-11T23:03:03.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.404999+0000", 2024-05-11T23:03:03.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.96e-07 2024-05-11T23:03:03.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405001+0000", 2024-05-11T23:03:03.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1469999999999999e-06 2024-05-11T23:03:03.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405008+0000", 2024-05-11T23:03:03.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4039999999999998e-06 2024-05-11T23:03:03.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405013+0000", 2024-05-11T23:03:03.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0499999999999999e-06 2024-05-11T23:03:03.780 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.780 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.780 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.781 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238526 3.1 3:8135c7ce:::rbd_data.111ebf0f641b.000000000000000d:2e [read 1360896~1024] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.781 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.392500+0000", 2024-05-11T23:03:03.781 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013926565, 2024-05-11T23:03:03.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02075507, 2024-05-11T23:03:03.781 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.781 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.781 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:03.781 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.781 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.781 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.781 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238526 2024-05-11T23:03:03.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.781 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392500+0000", 2024-05-11T23:03:03.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392500+0000", 2024-05-11T23:03:03.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.782 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392500+0000", 2024-05-11T23:03:03.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9900000000000001e-07 2024-05-11T23:03:03.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.782 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392502+0000", 2024-05-11T23:03:03.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7999999999999999e-06 2024-05-11T23:03:03.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392507+0000", 2024-05-11T23:03:03.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7289999999999997e-06 2024-05-11T23:03:03.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.392515+0000", 2024-05-11T23:03:03.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4420000000000001e-06 2024-05-11T23:03:03.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:03.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.413233+0000", 2024-05-11T23:03:03.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020718574999999999 2024-05-11T23:03:03.784 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.784 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.784 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.784 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238558 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.784 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393224+0000", 2024-05-11T23:03:03.784 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013202599000000001, 2024-05-11T23:03:03.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020100005000000001, 2024-05-11T23:03:03.784 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.784 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.784 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.784 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.785 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.785 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.785 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238558 2024-05-11T23:03:03.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.785 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393224+0000", 2024-05-11T23:03:03.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393224+0000", 2024-05-11T23:03:03.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393224+0000", 2024-05-11T23:03:03.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4400000000000001e-07 2024-05-11T23:03:03.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393225+0000", 2024-05-11T23:03:03.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.207e-06 2024-05-11T23:03:03.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393228+0000", 2024-05-11T23:03:03.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3979999999999999e-06 2024-05-11T23:03:03.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393231+0000", 2024-05-11T23:03:03.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.728e-06 2024-05-11T23:03:03.787 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.787 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.787 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.787 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238590 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.787 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393922+0000", 2024-05-11T23:03:03.787 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012503626, 2024-05-11T23:03:03.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019460680000000001, 2024-05-11T23:03:03.787 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.787 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.787 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.788 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.788 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.788 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.788 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238590 2024-05-11T23:03:03.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.788 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393922+0000", 2024-05-11T23:03:03.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393922+0000", 2024-05-11T23:03:03.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393923+0000", 2024-05-11T23:03:03.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:03:03.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393925+0000", 2024-05-11T23:03:03.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.956e-06 2024-05-11T23:03:03.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393935+0000", 2024-05-11T23:03:03.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0114e-05 2024-05-11T23:03:03.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393939+0000", 2024-05-11T23:03:03.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1160000000000001e-06 2024-05-11T23:03:03.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.790 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.790 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238624 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.790 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394595+0000", 2024-05-11T23:03:03.790 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.01183153, 2024-05-11T23:03:03.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018851548999999999, 2024-05-11T23:03:03.791 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.791 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.791 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.791 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.791 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.791 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238624 2024-05-11T23:03:03.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.791 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394595+0000", 2024-05-11T23:03:03.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394595+0000", 2024-05-11T23:03:03.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394595+0000", 2024-05-11T23:03:03.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9499999999999998e-07 2024-05-11T23:03:03.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394597+0000", 2024-05-11T23:03:03.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2900000000000001e-06 2024-05-11T23:03:03.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394604+0000", 2024-05-11T23:03:03.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3999999999999997e-06 2024-05-11T23:03:03.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394610+0000", 2024-05-11T23:03:03.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5930000000000002e-06 2024-05-11T23:03:03.793 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.793 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.793 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.794 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238656 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.794 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395305+0000", 2024-05-11T23:03:03.794 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011120628, 2024-05-11T23:03:03.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018199647999999999, 2024-05-11T23:03:03.794 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.794 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.794 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.794 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.794 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238656 2024-05-11T23:03:03.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.794 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395305+0000", 2024-05-11T23:03:03.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395305+0000", 2024-05-11T23:03:03.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395306+0000", 2024-05-11T23:03:03.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5999999999999999e-07 2024-05-11T23:03:03.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395308+0000", 2024-05-11T23:03:03.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0679999999999999e-06 2024-05-11T23:03:03.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395320+0000", 2024-05-11T23:03:03.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2405999999999999e-05 2024-05-11T23:03:03.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395329+0000", 2024-05-11T23:03:03.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6519999999999995e-06 2024-05-11T23:03:03.796 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.797 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.797 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.797 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238688 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.797 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.401014+0000", 2024-05-11T23:03:03.797 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0054123009999999996, 2024-05-11T23:03:03.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012553348000000001, 2024-05-11T23:03:03.797 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.797 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.797 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.797 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.797 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.798 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.798 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238688 2024-05-11T23:03:03.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.798 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401014+0000", 2024-05-11T23:03:03.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401014+0000", 2024-05-11T23:03:03.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401014+0000", 2024-05-11T23:03:03.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.37e-07 2024-05-11T23:03:03.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401017+0000", 2024-05-11T23:03:03.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.83e-06 2024-05-11T23:03:03.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401025+0000", 2024-05-11T23:03:03.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3799999999999996e-06 2024-05-11T23:03:03.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401031+0000", 2024-05-11T23:03:03.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0669999999999997e-06 2024-05-11T23:03:03.800 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.800 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.800 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.800 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238723 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.800 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.405021+0000", 2024-05-11T23:03:03.800 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.001405519, 2024-05-11T23:03:03.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0086041399999999997, 2024-05-11T23:03:03.800 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.801 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.801 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.801 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.801 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.801 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238723 2024-05-11T23:03:03.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.801 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405021+0000", 2024-05-11T23:03:03.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405021+0000", 2024-05-11T23:03:03.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405021+0000", 2024-05-11T23:03:03.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9100000000000004e-07 2024-05-11T23:03:03.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405023+0000", 2024-05-11T23:03:03.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.368e-06 2024-05-11T23:03:03.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405030+0000", 2024-05-11T23:03:03.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4849999999999997e-06 2024-05-11T23:03:03.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405035+0000", 2024-05-11T23:03:03.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0259999999999998e-06 2024-05-11T23:03:03.803 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.803 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.803 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.803 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238559 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.804 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393241+0000", 2024-05-11T23:03:03.804 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013184842, 2024-05-11T23:03:03.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020446840000000001, 2024-05-11T23:03:03.804 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.804 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.804 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.804 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.804 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.804 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238559 2024-05-11T23:03:03.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.804 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393241+0000", 2024-05-11T23:03:03.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393241+0000", 2024-05-11T23:03:03.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393241+0000", 2024-05-11T23:03:03.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.36e-07 2024-05-11T23:03:03.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393243+0000", 2024-05-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.39e-06 2024-05-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393250+0000", 2024-05-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0149999999999998e-06 2024-05-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393253+0000", 2024-05-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1920000000000001e-06 2024-05-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238591 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393945+0000", 2024-05-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012481447, 2024-05-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019801255, 2024-05-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238591 2024-05-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393945+0000", 2024-05-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393945+0000", 2024-05-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393945+0000", 2024-05-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1e-07 2024-05-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393946+0000", 2024-05-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.451e-06 2024-05-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393951+0000", 2024-05-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9309999999999999e-06 2024-05-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393954+0000", 2024-05-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.723e-06 2024-05-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238625 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394617+0000", 2024-05-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011809327, 2024-05-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019190762, 2024-05-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238625 2024-05-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394617+0000", 2024-05-11T23:03:03.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394617+0000", 2024-05-11T23:03:03.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394617+0000", 2024-05-11T23:03:03.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:03:03.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394619+0000", 2024-05-11T23:03:03.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0279999999999999e-06 2024-05-11T23:03:03.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394627+0000", 2024-05-11T23:03:03.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9310000000000008e-06 2024-05-11T23:03:03.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394632+0000", 2024-05-11T23:03:03.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6500000000000004e-06 2024-05-11T23:03:03.813 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.813 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.813 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.813 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238657 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.813 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395339+0000", 2024-05-11T23:03:03.813 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011087194999999999, 2024-05-11T23:03:03.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018528258999999998, 2024-05-11T23:03:03.813 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.813 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.813 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.813 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.813 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.813 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.814 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238657 2024-05-11T23:03:03.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.814 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395339+0000", 2024-05-11T23:03:03.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395339+0000", 2024-05-11T23:03:03.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395339+0000", 2024-05-11T23:03:03.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9400000000000001e-07 2024-05-11T23:03:03.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395342+0000", 2024-05-11T23:03:03.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5210000000000001e-06 2024-05-11T23:03:03.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395349+0000", 2024-05-11T23:03:03.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.029e-06 2024-05-11T23:03:03.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395356+0000", 2024-05-11T23:03:03.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7270000000000001e-06 2024-05-11T23:03:03.816 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.816 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.816 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.817 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238689 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.817 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.401039+0000", 2024-05-11T23:03:03.817 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0053874719999999999, 2024-05-11T23:03:03.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012896774, 2024-05-11T23:03:03.817 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.817 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.817 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.817 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.817 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.817 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.817 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238689 2024-05-11T23:03:03.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.817 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401039+0000", 2024-05-11T23:03:03.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401039+0000", 2024-05-11T23:03:03.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401039+0000", 2024-05-11T23:03:03.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:03:03.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401042+0000", 2024-05-11T23:03:03.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4380000000000002e-06 2024-05-11T23:03:03.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401048+0000", 2024-05-11T23:03:03.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5509999999999996e-06 2024-05-11T23:03:03.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401054+0000", 2024-05-11T23:03:03.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5160000000000002e-06 2024-05-11T23:03:03.820 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.820 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.820 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.820 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238724 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.820 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.405043+0000", 2024-05-11T23:03:03.820 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.001382751, 2024-05-11T23:03:03.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.008953562, 2024-05-11T23:03:03.820 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.821 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.821 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.821 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.821 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238724 2024-05-11T23:03:03.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.821 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405043+0000", 2024-05-11T23:03:03.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405043+0000", 2024-05-11T23:03:03.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405044+0000", 2024-05-11T23:03:03.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5700000000000002e-07 2024-05-11T23:03:03.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405046+0000", 2024-05-11T23:03:03.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2900000000000001e-06 2024-05-11T23:03:03.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405055+0000", 2024-05-11T23:03:03.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5159999999999994e-06 2024-05-11T23:03:03.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405060+0000", 2024-05-11T23:03:03.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5180000000000004e-06 2024-05-11T23:03:03.823 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.823 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.823 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.823 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238560 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.823 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393258+0000", 2024-05-11T23:03:03.823 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013168403, 2024-05-11T23:03:03.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020801219999999999, 2024-05-11T23:03:03.823 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.824 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238560 2024-05-11T23:03:03.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.824 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393258+0000", 2024-05-11T23:03:03.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393258+0000", 2024-05-11T23:03:03.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393258+0000", 2024-05-11T23:03:03.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1400000000000001e-07 2024-05-11T23:03:03.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393259+0000", 2024-05-11T23:03:03.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5269999999999999e-06 2024-05-11T23:03:03.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393263+0000", 2024-05-11T23:03:03.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4829999999999999e-06 2024-05-11T23:03:03.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393265+0000", 2024-05-11T23:03:03.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5069999999999999e-06 2024-05-11T23:03:03.826 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.826 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.826 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.826 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238592 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.826 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393958+0000", 2024-05-11T23:03:03.827 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012468572000000001, 2024-05-11T23:03:03.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020164075, 2024-05-11T23:03:03.827 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.827 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.827 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.827 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.827 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.827 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.827 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238592 2024-05-11T23:03:03.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.827 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393958+0000", 2024-05-11T23:03:03.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393958+0000", 2024-05-11T23:03:03.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393958+0000", 2024-05-11T23:03:03.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.23e-07 2024-05-11T23:03:03.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393959+0000", 2024-05-11T23:03:03.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3820000000000001e-06 2024-05-11T23:03:03.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393962+0000", 2024-05-11T23:03:03.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2430000000000001e-06 2024-05-11T23:03:03.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393965+0000", 2024-05-11T23:03:03.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4719999999999998e-06 2024-05-11T23:03:03.829 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.829 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.829 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.830 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238626 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.830 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394638+0000", 2024-05-11T23:03:03.830 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011787918999999999, 2024-05-11T23:03:03.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019528585000000001, 2024-05-11T23:03:03.830 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.830 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.830 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.830 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.830 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.830 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.831 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238626 2024-05-11T23:03:03.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.831 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394638+0000", 2024-05-11T23:03:03.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394638+0000", 2024-05-11T23:03:03.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394639+0000", 2024-05-11T23:03:03.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9299999999999999e-07 2024-05-11T23:03:03.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394641+0000", 2024-05-11T23:03:03.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4260000000000002e-06 2024-05-11T23:03:03.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394649+0000", 2024-05-11T23:03:03.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9039999999999992e-06 2024-05-11T23:03:03.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394654+0000", 2024-05-11T23:03:03.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8439999999999997e-06 2024-05-11T23:03:03.833 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.833 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.833 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.833 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238658 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.833 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395363+0000", 2024-05-11T23:03:03.833 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011063523, 2024-05-11T23:03:03.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018861328, 2024-05-11T23:03:03.833 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.834 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.834 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.834 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.834 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.834 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.834 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238658 2024-05-11T23:03:03.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.834 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395363+0000", 2024-05-11T23:03:03.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395363+0000", 2024-05-11T23:03:03.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395363+0000", 2024-05-11T23:03:03.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:03:03.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395365+0000", 2024-05-11T23:03:03.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4229999999999999e-06 2024-05-11T23:03:03.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395378+0000", 2024-05-11T23:03:03.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2248e-05 2024-05-11T23:03:03.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395383+0000", 2024-05-11T23:03:03.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.626e-06 2024-05-11T23:03:03.836 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.836 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.836 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.837 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238690 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.837 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.401061+0000", 2024-05-11T23:03:03.837 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0053647139999999996, 2024-05-11T23:03:03.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013221772, 2024-05-11T23:03:03.837 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.837 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.837 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238690 2024-05-11T23:03:03.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.837 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401061+0000", 2024-05-11T23:03:03.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401061+0000", 2024-05-11T23:03:03.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401062+0000", 2024-05-11T23:03:03.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0399999999999996e-07 2024-05-11T23:03:03.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401069+0000", 2024-05-11T23:03:03.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7700000000000004e-06 2024-05-11T23:03:03.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401077+0000", 2024-05-11T23:03:03.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3999999999999992e-06 2024-05-11T23:03:03.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401083+0000", 2024-05-11T23:03:03.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1659999999999999e-06 2024-05-11T23:03:03.839 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.839 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.839 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.840 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238725 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.840 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.405067+0000", 2024-05-11T23:03:03.840 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.00135871, 2024-05-11T23:03:03.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0092789629999999994, 2024-05-11T23:03:03.840 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.840 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.840 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.840 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.840 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.840 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238725 2024-05-11T23:03:03.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.841 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405067+0000", 2024-05-11T23:03:03.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405067+0000", 2024-05-11T23:03:03.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405068+0000", 2024-05-11T23:03:03.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7100000000000002e-07 2024-05-11T23:03:03.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405070+0000", 2024-05-11T23:03:03.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.435e-06 2024-05-11T23:03:03.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405077+0000", 2024-05-11T23:03:03.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.669e-06 2024-05-11T23:03:03.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405083+0000", 2024-05-11T23:03:03.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8159999999999999e-06 2024-05-11T23:03:03.843 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.843 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.843 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.843 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238561 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.843 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393269+0000", 2024-05-11T23:03:03.843 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.01315732, 2024-05-11T23:03:03.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021138454000000001, 2024-05-11T23:03:03.843 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.843 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.843 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238561 2024-05-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393269+0000", 2024-05-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393269+0000", 2024-05-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393269+0000", 2024-05-11T23:03:03.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-05-11T23:03:03.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393270+0000", 2024-05-11T23:03:03.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1349999999999999e-06 2024-05-11T23:03:03.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393277+0000", 2024-05-11T23:03:03.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0029999999999997e-06 2024-05-11T23:03:03.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393280+0000", 2024-05-11T23:03:03.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1810000000000001e-06 2024-05-11T23:03:03.846 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.846 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.846 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.846 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238593 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.846 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393968+0000", 2024-05-11T23:03:03.846 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012458123, 2024-05-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020501543000000001, 2024-05-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238593 2024-05-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393968+0000", 2024-05-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393968+0000", 2024-05-11T23:03:03.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393968+0000", 2024-05-11T23:03:03.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.05e-07 2024-05-11T23:03:03.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393972+0000", 2024-05-11T23:03:03.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2320000000000003e-06 2024-05-11T23:03:03.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393976+0000", 2024-05-11T23:03:03.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5109999999999999e-06 2024-05-11T23:03:03.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393979+0000", 2024-05-11T23:03:03.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6110000000000001e-06 2024-05-11T23:03:03.849 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.850 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.850 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238627 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.850 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394668+0000", 2024-05-11T23:03:03.850 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011758248000000001, 2024-05-11T23:03:03.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019863208, 2024-05-11T23:03:03.850 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.850 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.850 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.850 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.850 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.851 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.851 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238627 2024-05-11T23:03:03.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.851 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394668+0000", 2024-05-11T23:03:03.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394668+0000", 2024-05-11T23:03:03.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394668+0000", 2024-05-11T23:03:03.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2e-07 2024-05-11T23:03:03.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394671+0000", 2024-05-11T23:03:03.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4099999999999998e-06 2024-05-11T23:03:03.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394677+0000", 2024-05-11T23:03:03.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5869999999999998e-06 2024-05-11T23:03:03.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394683+0000", 2024-05-11T23:03:03.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3630000000000002e-06 2024-05-11T23:03:03.853 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.853 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.853 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.853 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238659 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395392+0000", 2024-05-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011033661, 2024-05-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019201391000000002, 2024-05-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238659 2024-05-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395392+0000", 2024-05-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395392+0000", 2024-05-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395393+0000", 2024-05-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3099999999999998e-07 2024-05-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395395+0000", 2024-05-11T23:03:03.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3769999999999999e-06 2024-05-11T23:03:03.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395402+0000", 2024-05-11T23:03:03.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9269999999999999e-06 2024-05-11T23:03:03.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395408+0000", 2024-05-11T23:03:03.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3550000000000004e-06 2024-05-11T23:03:03.856 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.856 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.856 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238691 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.401091+0000", 2024-05-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0053354209999999999, 2024-05-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.013562687, 2024-05-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238691 2024-05-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401091+0000", 2024-05-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401091+0000", 2024-05-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401091+0000", 2024-05-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4700000000000002e-07 2024-05-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401093+0000", 2024-05-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3020000000000002e-06 2024-05-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401100+0000", 2024-05-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7170000000000002e-06 2024-05-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401106+0000", 2024-05-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3730000000000001e-06 2024-05-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.859 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238726 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.405090+0000", 2024-05-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.001335908, 2024-05-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0096257040000000006, 2024-05-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.860 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238726 2024-05-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405090+0000", 2024-05-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405090+0000", 2024-05-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405091+0000", 2024-05-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4000000000000002e-07 2024-05-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405093+0000", 2024-05-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4509999999999999e-06 2024-05-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405100+0000", 2024-05-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6980000000000001e-06 2024-05-11T23:03:03.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405105+0000", 2024-05-11T23:03:03.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9810000000000003e-06 2024-05-11T23:03:03.863 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.863 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.863 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.863 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238562 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.863 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393284+0000", 2024-05-11T23:03:03.863 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.01314212, 2024-05-11T23:03:03.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021494866000000001, 2024-05-11T23:03:03.863 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.864 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238562 2024-05-11T23:03:03.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.864 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393284+0000", 2024-05-11T23:03:03.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393284+0000", 2024-05-11T23:03:03.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393284+0000", 2024-05-11T23:03:03.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2999999999999999e-07 2024-05-11T23:03:03.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393285+0000", 2024-05-11T23:03:03.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.229e-06 2024-05-11T23:03:03.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393289+0000", 2024-05-11T23:03:03.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.331e-06 2024-05-11T23:03:03.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393292+0000", 2024-05-11T23:03:03.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1939999999999998e-06 2024-05-11T23:03:03.866 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.866 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.866 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.867 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238594 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.867 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393982+0000", 2024-05-11T23:03:03.867 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012444162999999999, 2024-05-11T23:03:03.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020858397000000001, 2024-05-11T23:03:03.867 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.867 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.867 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.867 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.867 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.867 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.867 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238594 2024-05-11T23:03:03.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.867 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393982+0000", 2024-05-11T23:03:03.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393982+0000", 2024-05-11T23:03:03.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393982+0000", 2024-05-11T23:03:03.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3799999999999999e-07 2024-05-11T23:03:03.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393983+0000", 2024-05-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0690000000000001e-06 2024-05-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393986+0000", 2024-05-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2119999999999999e-06 2024-05-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393989+0000", 2024-05-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4380000000000002e-06 2024-05-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.869 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238628 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394690+0000", 2024-05-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011736526000000001, 2024-05-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020210242999999999, 2024-05-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.870 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238628 2024-05-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394690+0000", 2024-05-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394690+0000", 2024-05-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394690+0000", 2024-05-11T23:03:03.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.15e-07 2024-05-11T23:03:03.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394692+0000", 2024-05-11T23:03:03.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.041e-06 2024-05-11T23:03:03.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394702+0000", 2024-05-11T23:03:03.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7200000000000001e-06 2024-05-11T23:03:03.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394707+0000", 2024-05-11T23:03:03.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7270000000000004e-06 2024-05-11T23:03:03.873 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.873 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.873 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.873 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238660 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.873 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395414+0000", 2024-05-11T23:03:03.873 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.01101189, 2024-05-11T23:03:03.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.019695833999999999, 2024-05-11T23:03:03.873 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.873 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.873 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.873 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.873 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.873 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.874 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238660 2024-05-11T23:03:03.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.874 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395414+0000", 2024-05-11T23:03:03.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395414+0000", 2024-05-11T23:03:03.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395415+0000", 2024-05-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395417+0000", 2024-05-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0619999999999999e-06 2024-05-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395424+0000", 2024-05-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.25e-06 2024-05-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395429+0000", 2024-05-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6979999999999997e-06 2024-05-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238692 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.401129+0000", 2024-05-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0052973250000000003, 2024-05-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014043646, 2024-05-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.876 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238692 2024-05-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401129+0000", 2024-05-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401129+0000", 2024-05-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401129+0000", 2024-05-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0399999999999996e-07 2024-05-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401132+0000", 2024-05-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8550000000000002e-06 2024-05-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401140+0000", 2024-05-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6760000000000007e-06 2024-05-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401146+0000", 2024-05-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8139999999999997e-06 2024-05-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238727 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.405111+0000", 2024-05-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0013147720000000001, 2024-05-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.010121642, 2024-05-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.879 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.880 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.880 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.880 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238727 2024-05-11T23:03:03.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.880 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405111+0000", 2024-05-11T23:03:03.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405111+0000", 2024-05-11T23:03:03.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405112+0000", 2024-05-11T23:03:03.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7099999999999997e-07 2024-05-11T23:03:03.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405114+0000", 2024-05-11T23:03:03.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1509999999999998e-06 2024-05-11T23:03:03.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405120+0000", 2024-05-11T23:03:03.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2199999999999997e-06 2024-05-11T23:03:03.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405125+0000", 2024-05-11T23:03:03.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1540000000000001e-06 2024-05-11T23:03:03.882 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.882 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.882 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.882 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238563 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.882 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393302+0000", 2024-05-11T23:03:03.883 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013123747, 2024-05-11T23:03:03.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021996833, 2024-05-11T23:03:03.883 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.883 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.883 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.883 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.883 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.883 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.883 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238563 2024-05-11T23:03:03.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.883 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393302+0000", 2024-05-11T23:03:03.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393302+0000", 2024-05-11T23:03:03.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393303+0000", 2024-05-11T23:03:03.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1200000000000003e-07 2024-05-11T23:03:03.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393305+0000", 2024-05-11T23:03:03.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0930000000000001e-06 2024-05-11T23:03:03.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393311+0000", 2024-05-11T23:03:03.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.426e-06 2024-05-11T23:03:03.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393316+0000", 2024-05-11T23:03:03.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0660000000000003e-06 2024-05-11T23:03:03.885 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.885 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.885 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.886 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238595 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.886 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393993+0000", 2024-05-11T23:03:03.886 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012433244, 2024-05-11T23:03:03.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021365681000000001, 2024-05-11T23:03:03.886 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.886 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.886 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.886 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.886 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.886 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.887 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238595 2024-05-11T23:03:03.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.887 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393993+0000", 2024-05-11T23:03:03.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393993+0000", 2024-05-11T23:03:03.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393993+0000", 2024-05-11T23:03:03.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.16e-07 2024-05-11T23:03:03.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393994+0000", 2024-05-11T23:03:03.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1119999999999999e-06 2024-05-11T23:03:03.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393998+0000", 2024-05-11T23:03:03.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1790000000000001e-06 2024-05-11T23:03:03.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394001+0000", 2024-05-11T23:03:03.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5600000000000001e-06 2024-05-11T23:03:03.889 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.889 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.889 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.890 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238629 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.890 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394715+0000", 2024-05-11T23:03:03.890 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011711351, 2024-05-11T23:03:03.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020705596, 2024-05-11T23:03:03.890 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.890 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.890 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.890 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.890 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.890 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.890 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238629 2024-05-11T23:03:03.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.890 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394715+0000", 2024-05-11T23:03:03.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394715+0000", 2024-05-11T23:03:03.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394715+0000", 2024-05-11T23:03:03.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1399999999999997e-07 2024-05-11T23:03:03.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394718+0000", 2024-05-11T23:03:03.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3360000000000002e-06 2024-05-11T23:03:03.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394726+0000", 2024-05-11T23:03:03.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0469999999999994e-06 2024-05-11T23:03:03.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394732+0000", 2024-05-11T23:03:03.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4710000000000004e-06 2024-05-11T23:03:03.892 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.893 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.893 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.893 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238661 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.893 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395437+0000", 2024-05-11T23:03:03.893 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.010989581999999999, 2024-05-11T23:03:03.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020042912999999999, 2024-05-11T23:03:03.893 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.893 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.893 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.893 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.893 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.893 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.893 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238661 2024-05-11T23:03:03.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.894 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395437+0000", 2024-05-11T23:03:03.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395437+0000", 2024-05-11T23:03:03.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395437+0000", 2024-05-11T23:03:03.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0900000000000002e-07 2024-05-11T23:03:03.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395439+0000", 2024-05-11T23:03:03.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2110000000000001e-06 2024-05-11T23:03:03.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395446+0000", 2024-05-11T23:03:03.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5390000000000004e-06 2024-05-11T23:03:03.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395451+0000", 2024-05-11T23:03:03.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9679999999999997e-06 2024-05-11T23:03:03.896 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.896 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.896 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.896 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238693 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.896 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.401154+0000", 2024-05-11T23:03:03.896 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0052720400000000004, 2024-05-11T23:03:03.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014389311, 2024-05-11T23:03:03.896 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.896 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.896 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.896 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.896 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.897 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238693 2024-05-11T23:03:03.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.897 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401154+0000", 2024-05-11T23:03:03.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401154+0000", 2024-05-11T23:03:03.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401155+0000", 2024-05-11T23:03:03.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7700000000000005e-07 2024-05-11T23:03:03.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401157+0000", 2024-05-11T23:03:03.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3580000000000001e-06 2024-05-11T23:03:03.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401164+0000", 2024-05-11T23:03:03.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6019999999999996e-06 2024-05-11T23:03:03.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401169+0000", 2024-05-11T23:03:03.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4389999999999999e-06 2024-05-11T23:03:03.899 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.899 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.899 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.899 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238728 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.899 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.405132+0000", 2024-05-11T23:03:03.899 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0012943119999999999, 2024-05-11T23:03:03.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.010471013, 2024-05-11T23:03:03.900 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.900 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.900 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.900 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238728 2024-05-11T23:03:03.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.900 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405132+0000", 2024-05-11T23:03:03.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405132+0000", 2024-05-11T23:03:03.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405132+0000", 2024-05-11T23:03:03.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8999999999999996e-07 2024-05-11T23:03:03.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405135+0000", 2024-05-11T23:03:03.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2400000000000002e-06 2024-05-11T23:03:03.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405141+0000", 2024-05-11T23:03:03.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6139999999999997e-06 2024-05-11T23:03:03.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405146+0000", 2024-05-11T23:03:03.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0159999999999999e-06 2024-05-11T23:03:03.903 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.903 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.903 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.903 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238564 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.903 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393323+0000", 2024-05-11T23:03:03.903 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013102926000000001, 2024-05-11T23:03:03.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022344798999999999, 2024-05-11T23:03:03.903 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.903 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.903 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.903 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.903 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.903 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.903 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238564 2024-05-11T23:03:03.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.904 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393323+0000", 2024-05-11T23:03:03.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393323+0000", 2024-05-11T23:03:03.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393324+0000", 2024-05-11T23:03:03.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.65e-07 2024-05-11T23:03:03.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393326+0000", 2024-05-11T23:03:03.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9759999999999998e-06 2024-05-11T23:03:03.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393335+0000", 2024-05-11T23:03:03.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3500000000000003e-06 2024-05-11T23:03:03.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393341+0000", 2024-05-11T23:03:03.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8329999999999999e-06 2024-05-11T23:03:03.906 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.906 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.906 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.906 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238596 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.906 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394004+0000", 2024-05-11T23:03:03.906 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012421985999999999, 2024-05-11T23:03:03.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021724786999999999, 2024-05-11T23:03:03.907 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.907 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.907 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.907 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.907 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.907 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238596 2024-05-11T23:03:03.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.907 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394004+0000", 2024-05-11T23:03:03.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394004+0000", 2024-05-11T23:03:03.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394004+0000", 2024-05-11T23:03:03.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1e-07 2024-05-11T23:03:03.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394006+0000", 2024-05-11T23:03:03.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.238e-06 2024-05-11T23:03:03.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394009+0000", 2024-05-11T23:03:03.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.788e-06 2024-05-11T23:03:03.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394012+0000", 2024-05-11T23:03:03.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.531e-06 2024-05-11T23:03:03.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.909 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.909 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238630 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.910 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394740+0000", 2024-05-11T23:03:03.910 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011685803, 2024-05-11T23:03:03.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021049940999999999, 2024-05-11T23:03:03.910 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.910 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.910 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.910 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.910 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.910 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.910 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238630 2024-05-11T23:03:03.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.910 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394740+0000", 2024-05-11T23:03:03.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394740+0000", 2024-05-11T23:03:03.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394741+0000", 2024-05-11T23:03:03.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1600000000000002e-07 2024-05-11T23:03:03.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394743+0000", 2024-05-11T23:03:03.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1969999999999999e-06 2024-05-11T23:03:03.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394751+0000", 2024-05-11T23:03:03.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9729999999999997e-06 2024-05-11T23:03:03.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394759+0000", 2024-05-11T23:03:03.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7859999999999997e-06 2024-05-11T23:03:03.912 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.912 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.912 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.913 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238662 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.913 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395457+0000", 2024-05-11T23:03:03.913 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.010969045, 2024-05-11T23:03:03.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020392501, 2024-05-11T23:03:03.913 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.913 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.913 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.913 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.913 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.913 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.913 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238662 2024-05-11T23:03:03.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.913 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395457+0000", 2024-05-11T23:03:03.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395457+0000", 2024-05-11T23:03:03.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395457+0000", 2024-05-11T23:03:03.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6899999999999998e-07 2024-05-11T23:03:03.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395460+0000", 2024-05-11T23:03:03.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.142e-06 2024-05-11T23:03:03.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395466+0000", 2024-05-11T23:03:03.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3980000000000004e-06 2024-05-11T23:03:03.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395471+0000", 2024-05-11T23:03:03.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7389999999999999e-06 2024-05-11T23:03:03.915 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.915 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.915 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.916 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238694 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.916 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.401176+0000", 2024-05-11T23:03:03.916 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.005250177, 2024-05-11T23:03:03.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01473927, 2024-05-11T23:03:03.916 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.916 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.916 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.916 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.916 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.916 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.916 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238694 2024-05-11T23:03:03.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.916 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401176+0000", 2024-05-11T23:03:03.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401176+0000", 2024-05-11T23:03:03.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401176+0000", 2024-05-11T23:03:03.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4499999999999997e-07 2024-05-11T23:03:03.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401179+0000", 2024-05-11T23:03:03.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.17e-06 2024-05-11T23:03:03.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401185+0000", 2024-05-11T23:03:03.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6669999999999999e-06 2024-05-11T23:03:03.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401190+0000", 2024-05-11T23:03:03.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.091e-06 2024-05-11T23:03:03.919 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.919 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.919 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.919 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238729 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.919 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.405152+0000", 2024-05-11T23:03:03.919 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0012736080000000001, 2024-05-11T23:03:03.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.010822490000000001, 2024-05-11T23:03:03.919 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.919 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.919 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.919 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.920 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.920 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.920 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238729 2024-05-11T23:03:03.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.920 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405152+0000", 2024-05-11T23:03:03.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405152+0000", 2024-05-11T23:03:03.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405153+0000", 2024-05-11T23:03:03.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5900000000000003e-07 2024-05-11T23:03:03.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405155+0000", 2024-05-11T23:03:03.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1730000000000002e-06 2024-05-11T23:03:03.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405165+0000", 2024-05-11T23:03:03.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.031e-05 2024-05-11T23:03:03.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405170+0000", 2024-05-11T23:03:03.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.147e-06 2024-05-11T23:03:03.922 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.922 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.922 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.923 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238565 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.923 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393349+0000", 2024-05-11T23:03:03.923 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013077016, 2024-05-11T23:03:03.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022689621, 2024-05-11T23:03:03.923 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.923 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.923 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.923 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.923 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.923 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.923 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238565 2024-05-11T23:03:03.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.924 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393349+0000", 2024-05-11T23:03:03.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393349+0000", 2024-05-11T23:03:03.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393349+0000", 2024-05-11T23:03:03.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.01e-07 2024-05-11T23:03:03.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393352+0000", 2024-05-11T23:03:03.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2299999999999998e-06 2024-05-11T23:03:03.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393358+0000", 2024-05-11T23:03:03.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5139999999999998e-06 2024-05-11T23:03:03.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393364+0000", 2024-05-11T23:03:03.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7749999999999998e-06 2024-05-11T23:03:03.926 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.926 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.926 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.926 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238597 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.926 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394015+0000", 2024-05-11T23:03:03.926 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012411044, 2024-05-11T23:03:03.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022086002, 2024-05-11T23:03:03.926 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.927 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.927 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.927 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.927 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.927 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.927 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238597 2024-05-11T23:03:03.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.927 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394015+0000", 2024-05-11T23:03:03.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394015+0000", 2024-05-11T23:03:03.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394015+0000", 2024-05-11T23:03:03.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2700000000000001e-07 2024-05-11T23:03:03.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394016+0000", 2024-05-11T23:03:03.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.097e-06 2024-05-11T23:03:03.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394024+0000", 2024-05-11T23:03:03.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2309999999999999e-06 2024-05-11T23:03:03.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394026+0000", 2024-05-11T23:03:03.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7310000000000002e-06 2024-05-11T23:03:03.929 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.929 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.929 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.929 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238631 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.929 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394767+0000", 2024-05-11T23:03:03.930 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011659187999999999, 2024-05-11T23:03:03.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021393045999999999, 2024-05-11T23:03:03.930 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.930 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.930 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.930 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.930 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.930 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.930 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238631 2024-05-11T23:03:03.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.930 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394767+0000", 2024-05-11T23:03:03.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394767+0000", 2024-05-11T23:03:03.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394767+0000", 2024-05-11T23:03:03.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.08e-07 2024-05-11T23:03:03.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394774+0000", 2024-05-11T23:03:03.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3879999999999997e-06 2024-05-11T23:03:03.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394783+0000", 2024-05-11T23:03:03.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.73e-06 2024-05-11T23:03:03.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394790+0000", 2024-05-11T23:03:03.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3439999999999997e-06 2024-05-11T23:03:03.932 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.932 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.933 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.933 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238663 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.933 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395478+0000", 2024-05-11T23:03:03.933 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.010948505000000001, 2024-05-11T23:03:03.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020744553999999998, 2024-05-11T23:03:03.933 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.933 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.933 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.933 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.933 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.933 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.934 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238663 2024-05-11T23:03:03.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.934 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395478+0000", 2024-05-11T23:03:03.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395478+0000", 2024-05-11T23:03:03.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395478+0000", 2024-05-11T23:03:03.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.77e-07 2024-05-11T23:03:03.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395480+0000", 2024-05-11T23:03:03.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2129999999999998e-06 2024-05-11T23:03:03.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395486+0000", 2024-05-11T23:03:03.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1600000000000003e-06 2024-05-11T23:03:03.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395493+0000", 2024-05-11T23:03:03.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.832e-06 2024-05-11T23:03:03.936 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.936 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.936 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.936 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238695 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.936 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.401197+0000", 2024-05-11T23:03:03.936 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0052291830000000001, 2024-05-11T23:03:03.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015083740999999999, 2024-05-11T23:03:03.936 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.936 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.937 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.937 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.937 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.937 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.937 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238695 2024-05-11T23:03:03.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.937 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401197+0000", 2024-05-11T23:03:03.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401197+0000", 2024-05-11T23:03:03.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401197+0000", 2024-05-11T23:03:03.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8200000000000001e-07 2024-05-11T23:03:03.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401200+0000", 2024-05-11T23:03:03.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.21e-06 2024-05-11T23:03:03.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401206+0000", 2024-05-11T23:03:03.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3459999999999999e-06 2024-05-11T23:03:03.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401210+0000", 2024-05-11T23:03:03.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6179999999999997e-06 2024-05-11T23:03:03.939 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.939 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.939 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.939 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238730 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.939 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.405178+0000", 2024-05-11T23:03:03.940 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.001248381, 2024-05-11T23:03:03.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011165860999999999, 2024-05-11T23:03:03.940 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.940 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.940 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.940 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.940 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238730 2024-05-11T23:03:03.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.940 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405178+0000", 2024-05-11T23:03:03.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405178+0000", 2024-05-11T23:03:03.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405178+0000", 2024-05-11T23:03:03.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5999999999999999e-07 2024-05-11T23:03:03.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405185+0000", 2024-05-11T23:03:03.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7179999999999999e-06 2024-05-11T23:03:03.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405192+0000", 2024-05-11T23:03:03.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.25e-06 2024-05-11T23:03:03.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405197+0000", 2024-05-11T23:03:03.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9529999999999998e-06 2024-05-11T23:03:03.942 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.942 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.943 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.943 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238566 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.943 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393372+0000", 2024-05-11T23:03:03.943 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013053834, 2024-05-11T23:03:03.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023032161999999998, 2024-05-11T23:03:03.943 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.943 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.943 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.943 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.943 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.943 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.943 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238566 2024-05-11T23:03:03.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.944 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393372+0000", 2024-05-11T23:03:03.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393372+0000", 2024-05-11T23:03:03.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393373+0000", 2024-05-11T23:03:03.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1699999999999999e-07 2024-05-11T23:03:03.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393375+0000", 2024-05-11T23:03:03.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9889999999999999e-06 2024-05-11T23:03:03.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393382+0000", 2024-05-11T23:03:03.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1829999999999997e-06 2024-05-11T23:03:03.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393388+0000", 2024-05-11T23:03:03.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0929999999999999e-06 2024-05-11T23:03:03.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.946 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.946 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238600 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.946 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394030+0000", 2024-05-11T23:03:03.946 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012396364999999999, 2024-05-11T23:03:03.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022436810000000001, 2024-05-11T23:03:03.946 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.946 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.946 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.946 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.946 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.947 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238600 2024-05-11T23:03:03.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.947 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394030+0000", 2024-05-11T23:03:03.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.947 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394030+0000", 2024-05-11T23:03:03.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.947 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394030+0000", 2024-05-11T23:03:03.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:03:03.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394031+0000", 2024-05-11T23:03:03.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.251e-06 2024-05-11T23:03:03.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394038+0000", 2024-05-11T23:03:03.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5069999999999997e-06 2024-05-11T23:03:03.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394045+0000", 2024-05-11T23:03:03.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3379999999999999e-06 2024-05-11T23:03:03.949 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.949 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.949 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.949 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238632 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.949 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394799+0000", 2024-05-11T23:03:03.949 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011627238, 2024-05-11T23:03:03.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021728849000000001, 2024-05-11T23:03:03.949 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.949 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.949 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.949 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.949 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.950 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238632 2024-05-11T23:03:03.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.950 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394799+0000", 2024-05-11T23:03:03.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394799+0000", 2024-05-11T23:03:03.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394799+0000", 2024-05-11T23:03:03.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2300000000000002e-07 2024-05-11T23:03:03.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394802+0000", 2024-05-11T23:03:03.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4509999999999999e-06 2024-05-11T23:03:03.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394809+0000", 2024-05-11T23:03:03.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1029999999999996e-06 2024-05-11T23:03:03.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394821+0000", 2024-05-11T23:03:03.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2078e-05 2024-05-11T23:03:03.952 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.952 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.952 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.952 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238664 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.952 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395519+0000", 2024-05-11T23:03:03.952 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.010907218999999999, 2024-05-11T23:03:03.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021072809000000001, 2024-05-11T23:03:03.953 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.953 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.953 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.953 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.953 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.953 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.953 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238664 2024-05-11T23:03:03.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.953 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395519+0000", 2024-05-11T23:03:03.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395519+0000", 2024-05-11T23:03:03.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395519+0000", 2024-05-11T23:03:03.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4200000000000002e-07 2024-05-11T23:03:03.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395521+0000", 2024-05-11T23:03:03.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.945e-06 2024-05-11T23:03:03.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395526+0000", 2024-05-11T23:03:03.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0250000000000002e-06 2024-05-11T23:03:03.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395529+0000", 2024-05-11T23:03:03.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1590000000000002e-06 2024-05-11T23:03:03.955 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.956 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.956 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.956 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238696 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.956 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.401217+0000", 2024-05-11T23:03:03.956 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.005209277, 2024-05-11T23:03:03.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015434217, 2024-05-11T23:03:03.956 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.956 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.956 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.956 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238696 2024-05-11T23:03:03.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.957 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401217+0000", 2024-05-11T23:03:03.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.957 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401217+0000", 2024-05-11T23:03:03.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401217+0000", 2024-05-11T23:03:03.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.39e-07 2024-05-11T23:03:03.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401219+0000", 2024-05-11T23:03:03.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2110000000000001e-06 2024-05-11T23:03:03.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401226+0000", 2024-05-11T23:03:03.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6610000000000003e-06 2024-05-11T23:03:03.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401232+0000", 2024-05-11T23:03:03.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1850000000000001e-06 2024-05-11T23:03:03.959 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.959 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.959 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.959 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238567 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.959 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393395+0000", 2024-05-11T23:03:03.959 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013031575, 2024-05-11T23:03:03.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023322227000000001, 2024-05-11T23:03:03.959 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.960 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.960 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.960 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.960 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.960 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.960 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238567 2024-05-11T23:03:03.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.960 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393395+0000", 2024-05-11T23:03:03.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393395+0000", 2024-05-11T23:03:03.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393395+0000", 2024-05-11T23:03:03.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:03:03.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393397+0000", 2024-05-11T23:03:03.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9740000000000001e-06 2024-05-11T23:03:03.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393403+0000", 2024-05-11T23:03:03.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5939999999999999e-06 2024-05-11T23:03:03.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393408+0000", 2024-05-11T23:03:03.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6280000000000004e-06 2024-05-11T23:03:03.962 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.962 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.962 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.962 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238601 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.962 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394050+0000", 2024-05-11T23:03:03.963 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012376557999999999, 2024-05-11T23:03:03.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022725683, 2024-05-11T23:03:03.963 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.963 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.963 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.963 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.963 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.963 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.963 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238601 2024-05-11T23:03:03.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.963 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394050+0000", 2024-05-11T23:03:03.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394050+0000", 2024-05-11T23:03:03.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394050+0000", 2024-05-11T23:03:03.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4999999999999999e-07 2024-05-11T23:03:03.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394051+0000", 2024-05-11T23:03:03.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4419999999999999e-06 2024-05-11T23:03:03.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394056+0000", 2024-05-11T23:03:03.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0479999999999998e-06 2024-05-11T23:03:03.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394059+0000", 2024-05-11T23:03:03.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0939999999999999e-06 2024-05-11T23:03:03.965 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.966 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.966 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.966 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238633 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.966 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394829+0000", 2024-05-11T23:03:03.966 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011597555000000001, 2024-05-11T23:03:03.966 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022008181000000002, 2024-05-11T23:03:03.966 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.966 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.966 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.966 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.966 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.966 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.966 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238633 2024-05-11T23:03:03.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.967 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394829+0000", 2024-05-11T23:03:03.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394829+0000", 2024-05-11T23:03:03.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.967 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.967 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394829+0000", 2024-05-11T23:03:03.967 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5199999999999998e-07 2024-05-11T23:03:03.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394831+0000", 2024-05-11T23:03:03.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7910000000000001e-06 2024-05-11T23:03:03.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394835+0000", 2024-05-11T23:03:03.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0219999999999998e-06 2024-05-11T23:03:03.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394838+0000", 2024-05-11T23:03:03.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9000000000000002e-06 2024-05-11T23:03:03.968 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.969 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.969 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.969 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238665 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.969 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.395534+0000", 2024-05-11T23:03:03.969 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.010891853999999999, 2024-05-11T23:03:03.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021361419, 2024-05-11T23:03:03.969 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.969 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.969 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.969 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.969 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.969 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.969 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238665 2024-05-11T23:03:03.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.970 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395534+0000", 2024-05-11T23:03:03.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.395534+0000", 2024-05-11T23:03:03.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.396914+0000", 2024-05-11T23:03:03.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0013799909999999999 2024-05-11T23:03:03.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.396919+0000", 2024-05-11T23:03:03.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5210000000000004e-06 2024-05-11T23:03:03.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.396930+0000", 2024-05-11T23:03:03.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1046000000000001e-05 2024-05-11T23:03:03.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.971 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.396938+0000", 2024-05-11T23:03:03.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1499999999999999e-06 2024-05-11T23:03:03.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.972 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.972 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238697 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.972 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.401239+0000", 2024-05-11T23:03:03.972 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0051872929999999999, 2024-05-11T23:03:03.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015725034999999998, 2024-05-11T23:03:03.972 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.973 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.973 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.973 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.973 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.973 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.973 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238697 2024-05-11T23:03:03.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.973 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401239+0000", 2024-05-11T23:03:03.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401239+0000", 2024-05-11T23:03:03.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401239+0000", 2024-05-11T23:03:03.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.75e-07 2024-05-11T23:03:03.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.974 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.974 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401242+0000", 2024-05-11T23:03:03.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3800000000000001e-06 2024-05-11T23:03:03.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401248+0000", 2024-05-11T23:03:03.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.691e-06 2024-05-11T23:03:03.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401253+0000", 2024-05-11T23:03:03.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0100000000000003e-06 2024-05-11T23:03:03.975 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.976 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.976 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.976 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238568 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.976 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393417+0000", 2024-05-11T23:03:03.976 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.013009256, 2024-05-11T23:03:03.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023607446000000001, 2024-05-11T23:03:03.976 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.976 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.976 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.976 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.976 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.976 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.977 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238568 2024-05-11T23:03:03.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.977 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393417+0000", 2024-05-11T23:03:03.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393417+0000", 2024-05-11T23:03:03.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393417+0000", 2024-05-11T23:03:03.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8099999999999998e-07 2024-05-11T23:03:03.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393419+0000", 2024-05-11T23:03:03.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2479999999999999e-06 2024-05-11T23:03:03.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393428+0000", 2024-05-11T23:03:03.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0390000000000005e-06 2024-05-11T23:03:03.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393436+0000", 2024-05-11T23:03:03.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3610000000000001e-06 2024-05-11T23:03:03.979 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.979 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.979 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.979 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238602 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.979 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394087+0000", 2024-05-11T23:03:03.979 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.01233951, 2024-05-11T23:03:03.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023000102000000001, 2024-05-11T23:03:03.980 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.980 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.980 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.980 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.980 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.980 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.980 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238602 2024-05-11T23:03:03.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.980 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394087+0000", 2024-05-11T23:03:03.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394087+0000", 2024-05-11T23:03:03.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394087+0000", 2024-05-11T23:03:03.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2e-07 2024-05-11T23:03:03.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394090+0000", 2024-05-11T23:03:03.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5380000000000001e-06 2024-05-11T23:03:03.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394097+0000", 2024-05-11T23:03:03.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2699999999999999e-06 2024-05-11T23:03:03.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394104+0000", 2024-05-11T23:03:03.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7050000000000002e-06 2024-05-11T23:03:03.982 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.982 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.982 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.983 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238634 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.983 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394841+0000", 2024-05-11T23:03:03.983 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011585023999999999, 2024-05-11T23:03:03.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022305175, 2024-05-11T23:03:03.983 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.983 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.983 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.983 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.983 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.983 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.983 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238634 2024-05-11T23:03:03.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.983 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394841+0000", 2024-05-11T23:03:03.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394841+0000", 2024-05-11T23:03:03.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394841+0000", 2024-05-11T23:03:03.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.72e-07 2024-05-11T23:03:03.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394843+0000", 2024-05-11T23:03:03.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.17e-06 2024-05-11T23:03:03.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394849+0000", 2024-05-11T23:03:03.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9480000000000002e-06 2024-05-11T23:03:03.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394852+0000", 2024-05-11T23:03:03.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8499999999999998e-06 2024-05-11T23:03:03.985 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.985 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.985 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.986 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238666 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.986 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.396950+0000", 2024-05-11T23:03:03.986 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0094764399999999992, 2024-05-11T23:03:03.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020258149999999999, 2024-05-11T23:03:03.986 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.986 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.986 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.986 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.986 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.986 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.986 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238666 2024-05-11T23:03:03.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.987 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.396950+0000", 2024-05-11T23:03:03.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.396950+0000", 2024-05-11T23:03:03.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.396950+0000", 2024-05-11T23:03:03.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8699999999999995e-07 2024-05-11T23:03:03.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.396957+0000", 2024-05-11T23:03:03.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9330000000000004e-06 2024-05-11T23:03:03.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.396965+0000", 2024-05-11T23:03:03.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.678e-06 2024-05-11T23:03:03.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.396970+0000", 2024-05-11T23:03:03.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3310000000000003e-06 2024-05-11T23:03:03.988 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.989 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.989 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.989 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238698 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.989 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.401260+0000", 2024-05-11T23:03:03.989 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0051659729999999999, 2024-05-11T23:03:03.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016010924999999999, 2024-05-11T23:03:03.989 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.989 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.989 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.989 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.989 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.989 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.989 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238698 2024-05-11T23:03:03.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.990 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401260+0000", 2024-05-11T23:03:03.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.401260+0000", 2024-05-11T23:03:03.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.402914+0000", 2024-05-11T23:03:03.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0016537769999999999 2024-05-11T23:03:03.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.402918+0000", 2024-05-11T23:03:03.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9169999999999999e-06 2024-05-11T23:03:03.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.402928+0000", 2024-05-11T23:03:03.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0521e-05 2024-05-11T23:03:03.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.402935+0000", 2024-05-11T23:03:03.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8229999999999997e-06 2024-05-11T23:03:03.992 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.992 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.992 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.993 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238731 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.993 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.405204+0000", 2024-05-11T23:03:03.993 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0012223519999999999, 2024-05-11T23:03:03.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012125858, 2024-05-11T23:03:03.993 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.993 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.993 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.993 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.993 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.993 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.993 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238731 2024-05-11T23:03:03.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.994 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405204+0000", 2024-05-11T23:03:03.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.405204+0000", 2024-05-11T23:03:03.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.406914+0000", 2024-05-11T23:03:03.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001710666 2024-05-11T23:03:03.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.406919+0000", 2024-05-11T23:03:03.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9559999999999996e-06 2024-05-11T23:03:03.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.406930+0000", 2024-05-11T23:03:03.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1097000000000001e-05 2024-05-11T23:03:03.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:03.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.406939+0000", 2024-05-11T23:03:03.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1130000000000001e-06 2024-05-11T23:03:03.996 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.996 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:03.996 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:03.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.997 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238569 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:03.997 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393444+0000", 2024-05-11T23:03:03.997 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012982363, 2024-05-11T23:03:03.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023950326000000001, 2024-05-11T23:03:03.997 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:03.997 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:03.997 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:03.997 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:03.997 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:03.997 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:03.998 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238569 2024-05-11T23:03:03.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.998 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:03.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:03.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393444+0000", 2024-05-11T23:03:03.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:03.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393444+0000", 2024-05-11T23:03:03.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:03.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:03.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393444+0000", 2024-05-11T23:03:03.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.08e-07 2024-05-11T23:03:03.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:03.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393450+0000", 2024-05-11T23:03:03.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1099999999999999e-06 2024-05-11T23:03:03.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:03.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:03.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:03.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393467+0000", 2024-05-11T23:03:03.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7068e-05 2024-05-11T23:03:03.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393473+0000", 2024-05-11T23:03:04.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2480000000000004e-06 2024-05-11T23:03:04.000 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.000 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.000 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.000 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238603 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.000 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394113+0000", 2024-05-11T23:03:04.000 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012313121999999999, 2024-05-11T23:03:04.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023339537, 2024-05-11T23:03:04.000 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.000 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.001 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.001 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.001 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.001 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238603 2024-05-11T23:03:04.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.001 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394113+0000", 2024-05-11T23:03:04.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394113+0000", 2024-05-11T23:03:04.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394114+0000", 2024-05-11T23:03:04.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6100000000000001e-07 2024-05-11T23:03:04.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394116+0000", 2024-05-11T23:03:04.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1260000000000001e-06 2024-05-11T23:03:04.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394122+0000", 2024-05-11T23:03:04.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6059999999999999e-06 2024-05-11T23:03:04.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394129+0000", 2024-05-11T23:03:04.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.528e-06 2024-05-11T23:03:04.003 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.003 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.003 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.003 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238635 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.003 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394856+0000", 2024-05-11T23:03:04.003 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011569869999999999, 2024-05-11T23:03:04.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022657755000000002, 2024-05-11T23:03:04.003 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.004 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.004 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.004 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.004 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.004 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.004 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238635 2024-05-11T23:03:04.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.004 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394856+0000", 2024-05-11T23:03:04.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394856+0000", 2024-05-11T23:03:04.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394856+0000", 2024-05-11T23:03:04.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5899999999999998e-07 2024-05-11T23:03:04.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394858+0000", 2024-05-11T23:03:04.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.285e-06 2024-05-11T23:03:04.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394861+0000", 2024-05-11T23:03:04.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4460000000000001e-06 2024-05-11T23:03:04.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394864+0000", 2024-05-11T23:03:04.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.475e-06 2024-05-11T23:03:04.007 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.007 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.007 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.007 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238667 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.007 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.396978+0000", 2024-05-11T23:03:04.007 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0094483129999999999, 2024-05-11T23:03:04.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020595183999999999, 2024-05-11T23:03:04.007 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.007 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.008 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.008 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.008 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238667 2024-05-11T23:03:04.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.008 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.396978+0000", 2024-05-11T23:03:04.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.396978+0000", 2024-05-11T23:03:04.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.396978+0000", 2024-05-11T23:03:04.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4000000000000002e-07 2024-05-11T23:03:04.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.396981+0000", 2024-05-11T23:03:04.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2919999999999998e-06 2024-05-11T23:03:04.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.396988+0000", 2024-05-11T23:03:04.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1999999999999997e-06 2024-05-11T23:03:04.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.396995+0000", 2024-05-11T23:03:04.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7279999999999995e-06 2024-05-11T23:03:04.010 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.010 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.010 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.010 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238699 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.010 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.402947+0000", 2024-05-11T23:03:04.010 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0034793039999999999, 2024-05-11T23:03:04.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.014688425, 2024-05-11T23:03:04.011 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.011 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.011 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.011 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.011 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.011 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238699 2024-05-11T23:03:04.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.011 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.402947+0000", 2024-05-11T23:03:04.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.402947+0000", 2024-05-11T23:03:04.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.402947+0000", 2024-05-11T23:03:04.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-05-11T23:03:04.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.402950+0000", 2024-05-11T23:03:04.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5339999999999998e-06 2024-05-11T23:03:04.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.402957+0000", 2024-05-11T23:03:04.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3860000000000001e-06 2024-05-11T23:03:04.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.402962+0000", 2024-05-11T23:03:04.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2120000000000002e-06 2024-05-11T23:03:04.013 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.013 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.013 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.014 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238732 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.014 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.406951+0000", 2024-05-11T23:03:04.014 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4294967295.999475, 2024-05-11T23:03:04.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.010744792, 2024-05-11T23:03:04.014 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.014 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.014 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.014 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.014 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.014 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.014 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238732 2024-05-11T23:03:04.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.015 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.406951+0000", 2024-05-11T23:03:04.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.406951+0000", 2024-05-11T23:03:04.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.406951+0000", 2024-05-11T23:03:04.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8999999999999997e-07 2024-05-11T23:03:04.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.406954+0000", 2024-05-11T23:03:04.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6079999999999998e-06 2024-05-11T23:03:04.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.406961+0000", 2024-05-11T23:03:04.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4159999999999998e-06 2024-05-11T23:03:04.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.406967+0000", 2024-05-11T23:03:04.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4530000000000001e-06 2024-05-11T23:03:04.017 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.017 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.017 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.017 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238570 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.017 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393483+0000", 2024-05-11T23:03:04.017 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012943343, 2024-05-11T23:03:04.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024276346000000001, 2024-05-11T23:03:04.017 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.017 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.017 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.017 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.017 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.017 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238570 2024-05-11T23:03:04.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.018 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393483+0000", 2024-05-11T23:03:04.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393483+0000", 2024-05-11T23:03:04.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393483+0000", 2024-05-11T23:03:04.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:03:04.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393486+0000", 2024-05-11T23:03:04.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:03:04.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393498+0000", 2024-05-11T23:03:04.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2311e-05 2024-05-11T23:03:04.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393504+0000", 2024-05-11T23:03:04.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7030000000000003e-06 2024-05-11T23:03:04.020 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.020 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.020 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.020 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238604 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.020 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394136+0000", 2024-05-11T23:03:04.020 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012289906999999999, 2024-05-11T23:03:04.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023683359000000001, 2024-05-11T23:03:04.020 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.020 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.020 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.020 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.020 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.020 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.021 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238604 2024-05-11T23:03:04.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.021 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394136+0000", 2024-05-11T23:03:04.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394136+0000", 2024-05-11T23:03:04.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394137+0000", 2024-05-11T23:03:04.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:03:04.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394139+0000", 2024-05-11T23:03:04.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1689999999999999e-06 2024-05-11T23:03:04.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394149+0000", 2024-05-11T23:03:04.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9180000000000006e-06 2024-05-11T23:03:04.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394154+0000", 2024-05-11T23:03:04.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4999999999999999e-06 2024-05-11T23:03:04.023 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.023 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.023 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.023 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238636 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.023 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394867+0000", 2024-05-11T23:03:04.023 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011559432, 2024-05-11T23:03:04.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023015486000000002, 2024-05-11T23:03:04.023 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.023 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.023 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.023 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.023 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.024 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.024 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238636 2024-05-11T23:03:04.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.024 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394867+0000", 2024-05-11T23:03:04.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394867+0000", 2024-05-11T23:03:04.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394867+0000", 2024-05-11T23:03:04.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.17e-07 2024-05-11T23:03:04.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394868+0000", 2024-05-11T23:03:04.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.161e-06 2024-05-11T23:03:04.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394871+0000", 2024-05-11T23:03:04.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2169999999999999e-06 2024-05-11T23:03:04.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394875+0000", 2024-05-11T23:03:04.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2820000000000001e-06 2024-05-11T23:03:04.026 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.026 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.026 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.026 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238668 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.026 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.397003+0000", 2024-05-11T23:03:04.026 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0094229120000000003, 2024-05-11T23:03:04.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020942582000000001, 2024-05-11T23:03:04.027 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.027 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.027 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.027 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.027 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.027 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.027 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238668 2024-05-11T23:03:04.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.027 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397003+0000", 2024-05-11T23:03:04.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397003+0000", 2024-05-11T23:03:04.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397004+0000", 2024-05-11T23:03:04.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:04.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397006+0000", 2024-05-11T23:03:04.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8789999999999999e-06 2024-05-11T23:03:04.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397014+0000", 2024-05-11T23:03:04.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0600000000000002e-06 2024-05-11T23:03:04.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397019+0000", 2024-05-11T23:03:04.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1850000000000003e-06 2024-05-11T23:03:04.030 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.030 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.030 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.030 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238700 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.030 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.402979+0000", 2024-05-11T23:03:04.030 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0034475880000000001, 2024-05-11T23:03:04.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015030462, 2024-05-11T23:03:04.030 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.030 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.030 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.030 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.030 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.031 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.031 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238700 2024-05-11T23:03:04.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.031 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.402979+0000", 2024-05-11T23:03:04.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.402979+0000", 2024-05-11T23:03:04.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.402979+0000", 2024-05-11T23:03:04.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1699999999999998e-07 2024-05-11T23:03:04.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.402982+0000", 2024-05-11T23:03:04.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1020000000000001e-06 2024-05-11T23:03:04.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.402991+0000", 2024-05-11T23:03:04.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4370000000000007e-06 2024-05-11T23:03:04.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.402996+0000", 2024-05-11T23:03:04.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6010000000000002e-06 2024-05-11T23:03:04.033 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.033 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.033 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.033 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238733 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.033 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.406974+0000", 2024-05-11T23:03:04.033 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4294967295.9994516, 2024-05-11T23:03:04.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011094255000000001, 2024-05-11T23:03:04.034 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.034 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.034 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.034 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.034 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.034 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.034 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238733 2024-05-11T23:03:04.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.034 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.406974+0000", 2024-05-11T23:03:04.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.406974+0000", 2024-05-11T23:03:04.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.406975+0000", 2024-05-11T23:03:04.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.96e-07 2024-05-11T23:03:04.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.406977+0000", 2024-05-11T23:03:04.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.272e-06 2024-05-11T23:03:04.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.406984+0000", 2024-05-11T23:03:04.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5899999999999996e-06 2024-05-11T23:03:04.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.406989+0000", 2024-05-11T23:03:04.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2789999999999998e-06 2024-05-11T23:03:04.036 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.036 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.036 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.037 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238571 3.1 3:8135c7ce:::rbd_data.111ebf0f641b.000000000000000d:2e [read 1821696~1024 out=1024b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.037 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.393513+0000", 2024-05-11T23:03:04.037 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012912666999999999, 2024-05-11T23:03:04.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024625971999999999, 2024-05-11T23:03:04.037 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.037 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.037 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:04.037 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.037 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.037 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.037 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238571 2024-05-11T23:03:04.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.037 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393513+0000", 2024-05-11T23:03:04.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393513+0000", 2024-05-11T23:03:04.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393514+0000", 2024-05-11T23:03:04.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6600000000000002e-07 2024-05-11T23:03:04.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393516+0000", 2024-05-11T23:03:04.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.48e-06 2024-05-11T23:03:04.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393523+0000", 2024-05-11T23:03:04.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5710000000000003e-06 2024-05-11T23:03:04.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.393528+0000", 2024-05-11T23:03:04.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0889999999999999e-06 2024-05-11T23:03:04.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:04.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.418058+0000", 2024-05-11T23:03:04.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024529893000000001 2024-05-11T23:03:04.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:04.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.418097+0000", 2024-05-11T23:03:04.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8828999999999997e-05 2024-05-11T23:03:04.040 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.040 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.040 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.040 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238605 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.040 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394163+0000", 2024-05-11T23:03:04.041 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012263481, 2024-05-11T23:03:04.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024049911, 2024-05-11T23:03:04.041 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.041 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.041 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.041 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.041 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.041 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.041 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238605 2024-05-11T23:03:04.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.041 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394163+0000", 2024-05-11T23:03:04.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394163+0000", 2024-05-11T23:03:04.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394163+0000", 2024-05-11T23:03:04.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4900000000000001e-07 2024-05-11T23:03:04.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394166+0000", 2024-05-11T23:03:04.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4310000000000001e-06 2024-05-11T23:03:04.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394172+0000", 2024-05-11T23:03:04.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9600000000000003e-06 2024-05-11T23:03:04.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394177+0000", 2024-05-11T23:03:04.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9389999999999996e-06 2024-05-11T23:03:04.043 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.043 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.044 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.044 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238637 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.044 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394878+0000", 2024-05-11T23:03:04.044 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011548526999999999, 2024-05-11T23:03:04.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023398499, 2024-05-11T23:03:04.044 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.044 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.044 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.044 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.044 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.044 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.045 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238637 2024-05-11T23:03:04.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.045 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394878+0000", 2024-05-11T23:03:04.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394878+0000", 2024-05-11T23:03:04.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394878+0000", 2024-05-11T23:03:04.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2700000000000001e-07 2024-05-11T23:03:04.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394879+0000", 2024-05-11T23:03:04.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.189e-06 2024-05-11T23:03:04.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394882+0000", 2024-05-11T23:03:04.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.18e-06 2024-05-11T23:03:04.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394885+0000", 2024-05-11T23:03:04.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3690000000000001e-06 2024-05-11T23:03:04.047 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.047 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.047 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.047 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238669 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.047 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.397026+0000", 2024-05-11T23:03:04.047 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0094006039999999999, 2024-05-11T23:03:04.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021309583999999999, 2024-05-11T23:03:04.047 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.047 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.048 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.048 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.048 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.048 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.048 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238669 2024-05-11T23:03:04.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.048 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397026+0000", 2024-05-11T23:03:04.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397026+0000", 2024-05-11T23:03:04.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397026+0000", 2024-05-11T23:03:04.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2e-07 2024-05-11T23:03:04.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397028+0000", 2024-05-11T23:03:04.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2759999999999999e-06 2024-05-11T23:03:04.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397035+0000", 2024-05-11T23:03:04.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4529999999999999e-06 2024-05-11T23:03:04.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397040+0000", 2024-05-11T23:03:04.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9590000000000003e-06 2024-05-11T23:03:04.050 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.050 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.050 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.050 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238701 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.051 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.403004+0000", 2024-05-11T23:03:04.051 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0034224379999999999, 2024-05-11T23:03:04.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015392123000000001, 2024-05-11T23:03:04.051 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.051 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.051 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.051 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.051 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.051 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238701 2024-05-11T23:03:04.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.051 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403004+0000", 2024-05-11T23:03:04.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403004+0000", 2024-05-11T23:03:04.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403004+0000", 2024-05-11T23:03:04.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8599999999999998e-07 2024-05-11T23:03:04.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403006+0000", 2024-05-11T23:03:04.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.345e-06 2024-05-11T23:03:04.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403013+0000", 2024-05-11T23:03:04.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7510000000000002e-06 2024-05-11T23:03:04.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403018+0000", 2024-05-11T23:03:04.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2460000000000003e-06 2024-05-11T23:03:04.054 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.054 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.054 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.054 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238734 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.054 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.406996+0000", 2024-05-11T23:03:04.054 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4294967295.9994302, 2024-05-11T23:03:04.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011459449999999999, 2024-05-11T23:03:04.054 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.054 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.054 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.054 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.055 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.055 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.055 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238734 2024-05-11T23:03:04.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.055 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.406996+0000", 2024-05-11T23:03:04.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.406996+0000", 2024-05-11T23:03:04.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.406996+0000", 2024-05-11T23:03:04.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0699999999999998e-07 2024-05-11T23:03:04.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.406999+0000", 2024-05-11T23:03:04.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4279999999999999e-06 2024-05-11T23:03:04.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.407005+0000", 2024-05-11T23:03:04.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.455e-06 2024-05-11T23:03:04.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.407012+0000", 2024-05-11T23:03:04.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4099999999999996e-06 2024-05-11T23:03:04.057 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.057 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.057 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.057 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238606 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.057 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394184+0000", 2024-05-11T23:03:04.057 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.01224243, 2024-05-11T23:03:04.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024331562000000001, 2024-05-11T23:03:04.057 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.057 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.058 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238606 2024-05-11T23:03:04.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.058 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394184+0000", 2024-05-11T23:03:04.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394184+0000", 2024-05-11T23:03:04.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394184+0000", 2024-05-11T23:03:04.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6699999999999999e-07 2024-05-11T23:03:04.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394186+0000", 2024-05-11T23:03:04.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1409999999999999e-06 2024-05-11T23:03:04.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394193+0000", 2024-05-11T23:03:04.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5599999999999999e-06 2024-05-11T23:03:04.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394198+0000", 2024-05-11T23:03:04.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.921e-06 2024-05-11T23:03:04.060 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.060 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.061 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238638 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.061 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394888+0000", 2024-05-11T23:03:04.061 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011538423000000001, 2024-05-11T23:03:04.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023688827999999999, 2024-05-11T23:03:04.061 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.061 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.061 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.061 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.061 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.061 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.062 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238638 2024-05-11T23:03:04.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.062 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394888+0000", 2024-05-11T23:03:04.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394888+0000", 2024-05-11T23:03:04.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394888+0000", 2024-05-11T23:03:04.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.23e-07 2024-05-11T23:03:04.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394889+0000", 2024-05-11T23:03:04.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0470000000000001e-06 2024-05-11T23:03:04.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394893+0000", 2024-05-11T23:03:04.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9319999999999998e-06 2024-05-11T23:03:04.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394896+0000", 2024-05-11T23:03:04.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.818e-06 2024-05-11T23:03:04.064 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.065 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.065 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.065 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238670 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.065 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.397046+0000", 2024-05-11T23:03:04.065 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0093799209999999994, 2024-05-11T23:03:04.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021588916999999999, 2024-05-11T23:03:04.065 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.065 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.065 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.066 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238670 2024-05-11T23:03:04.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.066 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397046+0000", 2024-05-11T23:03:04.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397046+0000", 2024-05-11T23:03:04.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397047+0000", 2024-05-11T23:03:04.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.53e-07 2024-05-11T23:03:04.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397049+0000", 2024-05-11T23:03:04.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1349999999999999e-06 2024-05-11T23:03:04.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397055+0000", 2024-05-11T23:03:04.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.426e-06 2024-05-11T23:03:04.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397060+0000", 2024-05-11T23:03:04.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9439999999999996e-06 2024-05-11T23:03:04.068 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.068 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.068 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.069 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238735 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.069 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.407019+0000", 2024-05-11T23:03:04.069 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4294967295.9994068, 2024-05-11T23:03:04.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011682087000000001, 2024-05-11T23:03:04.069 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.069 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.069 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.069 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.069 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.069 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.069 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238735 2024-05-11T23:03:04.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.070 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.407019+0000", 2024-05-11T23:03:04.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.407019+0000", 2024-05-11T23:03:04.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.407019+0000", 2024-05-11T23:03:04.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7399999999999999e-07 2024-05-11T23:03:04.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.407022+0000", 2024-05-11T23:03:04.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.367e-06 2024-05-11T23:03:04.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.407032+0000", 2024-05-11T23:03:04.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0268000000000001e-05 2024-05-11T23:03:04.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.407038+0000", 2024-05-11T23:03:04.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.541e-06 2024-05-11T23:03:04.072 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.072 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.072 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.072 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238607 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.072 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394205+0000", 2024-05-11T23:03:04.072 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012221184, 2024-05-11T23:03:04.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024558021999999999, 2024-05-11T23:03:04.072 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.072 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.072 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.073 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.073 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.073 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.073 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238607 2024-05-11T23:03:04.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.073 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394205+0000", 2024-05-11T23:03:04.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394205+0000", 2024-05-11T23:03:04.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394205+0000", 2024-05-11T23:03:04.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8599999999999999e-07 2024-05-11T23:03:04.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394212+0000", 2024-05-11T23:03:04.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.207e-06 2024-05-11T23:03:04.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394218+0000", 2024-05-11T23:03:04.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9589999999999998e-06 2024-05-11T23:03:04.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394223+0000", 2024-05-11T23:03:04.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6149999999999999e-06 2024-05-11T23:03:04.075 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.075 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.075 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.075 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238639 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.075 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394900+0000", 2024-05-11T23:03:04.076 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011526115999999999, 2024-05-11T23:03:04.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.023926149000000001, 2024-05-11T23:03:04.076 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.076 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.076 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.076 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.076 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.076 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.076 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238639 2024-05-11T23:03:04.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.076 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394900+0000", 2024-05-11T23:03:04.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394900+0000", 2024-05-11T23:03:04.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394900+0000", 2024-05-11T23:03:04.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1500000000000001e-07 2024-05-11T23:03:04.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394901+0000", 2024-05-11T23:03:04.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.229e-06 2024-05-11T23:03:04.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394905+0000", 2024-05-11T23:03:04.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1760000000000002e-06 2024-05-11T23:03:04.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394907+0000", 2024-05-11T23:03:04.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5799999999999999e-06 2024-05-11T23:03:04.078 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.078 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.078 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.079 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238671 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.079 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.397067+0000", 2024-05-11T23:03:04.079 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0093591279999999995, 2024-05-11T23:03:04.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021821393000000001, 2024-05-11T23:03:04.079 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.079 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.079 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.079 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.079 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.079 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.079 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238671 2024-05-11T23:03:04.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.079 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397067+0000", 2024-05-11T23:03:04.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397067+0000", 2024-05-11T23:03:04.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397067+0000", 2024-05-11T23:03:04.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:03:04.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397070+0000", 2024-05-11T23:03:04.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1780000000000002e-06 2024-05-11T23:03:04.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397076+0000", 2024-05-11T23:03:04.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2280000000000003e-06 2024-05-11T23:03:04.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397082+0000", 2024-05-11T23:03:04.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8730000000000004e-06 2024-05-11T23:03:04.081 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.081 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.082 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.082 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238707 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.082 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.403059+0000", 2024-05-11T23:03:04.082 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0033667929999999999, 2024-05-11T23:03:04.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.015898427999999999, 2024-05-11T23:03:04.082 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.082 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.082 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.082 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.082 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.082 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.082 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238707 2024-05-11T23:03:04.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.083 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403059+0000", 2024-05-11T23:03:04.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403059+0000", 2024-05-11T23:03:04.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403060+0000", 2024-05-11T23:03:04.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3399999999999999e-07 2024-05-11T23:03:04.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403062+0000", 2024-05-11T23:03:04.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6280000000000001e-06 2024-05-11T23:03:04.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403071+0000", 2024-05-11T23:03:04.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0100000000000001e-06 2024-05-11T23:03:04.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403077+0000", 2024-05-11T23:03:04.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4280000000000004e-06 2024-05-11T23:03:04.085 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.085 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.085 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.085 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238736 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.085 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.407045+0000", 2024-05-11T23:03:04.085 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4294967295.9993811, 2024-05-11T23:03:04.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.011974063, 2024-05-11T23:03:04.085 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.085 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.085 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.086 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.086 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.086 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.086 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238736 2024-05-11T23:03:04.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.086 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.407045+0000", 2024-05-11T23:03:04.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.407045+0000", 2024-05-11T23:03:04.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.407046+0000", 2024-05-11T23:03:04.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:04.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.407048+0000", 2024-05-11T23:03:04.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:03:04.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.407055+0000", 2024-05-11T23:03:04.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9299999999999997e-06 2024-05-11T23:03:04.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.407060+0000", 2024-05-11T23:03:04.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7539999999999997e-06 2024-05-11T23:03:04.088 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.088 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.088 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.088 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238608 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.088 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394233+0000", 2024-05-11T23:03:04.088 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.012192873999999999, 2024-05-11T23:03:04.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.02485045, 2024-05-11T23:03:04.089 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.089 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.089 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.089 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.089 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.089 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.089 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238608 2024-05-11T23:03:04.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.089 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394233+0000", 2024-05-11T23:03:04.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394233+0000", 2024-05-11T23:03:04.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394234+0000", 2024-05-11T23:03:04.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1300000000000001e-07 2024-05-11T23:03:04.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394236+0000", 2024-05-11T23:03:04.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5660000000000001e-06 2024-05-11T23:03:04.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394244+0000", 2024-05-11T23:03:04.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9030000000000004e-06 2024-05-11T23:03:04.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394249+0000", 2024-05-11T23:03:04.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2320000000000001e-06 2024-05-11T23:03:04.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.091 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.092 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238640 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.092 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.394921+0000", 2024-05-11T23:03:04.092 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.011505425, 2024-05-11T23:03:04.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.024223166000000001, 2024-05-11T23:03:04.092 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.092 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.092 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.092 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.092 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.092 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.092 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238640 2024-05-11T23:03:04.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.092 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394921+0000", 2024-05-11T23:03:04.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394921+0000", 2024-05-11T23:03:04.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394921+0000", 2024-05-11T23:03:04.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5400000000000002e-07 2024-05-11T23:03:04.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394922+0000", 2024-05-11T23:03:04.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4670000000000001e-06 2024-05-11T23:03:04.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394926+0000", 2024-05-11T23:03:04.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5480000000000002e-06 2024-05-11T23:03:04.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.394930+0000", 2024-05-11T23:03:04.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0890000000000002e-06 2024-05-11T23:03:04.095 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.095 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.095 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.095 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238672 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.095 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.397107+0000", 2024-05-11T23:03:04.095 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0093193689999999992, 2024-05-11T23:03:04.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.022100970000000001, 2024-05-11T23:03:04.095 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.095 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.095 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.095 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.095 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.095 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.096 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238672 2024-05-11T23:03:04.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.096 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397107+0000", 2024-05-11T23:03:04.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397107+0000", 2024-05-11T23:03:04.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397107+0000", 2024-05-11T23:03:04.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.72e-07 2024-05-11T23:03:04.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397109+0000", 2024-05-11T23:03:04.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0279999999999999e-06 2024-05-11T23:03:04.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397119+0000", 2024-05-11T23:03:04.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4350000000000003e-06 2024-05-11T23:03:04.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.397124+0000", 2024-05-11T23:03:04.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1309999999999997e-06 2024-05-11T23:03:04.098 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.098 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.098 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.098 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238708 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.098 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.403093+0000", 2024-05-11T23:03:04.098 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0033333999999999998, 2024-05-11T23:03:04.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016175881, 2024-05-11T23:03:04.098 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.098 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.098 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.098 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.098 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.099 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.099 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238708 2024-05-11T23:03:04.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.099 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403093+0000", 2024-05-11T23:03:04.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403093+0000", 2024-05-11T23:03:04.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403093+0000", 2024-05-11T23:03:04.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5000000000000003e-07 2024-05-11T23:03:04.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403096+0000", 2024-05-11T23:03:04.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.728e-06 2024-05-11T23:03:04.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403103+0000", 2024-05-11T23:03:04.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.379e-06 2024-05-11T23:03:04.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.403109+0000", 2024-05-11T23:03:04.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9839999999999998e-06 2024-05-11T23:03:04.101 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.101 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.101 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.101 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:238737 3.1 3.73e3ac81 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:04.101 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.407067+0000", 2024-05-11T23:03:04.101 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4294967295.9993587, 2024-05-11T23:03:04.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.012265023999999999, 2024-05-11T23:03:04.101 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:04.101 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:04.102 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "queued for pg", 2024-05-11T23:03:04.102 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:04.102 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:04.102 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:04.102 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 238737 2024-05-11T23:03:04.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.102 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:04.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:04.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.407067+0000", 2024-05-11T23:03:04.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:04.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.407067+0000", 2024-05-11T23:03:04.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:04.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:04.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.407068+0000", 2024-05-11T23:03:04.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4299999999999998e-07 2024-05-11T23:03:04.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:04.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.407070+0000", 2024-05-11T23:03:04.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4889999999999998e-06 2024-05-11T23:03:04.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:04.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.407078+0000", 2024-05-11T23:03:04.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2899999999999997e-06 2024-05-11T23:03:04.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:04.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:04.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:04.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.407083+0000", 2024-05-11T23:03:04.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2000000000000002e-06 2024-05-11T23:03:04.104 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.104 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:04.104 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.104 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:04.104 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:03:04.105 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 205 2024-05-11T23:03:04.105 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:04.105 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:03:04.106 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:04.105+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:04.107 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:03:04.206 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:04.206+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:04.273 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:04.273 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:04.273 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:04.273 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:04.273 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:04.283 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:03:04.286 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-05-11T23:03:04.293 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:04.293 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:04.297 INFO:teuthology.orchestra.run.smithi005.stdout:4744 punch from 0x8b84c9 to 0x8bb371, (0x2ea8 bytes) 2024-05-11T23:03:04.305 INFO:teuthology.orchestra.run.smithi005.stdout:4746 trunc from 0x5cdc533 to 0x8ef0794 2024-05-11T23:03:04.307 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:04.307+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:04.311 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:04.311 INFO:teuthology.orchestra.run.smithi005.stdout:4748 read 0x7f50619 thru 0x7f5d0ff (0xcae7 bytes) 2024-05-11T23:03:04.311 INFO:teuthology.orchestra.run.smithi005.stdout:4749 read 0x37606f0 thru 0x376e175 (0xda86 bytes) 2024-05-11T23:03:04.316 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:04.316 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:04.328 INFO:teuthology.orchestra.run.smithi005.stdout:4752 writesame 0x93572b9 thru 0x935733c data_size 0x6(0x84 bytes) 2024-05-11T23:03:04.329 INFO:teuthology.orchestra.run.smithi005.stdout:4753 write 0x55aebc2 thru 0x55bb496 (0xc8d5 bytes) 2024-05-11T23:03:04.351 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:04.351 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:04.368 INFO:teuthology.orchestra.run.smithi005.stdout:4754 write 0x3d6abea thru 0x3d75200 (0xa617 bytes) 2024-05-11T23:03:04.388 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:03:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:03:04.396 INFO:teuthology.orchestra.run.smithi005.stdout:4756 read 0x552ed69 thru 0x5531ddd (0x3075 bytes) 2024-05-11T23:03:04.405 INFO:teuthology.orchestra.run.smithi005.stdout:4757 write 0xa9ce2d4 thru 0xa9d9298 (0xafc5 bytes) 2024-05-11T23:03:04.408 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:04.407+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:04.412 INFO:teuthology.orchestra.run.smithi005.stdout:4761 punch from 0x3d1a3ec to 0x3d24da3, (0xa9b7 bytes) 2024-05-11T23:03:04.412 INFO:teuthology.orchestra.run.smithi005.stdout:4762 read 0x5bfd89 thru 0x5cb938 (0xbbb0 bytes) 2024-05-11T23:03:04.414 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:04.414 INFO:teuthology.orchestra.run.smithi005.stdout:4764 punch from 0x2c7e63a to 0x2c83d67, (0x572d bytes) 2024-05-11T23:03:04.414 INFO:teuthology.orchestra.run.smithi005.stdout:4765 read 0x613f64b thru 0x614ad89 (0xb73f bytes) 2024-05-11T23:03:04.416 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:04.422 INFO:teuthology.orchestra.run.smithi005.stdout:4766 write 0xc6ee53c thru 0xc6ef9eb (0x14b0 bytes) 2024-05-11T23:03:04.425 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:04.425 INFO:teuthology.orchestra.run.smithi005.stdout:4769 punch from 0x924db45 to 0x925b0aa, (0xd565 bytes) 2024-05-11T23:03:04.433 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:04.433 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:04.435 INFO:teuthology.orchestra.run.smithi005.stdout:4770 trunc from 0xc6ef9ec to 0x4d074be 2024-05-11T23:03:04.444 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:04.444 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:04.447 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:03:04.473 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:04.474 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:04.478 INFO:teuthology.orchestra.run.smithi005.stdout:4771 writesame 0xa301a7c thru 0xa30d747 data_size 0x1(0xbccc bytes) 2024-05-11T23:03:04.478 INFO:teuthology.orchestra.run.smithi005.stdout:4772 read 0x1b612f thru 0x1b9f57 (0x3e29 bytes) 2024-05-11T23:03:04.488 INFO:teuthology.orchestra.run.smithi005.stdout:4774 write 0xc700fbc thru 0xc707141 (0x6186 bytes) 2024-05-11T23:03:04.492 INFO:teuthology.orchestra.run.smithi005.stdout:4776 read 0xb4f630 thru 0xb5939f (0x9d70 bytes) 2024-05-11T23:03:04.493 INFO:teuthology.orchestra.run.smithi005.stdout:4777 write 0x1dd2e06 thru 0x1ddab26 (0x7d21 bytes) 2024-05-11T23:03:04.499 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:04.508 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:04.508+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:04.556 INFO:teuthology.orchestra.run.smithi005.stdout:4778 read 0x73efa4c thru 0x73f5284 (0x5839 bytes) 2024-05-11T23:03:04.557 INFO:teuthology.orchestra.run.smithi005.stdout:4780 write 0x1805325 thru 0x180bf09 (0x6be5 bytes) 2024-05-11T23:03:04.580 INFO:teuthology.orchestra.run.smithi005.stdout:4781 read 0x2a879fb thru 0x2a93c5c (0xc262 bytes) 2024-05-11T23:03:04.581 INFO:teuthology.orchestra.run.smithi005.stdout:4782 read 0x4c63f93 thru 0x4c6b275 (0x72e3 bytes) 2024-05-11T23:03:04.581 INFO:teuthology.orchestra.run.smithi005.stdout:4783 read 0x411c0e5 thru 0x41266d8 (0xa5f4 bytes) 2024-05-11T23:03:04.593 INFO:teuthology.orchestra.run.smithi005.stdout:4784 trunc from 0xc707142 to 0x8131e68 2024-05-11T23:03:04.602 INFO:teuthology.orchestra.run.smithi005.stdout:4785 read 0x7257fe1 thru 0x7266ec7 (0xeee7 bytes) 2024-05-11T23:03:04.602 INFO:teuthology.orchestra.run.smithi005.stdout:4786 writesame 0x7128f4c thru 0x7130a70 data_size 0x41(0x7b25 bytes) 2024-05-11T23:03:04.607 INFO:teuthology.orchestra.run.smithi005.stdout:4787 write 0x318c42e thru 0x31905aa (0x417d bytes) 2024-05-11T23:03:04.609 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:04.608+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:04.616 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:04.616 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:04.627 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:03:04.678 INFO:teuthology.orchestra.run.smithi005.stdout:4788 read 0x4cbc596 thru 0x4cc7fc9 (0xba34 bytes) 2024-05-11T23:03:04.679 INFO:teuthology.orchestra.run.smithi005.stdout:4790 punch from 0x40f9778 to 0x40f9d9a, (0x622 bytes) 2024-05-11T23:03:04.679 INFO:teuthology.orchestra.run.smithi005.stdout:4791 read 0x6d7ccb0 thru 0x6d88770 (0xbac1 bytes) 2024-05-11T23:03:04.691 INFO:teuthology.orchestra.run.smithi005.stdout:4793 write 0xdf9bbdb thru 0xdf9cf7b (0x13a1 bytes) 2024-05-11T23:03:04.701 INFO:teuthology.orchestra.run.smithi005.stdout:4794 write 0xec5b43f thru 0xec68a06 (0xd5c8 bytes) 2024-05-11T23:03:04.709 INFO:teuthology.orchestra.run.smithi005.stdout:4795 read 0xa78d81d thru 0xa79bbfd (0xe3e1 bytes) 2024-05-11T23:03:04.709 INFO:teuthology.orchestra.run.smithi005.stdout:4797 write 0xdbb9172 thru 0xdbb9e25 (0xcb4 bytes) 2024-05-11T23:03:04.710 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:04.709+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:04.712 INFO:teuthology.orchestra.run.smithi005.stdout:4798 writesame 0x7f535d7 thru 0x7f550af data_size 0x1d(0x1ad9 bytes) 2024-05-11T23:03:04.716 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:04.717 INFO:teuthology.orchestra.run.smithi005.stdout:4800 read 0xb4873b3 thru 0xb48f274 (0x7ec2 bytes) 2024-05-11T23:03:04.727 INFO:teuthology.orchestra.run.smithi005.stdout:4802 trunc from 0xec68a07 to 0x5501080 2024-05-11T23:03:04.745 INFO:teuthology.orchestra.run.smithi005.stdout:4803 write 0x590f2cb thru 0x5912933 (0x3669 bytes) 2024-05-11T23:03:04.761 INFO:teuthology.orchestra.run.smithi005.stdout:4805 writesame 0xba4cf67 thru 0xba54a0b data_size 0x1(0x7aa5 bytes) 2024-05-11T23:03:04.770 INFO:teuthology.orchestra.run.smithi005.stdout:4808 trunc from 0xba54a0c to 0x403c3b7 2024-05-11T23:03:04.797 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:04.797 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:04.807 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:03:04.810 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:04.810+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:04.911 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:04.911+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:04.962 INFO:teuthology.orchestra.run.smithi005.stdout:4810 punch from 0x438f0d to 0x441635, (0x8728 bytes) 2024-05-11T23:03:04.963 INFO:teuthology.orchestra.run.smithi005.stdout:4811 write 0x3513885 thru 0x351a88a (0x7006 bytes) 2024-05-11T23:03:04.971 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:04.971 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:04.981 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:03:05.012 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:05.012+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:05.059 INFO:teuthology.orchestra.run.smithi005.stdout:4813 trunc from 0x403c3b7 to 0xe0a50d3 2024-05-11T23:03:05.064 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:05.064 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:05.064 INFO:teuthology.orchestra.run.smithi005.stdout:4817 writesame 0x59d2e0 thru 0x5a4f55 data_size 0x1(0x7c76 bytes) 2024-05-11T23:03:05.109 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:05.110 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:05.113 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:05.112+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:05.120 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:03:05.121 INFO:teuthology.orchestra.run.smithi005.stdout:4818 write 0x6522e1f thru 0x652fa7f (0xcc61 bytes) 2024-05-11T23:03:05.128 INFO:teuthology.orchestra.run.smithi005.stdout:4819 write 0x444c480 thru 0x44501e7 (0x3d68 bytes) 2024-05-11T23:03:05.147 INFO:teuthology.orchestra.run.smithi005.stdout:4820 trunc from 0xe0a50d3 to 0xff112b 2024-05-11T23:03:05.213 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:05.213+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:05.215 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-11T23:03:05.271 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:05.271 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:05.280 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:03:05.309 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:05.309 INFO:teuthology.orchestra.run.smithi005.stdout:4822 read 0xa92c8c thru 0xaa05f4 (0xd969 bytes) 2024-05-11T23:03:05.315 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:05.314+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:05.326 INFO:teuthology.orchestra.run.smithi005.stdout:4823 writesame 0x7a1aa53 thru 0x7a20390 data_size 0x1(0x593e bytes) 2024-05-11T23:03:05.326 INFO:teuthology.orchestra.run.smithi005.stdout:4824 write 0x1ec1dfb thru 0x1ec2a24 (0xc2a bytes) 2024-05-11T23:03:05.329 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:05.338 INFO:teuthology.orchestra.run.smithi005.stdout:4827 write 0xa6e5e3a thru 0xa6f290c (0xcad3 bytes) 2024-05-11T23:03:05.345 INFO:teuthology.orchestra.run.smithi005.stdout:4828 read 0x13de923 thru 0x13df65d (0xd3b bytes) 2024-05-11T23:03:05.346 INFO:teuthology.orchestra.run.smithi005.stdout:4829 write 0x83e9344 thru 0x83f73da (0xe097 bytes) 2024-05-11T23:03:05.363 INFO:teuthology.orchestra.run.smithi005.stdout:4830 trunc from 0xa6f290d to 0xbadc87 2024-05-11T23:03:05.416 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:05.415+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:05.431 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:05.431 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:05.442 INFO:teuthology.orchestra.run.smithi005.stdout:4831 trunc from 0xbadc87 to 0xb76cb34 2024-05-11T23:03:05.445 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:03:05.453 INFO:teuthology.orchestra.run.smithi005.stdout:4832 write 0xbc2a0f7 thru 0xbc326ba (0x85c4 bytes) 2024-05-11T23:03:05.459 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:05.459 INFO:teuthology.orchestra.run.smithi005.stdout:4834 punch from 0x8ee8fe5 to 0x8ef6692, (0xd6ad bytes) 2024-05-11T23:03:05.462 INFO:teuthology.orchestra.run.smithi005.stdout:4835 trunc from 0xbc326bb to 0x8e01f0a 2024-05-11T23:03:05.474 INFO:teuthology.orchestra.run.smithi005.stdout:4836 write 0x5974e8f thru 0x59768e9 (0x1a5b bytes) 2024-05-11T23:03:05.480 INFO:teuthology.orchestra.run.smithi005.stdout:4837 trunc from 0x8e01f0a to 0x70c7f44 2024-05-11T23:03:05.492 INFO:teuthology.orchestra.run.smithi005.stdout:4838 write 0x73dbcf6 thru 0x73e551c (0x9827 bytes) 2024-05-11T23:03:05.499 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:05.499 INFO:teuthology.orchestra.run.smithi005.stdout:4841 writesame 0x38a2428 thru 0x38b1cc2 data_size 0x1(0xf89b bytes) 2024-05-11T23:03:05.516 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:05.516+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:05.524 INFO:teuthology.orchestra.run.smithi005.stdout:4842 read 0x8283ce thru 0x82a8ac (0x24df bytes) 2024-05-11T23:03:05.525 INFO:teuthology.orchestra.run.smithi005.stdout:4844 write 0x3550ba0 thru 0x3557b98 (0x6ff9 bytes) 2024-05-11T23:03:05.530 INFO:teuthology.orchestra.run.smithi005.stdout:4846 punch from 0x664cd0a to 0x6656857, (0x9b4d bytes) 2024-05-11T23:03:05.543 INFO:teuthology.orchestra.run.smithi005.stdout:4848 write 0xcff931d thru 0xcff97db (0x4bf bytes) 2024-05-11T23:03:05.547 INFO:teuthology.orchestra.run.smithi005.stdout:4851 read 0x58ea2cd thru 0x58ee38f (0x40c3 bytes) 2024-05-11T23:03:05.547 INFO:teuthology.orchestra.run.smithi005.stdout:4852 write 0x344a506 thru 0x345a066 (0xfb61 bytes) 2024-05-11T23:03:05.555 INFO:teuthology.orchestra.run.smithi005.stdout:4853 write 0xa10deeb thru 0xa112041 (0x4157 bytes) 2024-05-11T23:03:05.568 INFO:teuthology.orchestra.run.smithi005.stdout:4854 write 0xe6f63ff thru 0xe70021b (0x9e1d bytes) 2024-05-11T23:03:05.576 INFO:teuthology.orchestra.run.smithi005.stdout:4856 read 0xdd8e17c thru 0xdd9a4c2 (0xc347 bytes) 2024-05-11T23:03:05.576 INFO:teuthology.orchestra.run.smithi005.stdout:4857 write 0xd9b62f5 thru 0xd9b91d8 (0x2ee4 bytes) 2024-05-11T23:03:05.581 INFO:teuthology.orchestra.run.smithi005.stdout:4858 write 0x9bbec50 thru 0x9bbfcc5 (0x1076 bytes) 2024-05-11T23:03:05.584 INFO:teuthology.orchestra.run.smithi005.stdout:4859 writesame 0x3380678 thru 0x338d7e6 data_size 0x1(0xd16f bytes) 2024-05-11T23:03:05.597 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:05.597 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:05.597 INFO:teuthology.orchestra.run.smithi005.stdout:4860 write 0x5a514d0 thru 0x5a5ee56 (0xd987 bytes) 2024-05-11T23:03:05.605 INFO:teuthology.orchestra.run.smithi005.stdout:4861 write 0x37c2862 thru 0x37cc005 (0x97a4 bytes) 2024-05-11T23:03:05.609 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:03:05.609 INFO:teuthology.orchestra.run.smithi005.stdout:4862 write 0x4cbce2f thru 0x4cbfbe2 (0x2db4 bytes) 2024-05-11T23:03:05.612 INFO:teuthology.orchestra.run.smithi005.stdout:4863 write 0xb3f9297 thru 0xb40126c (0x7fd6 bytes) 2024-05-11T23:03:05.616 INFO:teuthology.orchestra.run.smithi005.stdout:4864 write 0x6d8c0ba thru 0x6d96920 (0xa867 bytes) 2024-05-11T23:03:05.617 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:05.616+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:05.623 INFO:teuthology.orchestra.run.smithi005.stdout:4866 writesame 0x8b104a8 thru 0x8b2046a data_size 0x87(0xffc3 bytes) 2024-05-11T23:03:05.632 INFO:teuthology.orchestra.run.smithi005.stdout:4868 read 0xdb7a320 thru 0xdb7dcd6 (0x39b7 bytes) 2024-05-11T23:03:05.632 INFO:teuthology.orchestra.run.smithi005.stdout:4869 write 0x2486b94 thru 0x248c779 (0x5be6 bytes) 2024-05-11T23:03:05.637 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:05.638 INFO:teuthology.orchestra.run.smithi005.stdout:4872 read 0xc57a20c thru 0xc58948f (0xf284 bytes) 2024-05-11T23:03:05.652 INFO:teuthology.orchestra.run.smithi005.stdout:4874 trunc from 0xe70021c to 0x21d2474 2024-05-11T23:03:05.676 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:05.676 INFO:teuthology.orchestra.run.smithi005.stdout:4878 read 0x63b0bc thru 0x63bddd (0xd22 bytes) 2024-05-11T23:03:05.676 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:05.690 INFO:teuthology.orchestra.run.smithi005.stdout:4880 writesame 0x95374ab thru 0x953bd83 data_size 0x1(0x48d9 bytes) 2024-05-11T23:03:05.690 INFO:teuthology.orchestra.run.smithi005.stdout:4881 read 0x7c6a58 thru 0x7cd17d (0x6726 bytes) 2024-05-11T23:03:05.692 INFO:teuthology.orchestra.run.smithi005.stdout:4882 read 0x1949933 thru 0x194e726 (0x4df4 bytes) 2024-05-11T23:03:05.692 INFO:teuthology.orchestra.run.smithi005.stdout:4883 punch from 0x57d078 to 0x582f37, (0x5ebf bytes) 2024-05-11T23:03:05.692 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:05.692 INFO:teuthology.orchestra.run.smithi005.stdout:4885 read 0x7a66d1e thru 0x7a7658f (0xf872 bytes) 2024-05-11T23:03:05.692 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:05.694 INFO:teuthology.orchestra.run.smithi005.stdout:4887 trunc from 0x953bd84 to 0xbac6dc0 2024-05-11T23:03:05.700 INFO:teuthology.orchestra.run.smithi005.stdout:4888 read 0x86d13b6 thru 0x86d60de (0x4d29 bytes) 2024-05-11T23:03:05.708 INFO:teuthology.orchestra.run.smithi005.stdout:4890 trunc from 0xbac6dc0 to 0x4e5f6b1 2024-05-11T23:03:05.714 INFO:teuthology.orchestra.run.smithi005.stdout:4891 writesame 0x18e6129 thru 0x18f5f7d data_size 0xb(0xfe55 bytes) 2024-05-11T23:03:05.717 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:05.717+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:05.722 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:05.727 INFO:teuthology.orchestra.run.smithi005.stdout:4894 trunc from 0x4e5f6b1 to 0x935e8c1 2024-05-11T23:03:05.734 INFO:teuthology.orchestra.run.smithi005.stdout:4897 read 0x60e8c2e thru 0x60f6b0e (0xdee1 bytes) 2024-05-11T23:03:05.734 INFO:teuthology.orchestra.run.smithi005.stdout:4898 write 0x717b551 thru 0x7186a46 (0xb4f6 bytes) 2024-05-11T23:03:05.740 INFO:teuthology.orchestra.run.smithi005.stdout:4899 write 0x5d78df7 thru 0x5d7e8f2 (0x5afc bytes) 2024-05-11T23:03:05.743 INFO:teuthology.orchestra.run.smithi005.stdout:4900 write 0x3b5cae9 thru 0x3b5ea0d (0x1f25 bytes) 2024-05-11T23:03:05.753 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:05.753 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:05.755 INFO:teuthology.orchestra.run.smithi005.stdout:4901 trunc from 0x935e8c1 to 0x1c96674 2024-05-11T23:03:05.765 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:03:05.775 INFO:teuthology.orchestra.run.smithi005.stdout:4902 trunc from 0x1c96674 to 0xab80841 2024-05-11T23:03:05.792 INFO:teuthology.orchestra.run.smithi005.stdout:4903 trunc from 0xab80841 to 0x167c4e8 2024-05-11T23:03:05.803 INFO:teuthology.orchestra.run.smithi005.stdout:4905 punch from 0xb31532 to 0xb3b1f0, (0x9cbe bytes) 2024-05-11T23:03:05.804 INFO:teuthology.orchestra.run.smithi005.stdout:4906 read 0x2de2ab thru 0x2e0a75 (0x27cb bytes) 2024-05-11T23:03:05.806 INFO:teuthology.orchestra.run.smithi005.stdout:4908 read 0x9baaa0 thru 0x9c466c (0x9bcd bytes) 2024-05-11T23:03:05.818 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:05.817+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:05.826 INFO:teuthology.orchestra.run.smithi005.stdout:4909 write 0xbafe50a thru 0xbb0cc46 (0xe73d bytes) 2024-05-11T23:03:05.832 INFO:teuthology.orchestra.run.smithi005.stdout:4910 write 0x5cdbbf2 thru 0x5cdff58 (0x4367 bytes) 2024-05-11T23:03:05.843 INFO:teuthology.orchestra.run.smithi005.stdout:4911 writesame 0xcdd3210 thru 0xcdd7a06 data_size 0x45(0x47f7 bytes) 2024-05-11T23:03:05.843 INFO:teuthology.orchestra.run.smithi005.stdout:4912 read 0x4f548d thru 0x4ff818 (0xa38c bytes) 2024-05-11T23:03:05.843 INFO:teuthology.orchestra.run.smithi005.stdout:4913 write 0x242e75d thru 0x2436019 (0x78bd bytes) 2024-05-11T23:03:05.848 INFO:teuthology.orchestra.run.smithi005.stdout:4917 write 0xb2c4084 thru 0xb2c8d5e (0x4cdb bytes) 2024-05-11T23:03:05.852 INFO:teuthology.orchestra.run.smithi005.stdout:4918 punch from 0x90ebd10 to 0x90f5046, (0x9336 bytes) 2024-05-11T23:03:05.859 INFO:teuthology.orchestra.run.smithi005.stdout:4919 write 0xd791a91 thru 0xd797800 (0x5d70 bytes) 2024-05-11T23:03:05.864 INFO:teuthology.orchestra.run.smithi005.stdout:4920 write 0x13187f7 thru 0x131d2a0 (0x4aaa bytes) 2024-05-11T23:03:05.868 INFO:teuthology.orchestra.run.smithi005.stdout:4921 writesame 0x86a8f13 thru 0x86b5b4a data_size 0x14(0xcc38 bytes) 2024-05-11T23:03:05.874 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:05.874 INFO:teuthology.orchestra.run.smithi005.stdout:4923 punch from 0x1822ae1 to 0x182b783, (0x8ca2 bytes) 2024-05-11T23:03:05.875 INFO:teuthology.orchestra.run.smithi005.stdout:4925 read 0x7b8e7dc thru 0x7b8fb10 (0x1335 bytes) 2024-05-11T23:03:05.875 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:05.875 INFO:teuthology.orchestra.run.smithi005.stdout:4927 write 0xacf11d2 thru 0xacff8cc (0xe6fb bytes) 2024-05-11T23:03:05.880 INFO:teuthology.orchestra.run.smithi005.stdout:4928 write 0xd5c09c5 thru 0xd5cd56e (0xcbaa bytes) 2024-05-11T23:03:05.887 INFO:teuthology.orchestra.run.smithi005.stdout:4929 punch from 0xd54f3a7 to 0xd5569b2, (0x760b bytes) 2024-05-11T23:03:05.887 INFO:teuthology.orchestra.run.smithi005.stdout:4930 read 0x5370c13 thru 0x53794fb (0x88e9 bytes) 2024-05-11T23:03:05.887 INFO:teuthology.orchestra.run.smithi005.stdout:4932 write 0x54807c9 thru 0x5481a7c (0x12b4 bytes) 2024-05-11T23:03:05.890 INFO:teuthology.orchestra.run.smithi005.stdout:4933 punch from 0x3b18b04 to 0x3b24333, (0xb82f bytes) 2024-05-11T23:03:05.890 INFO:teuthology.orchestra.run.smithi005.stdout:4934 punch from 0x6353c0e to 0x6356eab, (0x329d bytes) 2024-05-11T23:03:05.890 INFO:teuthology.orchestra.run.smithi005.stdout:4936 read 0x40d4440 thru 0x40d9ad8 (0x5699 bytes) 2024-05-11T23:03:05.890 INFO:teuthology.orchestra.run.smithi005.stdout:4937 write 0xd5110ee thru 0xd51ee49 (0xdd5c bytes) 2024-05-11T23:03:05.894 INFO:teuthology.orchestra.run.smithi005.stdout:4938 read 0x5094c7b thru 0x509faad (0xae33 bytes) 2024-05-11T23:03:05.894 INFO:teuthology.orchestra.run.smithi005.stdout:4939 writesame 0x8dbc0a9 thru 0x8dc6d51 data_size 0x1(0xaca9 bytes) 2024-05-11T23:03:05.908 INFO:teuthology.orchestra.run.smithi005.stdout:4940 writesame 0x5629d77 thru 0x5639c76 data_size 0x88(0xff00 bytes) 2024-05-11T23:03:05.917 INFO:teuthology.orchestra.run.smithi005.stdout:4941 punch from 0x48d906f to 0x48e2086, (0x9017 bytes) 2024-05-11T23:03:05.918 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:05.918 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:05.919 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:05.918+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:05.928 INFO:teuthology.orchestra.run.smithi005.stdout:4942 trunc from 0xd797801 to 0x44d7e5e 2024-05-11T23:03:05.933 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:03:05.950 INFO:teuthology.orchestra.run.smithi005.stdout:4943 trunc from 0x44d7e5e to 0x9580fda 2024-05-11T23:03:05.955 INFO:teuthology.orchestra.run.smithi005.stdout:4944 write 0x85d606b thru 0x85dfcb6 (0x9c4c bytes) 2024-05-11T23:03:05.961 INFO:teuthology.orchestra.run.smithi005.stdout:4945 write 0x61b0189 thru 0x61bfc49 (0xfac1 bytes) 2024-05-11T23:03:05.967 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:05.967 INFO:teuthology.orchestra.run.smithi005.stdout:4947 read 0x5638b02 thru 0x563d8e7 (0x4de6 bytes) 2024-05-11T23:03:05.967 INFO:teuthology.orchestra.run.smithi005.stdout:4949 punch from 0x772f42 to 0x77b5f7, (0x86b5 bytes) 2024-05-11T23:03:05.968 INFO:teuthology.orchestra.run.smithi005.stdout:4952 write 0x787e17e thru 0x7885927 (0x77aa bytes) 2024-05-11T23:03:05.973 INFO:teuthology.orchestra.run.smithi005.stdout:4953 read 0x4f07be4 thru 0x4f13c26 (0xc043 bytes) 2024-05-11T23:03:05.973 INFO:teuthology.orchestra.run.smithi005.stdout:4955 punch from 0x5c6884a to 0x5c6937d, (0xb33 bytes) 2024-05-11T23:03:05.973 INFO:teuthology.orchestra.run.smithi005.stdout:4956 read 0x406c6d2 thru 0x406e8d6 (0x2205 bytes) 2024-05-11T23:03:05.978 INFO:teuthology.orchestra.run.smithi005.stdout:4957 trunc from 0x9580fda to 0x4ab0d17 2024-05-11T23:03:05.993 INFO:teuthology.orchestra.run.smithi005.stdout:4958 read 0x3d1fa38 thru 0x3d2cf7b (0xd544 bytes) 2024-05-11T23:03:06.005 INFO:teuthology.orchestra.run.smithi005.stdout:4960 writesame 0xaaa8aea thru 0xaaaefff data_size 0x6(0x6516 bytes) 2024-05-11T23:03:06.005 INFO:teuthology.orchestra.run.smithi005.stdout:4961 punch from 0x733e88a to 0x7342ee0, (0x4656 bytes) 2024-05-11T23:03:06.005 INFO:teuthology.orchestra.run.smithi005.stdout:4963 writesame 0xa7ea104 thru 0xa7eaab9 data_size 0x2(0x9b6 bytes) 2024-05-11T23:03:06.009 INFO:teuthology.orchestra.run.smithi005.stdout:4964 read 0x4ac24ac thru 0x4acf0cd (0xcc22 bytes) 2024-05-11T23:03:06.009 INFO:teuthology.orchestra.run.smithi005.stdout:4966 write 0x987d4f8 thru 0x9880dcd (0x38d6 bytes) 2024-05-11T23:03:06.013 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:06.013 INFO:teuthology.orchestra.run.smithi005.stdout:4969 punch from 0x3a7e85c to 0x3a7fdd0, (0x1574 bytes) 2024-05-11T23:03:06.013 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:06.013 INFO:teuthology.orchestra.run.smithi005.stdout:4972 writesame 0x9ab1429 thru 0x9ab90ee data_size 0x1(0x7cc6 bytes) 2024-05-11T23:03:06.019 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:06.018+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:06.030 INFO:teuthology.orchestra.run.smithi005.stdout:4973 trunc from 0xaaaf000 to 0xeb9b43b 2024-05-11T23:03:06.037 INFO:teuthology.orchestra.run.smithi005.stdout:4974 punch from 0x71c0e6d to 0x71c89b0, (0x7b43 bytes) 2024-05-11T23:03:06.037 INFO:teuthology.orchestra.run.smithi005.stdout:4975 punch from 0x45102e7 to 0x451c1fb, (0xbf14 bytes) 2024-05-11T23:03:06.037 INFO:teuthology.orchestra.run.smithi005.stdout:4976 writesame 0x4f1c519 thru 0x4f244bd data_size 0x1(0x7fa5 bytes) 2024-05-11T23:03:06.046 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:06.046 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:06.048 INFO:teuthology.orchestra.run.smithi005.stdout:4977 write 0xb8cdf19 thru 0xb8d7ce3 (0x9dcb bytes) 2024-05-11T23:03:06.053 INFO:teuthology.orchestra.run.smithi005.stdout:4978 read 0xdfc2668 thru 0xdfd1d7e (0xf717 bytes) 2024-05-11T23:03:06.053 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:06.053 INFO:teuthology.orchestra.run.smithi005.stdout:4980 punch from 0x8eee2a3 to 0x8eef5a9, (0x1306 bytes) 2024-05-11T23:03:06.054 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:06.054 INFO:teuthology.orchestra.run.smithi005.stdout:4983 writesame 0x4d2720e thru 0x4d29284 data_size 0x1(0x2077 bytes) 2024-05-11T23:03:06.056 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:03:06.058 INFO:teuthology.orchestra.run.smithi005.stdout:4984 read 0x66725ac thru 0x66779ab (0x5400 bytes) 2024-05-11T23:03:06.058 INFO:teuthology.orchestra.run.smithi005.stdout:4985 writesame 0x235ebf1 thru 0x23665ad data_size 0x17(0x79bd bytes) 2024-05-11T23:03:06.063 INFO:teuthology.orchestra.run.smithi005.stdout:4986 punch from 0x67f22ba to 0x67fc0b5, (0x9dfb bytes) 2024-05-11T23:03:06.063 INFO:teuthology.orchestra.run.smithi005.stdout:4987 read 0x1314038 thru 0x131f36e (0xb337 bytes) 2024-05-11T23:03:06.063 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:06.079 INFO:teuthology.orchestra.run.smithi005.stdout:4989 trunc from 0xeb9b43b to 0xb91d1a 2024-05-11T23:03:06.118 INFO:teuthology.orchestra.run.smithi005.stdout:4990 writesame 0x9e2bf2b thru 0x9e39953 data_size 0x1(0xda29 bytes) 2024-05-11T23:03:06.118 INFO:teuthology.orchestra.run.smithi005.stdout:4995 read 0x8530ea8 thru 0x853d745 (0xc89e bytes) 2024-05-11T23:03:06.120 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:06.119+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:06.124 INFO:teuthology.orchestra.run.smithi005.stdout:4996 trunc from 0x9e39954 to 0x4942ce3 2024-05-11T23:03:06.129 INFO:teuthology.orchestra.run.smithi005.stdout:4997 read 0x38241c4 thru 0x382948a (0x52c7 bytes) 2024-05-11T23:03:06.129 INFO:teuthology.orchestra.run.smithi005.stdout:4998 write 0x32a46e7 thru 0x32afc92 (0xb5ac bytes) 2024-05-11T23:03:06.137 INFO:teuthology.orchestra.run.smithi005.stdout:4999 read 0x2fb2deb thru 0x2fc14cd (0xe6e3 bytes) 2024-05-11T23:03:06.137 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:06.137 INFO:teuthology.orchestra.run.smithi005.stdout:5002 punch from 0x12727b8 to 0x128272b, (0xff73 bytes) 2024-05-11T23:03:06.145 INFO:teuthology.orchestra.run.smithi005.stdout:5003 write 0x6d2c6e5 thru 0x6d3c5c8 (0xfee4 bytes) 2024-05-11T23:03:06.153 INFO:teuthology.orchestra.run.smithi005.stdout:5004 flatten 2024-05-11T23:03:06.161 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-05-11T23:03:06.171 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:06.171 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:06.180 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:03:06.223 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:06.223+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:06.323 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:06.322+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:06.335 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:06.335 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:06.345 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:03:06.429 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:06.429+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:06.516 INFO:teuthology.orchestra.run.smithi005.stdout:5005 read 0x6cf686 thru 0x6def2c (0xf8a7 bytes) 2024-05-11T23:03:06.516 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:03:06.517 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:06.517 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:06.519 INFO:teuthology.orchestra.run.smithi005.stdout:5006 trunc from 0x6d3c5c9 to 0x4dc4b48 2024-05-11T23:03:06.524 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:06.524+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:06.525 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:03:06.531 INFO:teuthology.orchestra.run.smithi005.stdout:5007 read 0x466f097 thru 0x467b970 (0xc8da bytes) 2024-05-11T23:03:06.548 INFO:teuthology.orchestra.run.smithi005.stdout:5008 writesame 0xe00638e thru 0xe00c6f9 data_size 0x1c(0x636c bytes) 2024-05-11T23:03:06.549 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:06.549 INFO:teuthology.orchestra.run.smithi005.stdout:5012 punch from 0x5648345 to 0x564dabe, (0x5779 bytes) 2024-05-11T23:03:06.549 INFO:teuthology.orchestra.run.smithi005.stdout:5014 read 0x68788ee thru 0x688009d (0x77b0 bytes) 2024-05-11T23:03:06.549 INFO:teuthology.orchestra.run.smithi005.stdout:5015 read 0xc8dfc58 thru 0xc8e6d0b (0x70b4 bytes) 2024-05-11T23:03:06.549 INFO:teuthology.orchestra.run.smithi005.stdout:5016 write 0x65fe900 thru 0x6600071 (0x1772 bytes) 2024-05-11T23:03:06.553 INFO:teuthology.orchestra.run.smithi005.stdout:5017 writesame 0x74d5801 thru 0x74d97ce data_size 0x1(0x3fce bytes) 2024-05-11T23:03:06.561 INFO:teuthology.orchestra.run.smithi005.stdout:5018 write 0x9471fba thru 0x9479125 (0x716c bytes) 2024-05-11T23:03:06.566 INFO:teuthology.orchestra.run.smithi005.stdout:5019 read 0xcd69460 thru 0xcd731c8 (0x9d69 bytes) 2024-05-11T23:03:06.566 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:06.568 INFO:teuthology.orchestra.run.smithi005.stdout:5022 trunc from 0xe00c6fa to 0xc27e78c 2024-05-11T23:03:06.574 INFO:teuthology.orchestra.run.smithi005.stdout:5023 read 0x5e2e645 thru 0x5e3c171 (0xdb2d bytes) 2024-05-11T23:03:06.586 INFO:teuthology.orchestra.run.smithi005.stdout:5025 trunc from 0xc27e78c to 0x1598a2e 2024-05-11T23:03:06.609 INFO:teuthology.orchestra.run.smithi005.stdout:5027 writesame 0x591f746 thru 0x59201a5 data_size 0x8(0xa60 bytes) 2024-05-11T23:03:06.610 INFO:teuthology.orchestra.run.smithi005.stdout:5029 read 0x134ba47 thru 0x1359ad6 (0xe090 bytes) 2024-05-11T23:03:06.610 INFO:teuthology.orchestra.run.smithi005.stdout:5030 punch from 0x4ad8947 to 0x4ae5f0e, (0xd5c7 bytes) 2024-05-11T23:03:06.610 INFO:teuthology.orchestra.run.smithi005.stdout:5031 writesame 0x441c547 thru 0x4423f3d data_size 0x1(0x79f7 bytes) 2024-05-11T23:03:06.625 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:06.625+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:06.637 INFO:teuthology.orchestra.run.smithi005.stdout:5032 writesame 0xe4db010 thru 0xe4e6248 data_size 0x19(0xb239 bytes) 2024-05-11T23:03:06.638 INFO:teuthology.orchestra.run.smithi005.stdout:5033 writesame 0xc5edb5e thru 0xc5fcacf data_size 0x1(0xef72 bytes) 2024-05-11T23:03:06.671 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:06.671 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:06.675 INFO:teuthology.orchestra.run.smithi005.stdout:5034 trunc from 0xe4e6249 to 0x3e76af 2024-05-11T23:03:06.680 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:06.680 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:06.691 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:03:06.691 INFO:teuthology.orchestra.run.smithi005.stdout:5035 punch from 0x3c1935 to 0x3c730f, (0x59da bytes) 2024-05-11T23:03:06.696 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:06.696 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:06.698 INFO:teuthology.orchestra.run.smithi005.stdout:5037 write 0x245bbd1 thru 0x245de7e (0x22ae bytes) 2024-05-11T23:03:06.703 INFO:teuthology.orchestra.run.smithi005.stdout:5038 read 0x3e434 thru 0x43a4e (0x561b bytes) 2024-05-11T23:03:06.703 INFO:teuthology.orchestra.run.smithi005.stdout:5039 read 0x763535 thru 0x76b46f (0x7f3b bytes) 2024-05-11T23:03:06.703 INFO:teuthology.orchestra.run.smithi005.stdout:5040 read 0xb95fce thru 0xb99a1b (0x3a4e bytes) 2024-05-11T23:03:06.704 INFO:teuthology.orchestra.run.smithi005.stdout:5042 read 0x1f858e9 thru 0x1f90952 (0xb06a bytes) 2024-05-11T23:03:06.706 INFO:teuthology.orchestra.run.smithi005.stdout:5044 trunc from 0x245de7f to 0x47ca2ff 2024-05-11T23:03:06.712 INFO:teuthology.orchestra.run.smithi005.stdout:5045 punch from 0x43fc5bc to 0x440a3dd, (0xde21 bytes) 2024-05-11T23:03:06.721 INFO:teuthology.orchestra.run.smithi005.stdout:5046 trunc from 0x47ca2ff to 0xb2cff1e 2024-05-11T23:03:06.725 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:06.726 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:06.724+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:06.736 INFO:teuthology.orchestra.run.smithi005.stdout:5048 trunc from 0xb2cff1e to 0x2d9a39e 2024-05-11T23:03:06.744 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:03:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:03:06.750 INFO:teuthology.orchestra.run.smithi005.stdout:5049 write 0x460c0b7 thru 0x461a248 (0xe192 bytes) 2024-05-11T23:03:06.757 INFO:teuthology.orchestra.run.smithi005.stdout:5051 writesame 0xdc04b2 thru 0xdca99d data_size 0xa(0xa4ec bytes) 2024-05-11T23:03:06.763 INFO:teuthology.orchestra.run.smithi005.stdout:5052 read 0x450f1bc thru 0x450ff6c (0xdb1 bytes) 2024-05-11T23:03:06.770 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:06.770 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:06.782 INFO:teuthology.orchestra.run.smithi005.stdout:5055 writesame 0xe09eb45 thru 0xe0a4fb5 data_size 0x3(0x6471 bytes) 2024-05-11T23:03:06.782 INFO:teuthology.orchestra.run.smithi005.stdout:5058 write 0xa35d4ed thru 0xa3672f4 (0x9e08 bytes) 2024-05-11T23:03:06.788 INFO:teuthology.orchestra.run.smithi005.stdout:5059 write 0x1a81055 thru 0x1a8cd1b (0xbcc7 bytes) 2024-05-11T23:03:06.794 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:06.795 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:06.795 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:06.804 INFO:teuthology.orchestra.run.smithi005.stdout:5063 trunc from 0xe0a4fb6 to 0x6552de0 2024-05-11T23:03:06.826 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:06.826 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:06.826 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:06.825+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:06.829 INFO:teuthology.orchestra.run.smithi005.stdout:5064 write 0xdae8f0f thru 0xdaf2eb9 (0x9fab bytes) 2024-05-11T23:03:06.835 INFO:teuthology.orchestra.run.smithi005.stdout:5066 write 0x7790102 thru 0x779d331 (0xd230 bytes) 2024-05-11T23:03:06.840 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:06.840 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:06.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:06.840 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:06.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.840 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67817 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3526656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:06.841 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:03:06.841 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.219953461999999, 2024-05-11T23:03:06.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044148845999999999, 2024-05-11T23:03:06.841 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:06.841 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:06.841 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:06.841 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:06.841 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:06.841 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:06.841 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67817 2024-05-11T23:03:06.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.842 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:06.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:06.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:03:06.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:06.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.619760+0000", 2024-05-11T23:03:06.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:06.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620227+0000", 2024-05-11T23:03:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00046792099999999999 2024-05-11T23:03:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620232+0000", 2024-05-11T23:03:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3050000000000003e-06 2024-05-11T23:03:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620243+0000", 2024-05-11T23:03:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1289e-05 2024-05-11T23:03:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:06.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620250+0000", 2024-05-11T23:03:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2130000000000002e-06 2024-05-11T23:03:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663745+0000", 2024-05-11T23:03:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043494865000000001 2024-05-11T23:03:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663815+0000", 2024-05-11T23:03:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0284999999999995e-05 2024-05-11T23:03:06.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.663908+0000", 2024-05-11T23:03:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2967999999999995e-05 2024-05-11T23:03:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67820 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4902912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:03:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.219403954999997, 2024-05-11T23:03:06.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044096781000000002, 2024-05-11T23:03:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67820 2024-05-11T23:03:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:03:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620309+0000", 2024-05-11T23:03:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620310+0000", 2024-05-11T23:03:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9800000000000004e-07 2024-05-11T23:03:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620312+0000", 2024-05-11T23:03:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.272e-06 2024-05-11T23:03:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620319+0000", 2024-05-11T23:03:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9310000000000002e-06 2024-05-11T23:03:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:06.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620323+0000", 2024-05-11T23:03:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:03:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664284+0000", 2024-05-11T23:03:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043960908999999999 2024-05-11T23:03:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664327+0000", 2024-05-11T23:03:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2811000000000001e-05 2024-05-11T23:03:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664406+0000", 2024-05-11T23:03:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8588e-05 2024-05-11T23:03:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67821 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4927488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:03:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.219383544999999, 2024-05-11T23:03:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044203340000000001, 2024-05-11T23:03:06.850 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67821 2024-05-11T23:03:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:03:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620329+0000", 2024-05-11T23:03:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620330+0000", 2024-05-11T23:03:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2899999999999999e-07 2024-05-11T23:03:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620332+0000", 2024-05-11T23:03:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:03:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620338+0000", 2024-05-11T23:03:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6250000000000004e-06 2024-05-11T23:03:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620342+0000", 2024-05-11T23:03:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7609999999999998e-06 2024-05-11T23:03:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664433+0000", 2024-05-11T23:03:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044090455000000001 2024-05-11T23:03:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664476+0000", 2024-05-11T23:03:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2861999999999997e-05 2024-05-11T23:03:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664533+0000", 2024-05-11T23:03:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7156e-05 2024-05-11T23:03:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67822 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4976640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:06.855 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:03:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.219364507000002, 2024-05-11T23:03:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044409416, 2024-05-11T23:03:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67822 2024-05-11T23:03:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:06.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:03:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620348+0000", 2024-05-11T23:03:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620349+0000", 2024-05-11T23:03:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.51e-07 2024-05-11T23:03:06.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620351+0000", 2024-05-11T23:03:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.181e-06 2024-05-11T23:03:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620358+0000", 2024-05-11T23:03:06.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3060000000000003e-06 2024-05-11T23:03:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620362+0000", 2024-05-11T23:03:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7949999999999998e-06 2024-05-11T23:03:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664572+0000", 2024-05-11T23:03:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044209318999999997 2024-05-11T23:03:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664654+0000", 2024-05-11T23:03:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2367999999999995e-05 2024-05-11T23:03:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664758+0000", 2024-05-11T23:03:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000103896 2024-05-11T23:03:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67823 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4952064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:03:06.860 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.219344456999998, 2024-05-11T23:03:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044623293000000001, 2024-05-11T23:03:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67823 2024-05-11T23:03:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:03:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.620369+0000", 2024-05-11T23:03:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622228+0000", 2024-05-11T23:03:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0018592280000000001 2024-05-11T23:03:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:06.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622232+0000", 2024-05-11T23:03:06.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.403e-06 2024-05-11T23:03:06.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:06.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622248+0000", 2024-05-11T23:03:06.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5449000000000001e-05 2024-05-11T23:03:06.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:06.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622254+0000", 2024-05-11T23:03:06.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5819999999999998e-06 2024-05-11T23:03:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664800+0000", 2024-05-11T23:03:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042546048000000003 2024-05-11T23:03:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664886+0000", 2024-05-11T23:03:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5525000000000005e-05 2024-05-11T23:03:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.664992+0000", 2024-05-11T23:03:06.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010605800000000001 2024-05-11T23:03:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67833 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5099520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:03:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.217237984, 2024-05-11T23:03:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044076606999999997, 2024-05-11T23:03:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:06.865 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67833 2024-05-11T23:03:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:03:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:03:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622475+0000", 2024-05-11T23:03:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8099999999999998e-07 2024-05-11T23:03:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622477+0000", 2024-05-11T23:03:06.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0700000000000001e-06 2024-05-11T23:03:06.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:06.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622483+0000", 2024-05-11T23:03:06.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0780000000000003e-06 2024-05-11T23:03:06.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:06.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622487+0000", 2024-05-11T23:03:06.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.352e-06 2024-05-11T23:03:06.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:06.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666460+0000", 2024-05-11T23:03:06.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043973085000000002 2024-05-11T23:03:06.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:06.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666498+0000", 2024-05-11T23:03:06.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8441e-05 2024-05-11T23:03:06.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666552+0000", 2024-05-11T23:03:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3199999999999999e-05 2024-05-11T23:03:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67834 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3944448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:03:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.217199706000002, 2024-05-11T23:03:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044166551999999998, 2024-05-11T23:03:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:06.870 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67834 2024-05-11T23:03:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:03:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622513+0000", 2024-05-11T23:03:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622514+0000", 2024-05-11T23:03:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1100000000000002e-07 2024-05-11T23:03:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622515+0000", 2024-05-11T23:03:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:03:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622519+0000", 2024-05-11T23:03:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8260000000000003e-06 2024-05-11T23:03:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622522+0000", 2024-05-11T23:03:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8899999999999999e-06 2024-05-11T23:03:06.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666580+0000", 2024-05-11T23:03:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044058235000000001 2024-05-11T23:03:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666624+0000", 2024-05-11T23:03:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3501999999999998e-05 2024-05-11T23:03:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666680+0000", 2024-05-11T23:03:06.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6042999999999999e-05 2024-05-11T23:03:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67835 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3969024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:03:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.217187170000003, 2024-05-11T23:03:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044272019000000003, 2024-05-11T23:03:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:06.875 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67835 2024-05-11T23:03:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:03:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:06.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:03:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622526+0000", 2024-05-11T23:03:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1699999999999999e-07 2024-05-11T23:03:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622527+0000", 2024-05-11T23:03:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2109999999999999e-06 2024-05-11T23:03:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:06.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622531+0000", 2024-05-11T23:03:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4910000000000001e-06 2024-05-11T23:03:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622534+0000", 2024-05-11T23:03:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7080000000000002e-06 2024-05-11T23:03:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666703+0000", 2024-05-11T23:03:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044169377000000003 2024-05-11T23:03:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:06.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666745+0000", 2024-05-11T23:03:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1616000000000003e-05 2024-05-11T23:03:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666798+0000", 2024-05-11T23:03:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3298999999999999e-05 2024-05-11T23:03:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67836 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 3993600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:06.879 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:03:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.217176510000002, 2024-05-11T23:03:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044373692999999999, 2024-05-11T23:03:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67836 2024-05-11T23:03:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:03:06.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622536+0000", 2024-05-11T23:03:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622537+0000", 2024-05-11T23:03:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:03:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:06.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622538+0000", 2024-05-11T23:03:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1510000000000001e-06 2024-05-11T23:03:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622541+0000", 2024-05-11T23:03:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:03:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622543+0000", 2024-05-11T23:03:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4870000000000001e-06 2024-05-11T23:03:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666821+0000", 2024-05-11T23:03:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044278038999999998 2024-05-11T23:03:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666859+0000", 2024-05-11T23:03:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7224000000000001e-05 2024-05-11T23:03:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666910+0000", 2024-05-11T23:03:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1629999999999999e-05 2024-05-11T23:03:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.883 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67837 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4018176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:03:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.217166783000003, 2024-05-11T23:03:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044477883000000003, 2024-05-11T23:03:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:06.884 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67837 2024-05-11T23:03:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:03:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:03:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622546+0000", 2024-05-11T23:03:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622548+0000", 2024-05-11T23:03:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:03:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622550+0000", 2024-05-11T23:03:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8870000000000001e-06 2024-05-11T23:03:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622553+0000", 2024-05-11T23:03:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.43e-06 2024-05-11T23:03:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666932+0000", 2024-05-11T23:03:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044379584 2024-05-11T23:03:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.666971+0000", 2024-05-11T23:03:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8427000000000003e-05 2024-05-11T23:03:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:06.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667024+0000", 2024-05-11T23:03:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3226000000000002e-05 2024-05-11T23:03:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67838 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4042752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:03:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.217157266000001, 2024-05-11T23:03:06.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044580374999999998, 2024-05-11T23:03:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67838 2024-05-11T23:03:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:06.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:03:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:03:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:06.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622556+0000", 2024-05-11T23:03:06.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:03:06.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:06.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622557+0000", 2024-05-11T23:03:06.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.102e-06 2024-05-11T23:03:06.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:06.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622560+0000", 2024-05-11T23:03:06.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:03:06.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:06.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622562+0000", 2024-05-11T23:03:06.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:03:06.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:06.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667045+0000", 2024-05-11T23:03:06.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044482936000000001 2024-05-11T23:03:06.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:06.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667083+0000", 2024-05-11T23:03:06.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7443e-05 2024-05-11T23:03:06.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:06.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667136+0000", 2024-05-11T23:03:06.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3257000000000002e-05 2024-05-11T23:03:06.894 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.894 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.894 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.894 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67839 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5394432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:06.894 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:03:06.894 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.217147523999998, 2024-05-11T23:03:06.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04470263, 2024-05-11T23:03:06.894 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:06.894 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:06.894 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:06.895 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:06.895 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:06.895 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:06.895 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67839 2024-05-11T23:03:06.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.895 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:06.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:06.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:03:06.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:06.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622565+0000", 2024-05-11T23:03:06.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:06.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622566+0000", 2024-05-11T23:03:06.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1799999999999999e-07 2024-05-11T23:03:06.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:06.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622572+0000", 2024-05-11T23:03:06.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5610000000000004e-06 2024-05-11T23:03:06.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:06.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622576+0000", 2024-05-11T23:03:06.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3210000000000001e-06 2024-05-11T23:03:06.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:06.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622578+0000", 2024-05-11T23:03:06.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4710000000000001e-06 2024-05-11T23:03:06.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:06.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667160+0000", 2024-05-11T23:03:06.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044582220999999998 2024-05-11T23:03:06.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:06.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667205+0000", 2024-05-11T23:03:06.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5049000000000002e-05 2024-05-11T23:03:06.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:06.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667268+0000", 2024-05-11T23:03:06.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2788999999999998e-05 2024-05-11T23:03:06.898 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.898 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.898 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.899 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67840 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5419008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:06.899 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:03:06.899 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.217131893999998, 2024-05-11T23:03:06.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044796006999999999, 2024-05-11T23:03:06.899 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:06.899 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:06.899 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:06.899 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:06.899 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:06.899 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:06.899 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67840 2024-05-11T23:03:06.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.899 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:06.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:06.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:03:06.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:06.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:03:06.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:06.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622581+0000", 2024-05-11T23:03:06.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:03:06.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:06.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622583+0000", 2024-05-11T23:03:06.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4389999999999999e-06 2024-05-11T23:03:06.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:06.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622586+0000", 2024-05-11T23:03:06.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8949999999999998e-06 2024-05-11T23:03:06.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:06.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622588+0000", 2024-05-11T23:03:06.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4540000000000001e-06 2024-05-11T23:03:06.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:06.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667287+0000", 2024-05-11T23:03:06.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044698882000000002 2024-05-11T23:03:06.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:06.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667324+0000", 2024-05-11T23:03:06.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7196000000000001e-05 2024-05-11T23:03:06.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:06.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667377+0000", 2024-05-11T23:03:06.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2927999999999999e-05 2024-05-11T23:03:06.904 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.904 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.904 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.904 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67841 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5443584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:06.904 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:03:06.904 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.217122121000003, 2024-05-11T23:03:06.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044907337999999998, 2024-05-11T23:03:06.904 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:06.904 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:06.904 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:06.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:06.905 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:06.905 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:06.905 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67841 2024-05-11T23:03:06.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.905 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:06.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:06.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:03:06.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:06.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:03:06.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:06.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622591+0000", 2024-05-11T23:03:06.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:06.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:06.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622592+0000", 2024-05-11T23:03:06.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1060000000000001e-06 2024-05-11T23:03:06.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:06.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622595+0000", 2024-05-11T23:03:06.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:03:06.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:06.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622598+0000", 2024-05-11T23:03:06.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5909999999999998e-06 2024-05-11T23:03:06.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:06.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667401+0000", 2024-05-11T23:03:06.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044802980999999999 2024-05-11T23:03:06.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:06.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667439+0000", 2024-05-11T23:03:06.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8840000000000001e-05 2024-05-11T23:03:06.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:06.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667498+0000", 2024-05-11T23:03:06.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8709000000000003e-05 2024-05-11T23:03:06.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.908 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.909 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.909 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67842 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5468160~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:06.909 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:03:06.909 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.217112514, 2024-05-11T23:03:06.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045005876, 2024-05-11T23:03:06.909 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:06.909 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:06.909 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:06.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:06.910 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:06.910 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:06.910 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67842 2024-05-11T23:03:06.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.910 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:06.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:06.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:03:06.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:06.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622600+0000", 2024-05-11T23:03:06.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:06.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622601+0000", 2024-05-11T23:03:06.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1899999999999999e-07 2024-05-11T23:03:06.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:06.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622602+0000", 2024-05-11T23:03:06.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0780000000000001e-06 2024-05-11T23:03:06.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:06.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622605+0000", 2024-05-11T23:03:06.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.914e-06 2024-05-11T23:03:06.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:06.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622607+0000", 2024-05-11T23:03:06.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:03:06.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:06.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667516+0000", 2024-05-11T23:03:06.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044909156999999998 2024-05-11T23:03:06.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:06.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667554+0000", 2024-05-11T23:03:06.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7920000000000003e-05 2024-05-11T23:03:06.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:06.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667606+0000", 2024-05-11T23:03:06.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2111000000000003e-05 2024-05-11T23:03:06.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.913 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.914 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.914 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67843 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4067328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:06.914 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:03:06.914 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.217102988000001, 2024-05-11T23:03:06.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045119014999999998, 2024-05-11T23:03:06.914 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:06.914 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:06.914 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:06.914 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:06.915 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:06.915 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:06.915 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67843 2024-05-11T23:03:06.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.915 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:06.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:06.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:03:06.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:06.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.622610+0000", 2024-05-11T23:03:06.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:06.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624228+0000", 2024-05-11T23:03:06.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001617867 2024-05-11T23:03:06.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:06.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624232+0000", 2024-05-11T23:03:06.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9779999999999999e-06 2024-05-11T23:03:06.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:06.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624240+0000", 2024-05-11T23:03:06.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4589999999999998e-06 2024-05-11T23:03:06.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:06.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624247+0000", 2024-05-11T23:03:06.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4400000000000002e-06 2024-05-11T23:03:06.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:06.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667629+0000", 2024-05-11T23:03:06.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043382012999999997 2024-05-11T23:03:06.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:06.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667674+0000", 2024-05-11T23:03:06.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5172000000000001e-05 2024-05-11T23:03:06.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:06.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.667729+0000", 2024-05-11T23:03:06.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5086000000000001e-05 2024-05-11T23:03:06.918 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.918 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.918 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.919 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67849 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 4190208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:06.919 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:03:06.919 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.215344866999999, 2024-05-11T23:03:06.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044061433999999997, 2024-05-11T23:03:06.919 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:06.919 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:06.919 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:06.919 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:06.919 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:06.919 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:06.919 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67849 2024-05-11T23:03:06.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.920 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:06.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:06.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:03:06.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:06.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624368+0000", 2024-05-11T23:03:06.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:06.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624369+0000", 2024-05-11T23:03:06.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5900000000000002e-07 2024-05-11T23:03:06.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:06.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624371+0000", 2024-05-11T23:03:06.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0310000000000001e-06 2024-05-11T23:03:06.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:06.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624376+0000", 2024-05-11T23:03:06.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4029999999999998e-06 2024-05-11T23:03:06.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:06.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624381+0000", 2024-05-11T23:03:06.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4349999999999996e-06 2024-05-11T23:03:06.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:06.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668331+0000", 2024-05-11T23:03:06.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043949191999999998 2024-05-11T23:03:06.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:06.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668370+0000", 2024-05-11T23:03:06.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9227999999999998e-05 2024-05-11T23:03:06.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:06.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668430+0000", 2024-05-11T23:03:06.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9685999999999997e-05 2024-05-11T23:03:06.923 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.923 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.923 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.923 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67850 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5517312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:06.923 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:03:06.923 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.215323634999997, 2024-05-11T23:03:06.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044151953000000001, 2024-05-11T23:03:06.923 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:06.923 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:06.923 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:06.923 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:06.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:06.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:06.924 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67850 2024-05-11T23:03:06.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.924 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:06.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:06.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:03:06.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:06.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624389+0000", 2024-05-11T23:03:06.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:06.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624390+0000", 2024-05-11T23:03:06.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:03:06.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:06.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624392+0000", 2024-05-11T23:03:06.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1880000000000001e-06 2024-05-11T23:03:06.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:06.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624399+0000", 2024-05-11T23:03:06.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7390000000000002e-06 2024-05-11T23:03:06.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:06.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624405+0000", 2024-05-11T23:03:06.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1059999999999996e-06 2024-05-11T23:03:06.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:06.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668450+0000", 2024-05-11T23:03:06.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044045147999999999 2024-05-11T23:03:06.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:06.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668487+0000", 2024-05-11T23:03:06.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7450000000000002e-05 2024-05-11T23:03:06.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:06.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668541+0000", 2024-05-11T23:03:06.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3952000000000001e-05 2024-05-11T23:03:06.927 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.927 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.927 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.927 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67851 3.16 3:6f525258:::rbd_data.111ecf60147b.0000000000000000:17 [read 5541888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:06.927 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:03:06.927 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.215297053, 2024-05-11T23:03:06.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044264413000000002, 2024-05-11T23:03:06.927 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:06.927 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:06.927 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:06.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:06.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:06.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:06.928 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67851 2024-05-11T23:03:06.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.928 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:06.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:06.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:03:06.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:06.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:03:06.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:06.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624416+0000", 2024-05-11T23:03:06.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9099999999999999e-07 2024-05-11T23:03:06.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:06.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624419+0000", 2024-05-11T23:03:06.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.21e-06 2024-05-11T23:03:06.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:06.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624424+0000", 2024-05-11T23:03:06.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9050000000000002e-06 2024-05-11T23:03:06.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:06.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624430+0000", 2024-05-11T23:03:06.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4949999999999999e-06 2024-05-11T23:03:06.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:06.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668566+0000", 2024-05-11T23:03:06.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044136208000000003 2024-05-11T23:03:06.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:06.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668614+0000", 2024-05-11T23:03:06.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8022000000000002e-05 2024-05-11T23:03:06.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:06.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668680+0000", 2024-05-11T23:03:06.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6181999999999998e-05 2024-05-11T23:03:06.931 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.931 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.931 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.931 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:67852 3.16 3.1a4a4af6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-05-11T23:03:06.931 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:03:06.931 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.215276586000002, 2024-05-11T23:03:06.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044278287999999999, 2024-05-11T23:03:06.932 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:06.932 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:06.932 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "reached pg", 2024-05-11T23:03:06.932 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:06.932 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:06.932 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:06.932 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 67852 2024-05-11T23:03:06.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.932 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:06.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:06.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:03:06.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:06.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.624436+0000", 2024-05-11T23:03:06.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:06.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:06.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626226+0000", 2024-05-11T23:03:06.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017893 2024-05-11T23:03:06.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:06.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626229+0000", 2024-05-11T23:03:06.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0709999999999999e-06 2024-05-11T23:03:06.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:06.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626239+0000", 2024-05-11T23:03:06.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0507e-05 2024-05-11T23:03:06.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:06.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.626244+0000", 2024-05-11T23:03:06.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4880000000000001e-06 2024-05-11T23:03:06.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:06.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668706+0000", 2024-05-11T23:03:06.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042462054999999999 2024-05-11T23:03:06.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:06.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:06.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:06.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:06.668715+0000", 2024-05-11T23:03:06.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.867e-06 2024-05-11T23:03:06.935 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.935 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.935 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.935 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:06.935 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:06.935 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:06.936 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:06.936 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:06.936 INFO:teuthology.orchestra.run.smithi005.stdout:5069 read 0xccfb66e thru 0xccfc8cb (0x125e bytes) 2024-05-11T23:03:06.937 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:06.937 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:06.937 INFO:teuthology.orchestra.run.smithi005.stdout:5070 trunc from 0xdaf2eba to 0x3350124 2024-05-11T23:03:06.938 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:03:06.939 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:06.936+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:06.940 INFO:teuthology.orchestra.run.smithi005.stdout:5071 read 0x1087a8f thru 0x1096239 (0xe7ab bytes) 2024-05-11T23:03:06.940 INFO:teuthology.orchestra.run.smithi005.stdout:5072 punch from 0x14d9a1f to 0x14dda83, (0x4064 bytes) 2024-05-11T23:03:06.940 INFO:teuthology.orchestra.run.smithi005.stdout:5074 write 0x2f9b09 thru 0x308042 (0xe53a bytes) 2024-05-11T23:03:06.940 INFO:teuthology.orchestra.run.smithi005.stdout:5075 write 0xcebc602 thru 0xcec126d (0x4c6c bytes) 2024-05-11T23:03:06.940 INFO:teuthology.orchestra.run.smithi005.stdout:5076 write 0x77cb11b thru 0x77d3c93 (0x8b79 bytes) 2024-05-11T23:03:06.940 INFO:teuthology.orchestra.run.smithi005.stdout:5077 trunc from 0xcec126e to 0x1d70907 2024-05-11T23:03:06.940 INFO:teuthology.orchestra.run.smithi005.stdout:5078 write 0xbe1e5fa thru 0xbe29e9e (0xb8a5 bytes) 2024-05-11T23:03:06.945 INFO:teuthology.orchestra.run.smithi005.stdout:5079 write 0x3e3c189 thru 0x3e466a4 (0xa51c bytes) 2024-05-11T23:03:06.958 INFO:teuthology.orchestra.run.smithi005.stdout:5080 write 0xe25375c thru 0xe253e60 (0x705 bytes) 2024-05-11T23:03:06.961 INFO:teuthology.orchestra.run.smithi005.stdout:5081 write 0x37b296a thru 0x37b9804 (0x6e9b bytes) 2024-05-11T23:03:06.965 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:06.966 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:06.967 INFO:teuthology.orchestra.run.smithi005.stdout:5087 trunc from 0xe253e61 to 0xcd6b7ca 2024-05-11T23:03:06.976 INFO:teuthology.orchestra.run.smithi005.stdout:5088 clone 46 order 18 su 65536 sc 8 2024-05-11T23:03:07.037 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:07.037+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:07.084 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:07.084 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:07.084 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:07.084 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:07.092 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:03:07.138 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:07.138+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:07.200 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:07.200 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:07.201 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:07.201 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:07.210 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:03:07.253 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:07.253+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:07.316 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:07.316 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:07.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:07.316 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:07.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.316 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:07.317 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:07.317 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.661501959000006, 2024-05-11T23:03:07.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:03:07.317 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.317 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.317 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.317 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.317 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.317 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.317 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:03:07.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.317 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:07.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:07.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:07.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:03:07.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:03:07.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:03:07.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:03:07.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:03:07.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:03:07.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:03:07.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:03:07.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:03:07.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:03:07.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:03:07.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:03:07.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:03:07.320 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.321 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.321 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.321 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:07.321 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:07.321 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.658120769999996, 2024-05-11T23:03:07.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:03:07.321 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.321 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.321 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.321 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:03:07.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.322 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:07.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:07.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:07.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:07.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:03:07.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:03:07.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:03:07.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:03:07.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:03:07.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:03:07.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:03:07.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:03:07.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:03:07.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:03:07.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:03:07.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:03:07.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.324 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.325 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.325 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:07.325 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:07.325 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.658102063000001, 2024-05-11T23:03:07.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:03:07.325 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.325 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.325 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.325 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:03:07.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.326 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:07.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:07.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:03:07.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:03:07.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:03:07.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:03:07.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:03:07.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:03:07.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:03:07.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:03:07.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:03:07.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:03:07.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:03:07.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:03:07.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:03:07.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:03:07.329 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.329 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.329 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.329 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:07.329 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:07.329 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.658083919999996, 2024-05-11T23:03:07.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:03:07.329 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.329 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.329 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.329 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.330 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:03:07.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.330 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:07.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:07.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:07.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:03:07.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:03:07.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:03:07.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:03:07.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:03:07.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:03:07.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:03:07.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:03:07.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:03:07.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:03:07.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:03:07.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:03:07.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:03:07.333 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.333 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.333 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.333 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:07.333 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:07.333 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.658056432999999, 2024-05-11T23:03:07.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:03:07.333 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.333 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.334 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.334 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:03:07.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.334 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:07.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:07.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:03:07.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:03:07.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:03:07.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:03:07.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:03:07.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:03:07.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:03:07.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:03:07.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:03:07.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:03:07.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:03:07.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:03:07.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:03:07.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:03:07.337 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.337 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.337 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.337 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:07.337 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:07.337 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.658035905000006, 2024-05-11T23:03:07.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:03:07.338 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.338 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.338 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.338 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:03:07.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.338 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:07.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:07.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:07.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:03:07.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:03:07.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:03:07.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:03:07.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:03:07.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:03:07.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:03:07.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:03:07.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:03:07.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:03:07.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:03:07.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:03:07.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:03:07.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.341 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.341 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:07.341 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:07.342 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.658016129999993, 2024-05-11T23:03:07.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:03:07.342 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.342 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.342 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.342 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:03:07.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.342 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:07.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:07.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:03:07.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:03:07.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:03:07.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:03:07.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:03:07.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:03:07.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:03:07.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:03:07.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:03:07.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:03:07.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:03:07.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:03:07.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:03:07.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:03:07.347 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.347 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.347 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.347 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:07.347 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:07.347 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.657997323999993, 2024-05-11T23:03:07.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:03:07.347 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.347 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.347 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.348 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.348 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:03:07.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.348 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:07.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:07.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:03:07.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:03:07.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:03:07.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:03:07.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:03:07.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:03:07.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:03:07.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:03:07.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:03:07.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:03:07.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:03:07.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:03:07.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:03:07.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:03:07.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.351 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.351 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:07.351 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:07.351 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.657978400999994, 2024-05-11T23:03:07.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:03:07.351 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.351 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.351 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.352 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:03:07.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.352 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:07.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:07.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:07.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:03:07.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:03:07.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:03:07.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:03:07.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:03:07.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:03:07.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:03:07.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:03:07.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:03:07.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:03:07.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:03:07.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:03:07.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:03:07.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.355 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.355 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:07.355 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:07.355 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.657959739000006, 2024-05-11T23:03:07.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:03:07.355 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.355 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.355 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.355 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.355 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.356 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:03:07.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.356 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:07.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:07.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:07.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:03:07.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:03:07.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:03:07.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:03:07.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:03:07.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:03:07.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:03:07.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:03:07.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:03:07.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:03:07.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:03:07.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:03:07.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:03:07.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.359 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.359 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:07.359 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:07.359 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.657614577000004, 2024-05-11T23:03:07.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:03:07.360 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.360 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.360 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.360 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:03:07.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.360 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:07.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:07.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:03:07.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:03:07.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:03:07.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:03:07.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:03:07.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:03:07.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:03:07.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:03:07.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:03:07.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:03:07.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:03:07.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:03:07.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:03:07.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:03:07.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.363 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.363 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:07.363 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:07.364 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.657589027, 2024-05-11T23:03:07.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:03:07.364 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.364 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.364 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.364 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:03:07.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.364 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:07.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:07.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:07.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:03:07.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:03:07.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:03:07.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:03:07.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:03:07.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:03:07.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:03:07.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:03:07.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:03:07.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:03:07.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:03:07.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:03:07.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:03:07.367 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.367 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.367 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.367 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:07.368 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:07.368 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.657506768999994, 2024-05-11T23:03:07.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:03:07.368 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.368 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.368 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.368 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:03:07.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.368 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:07.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:07.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:07.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:03:07.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:03:07.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:03:07.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:03:07.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:03:07.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:03:07.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:03:07.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:03:07.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:03:07.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:03:07.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:03:07.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:03:07.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:03:07.371 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.371 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.371 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.372 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:07.372 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:07.372 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.657484322000002, 2024-05-11T23:03:07.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:03:07.372 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.372 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.372 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.372 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:03:07.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.372 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:07.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:07.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:03:07.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:03:07.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:03:07.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:03:07.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:03:07.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:03:07.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:03:07.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:03:07.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:03:07.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:03:07.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:03:07.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:03:07.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:03:07.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:03:07.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.375 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.376 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:07.376 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:07.376 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.657464864000005, 2024-05-11T23:03:07.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:03:07.376 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.376 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.376 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.376 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:03:07.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.376 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:07.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:07.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:07.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:03:07.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:03:07.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:03:07.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:03:07.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:03:07.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:03:07.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:03:07.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:03:07.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:03:07.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:03:07.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:03:07.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:03:07.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:03:07.379 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.379 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.379 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.380 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:07.380 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:07.380 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.657444803999994, 2024-05-11T23:03:07.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:03:07.380 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.380 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.380 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.380 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:03:07.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.380 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:07.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:07.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:07.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:07.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:03:07.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:03:07.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:03:07.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:03:07.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:03:07.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:03:07.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:03:07.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:03:07.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:03:07.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:03:07.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:03:07.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:03:07.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.384 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.384 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:07.384 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:07.384 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.657425533999998, 2024-05-11T23:03:07.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:03:07.384 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.384 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.384 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.385 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:03:07.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.385 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:07.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:07.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:07.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:03:07.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:03:07.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:03:07.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:03:07.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:03:07.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:03:07.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:03:07.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:03:07.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:03:07.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:03:07.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:03:07.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:03:07.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:03:07.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.388 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.388 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:07.388 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:07.388 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.656879468, 2024-05-11T23:03:07.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:03:07.388 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.388 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.388 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.388 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.389 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:03:07.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.389 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:07.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:07.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:03:07.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:03:07.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:03:07.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:03:07.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:03:07.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:03:07.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:03:07.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:03:07.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:03:07.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:03:07.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:03:07.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:03:07.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:03:07.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:03:07.392 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.392 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.392 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.392 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:07.392 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:07.392 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.656847683999999, 2024-05-11T23:03:07.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:03:07.392 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.392 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.392 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.392 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.393 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:03:07.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.393 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:07.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:07.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:07.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:03:07.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:03:07.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:03:07.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:03:07.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:03:07.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:03:07.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:03:07.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:03:07.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:03:07.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:03:07.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:03:07.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:03:07.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:03:07.396 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.396 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.396 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.396 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:03:07.396 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:07.396 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.219602219999999, 2024-05-11T23:03:07.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:03:07.396 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.397 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.397 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:07.397 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.397 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.397 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.397 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:03:07.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.397 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:07.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:07.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:03:07.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:03:07.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:03:07.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:03:07.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:03:07.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:03:07.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:03:07.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:03:07.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:03:07.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:03:07.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:03:07.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:03:07.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:03:07.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:03:07.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:03:07.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:07.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:03:07.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:03:07.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:07.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:03:07.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:03:07.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:07.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:03:07.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:03:07.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:03:07.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:03:07.401 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.401 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.401 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.401 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.402 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.402 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:07.404 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:03:07.405 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:07.402+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-05-11T23:03:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:5089 punch from 0x2dd632 to 0x2e445a, (0x6e28 bytes) 2024-05-11T23:03:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:5090 punch from 0x658dcd1 to 0x6592002, (0x4331 bytes) 2024-05-11T23:03:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:5092 read 0x845b6fd thru 0x84622c6 (0x6bca bytes) 2024-05-11T23:03:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:5093 read 0x24892dc thru 0x2490626 (0x734b bytes) 2024-05-11T23:03:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:5094 write 0xcff221e thru 0xcff3beb (0x19ce bytes) 2024-05-11T23:03:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:5095 read 0xc3cd0ee thru 0xc3d540f (0x8322 bytes) 2024-05-11T23:03:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:5096 punch from 0x8e284c7 to 0x8e318b9, (0x93f2 bytes) 2024-05-11T23:03:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:5097 read 0x92ad299 thru 0x92ba6cd (0xd435 bytes) 2024-05-11T23:03:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:5098 read 0x7f1b91f thru 0x7f24017 (0x86f9 bytes) 2024-05-11T23:03:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:5099 read 0x42e4fc2 thru 0x42eed2a (0x9d69 bytes) 2024-05-11T23:03:07.407 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:07.407 INFO:teuthology.orchestra.run.smithi005.stdout:5101 write 0xa79fb3f thru 0xa7a8966 (0x8e28 bytes) 2024-05-11T23:03:07.407 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:07.407 INFO:teuthology.orchestra.run.smithi005.stdout:5104 write 0xb609c56 thru 0xb617829 (0xdbd4 bytes) 2024-05-11T23:03:07.407 INFO:teuthology.orchestra.run.smithi005.stdout:5105 trunc from 0xcff3bec to 0x241d57a 2024-05-11T23:03:07.407 INFO:teuthology.orchestra.run.smithi005.stdout:5106 writesame 0xa71ecec thru 0xa71f3ef data_size 0x4(0x704 bytes) 2024-05-11T23:03:07.414 INFO:teuthology.orchestra.run.smithi005.stdout:5108 writesame 0xe609152 thru 0xe61830f data_size 0x61(0xf1be bytes) 2024-05-11T23:03:07.414 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:07.414 INFO:teuthology.orchestra.run.smithi005.stdout:5110 read 0x106b463 thru 0x10780b1 (0xcc4f bytes) 2024-05-11T23:03:07.414 INFO:teuthology.orchestra.run.smithi005.stdout:5112 write 0xde8f5cd thru 0xde95d30 (0x6764 bytes) 2024-05-11T23:03:07.418 INFO:teuthology.orchestra.run.smithi005.stdout:5113 read 0xdd99df9 thru 0xdda22d5 (0x84dd bytes) 2024-05-11T23:03:07.418 INFO:teuthology.orchestra.run.smithi005.stdout:5114 punch from 0x8ad9a85 to 0x8ae53ee, (0xb969 bytes) 2024-05-11T23:03:07.418 INFO:teuthology.orchestra.run.smithi005.stdout:5115 read 0xc2a0785 thru 0xc2a0ce9 (0x565 bytes) 2024-05-11T23:03:07.419 INFO:teuthology.orchestra.run.smithi005.stdout:5118 writesame 0x3e970a6 thru 0x3e9aade data_size 0xb(0x3a39 bytes) 2024-05-11T23:03:07.422 INFO:teuthology.orchestra.run.smithi005.stdout:5120 read 0x7bcf6a6 thru 0x7bd9016 (0x9971 bytes) 2024-05-11T23:03:07.422 INFO:teuthology.orchestra.run.smithi005.stdout:5121 read 0xb75fb4 thru 0xb8561e (0xf66b bytes) 2024-05-11T23:03:07.422 INFO:teuthology.orchestra.run.smithi005.stdout:5122 write 0xe2a088d thru 0xe2a1434 (0xba8 bytes) 2024-05-11T23:03:07.424 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:07.424 INFO:teuthology.orchestra.run.smithi005.stdout:5124 clone 47 order 20 su 16384 sc 10 2024-05-11T23:03:07.504 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:07.504+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:07.547 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:07.547 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:07.548 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:07.548 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:07.556 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:03:07.605 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:07.605+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:07.660 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:07.660 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:07.661 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:07.661 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:07.661 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:07.669 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:03:07.707 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:07.706+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:07.775 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:07.775 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:07.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:07.775 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:07.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.776 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:03:07.776 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:07.776 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 110.61137076999999, 2024-05-11T23:03:07.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:03:07.776 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.776 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.776 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:07.776 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.776 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:03:07.776 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:03:07.776 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:03:07.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.776 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:07.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:07.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:03:07.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:03:07.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:03:07.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:03:07.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:03:07.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:03:07.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:03:07.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:03:07.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:03:07.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:03:07.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:03:07.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:03:07.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:03:07.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:03:07.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:03:07.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:03:07.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:03:07.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:03:07.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:03:07.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:03:07.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:07.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:03:07.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:03:07.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:07.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:03:07.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:03:07.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:07.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:03:07.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:03:07.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:03:07.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:03:07.781 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.782 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.782 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78429 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:07.782 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:07.782 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.810101428000003, 2024-05-11T23:03:07.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044563497000000001, 2024-05-11T23:03:07.782 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.782 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.782 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.782 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.782 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.782 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.783 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78429 2024-05-11T23:03:07.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.783 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:07.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:07.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964872+0000", 2024-05-11T23:03:07.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035070300000000001 2024-05-11T23:03:07.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964876+0000", 2024-05-11T23:03:07.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5009999999999998e-06 2024-05-11T23:03:07.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964902+0000", 2024-05-11T23:03:07.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5715999999999999e-05 2024-05-11T23:03:07.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964910+0000", 2024-05-11T23:03:07.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3930000000000002e-06 2024-05-11T23:03:07.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008898+0000", 2024-05-11T23:03:07.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043988772000000002 2024-05-11T23:03:07.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008981+0000", 2024-05-11T23:03:07.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3140999999999996e-05 2024-05-11T23:03:07.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009085+0000", 2024-05-11T23:03:07.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010327099999999999 2024-05-11T23:03:07.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.786 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.786 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.786 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78431 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:07.786 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:07.786 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.809641120000002, 2024-05-11T23:03:07.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044542033000000002, 2024-05-11T23:03:07.786 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.786 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.786 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.787 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.787 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78431 2024-05-11T23:03:07.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.787 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:07.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:07.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965038+0000", 2024-05-11T23:03:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6993000000000001e-05 2024-05-11T23:03:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965043+0000", 2024-05-11T23:03:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7040000000000002e-06 2024-05-11T23:03:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965053+0000", 2024-05-11T23:03:07.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8539999999999992e-06 2024-05-11T23:03:07.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965061+0000", 2024-05-11T23:03:07.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9810000000000003e-06 2024-05-11T23:03:07.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009350+0000", 2024-05-11T23:03:07.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044288739000000001 2024-05-11T23:03:07.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009429+0000", 2024-05-11T23:03:07.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9354e-05 2024-05-11T23:03:07.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009524+0000", 2024-05-11T23:03:07.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4407999999999995e-05 2024-05-11T23:03:07.790 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.790 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.790 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.790 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78432 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:07.790 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:07.790 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.809550272000003, 2024-05-11T23:03:07.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044635421000000002, 2024-05-11T23:03:07.791 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.791 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.791 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.791 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.791 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.791 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.791 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78432 2024-05-11T23:03:07.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.791 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:07.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:07.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965343+0000", 2024-05-11T23:03:07.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027056500000000001 2024-05-11T23:03:07.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965347+0000", 2024-05-11T23:03:07.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3540000000000002e-06 2024-05-11T23:03:07.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965356+0000", 2024-05-11T23:03:07.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2140000000000002e-06 2024-05-11T23:03:07.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965363+0000", 2024-05-11T23:03:07.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4150000000000004e-06 2024-05-11T23:03:07.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009551+0000", 2024-05-11T23:03:07.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044187917 2024-05-11T23:03:07.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009626+0000", 2024-05-11T23:03:07.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5024000000000005e-05 2024-05-11T23:03:07.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009708+0000", 2024-05-11T23:03:07.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1932000000000001e-05 2024-05-11T23:03:07.794 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.794 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.794 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.794 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78433 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:07.794 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:07.795 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.809249385000001, 2024-05-11T23:03:07.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044564864000000003, 2024-05-11T23:03:07.795 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.795 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.795 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.795 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.795 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.795 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.795 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78433 2024-05-11T23:03:07.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.795 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:07.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:07.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966225+0000", 2024-05-11T23:03:07.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085220700000000001 2024-05-11T23:03:07.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966230+0000", 2024-05-11T23:03:07.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0690000000000003e-06 2024-05-11T23:03:07.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966239+0000", 2024-05-11T23:03:07.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-05-11T23:03:07.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966245+0000", 2024-05-11T23:03:07.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5520000000000001e-06 2024-05-11T23:03:07.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009743+0000", 2024-05-11T23:03:07.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043498311999999997 2024-05-11T23:03:07.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009848+0000", 2024-05-11T23:03:07.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104885 2024-05-11T23:03:07.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009938+0000", 2024-05-11T23:03:07.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9812e-05 2024-05-11T23:03:07.798 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.798 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.799 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.799 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78580 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 999424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:07.799 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:07.799 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.806136702000003, 2024-05-11T23:03:07.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044493076999999999, 2024-05-11T23:03:07.799 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.799 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.799 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.799 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.799 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.799 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.799 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78580 2024-05-11T23:03:07.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.800 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:07.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:07.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:07.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:03:07.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968489+0000", 2024-05-11T23:03:07.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.52e-06 2024-05-11T23:03:07.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968496+0000", 2024-05-11T23:03:07.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9419999999999998e-06 2024-05-11T23:03:07.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968501+0000", 2024-05-11T23:03:07.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8720000000000001e-06 2024-05-11T23:03:07.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012690+0000", 2024-05-11T23:03:07.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044189655000000001 2024-05-11T23:03:07.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.802 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.802 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012848+0000", 2024-05-11T23:03:07.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015758100000000001 2024-05-11T23:03:07.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012979+0000", 2024-05-11T23:03:07.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000131066 2024-05-11T23:03:07.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.803 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.803 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.803 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78581 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:07.803 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:07.803 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.806115697999999, 2024-05-11T23:03:07.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044616243, 2024-05-11T23:03:07.803 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.803 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.803 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.804 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.804 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78581 2024-05-11T23:03:07.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.804 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:07.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:07.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970227+0000", 2024-05-11T23:03:07.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017200049999999999 2024-05-11T23:03:07.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970232+0000", 2024-05-11T23:03:07.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000003e-06 2024-05-11T23:03:07.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970242+0000", 2024-05-11T23:03:07.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0192e-05 2024-05-11T23:03:07.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970249+0000", 2024-05-11T23:03:07.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.888e-06 2024-05-11T23:03:07.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.806 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013008+0000", 2024-05-11T23:03:07.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042759495000000002 2024-05-11T23:03:07.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.806 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013063+0000", 2024-05-11T23:03:07.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5071999999999997e-05 2024-05-11T23:03:07.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013123+0000", 2024-05-11T23:03:07.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9930999999999999e-05 2024-05-11T23:03:07.807 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.807 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.807 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.807 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78589 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:07.807 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:07.807 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.804196454, 2024-05-11T23:03:07.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044587697000000003, 2024-05-11T23:03:07.808 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.808 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.808 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.808 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.808 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.808 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.808 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78589 2024-05-11T23:03:07.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.808 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:07.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:07.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970427+0000", 2024-05-11T23:03:07.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:03:07.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970429+0000", 2024-05-11T23:03:07.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3800000000000001e-06 2024-05-11T23:03:07.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970435+0000", 2024-05-11T23:03:07.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2530000000000001e-06 2024-05-11T23:03:07.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970441+0000", 2024-05-11T23:03:07.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-05-11T23:03:07.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014822+0000", 2024-05-11T23:03:07.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04438168 2024-05-11T23:03:07.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014911+0000", 2024-05-11T23:03:07.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8882999999999996e-05 2024-05-11T23:03:07.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015014+0000", 2024-05-11T23:03:07.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010263399999999999 2024-05-11T23:03:07.811 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.811 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.811 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.811 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78590 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:07.812 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:07.812 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.804175655000002, 2024-05-11T23:03:07.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044880120000000003, 2024-05-11T23:03:07.812 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.812 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.812 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.812 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.812 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.812 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.812 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78590 2024-05-11T23:03:07.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.812 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:07.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:07.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970918+0000", 2024-05-11T23:03:07.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000470834 2024-05-11T23:03:07.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970925+0000", 2024-05-11T23:03:07.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1690000000000003e-06 2024-05-11T23:03:07.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970934+0000", 2024-05-11T23:03:07.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9099999999999994e-06 2024-05-11T23:03:07.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970941+0000", 2024-05-11T23:03:07.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7410000000000003e-06 2024-05-11T23:03:07.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015049+0000", 2024-05-11T23:03:07.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044108392000000003 2024-05-11T23:03:07.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015133+0000", 2024-05-11T23:03:07.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3568999999999999e-05 2024-05-11T23:03:07.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015327+0000", 2024-05-11T23:03:07.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019450500000000001 2024-05-11T23:03:07.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.815 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.815 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.815 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78591 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:07.816 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:07.816 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.803672239000001, 2024-05-11T23:03:07.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044618497, 2024-05-11T23:03:07.816 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.816 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.816 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.816 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.816 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78591 2024-05-11T23:03:07.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.816 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:07.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:07.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970951+0000", 2024-05-11T23:03:07.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:03:07.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970953+0000", 2024-05-11T23:03:07.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:03:07.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970956+0000", 2024-05-11T23:03:07.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-05-11T23:03:07.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970959+0000", 2024-05-11T23:03:07.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9409999999999999e-06 2024-05-11T23:03:07.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015367+0000", 2024-05-11T23:03:07.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044407621000000001 2024-05-11T23:03:07.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015453+0000", 2024-05-11T23:03:07.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6685000000000001e-05 2024-05-11T23:03:07.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015569+0000", 2024-05-11T23:03:07.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115396 2024-05-11T23:03:07.819 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.820 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.820 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.820 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78592 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:07.820 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:07.820 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.803659431, 2024-05-11T23:03:07.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044831546999999999, 2024-05-11T23:03:07.820 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.820 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.820 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.821 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.821 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.821 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.821 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78592 2024-05-11T23:03:07.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.821 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:07.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:07.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972226+0000", 2024-05-11T23:03:07.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00126246 2024-05-11T23:03:07.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972230+0000", 2024-05-11T23:03:07.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1860000000000002e-06 2024-05-11T23:03:07.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972237+0000", 2024-05-11T23:03:07.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:03:07.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972241+0000", 2024-05-11T23:03:07.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.284e-06 2024-05-11T23:03:07.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015611+0000", 2024-05-11T23:03:07.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043369461999999998 2024-05-11T23:03:07.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015695+0000", 2024-05-11T23:03:07.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4458999999999999e-05 2024-05-11T23:03:07.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015795+0000", 2024-05-11T23:03:07.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9475999999999994e-05 2024-05-11T23:03:07.824 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.824 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.824 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.824 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78593 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:07.824 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:07.824 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.802372511999998, 2024-05-11T23:03:07.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046045214000000001, 2024-05-11T23:03:07.824 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.824 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.824 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.825 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.825 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.825 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.825 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78593 2024-05-11T23:03:07.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.825 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:07.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:07.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:07.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7599999999999998e-07 2024-05-11T23:03:07.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972252+0000", 2024-05-11T23:03:07.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7349999999999999e-06 2024-05-11T23:03:07.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972256+0000", 2024-05-11T23:03:07.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5659999999999998e-06 2024-05-11T23:03:07.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972259+0000", 2024-05-11T23:03:07.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8040000000000002e-06 2024-05-11T23:03:07.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018088+0000", 2024-05-11T23:03:07.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045829316000000002 2024-05-11T23:03:07.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018179+0000", 2024-05-11T23:03:07.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1299999999999997e-05 2024-05-11T23:03:07.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018295+0000", 2024-05-11T23:03:07.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116117 2024-05-11T23:03:07.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.828 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.828 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78594 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:07.828 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:07.828 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.802360151999999, 2024-05-11T23:03:07.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046245204999999998, 2024-05-11T23:03:07.828 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.828 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.829 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.829 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78594 2024-05-11T23:03:07.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.829 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:07.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:07.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972263+0000", 2024-05-11T23:03:07.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:03:07.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972264+0000", 2024-05-11T23:03:07.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.206e-06 2024-05-11T23:03:07.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972267+0000", 2024-05-11T23:03:07.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0690000000000002e-06 2024-05-11T23:03:07.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972270+0000", 2024-05-11T23:03:07.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5610000000000001e-06 2024-05-11T23:03:07.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018330+0000", 2024-05-11T23:03:07.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04606085 2024-05-11T23:03:07.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018414+0000", 2024-05-11T23:03:07.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3125000000000001e-05 2024-05-11T23:03:07.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018508+0000", 2024-05-11T23:03:07.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4153000000000005e-05 2024-05-11T23:03:07.832 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.832 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.832 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.834 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78595 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:07.834 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:07.834 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.802347287000003, 2024-05-11T23:03:07.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046396698, 2024-05-11T23:03:07.834 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.834 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.834 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.834 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.835 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.835 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.835 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78595 2024-05-11T23:03:07.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.835 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:07.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:07.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972276+0000", 2024-05-11T23:03:07.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5600000000000002e-07 2024-05-11T23:03:07.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972277+0000", 2024-05-11T23:03:07.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.576e-06 2024-05-11T23:03:07.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972280+0000", 2024-05-11T23:03:07.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-05-11T23:03:07.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972283+0000", 2024-05-11T23:03:07.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:03:07.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018538+0000", 2024-05-11T23:03:07.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046255255000000002 2024-05-11T23:03:07.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018605+0000", 2024-05-11T23:03:07.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6847000000000006e-05 2024-05-11T23:03:07.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018672+0000", 2024-05-11T23:03:07.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7045000000000006e-05 2024-05-11T23:03:07.838 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.838 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.838 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.838 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78596 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:07.838 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:07.838 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.802336437999998, 2024-05-11T23:03:07.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046586806000000001, 2024-05-11T23:03:07.838 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.838 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.838 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.839 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.839 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.839 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.839 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78596 2024-05-11T23:03:07.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.839 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:07.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:07.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:07.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:03:07.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972288+0000", 2024-05-11T23:03:07.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2550000000000001e-06 2024-05-11T23:03:07.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972291+0000", 2024-05-11T23:03:07.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9490000000000001e-06 2024-05-11T23:03:07.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972293+0000", 2024-05-11T23:03:07.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5069999999999999e-06 2024-05-11T23:03:07.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018707+0000", 2024-05-11T23:03:07.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046413604999999997 2024-05-11T23:03:07.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018782+0000", 2024-05-11T23:03:07.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5556000000000006e-05 2024-05-11T23:03:07.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018873+0000", 2024-05-11T23:03:07.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0703000000000002e-05 2024-05-11T23:03:07.842 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.842 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.842 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.842 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78597 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:07.842 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:07.842 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.802326665000002, 2024-05-11T23:03:07.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046825787000000001, 2024-05-11T23:03:07.843 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.843 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.843 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.843 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78597 2024-05-11T23:03:07.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.843 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:07.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:07.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:07.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-05-11T23:03:07.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972297+0000", 2024-05-11T23:03:07.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3069999999999999e-06 2024-05-11T23:03:07.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972300+0000", 2024-05-11T23:03:07.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:03:07.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972303+0000", 2024-05-11T23:03:07.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5509999999999998e-06 2024-05-11T23:03:07.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018911+0000", 2024-05-11T23:03:07.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046608194999999998 2024-05-11T23:03:07.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018986+0000", 2024-05-11T23:03:07.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5360999999999999e-05 2024-05-11T23:03:07.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019122+0000", 2024-05-11T23:03:07.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013526 2024-05-11T23:03:07.846 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.846 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.846 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.846 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78598 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:07.846 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:07.847 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 51.802316765, 2024-05-11T23:03:07.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046998572000000002, 2024-05-11T23:03:07.847 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.847 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.847 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:07.847 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.847 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.847 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.847 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78598 2024-05-11T23:03:07.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.847 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:07.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:07.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:07.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:03:07.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972307+0000", 2024-05-11T23:03:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1799999999999999e-06 2024-05-11T23:03:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972310+0000", 2024-05-11T23:03:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:03:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972313+0000", 2024-05-11T23:03:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4760000000000001e-06 2024-05-11T23:03:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019149+0000", 2024-05-11T23:03:07.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046836056000000001 2024-05-11T23:03:07.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019203+0000", 2024-05-11T23:03:07.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-05 2024-05-11T23:03:07.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019304+0000", 2024-05-11T23:03:07.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101537 2024-05-11T23:03:07.850 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.850 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.850 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.851 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:105234 3.15 3:adf885e5:::rbd_object_map.111e913a615f.0000000000000025:head [writefull 0~39] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-11T23:03:07.851 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:07.851 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 37.767001055000001, 2024-05-11T23:03:07.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77797599100000003, 2024-05-11T23:03:07.851 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.851 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.851 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:07.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.851 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 105234 2024-05-11T23:03:07.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.851 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:07.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:07.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007623+0000", 2024-05-11T23:03:07.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2240000000000001e-06 2024-05-11T23:03:07.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007627+0000", 2024-05-11T23:03:07.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:03:07.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007636+0000", 2024-05-11T23:03:07.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2409999999999995e-06 2024-05-11T23:03:07.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007641+0000", 2024-05-11T23:03:07.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6810000000000003e-06 2024-05-11T23:03:07.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007676+0000", 2024-05-11T23:03:07.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5061999999999997e-05 2024-05-11T23:03:07.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:03:07.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007682+0000", 2024-05-11T23:03:07.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0399999999999998e-06 2024-05-11T23:03:07.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783880+0000", 2024-05-11T23:03:07.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77619752900000005 2024-05-11T23:03:07.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783995+0000", 2024-05-11T23:03:07.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011550800000000001 2024-05-11T23:03:07.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:03:07.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784082+0000", 2024-05-11T23:03:07.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6347999999999994e-05 2024-05-11T23:03:07.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:07.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784682+0000", 2024-05-11T23:03:07.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000600162 2024-05-11T23:03:07.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:07.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785539+0000", 2024-05-11T23:03:07.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085699099999999998 2024-05-11T23:03:07.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:07.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785572+0000", 2024-05-11T23:03:07.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3182999999999997e-05 2024-05-11T23:03:07.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785598+0000", 2024-05-11T23:03:07.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5522000000000001e-05 2024-05-11T23:03:07.856 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.856 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.857 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.857 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113761 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [call rbd.sparse_copyup in=999496b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:03:07.857 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:07.857 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.244314136, 2024-05-11T23:03:07.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048572964000000003, 2024-05-11T23:03:07.857 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.857 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.857 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:07.857 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.857 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.857 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.857 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113761 2024-05-11T23:03:07.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.858 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:07.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:07.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530309+0000", 2024-05-11T23:03:07.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:03:07.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531242+0000", 2024-05-11T23:03:07.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093316899999999997 2024-05-11T23:03:07.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531249+0000", 2024-05-11T23:03:07.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8140000000000003e-06 2024-05-11T23:03:07.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531253+0000", 2024-05-11T23:03:07.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4290000000000003e-06 2024-05-11T23:03:07.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570325+0000", 2024-05-11T23:03:07.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039071379000000003 2024-05-11T23:03:07.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570382+0000", 2024-05-11T23:03:07.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6988999999999999e-05 2024-05-11T23:03:07.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:03:07.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570555+0000", 2024-05-11T23:03:07.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017379799999999999 2024-05-11T23:03:07.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:07.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.575953+0000", 2024-05-11T23:03:07.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053975539999999997 2024-05-11T23:03:07.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:07.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578778+0000", 2024-05-11T23:03:07.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002825616 2024-05-11T23:03:07.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:07.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578837+0000", 2024-05-11T23:03:07.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9005000000000002e-05 2024-05-11T23:03:07.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578881+0000", 2024-05-11T23:03:07.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3970000000000001e-05 2024-05-11T23:03:07.862 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.862 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.862 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.862 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113762 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [delete] snapc 27=[27] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:03:07.862 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:07.862 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.243360185, 2024-05-11T23:03:07.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047916260000000002, 2024-05-11T23:03:07.862 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:07.862 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:07.862 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:07.862 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:07.863 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:07.863 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:07.863 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113762 2024-05-11T23:03:07.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.863 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:07.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:07.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:07.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:07.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:07.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:07.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:07.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531263+0000", 2024-05-11T23:03:07.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.85e-07 2024-05-11T23:03:07.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:07.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531265+0000", 2024-05-11T23:03:07.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.827e-06 2024-05-11T23:03:07.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:07.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531268+0000", 2024-05-11T23:03:07.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3780000000000001e-06 2024-05-11T23:03:07.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:07.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531271+0000", 2024-05-11T23:03:07.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8509999999999999e-06 2024-05-11T23:03:07.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:07.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572298+0000", 2024-05-11T23:03:07.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041026990999999999 2024-05-11T23:03:07.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:07.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572345+0000", 2024-05-11T23:03:07.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6878000000000001e-05 2024-05-11T23:03:07.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:03:07.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572443+0000", 2024-05-11T23:03:07.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8184999999999996e-05 2024-05-11T23:03:07.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:07.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578585+0000", 2024-05-11T23:03:07.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006141919 2024-05-11T23:03:07.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:07.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579126+0000", 2024-05-11T23:03:07.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00054063799999999997 2024-05-11T23:03:07.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:07.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579154+0000", 2024-05-11T23:03:07.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8042999999999999e-05 2024-05-11T23:03:07.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:07.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:07.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:07.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579179+0000", 2024-05-11T23:03:07.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4765000000000001e-05 2024-05-11T23:03:07.867 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.867 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.867 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.867 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:07.867 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:07.867 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:07.869 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:07.868+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:07.869 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:03:07.970 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:07.969+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:07.984 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:07.984 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:07.984 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:07.984 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:07.993 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:03:08.077 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:08.077+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:08.142 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:08.143 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:08.143 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:08.143 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:08.143 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:08.151 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:03:08.185 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-11T23:03:08.192 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:08.191+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:08.218 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 5, 7, 3, 6] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:03:08.218 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:03:08.218 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-11T23:03:08.218 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.798659 2024-05-11T23:03:08.218 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0.7986592299191275 2024-05-11T23:03:08.225 INFO:teuthology.orchestra.run.smithi005.stdout:flattening image image_client.0-clone46 2024-05-11T23:03:08.272 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:08.272+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:08.328 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:08.328 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:08.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:08.328 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:08.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.328 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226051 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6049792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:08.328 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.0090913380000002, 2024-05-11T23:03:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050622418000000002, 2024-05-11T23:03:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226051 2024-05-11T23:03:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:08.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317156+0000", 2024-05-11T23:03:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317157+0000", 2024-05-11T23:03:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.144e-06 2024-05-11T23:03:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317160+0000", 2024-05-11T23:03:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1350000000000001e-06 2024-05-11T23:03:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317162+0000", 2024-05-11T23:03:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5050000000000002e-06 2024-05-11T23:03:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367689+0000", 2024-05-11T23:03:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526797999999998 2024-05-11T23:03:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367735+0000", 2024-05-11T23:03:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5698999999999997e-05 2024-05-11T23:03:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367778+0000", 2024-05-11T23:03:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2928e-05 2024-05-11T23:03:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226052 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6060032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.0090812820000004, 2024-05-11T23:03:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050830110999999997, 2024-05-11T23:03:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226052 2024-05-11T23:03:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317166+0000", 2024-05-11T23:03:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:03:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317167+0000", 2024-05-11T23:03:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.155e-06 2024-05-11T23:03:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317170+0000", 2024-05-11T23:03:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.061e-06 2024-05-11T23:03:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317172+0000", 2024-05-11T23:03:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4420000000000001e-06 2024-05-11T23:03:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367816+0000", 2024-05-11T23:03:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050644013000000002 2024-05-11T23:03:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367906+0000", 2024-05-11T23:03:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9739000000000003e-05 2024-05-11T23:03:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367995+0000", 2024-05-11T23:03:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9499000000000005e-05 2024-05-11T23:03:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226053 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6070272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.0090712430000002, 2024-05-11T23:03:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051106912999999997, 2024-05-11T23:03:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.337 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226053 2024-05-11T23:03:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317176+0000", 2024-05-11T23:03:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0100000000000001e-07 2024-05-11T23:03:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317179+0000", 2024-05-11T23:03:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-05-11T23:03:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317182+0000", 2024-05-11T23:03:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3589999999999999e-06 2024-05-11T23:03:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317184+0000", 2024-05-11T23:03:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4339999999999999e-06 2024-05-11T23:03:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368061+0000", 2024-05-11T23:03:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050876927000000002 2024-05-11T23:03:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368179+0000", 2024-05-11T23:03:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118001 2024-05-11T23:03:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368282+0000", 2024-05-11T23:03:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102942 2024-05-11T23:03:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.341 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226054 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6080512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.0090589210000003, 2024-05-11T23:03:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051322, 2024-05-11T23:03:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226054 2024-05-11T23:03:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.342 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317189+0000", 2024-05-11T23:03:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.217e-06 2024-05-11T23:03:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317192+0000", 2024-05-11T23:03:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0699999999999998e-06 2024-05-11T23:03:08.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317195+0000", 2024-05-11T23:03:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3829999999999999e-06 2024-05-11T23:03:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368328+0000", 2024-05-11T23:03:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051132999999999998 2024-05-11T23:03:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368429+0000", 2024-05-11T23:03:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010168800000000001 2024-05-11T23:03:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368510+0000", 2024-05-11T23:03:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0420999999999995e-05 2024-05-11T23:03:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226055 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6090752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.0090488530000004, 2024-05-11T23:03:08.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051562835000000001, 2024-05-11T23:03:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226055 2024-05-11T23:03:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317256+0000", 2024-05-11T23:03:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8047000000000002e-05 2024-05-11T23:03:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317259+0000", 2024-05-11T23:03:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2590000000000001e-06 2024-05-11T23:03:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317267+0000", 2024-05-11T23:03:08.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8769999999999993e-06 2024-05-11T23:03:08.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317274+0000", 2024-05-11T23:03:08.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.187e-06 2024-05-11T23:03:08.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368544+0000", 2024-05-11T23:03:08.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051270299999999998 2024-05-11T23:03:08.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368658+0000", 2024-05-11T23:03:08.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000113347 2024-05-11T23:03:08.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368761+0000", 2024-05-11T23:03:08.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102818 2024-05-11T23:03:08.350 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.350 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.350 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.350 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226056 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6100992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:08.350 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:08.350 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.0089630070000002, 2024-05-11T23:03:08.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051615576000000003, 2024-05-11T23:03:08.351 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.351 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.351 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.351 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.351 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.351 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.351 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226056 2024-05-11T23:03:08.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.351 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:08.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:08.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318915+0000", 2024-05-11T23:03:08.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001631291 2024-05-11T23:03:08.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318918+0000", 2024-05-11T23:03:08.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.512e-06 2024-05-11T23:03:08.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318927+0000", 2024-05-11T23:03:08.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1960000000000008e-06 2024-05-11T23:03:08.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318932+0000", 2024-05-11T23:03:08.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:03:08.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368805+0000", 2024-05-11T23:03:08.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049872998000000002 2024-05-11T23:03:08.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368853+0000", 2024-05-11T23:03:08.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8041999999999998e-05 2024-05-11T23:03:08.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368899+0000", 2024-05-11T23:03:08.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5766000000000002e-05 2024-05-11T23:03:08.354 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.354 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.354 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.355 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226061 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6152192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:08.355 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:08.355 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.0071719200000002, 2024-05-11T23:03:08.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050675395999999998, 2024-05-11T23:03:08.355 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.355 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.355 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.355 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.355 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.355 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226061 2024-05-11T23:03:08.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.355 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:08.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:08.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:08.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:08.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319078+0000", 2024-05-11T23:03:08.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5670000000000002e-06 2024-05-11T23:03:08.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319085+0000", 2024-05-11T23:03:08.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2570000000000001e-06 2024-05-11T23:03:08.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319100+0000", 2024-05-11T23:03:08.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.489e-05 2024-05-11T23:03:08.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369584+0000", 2024-05-11T23:03:08.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050484345999999999 2024-05-11T23:03:08.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369671+0000", 2024-05-11T23:03:08.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6654000000000001e-05 2024-05-11T23:03:08.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369750+0000", 2024-05-11T23:03:08.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9257000000000004e-05 2024-05-11T23:03:08.358 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.359 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.359 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.359 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226062 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6162432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:08.359 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:08.359 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.0071383760000003, 2024-05-11T23:03:08.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050761158000000001, 2024-05-11T23:03:08.359 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.359 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.359 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.359 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.359 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.359 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.359 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226062 2024-05-11T23:03:08.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.360 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:08.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:08.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319109+0000", 2024-05-11T23:03:08.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2800000000000002e-07 2024-05-11T23:03:08.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319111+0000", 2024-05-11T23:03:08.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4980000000000001e-06 2024-05-11T23:03:08.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319120+0000", 2024-05-11T23:03:08.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3999999999999992e-06 2024-05-11T23:03:08.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319129+0000", 2024-05-11T23:03:08.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7580000000000004e-06 2024-05-11T23:03:08.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369775+0000", 2024-05-11T23:03:08.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050645580000000003 2024-05-11T23:03:08.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369824+0000", 2024-05-11T23:03:08.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9506999999999999e-05 2024-05-11T23:03:08.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369869+0000", 2024-05-11T23:03:08.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4987000000000001e-05 2024-05-11T23:03:08.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.362 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.363 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.363 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226063 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6172672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:08.363 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:08.363 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.0071094189999998, 2024-05-11T23:03:08.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050939784000000002, 2024-05-11T23:03:08.363 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.363 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.363 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.363 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.363 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.363 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226063 2024-05-11T23:03:08.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.364 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:08.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:08.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319138+0000", 2024-05-11T23:03:08.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.39e-07 2024-05-11T23:03:08.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319140+0000", 2024-05-11T23:03:08.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4820000000000001e-06 2024-05-11T23:03:08.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319148+0000", 2024-05-11T23:03:08.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6599999999999995e-06 2024-05-11T23:03:08.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319154+0000", 2024-05-11T23:03:08.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4169999999999997e-06 2024-05-11T23:03:08.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369905+0000", 2024-05-11T23:03:08.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050751009999999999 2024-05-11T23:03:08.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369991+0000", 2024-05-11T23:03:08.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5909999999999996e-05 2024-05-11T23:03:08.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370077+0000", 2024-05-11T23:03:08.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5865999999999994e-05 2024-05-11T23:03:08.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.366 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.367 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226064 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6182912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:08.367 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:08.367 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.0070832230000004, 2024-05-11T23:03:08.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051038241999999998, 2024-05-11T23:03:08.367 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.367 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.367 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.367 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.367 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226064 2024-05-11T23:03:08.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.367 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:08.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:08.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319164+0000", 2024-05-11T23:03:08.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9299999999999999e-07 2024-05-11T23:03:08.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319166+0000", 2024-05-11T23:03:08.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.48e-06 2024-05-11T23:03:08.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319176+0000", 2024-05-11T23:03:08.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9669999999999996e-06 2024-05-11T23:03:08.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319180+0000", 2024-05-11T23:03:08.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8020000000000002e-06 2024-05-11T23:03:08.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370100+0000", 2024-05-11T23:03:08.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050919845999999998 2024-05-11T23:03:08.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370155+0000", 2024-05-11T23:03:08.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5040000000000002e-05 2024-05-11T23:03:08.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370202+0000", 2024-05-11T23:03:08.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6714e-05 2024-05-11T23:03:08.370 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.370 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.371 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.371 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226065 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6193152~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:08.371 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:08.371 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.0070606529999999, 2024-05-11T23:03:08.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051225087000000002, 2024-05-11T23:03:08.371 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.371 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.371 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.371 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.371 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.371 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226065 2024-05-11T23:03:08.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.372 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:08.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:08.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320913+0000", 2024-05-11T23:03:08.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001727492 2024-05-11T23:03:08.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320919+0000", 2024-05-11T23:03:08.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.964e-06 2024-05-11T23:03:08.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320927+0000", 2024-05-11T23:03:08.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2239999999999998e-06 2024-05-11T23:03:08.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320934+0000", 2024-05-11T23:03:08.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8469999999999998e-06 2024-05-11T23:03:08.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370240+0000", 2024-05-11T23:03:08.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049306562999999998 2024-05-11T23:03:08.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370328+0000", 2024-05-11T23:03:08.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8122999999999996e-05 2024-05-11T23:03:08.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370411+0000", 2024-05-11T23:03:08.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2874000000000006e-05 2024-05-11T23:03:08.375 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.375 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.375 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.375 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226073 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6275072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:08.375 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:08.375 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.0052222659999996, 2024-05-11T23:03:08.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050730475999999997, 2024-05-11T23:03:08.375 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.375 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.375 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.375 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.375 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.375 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.376 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226073 2024-05-11T23:03:08.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.376 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:08.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:08.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322914+0000", 2024-05-11T23:03:08.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001889153 2024-05-11T23:03:08.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322917+0000", 2024-05-11T23:03:08.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2100000000000002e-06 2024-05-11T23:03:08.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322924+0000", 2024-05-11T23:03:08.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9929999999999998e-06 2024-05-11T23:03:08.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322928+0000", 2024-05-11T23:03:08.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5549999999999996e-06 2024-05-11T23:03:08.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371578+0000", 2024-05-11T23:03:08.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048650001999999998 2024-05-11T23:03:08.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371671+0000", 2024-05-11T23:03:08.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2230000000000003e-05 2024-05-11T23:03:08.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371755+0000", 2024-05-11T23:03:08.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4332999999999994e-05 2024-05-11T23:03:08.379 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.379 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.379 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.379 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226085 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 5338112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:08.379 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:08.379 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 6.0031311670000003, 2024-05-11T23:03:08.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050727622, 2024-05-11T23:03:08.379 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.379 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.380 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.380 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226085 2024-05-11T23:03:08.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.380 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:08.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:08.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324914+0000", 2024-05-11T23:03:08.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001798298 2024-05-11T23:03:08.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324918+0000", 2024-05-11T23:03:08.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3889999999999998e-06 2024-05-11T23:03:08.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324928+0000", 2024-05-11T23:03:08.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4339999999999998e-06 2024-05-11T23:03:08.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324936+0000", 2024-05-11T23:03:08.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2770000000000005e-06 2024-05-11T23:03:08.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373664+0000", 2024-05-11T23:03:08.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048728207000000003 2024-05-11T23:03:08.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373757+0000", 2024-05-11T23:03:08.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3332999999999996e-05 2024-05-11T23:03:08.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373843+0000", 2024-05-11T23:03:08.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5684000000000002e-05 2024-05-11T23:03:08.383 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.383 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.383 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.383 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232555 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7789568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:08.383 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:08.383 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5261208420000001, 2024-05-11T23:03:08.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050583086999999999, 2024-05-11T23:03:08.383 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.383 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.384 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.384 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.384 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.384 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232555 2024-05-11T23:03:08.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.384 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:08.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:08.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:08.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:03:08.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800127+0000", 2024-05-11T23:03:08.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.207e-06 2024-05-11T23:03:08.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800132+0000", 2024-05-11T23:03:08.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:03:08.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800134+0000", 2024-05-11T23:03:08.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5459999999999998e-06 2024-05-11T23:03:08.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850548+0000", 2024-05-11T23:03:08.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050414182000000002 2024-05-11T23:03:08.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850635+0000", 2024-05-11T23:03:08.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6853000000000003e-05 2024-05-11T23:03:08.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850709+0000", 2024-05-11T23:03:08.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3585999999999996e-05 2024-05-11T23:03:08.387 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.387 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.387 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.387 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232556 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7799808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:08.387 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:08.387 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5261091879999995, 2024-05-11T23:03:08.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050749599999999999, 2024-05-11T23:03:08.387 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.387 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.387 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.388 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.388 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232556 2024-05-11T23:03:08.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.388 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:08.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:08.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801914+0000", 2024-05-11T23:03:08.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017766360000000001 2024-05-11T23:03:08.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801919+0000", 2024-05-11T23:03:08.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.42e-06 2024-05-11T23:03:08.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801927+0000", 2024-05-11T23:03:08.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1999999999999994e-06 2024-05-11T23:03:08.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801931+0000", 2024-05-11T23:03:08.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4279999999999998e-06 2024-05-11T23:03:08.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850740+0000", 2024-05-11T23:03:08.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048808658999999997 2024-05-11T23:03:08.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850819+0000", 2024-05-11T23:03:08.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9102000000000003e-05 2024-05-11T23:03:08.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850887+0000", 2024-05-11T23:03:08.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8155000000000001e-05 2024-05-11T23:03:08.391 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.391 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.391 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.391 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232578 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8025088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:08.391 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:08.391 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.52281595, 2024-05-11T23:03:08.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050679473000000003, 2024-05-11T23:03:08.391 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.391 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.391 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.392 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232578 2024-05-11T23:03:08.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.392 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:08.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:08.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:08.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:08.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803432+0000", 2024-05-11T23:03:08.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.139e-06 2024-05-11T23:03:08.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803435+0000", 2024-05-11T23:03:08.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2440000000000002e-06 2024-05-11T23:03:08.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803439+0000", 2024-05-11T23:03:08.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3129999999999999e-06 2024-05-11T23:03:08.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.853953+0000", 2024-05-11T23:03:08.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050514846000000002 2024-05-11T23:03:08.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854035+0000", 2024-05-11T23:03:08.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1148000000000003e-05 2024-05-11T23:03:08.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854110+0000", 2024-05-11T23:03:08.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5562000000000006e-05 2024-05-11T23:03:08.395 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.395 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.395 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.396 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232579 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8035328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:08.396 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:08.396 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5228033459999999, 2024-05-11T23:03:08.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050886656000000002, 2024-05-11T23:03:08.396 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.396 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.396 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.396 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.396 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.396 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.396 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232579 2024-05-11T23:03:08.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.396 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:08.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:08.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803914+0000", 2024-05-11T23:03:08.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00047026700000000002 2024-05-11T23:03:08.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803918+0000", 2024-05-11T23:03:08.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3460000000000004e-06 2024-05-11T23:03:08.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803927+0000", 2024-05-11T23:03:08.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5030000000000003e-06 2024-05-11T23:03:08.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803933+0000", 2024-05-11T23:03:08.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0310000000000004e-06 2024-05-11T23:03:08.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854153+0000", 2024-05-11T23:03:08.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050219321999999997 2024-05-11T23:03:08.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854242+0000", 2024-05-11T23:03:08.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9457999999999996e-05 2024-05-11T23:03:08.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854330+0000", 2024-05-11T23:03:08.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7729e-05 2024-05-11T23:03:08.400 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.400 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.400 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.400 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232581 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8055808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:08.400 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:08.400 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5222798690000001, 2024-05-11T23:03:08.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050664439999999998, 2024-05-11T23:03:08.400 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.401 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.401 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.401 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.401 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.401 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232581 2024-05-11T23:03:08.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.401 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:08.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:08.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:08.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:03:08.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803969+0000", 2024-05-11T23:03:08.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0889999999999998e-06 2024-05-11T23:03:08.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803978+0000", 2024-05-11T23:03:08.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3070000000000003e-06 2024-05-11T23:03:08.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803983+0000", 2024-05-11T23:03:08.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1599999999999997e-06 2024-05-11T23:03:08.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854509+0000", 2024-05-11T23:03:08.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526339000000003 2024-05-11T23:03:08.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854576+0000", 2024-05-11T23:03:08.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7102999999999998e-05 2024-05-11T23:03:08.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854631+0000", 2024-05-11T23:03:08.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4997999999999998e-05 2024-05-11T23:03:08.404 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.404 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.404 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.404 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232582 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8066048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:08.404 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:08.404 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5222571240000002, 2024-05-11T23:03:08.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050753983000000003, 2024-05-11T23:03:08.405 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.405 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.405 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.405 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.405 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.405 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.405 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232582 2024-05-11T23:03:08.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.405 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:08.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:08.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805917+0000", 2024-05-11T23:03:08.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001927432 2024-05-11T23:03:08.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805920+0000", 2024-05-11T23:03:08.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1219999999999999e-06 2024-05-11T23:03:08.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805933+0000", 2024-05-11T23:03:08.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2727e-05 2024-05-11T23:03:08.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805941+0000", 2024-05-11T23:03:08.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7090000000000002e-06 2024-05-11T23:03:08.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854657+0000", 2024-05-11T23:03:08.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048716164999999999 2024-05-11T23:03:08.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854699+0000", 2024-05-11T23:03:08.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2531999999999999e-05 2024-05-11T23:03:08.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854744+0000", 2024-05-11T23:03:08.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4295999999999997e-05 2024-05-11T23:03:08.408 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.408 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.408 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.408 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232593 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8178688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:08.408 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:08.408 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 5.5200577309999996, 2024-05-11T23:03:08.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050735178999999998, 2024-05-11T23:03:08.409 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.409 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.409 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.409 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.409 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.409 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.409 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232593 2024-05-11T23:03:08.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.409 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:08.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:08.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807913+0000", 2024-05-11T23:03:08.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017240319999999999 2024-05-11T23:03:08.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807916+0000", 2024-05-11T23:03:08.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4589999999999998e-06 2024-05-11T23:03:08.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807924+0000", 2024-05-11T23:03:08.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.272e-06 2024-05-11T23:03:08.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807928+0000", 2024-05-11T23:03:08.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5369999999999999e-06 2024-05-11T23:03:08.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856768+0000", 2024-05-11T23:03:08.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048839489999999999 2024-05-11T23:03:08.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856845+0000", 2024-05-11T23:03:08.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7746999999999999e-05 2024-05-11T23:03:08.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856924+0000", 2024-05-11T23:03:08.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8641999999999997e-05 2024-05-11T23:03:08.412 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.412 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.412 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.413 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.413 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.413 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:08.414 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:03:08.415 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:08.413+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:08.422 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-05-11T23:03:08.515 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:08.514+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:08.557 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:08.557 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:08.557 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:08.557 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:08.567 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:03:08.651 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:08.650+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:08.675 INFO:teuthology.orchestra.run.smithi005.stdout:5125 write 0x77eddcf thru 0x77f9ced (0xbf1f bytes) 2024-05-11T23:03:08.687 INFO:teuthology.orchestra.run.smithi005.stdout:5126 write 0x4df77f0 thru 0x4dfdcdf (0x64f0 bytes) 2024-05-11T23:03:08.693 INFO:teuthology.orchestra.run.smithi005.stdout:5127 write 0xde0de2 thru 0xde5e97 (0x50b6 bytes) 2024-05-11T23:03:08.699 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:08.699 INFO:teuthology.orchestra.run.smithi005.stdout:5129 read 0xa1eadbb thru 0xa1f97de (0xea24 bytes) 2024-05-11T23:03:08.699 INFO:teuthology.orchestra.run.smithi005.stdout:5131 read 0xd1f75ad thru 0xd204ff4 (0xda48 bytes) 2024-05-11T23:03:08.699 INFO:teuthology.orchestra.run.smithi005.stdout:5133 read 0x7c52e26 thru 0x7c56888 (0x3a63 bytes) 2024-05-11T23:03:08.699 INFO:teuthology.orchestra.run.smithi005.stdout:5134 writesame 0x9540321 thru 0x954f8b1 data_size 0x7f(0xf591 bytes) 2024-05-11T23:03:08.705 INFO:teuthology.orchestra.run.smithi005.stdout:5136 read 0x7cd63ab thru 0x7cde6d4 (0x832a bytes) 2024-05-11T23:03:08.710 INFO:teuthology.orchestra.run.smithi005.stdout:5137 trunc from 0xe618310 to 0xa185b9f 2024-05-11T23:03:08.717 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:08.717+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:08.728 INFO:teuthology.orchestra.run.smithi005.stdout:5138 punch from 0x71e84f5 to 0x71f1b87, (0x9692 bytes) 2024-05-11T23:03:08.728 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:08.728 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:08.728 INFO:teuthology.orchestra.run.smithi005.stdout:5142 write 0x56b5cd5 thru 0x56b6c0a (0xf36 bytes) 2024-05-11T23:03:08.728 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:08.728 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:08.729 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:08.729 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:08.729 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:08.734 INFO:teuthology.orchestra.run.smithi005.stdout:5144 trunc from 0xa185b9f to 0xca5bb85 2024-05-11T23:03:08.739 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:03:08.740 INFO:teuthology.orchestra.run.smithi005.stdout:5146 clone 48 order 22 su 524288 sc 13 2024-05-11T23:03:08.818 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:08.817+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:08.850 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:08.851 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:08.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:08.851 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:08.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.851 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:64049 3.8 3:11d98727:::rbd_object_map.111e8974a681:head [read 0~38] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-05-11T23:03:08.851 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:08.851 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.070085277999993, 2024-05-11T23:03:08.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99884604399999999, 2024-05-11T23:03:08.851 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.851 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.851 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:08.852 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.852 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.852 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.852 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 64049 2024-05-11T23:03:08.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.852 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:08.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:08.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779883+0000", 2024-05-11T23:03:08.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.57e-06 2024-05-11T23:03:08.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779887+0000", 2024-05-11T23:03:08.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5709999999999999e-06 2024-05-11T23:03:08.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779898+0000", 2024-05-11T23:03:08.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.042e-05 2024-05-11T23:03:08.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779905+0000", 2024-05-11T23:03:08.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0110000000000003e-06 2024-05-11T23:03:08.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779962+0000", 2024-05-11T23:03:08.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7395999999999997e-05 2024-05-11T23:03:08.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:08.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779969+0000", 2024-05-11T23:03:08.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2459999999999997e-06 2024-05-11T23:03:08.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778379+0000", 2024-05-11T23:03:08.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99840986300000001 2024-05-11T23:03:08.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:08.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778564+0000", 2024-05-11T23:03:08.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018420900000000001 2024-05-11T23:03:08.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778727+0000", 2024-05-11T23:03:08.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016375800000000001 2024-05-11T23:03:08.858 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.858 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.858 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.858 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172773 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:08.858 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:08.858 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.245978859999999, 2024-05-11T23:03:08.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058721316000000003, 2024-05-11T23:03:08.859 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.859 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:08.859 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.859 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.859 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.859 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172773 2024-05-11T23:03:08.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.859 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:08.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:08.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:08.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3500000000000002e-07 2024-05-11T23:03:08.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603990+0000", 2024-05-11T23:03:08.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3549999999999999e-06 2024-05-11T23:03:08.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603997+0000", 2024-05-11T23:03:08.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3400000000000003e-06 2024-05-11T23:03:08.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604002+0000", 2024-05-11T23:03:08.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0509999999999996e-06 2024-05-11T23:03:08.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662697+0000", 2024-05-11T23:03:08.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058695487999999997 2024-05-11T23:03:08.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662709+0000", 2024-05-11T23:03:08.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1647e-05 2024-05-11T23:03:08.863 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.863 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.863 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.863 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172774 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:08.864 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:08.864 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.245957477999999, 2024-05-11T23:03:08.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058745040999999998, 2024-05-11T23:03:08.864 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.864 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:08.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172774 2024-05-11T23:03:08.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.864 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:08.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:08.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:08.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:03:08.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604012+0000", 2024-05-11T23:03:08.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2199999999999999e-06 2024-05-11T23:03:08.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604018+0000", 2024-05-11T23:03:08.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1770000000000003e-06 2024-05-11T23:03:08.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604023+0000", 2024-05-11T23:03:08.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0089999999999998e-06 2024-05-11T23:03:08.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662742+0000", 2024-05-11T23:03:08.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058719187999999999 2024-05-11T23:03:08.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662754+0000", 2024-05-11T23:03:08.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2095e-05 2024-05-11T23:03:08.867 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.868 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.868 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.868 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172775 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:08.868 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:08.868 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.245937266, 2024-05-11T23:03:08.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058764743000000001, 2024-05-11T23:03:08.868 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.868 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.868 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:08.868 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.868 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.869 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.869 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172775 2024-05-11T23:03:08.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.869 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:08.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:08.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604030+0000", 2024-05-11T23:03:08.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6699999999999999e-07 2024-05-11T23:03:08.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604032+0000", 2024-05-11T23:03:08.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9530000000000002e-06 2024-05-11T23:03:08.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604037+0000", 2024-05-11T23:03:08.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8590000000000001e-06 2024-05-11T23:03:08.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604042+0000", 2024-05-11T23:03:08.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8609999999999997e-06 2024-05-11T23:03:08.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662782+0000", 2024-05-11T23:03:08.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058739731000000003 2024-05-11T23:03:08.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662794+0000", 2024-05-11T23:03:08.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1972e-05 2024-05-11T23:03:08.872 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.872 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.872 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.872 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172776 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:08.872 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:08.872 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.245917975999999, 2024-05-11T23:03:08.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058778696999999998, 2024-05-11T23:03:08.873 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.873 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.873 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:08.873 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.873 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.873 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.873 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172776 2024-05-11T23:03:08.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.873 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:08.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:08.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:08.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2000000000000001e-07 2024-05-11T23:03:08.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604051+0000", 2024-05-11T23:03:08.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2879999999999999e-06 2024-05-11T23:03:08.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604057+0000", 2024-05-11T23:03:08.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:03:08.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604062+0000", 2024-05-11T23:03:08.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6800000000000001e-06 2024-05-11T23:03:08.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662820+0000", 2024-05-11T23:03:08.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058758444999999999 2024-05-11T23:03:08.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662827+0000", 2024-05-11T23:03:08.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1930000000000004e-06 2024-05-11T23:03:08.876 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.876 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.876 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.877 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172777 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:08.877 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:08.877 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.245898670000001, 2024-05-11T23:03:08.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058799918999999999, 2024-05-11T23:03:08.877 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.877 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.877 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:08.877 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.877 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.877 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.877 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172777 2024-05-11T23:03:08.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.878 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:08.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:08.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:08.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:03:08.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604075+0000", 2024-05-11T23:03:08.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.109e-06 2024-05-11T23:03:08.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604083+0000", 2024-05-11T23:03:08.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9170000000000006e-06 2024-05-11T23:03:08.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604089+0000", 2024-05-11T23:03:08.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3399999999999997e-06 2024-05-11T23:03:08.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662859+0000", 2024-05-11T23:03:08.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058770172000000002 2024-05-11T23:03:08.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662868+0000", 2024-05-11T23:03:08.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0289999999999994e-06 2024-05-11T23:03:08.881 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.881 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.881 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.881 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172778 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:08.881 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:08.881 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.24587058, 2024-05-11T23:03:08.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058845435000000001, 2024-05-11T23:03:08.882 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.882 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.882 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:08.882 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.882 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.882 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.882 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172778 2024-05-11T23:03:08.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.883 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:08.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:08.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:08.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5400000000000002e-07 2024-05-11T23:03:08.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604099+0000", 2024-05-11T23:03:08.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4650000000000001e-06 2024-05-11T23:03:08.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604105+0000", 2024-05-11T23:03:08.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.314e-06 2024-05-11T23:03:08.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604110+0000", 2024-05-11T23:03:08.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6310000000000002e-06 2024-05-11T23:03:08.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662931+0000", 2024-05-11T23:03:08.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058820960999999998 2024-05-11T23:03:08.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662941+0000", 2024-05-11T23:03:08.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.061e-05 2024-05-11T23:03:08.886 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.886 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.886 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.886 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172779 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:08.886 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:08.886 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.24585042, 2024-05-11T23:03:08.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058859163999999999, 2024-05-11T23:03:08.887 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.887 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.887 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:08.887 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.887 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.887 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.887 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172779 2024-05-11T23:03:08.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.887 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:08.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:08.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:08.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:03:08.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604119+0000", 2024-05-11T23:03:08.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2529999999999999e-06 2024-05-11T23:03:08.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604125+0000", 2024-05-11T23:03:08.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1670000000000004e-06 2024-05-11T23:03:08.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604130+0000", 2024-05-11T23:03:08.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.702e-06 2024-05-11T23:03:08.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662967+0000", 2024-05-11T23:03:08.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058837792999999999 2024-05-11T23:03:08.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662975+0000", 2024-05-11T23:03:08.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8490000000000006e-06 2024-05-11T23:03:08.891 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.891 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.891 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.891 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172780 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:08.891 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:08.891 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.245830270000001, 2024-05-11T23:03:08.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058872696000000002, 2024-05-11T23:03:08.891 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.891 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.891 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:08.892 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.892 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.892 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.892 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172780 2024-05-11T23:03:08.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.892 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:08.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:08.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604137+0000", 2024-05-11T23:03:08.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4400000000000001e-07 2024-05-11T23:03:08.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604139+0000", 2024-05-11T23:03:08.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3429999999999998e-06 2024-05-11T23:03:08.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604145+0000", 2024-05-11T23:03:08.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1940000000000003e-06 2024-05-11T23:03:08.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604150+0000", 2024-05-11T23:03:08.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:03:08.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663002+0000", 2024-05-11T23:03:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058851829000000001 2024-05-11T23:03:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663009+0000", 2024-05-11T23:03:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.199e-06 2024-05-11T23:03:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172781 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:08.895 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.245798430000001, 2024-05-11T23:03:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058875553999999997, 2024-05-11T23:03:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172781 2024-05-11T23:03:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604171+0000", 2024-05-11T23:03:08.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.21e-06 2024-05-11T23:03:08.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604176+0000", 2024-05-11T23:03:08.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2970000000000003e-06 2024-05-11T23:03:08.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604179+0000", 2024-05-11T23:03:08.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8459999999999999e-06 2024-05-11T23:03:08.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663033+0000", 2024-05-11T23:03:08.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058854565999999997 2024-05-11T23:03:08.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663044+0000", 2024-05-11T23:03:08.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0229e-05 2024-05-11T23:03:08.899 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.899 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.900 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.900 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172782 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:08.900 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:08.900 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.245783682000001, 2024-05-11T23:03:08.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058894375999999998, 2024-05-11T23:03:08.900 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.900 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.900 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:08.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.900 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.901 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172782 2024-05-11T23:03:08.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.901 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:08.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:08.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:08.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-05-11T23:03:08.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604184+0000", 2024-05-11T23:03:08.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:03:08.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604187+0000", 2024-05-11T23:03:08.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.044e-06 2024-05-11T23:03:08.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604190+0000", 2024-05-11T23:03:08.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6460000000000002e-06 2024-05-11T23:03:08.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663070+0000", 2024-05-11T23:03:08.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058879510000000003 2024-05-11T23:03:08.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663077+0000", 2024-05-11T23:03:08.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6669999999999996e-06 2024-05-11T23:03:08.904 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.904 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.904 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.904 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172783 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:08.904 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:08.904 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.245773363, 2024-05-11T23:03:08.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058913946000000002, 2024-05-11T23:03:08.904 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.904 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.905 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:08.905 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.905 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.905 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.905 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172783 2024-05-11T23:03:08.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.905 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:08.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:08.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:08.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-05-11T23:03:08.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604195+0000", 2024-05-11T23:03:08.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.19e-06 2024-05-11T23:03:08.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604198+0000", 2024-05-11T23:03:08.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:03:08.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604200+0000", 2024-05-11T23:03:08.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4719999999999998e-06 2024-05-11T23:03:08.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663100+0000", 2024-05-11T23:03:08.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058900300000000003 2024-05-11T23:03:08.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663107+0000", 2024-05-11T23:03:08.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7449999999999998e-06 2024-05-11T23:03:08.908 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.908 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.908 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.909 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172784 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:08.909 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:08.909 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.245763344, 2024-05-11T23:03:08.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933728999999997, 2024-05-11T23:03:08.909 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.909 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:08.909 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.909 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.910 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.910 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172784 2024-05-11T23:03:08.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.910 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:08.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:08.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:08.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.03e-07 2024-05-11T23:03:08.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604205+0000", 2024-05-11T23:03:08.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2300000000000001e-06 2024-05-11T23:03:08.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604207+0000", 2024-05-11T23:03:08.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8729999999999999e-06 2024-05-11T23:03:08.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604210+0000", 2024-05-11T23:03:08.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.858e-06 2024-05-11T23:03:08.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663130+0000", 2024-05-11T23:03:08.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058919314 2024-05-11T23:03:08.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663137+0000", 2024-05-11T23:03:08.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2509999999999997e-06 2024-05-11T23:03:08.913 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.913 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.913 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.913 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172785 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:08.914 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:08.914 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.245751073999999, 2024-05-11T23:03:08.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058951664000000001, 2024-05-11T23:03:08.914 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.914 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.914 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:08.914 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.914 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.914 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.914 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172785 2024-05-11T23:03:08.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.914 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:08.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:08.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604216+0000", 2024-05-11T23:03:08.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:03:08.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604218+0000", 2024-05-11T23:03:08.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7549999999999999e-06 2024-05-11T23:03:08.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604222+0000", 2024-05-11T23:03:08.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6700000000000002e-06 2024-05-11T23:03:08.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604227+0000", 2024-05-11T23:03:08.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4399999999999998e-06 2024-05-11T23:03:08.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663160+0000", 2024-05-11T23:03:08.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933119999999999 2024-05-11T23:03:08.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663167+0000", 2024-05-11T23:03:08.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1840000000000002e-06 2024-05-11T23:03:08.918 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.918 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.918 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.918 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172786 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:08.918 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:08.918 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.24573402, 2024-05-11T23:03:08.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058966991000000003, 2024-05-11T23:03:08.918 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.918 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.918 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:08.918 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.919 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.919 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.919 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172786 2024-05-11T23:03:08.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.919 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:08.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:08.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604233+0000", 2024-05-11T23:03:08.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6899999999999999e-07 2024-05-11T23:03:08.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604235+0000", 2024-05-11T23:03:08.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8369999999999999e-06 2024-05-11T23:03:08.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604240+0000", 2024-05-11T23:03:08.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0329999999999999e-06 2024-05-11T23:03:08.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604244+0000", 2024-05-11T23:03:08.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9859999999999996e-06 2024-05-11T23:03:08.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663192+0000", 2024-05-11T23:03:08.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058948852000000003 2024-05-11T23:03:08.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663199+0000", 2024-05-11T23:03:08.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0140000000000001e-06 2024-05-11T23:03:08.922 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.922 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.922 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.922 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172787 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:08.923 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:08.923 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.245717674, 2024-05-11T23:03:08.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058982018999999997, 2024-05-11T23:03:08.923 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.923 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.923 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:08.923 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.923 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.923 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.923 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172787 2024-05-11T23:03:08.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.923 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:08.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:08.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605916+0000", 2024-05-11T23:03:08.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00166752 2024-05-11T23:03:08.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605921+0000", 2024-05-11T23:03:08.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0669999999999999e-06 2024-05-11T23:03:08.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605934+0000", 2024-05-11T23:03:08.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2271e-05 2024-05-11T23:03:08.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605940+0000", 2024-05-11T23:03:08.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6880000000000002e-06 2024-05-11T23:03:08.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663224+0000", 2024-05-11T23:03:08.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057283251 2024-05-11T23:03:08.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663231+0000", 2024-05-11T23:03:08.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2219999999999996e-06 2024-05-11T23:03:08.927 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.927 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.927 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.927 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172793 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:08.927 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:08.927 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.243950827000001, 2024-05-11T23:03:08.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057402373999999999, 2024-05-11T23:03:08.927 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.927 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.927 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:08.928 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.928 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.928 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.928 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172793 2024-05-11T23:03:08.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.928 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:08.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:08.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:08.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3300000000000001e-07 2024-05-11T23:03:08.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606017+0000", 2024-05-11T23:03:08.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4920000000000001e-06 2024-05-11T23:03:08.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606020+0000", 2024-05-11T23:03:08.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:03:08.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606023+0000", 2024-05-11T23:03:08.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6680000000000001e-06 2024-05-11T23:03:08.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663411+0000", 2024-05-11T23:03:08.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057388206999999997 2024-05-11T23:03:08.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663418+0000", 2024-05-11T23:03:08.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.815e-06 2024-05-11T23:03:08.931 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.931 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.932 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.932 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172795 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:08.932 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:08.932 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.2439094, 2024-05-11T23:03:08.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057423015000000001, 2024-05-11T23:03:08.932 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.932 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.932 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:08.932 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.932 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.933 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.933 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172795 2024-05-11T23:03:08.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.933 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:08.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:08.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:08.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9499999999999998e-07 2024-05-11T23:03:08.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606059+0000", 2024-05-11T23:03:08.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-05-11T23:03:08.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606062+0000", 2024-05-11T23:03:08.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.977e-06 2024-05-11T23:03:08.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606064+0000", 2024-05-11T23:03:08.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7410000000000001e-06 2024-05-11T23:03:08.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663473+0000", 2024-05-11T23:03:08.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057408624999999998 2024-05-11T23:03:08.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663480+0000", 2024-05-11T23:03:08.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1049999999999997e-06 2024-05-11T23:03:08.936 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.936 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.936 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.937 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172796 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:08.937 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:08.937 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.243898837, 2024-05-11T23:03:08.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057449235000000001, 2024-05-11T23:03:08.937 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.937 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.937 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:08.937 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.937 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.937 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.937 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172796 2024-05-11T23:03:08.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.937 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:08.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:08.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:08.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:08.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606069+0000", 2024-05-11T23:03:08.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:03:08.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606072+0000", 2024-05-11T23:03:08.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8700000000000001e-06 2024-05-11T23:03:08.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606074+0000", 2024-05-11T23:03:08.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6230000000000001e-06 2024-05-11T23:03:08.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663510+0000", 2024-05-11T23:03:08.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05743529 2024-05-11T23:03:08.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663517+0000", 2024-05-11T23:03:08.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1230000000000002e-06 2024-05-11T23:03:08.941 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.941 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.941 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.941 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172797 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:08.941 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:08.941 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 10.243888804999999, 2024-05-11T23:03:08.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057470420000000001, 2024-05-11T23:03:08.941 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:08.941 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:08.941 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:08.941 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:08.942 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:08.942 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:08.942 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172797 2024-05-11T23:03:08.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.942 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:08.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:08.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:08.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:08.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:08.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:08.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:08.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607915+0000", 2024-05-11T23:03:08.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018375609999999999 2024-05-11T23:03:08.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:08.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607920+0000", 2024-05-11T23:03:08.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8899999999999998e-06 2024-05-11T23:03:08.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:08.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607930+0000", 2024-05-11T23:03:08.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4429999999999992e-06 2024-05-11T23:03:08.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:08.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607936+0000", 2024-05-11T23:03:08.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7089999999999996e-06 2024-05-11T23:03:08.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:08.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663541+0000", 2024-05-11T23:03:08.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055604537000000002 2024-05-11T23:03:08.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:08.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:08.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:08.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663548+0000", 2024-05-11T23:03:08.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2799999999999998e-06 2024-05-11T23:03:08.945 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.945 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.945 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.945 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:08.946 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:08.946 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:08.947 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:03:08.949 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:03:08.949 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:08.947+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:09.048 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:09.048+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:09.102 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:09.102 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:09.102 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:09.103 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:09.112 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:03:09.118 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:09.119 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:09.138 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:09.138 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:09.149 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:09.149+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:09.155 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:09.156 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:09.173 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-05-11T23:03:09.174 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:03:09.174 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:03:09.177 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.2 primary-affinity to 0.798659 (8523402) 2024-05-11T23:03:09.197 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:09.197 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:09.215 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:09.229 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:09.229 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:09.233 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:09.239 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:03:09.250 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:09.250+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:09.253 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:09.253 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:09.351 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:09.351+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:09.362 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:09.362 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:09.371 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:03:09.452 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:09.452+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:09.479 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:09.480 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:09.489 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:03:09.553 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:09.553+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:09.639 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:09.639 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:09.648 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:03:09.654 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:09.654+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:09.754 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:09.754+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:09.757 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:09.758 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:09.766 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:03:09.856 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:09.855+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:09.884 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:09.884 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:09.893 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:03:09.957 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:09.957+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:10.040 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:10.041 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:10.049 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:03:10.093 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:10.093+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:10.153 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:10.153 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:10.158 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:10.158+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:10.162 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:03:10.235 INFO:teuthology.orchestra.run.smithi005.stdout:flattening image image_client.0-clone47 2024-05-11T23:03:10.258 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:10.258+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:10.318 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:10.319 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:10.327 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:03:10.359 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:10.359+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:10.413 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #46, image image_client.0-clone46 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent47 2024-05-11T23:03:10.460 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:10.460+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:10.483 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:10.483 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:10.492 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:03:10.576 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:10.576+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:10.637 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:10.637 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:10.646 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:03:10.662 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:10.662+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:10.678 INFO:teuthology.orchestra.run.smithi005.stdout:5147 read 0x437ad2 thru 0x4423a1 (0xa8d0 bytes) 2024-05-11T23:03:10.692 INFO:teuthology.orchestra.run.smithi005.stdout:5148 write 0xd8c0e74 thru 0xd8cac0e (0x9d9b bytes) 2024-05-11T23:03:10.696 INFO:teuthology.orchestra.run.smithi005.stdout:5149 writesame 0xcac5fd9 thru 0xcad3b1f data_size 0x67(0xdb47 bytes) 2024-05-11T23:03:10.700 INFO:teuthology.orchestra.run.smithi005.stdout:5150 read 0x8d50331 thru 0x8d5eacf (0xe79f bytes) 2024-05-11T23:03:10.707 INFO:teuthology.orchestra.run.smithi005.stdout:5152 trunc from 0xd8cac0f to 0x78e3e5e 2024-05-11T23:03:10.737 INFO:teuthology.orchestra.run.smithi005.stdout:5153 write 0x3eb6d19 thru 0x3ebb4ce (0x47b6 bytes) 2024-05-11T23:03:10.742 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:10.742 INFO:teuthology.orchestra.run.smithi005.stdout:5156 read 0x6b058d5 thru 0x6b0dcf0 (0x841c bytes) 2024-05-11T23:03:10.742 INFO:teuthology.orchestra.run.smithi005.stdout:5157 write 0x3b2d140 thru 0x3b2dddf (0xca0 bytes) 2024-05-11T23:03:10.749 INFO:teuthology.orchestra.run.smithi005.stdout:5158 trunc from 0x78e3e5e to 0x4e8a306 2024-05-11T23:03:10.764 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:10.764+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:10.779 INFO:teuthology.orchestra.run.smithi005.stdout:5159 punch from 0x2b80785 to 0x2b82cc0, (0x253b bytes) 2024-05-11T23:03:10.779 INFO:teuthology.orchestra.run.smithi005.stdout:5160 write 0x229999e thru 0x22a4df4 (0xb457 bytes) 2024-05-11T23:03:10.784 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:10.786 INFO:teuthology.orchestra.run.smithi005.stdout:5163 trunc from 0x4e8a306 to 0x64cb4e2 2024-05-11T23:03:10.792 INFO:teuthology.orchestra.run.smithi005.stdout:5165 write 0x273b9a6 thru 0x273fab4 (0x410f bytes) 2024-05-11T23:03:10.807 INFO:teuthology.orchestra.run.smithi005.stdout:5166 trunc from 0x64cb4e2 to 0x13996e4 2024-05-11T23:03:10.813 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:10.814 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:10.827 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:03:10.849 INFO:teuthology.orchestra.run.smithi005.stdout:5167 read 0x1101bd1 thru 0x110cf4e (0xb37e bytes) 2024-05-11T23:03:10.850 INFO:teuthology.orchestra.run.smithi005.stdout:5169 punch from 0x6c95d0 to 0x6d724d, (0xdc7d bytes) 2024-05-11T23:03:10.850 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:10.872 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:10.872+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:10.873 INFO:teuthology.orchestra.run.smithi005.stdout:5171 writesame 0xea710fe thru 0xea80883 data_size 0x76(0xf786 bytes) 2024-05-11T23:03:10.875 INFO:teuthology.orchestra.run.smithi005.stdout:5172 trunc from 0xea80884 to 0xc193b58 2024-05-11T23:03:10.883 INFO:teuthology.orchestra.run.smithi005.stdout:5175 read 0x777c5f5 thru 0x778458b (0x7f97 bytes) 2024-05-11T23:03:10.883 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:10.886 INFO:teuthology.orchestra.run.smithi005.stdout:5181 trunc from 0xc193b58 to 0x8fc9ac2 2024-05-11T23:03:10.896 INFO:teuthology.orchestra.run.smithi005.stdout:5183 trunc from 0x8fc9ac2 to 0xaa6fb67 2024-05-11T23:03:10.904 INFO:teuthology.orchestra.run.smithi005.stdout:5184 write 0x1890af2 thru 0x189bd3a (0xb249 bytes) 2024-05-11T23:03:10.916 INFO:teuthology.orchestra.run.smithi005.stdout:5185 writesame 0xdce62f4 thru 0xdceabb3 data_size 0x18(0x48c0 bytes) 2024-05-11T23:03:10.916 INFO:teuthology.orchestra.run.smithi005.stdout:5187 write 0x63f40fd thru 0x6403188 (0xf08c bytes) 2024-05-11T23:03:10.922 INFO:teuthology.orchestra.run.smithi005.stdout:5188 writesame 0x1aae442 thru 0x1aaffae data_size 0x11(0x1b6d bytes) 2024-05-11T23:03:10.936 INFO:teuthology.orchestra.run.smithi005.stdout:5189 trunc from 0xdceabb4 to 0x319d7eb 2024-05-11T23:03:10.943 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:10.943 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:10.948 INFO:teuthology.orchestra.run.smithi005.stdout:5190 read 0x8cff05 thru 0x8d80d3 (0x81cf bytes) 2024-05-11T23:03:10.948 INFO:teuthology.orchestra.run.smithi005.stdout:5191 trunc from 0x319d7eb to 0x3304593 2024-05-11T23:03:10.953 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:03:10.954 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:10.955 INFO:teuthology.orchestra.run.smithi005.stdout:5195 read 0x2a97a40 thru 0x2aa678b (0xed4c bytes) 2024-05-11T23:03:10.955 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:10.956 INFO:teuthology.orchestra.run.smithi005.stdout:5197 trunc from 0x3304593 to 0x3fdf717 2024-05-11T23:03:10.962 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:10.965 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:10.965+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:10.981 INFO:teuthology.orchestra.run.smithi005.stdout:5200 write 0xee3ec68 thru 0xee4e6a0 (0xfa39 bytes) 2024-05-11T23:03:10.984 INFO:teuthology.orchestra.run.smithi005.stdout:5201 read 0x327e3f7 thru 0x3283b38 (0x5742 bytes) 2024-05-11T23:03:10.984 INFO:teuthology.orchestra.run.smithi005.stdout:5203 write 0xdf20b0f thru 0xdf28e20 (0x8312 bytes) 2024-05-11T23:03:10.988 INFO:teuthology.orchestra.run.smithi005.stdout:5205 write 0xdd8c2d8 thru 0xdd9b562 (0xf28b bytes) 2024-05-11T23:03:10.991 INFO:teuthology.orchestra.run.smithi005.stdout:5207 write 0x5cfe66d thru 0x5d0a2f7 (0xbc8b bytes) 2024-05-11T23:03:10.995 INFO:teuthology.orchestra.run.smithi005.stdout:5208 writesame 0xccb4e71 thru 0xccc242c data_size 0x2(0xd5bc bytes) 2024-05-11T23:03:11.025 INFO:teuthology.orchestra.run.smithi005.stdout:5209 trunc from 0xee4e6a1 to 0x20521cf 2024-05-11T23:03:11.058 INFO:teuthology.orchestra.run.smithi005.stdout:5213 write 0xe2f8a6c thru 0xe3063a7 (0xd93c bytes) 2024-05-11T23:03:11.062 INFO:teuthology.orchestra.run.smithi005.stdout:5215 write 0x49f682a thru 0x49fb6d4 (0x4eab bytes) 2024-05-11T23:03:11.065 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:11.065 INFO:teuthology.orchestra.run.smithi005.stdout:5217 punch from 0xd228936 to 0xd22e78e, (0x5e58 bytes) 2024-05-11T23:03:11.065 INFO:teuthology.orchestra.run.smithi005.stdout:5218 read 0xa676064 thru 0xa678b8e (0x2b2b bytes) 2024-05-11T23:03:11.065 INFO:teuthology.orchestra.run.smithi005.stdout:5219 punch from 0x40483da to 0x40552f7, (0xcf1d bytes) 2024-05-11T23:03:11.065 INFO:teuthology.orchestra.run.smithi005.stdout:5220 punch from 0xd5a08ff to 0xd5ab20d, (0xa90e bytes) 2024-05-11T23:03:11.066 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:11.066 INFO:teuthology.orchestra.run.smithi005.stdout:5223 write 0xbb9f7a7 thru 0xbbac709 (0xcf63 bytes) 2024-05-11T23:03:11.066 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:11.067+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:11.068 INFO:teuthology.orchestra.run.smithi005.stdout:5224 writesame 0xb2f5108 thru 0xb3027d8 data_size 0x3(0xd6d1 bytes) 2024-05-11T23:03:11.076 INFO:teuthology.orchestra.run.smithi005.stdout:5225 writesame 0x4d31bda thru 0x4d39721 data_size 0x28(0x7b48 bytes) 2024-05-11T23:03:11.081 INFO:teuthology.orchestra.run.smithi005.stdout:5227 read 0x909cd51 thru 0x90ab0f7 (0xe3a7 bytes) 2024-05-11T23:03:11.081 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:11.081 INFO:teuthology.orchestra.run.smithi005.stdout:5229 punch from 0xce859d0 to 0xce875c6, (0x1bf6 bytes) 2024-05-11T23:03:11.081 INFO:teuthology.orchestra.run.smithi005.stdout:5230 read 0x4709eef thru 0x470c46d (0x257f bytes) 2024-05-11T23:03:11.081 INFO:teuthology.orchestra.run.smithi005.stdout:5231 read 0x5404a84 thru 0x5411c6e (0xd1eb bytes) 2024-05-11T23:03:11.081 INFO:teuthology.orchestra.run.smithi005.stdout:5232 read 0x18287d8 thru 0x183014e (0x7977 bytes) 2024-05-11T23:03:11.082 INFO:teuthology.orchestra.run.smithi005.stdout:5233 writesame 0x2a5d18c thru 0x2a5e6f2 data_size 0x1(0x1567 bytes) 2024-05-11T23:03:11.090 INFO:teuthology.orchestra.run.smithi005.stdout:5234 trunc from 0xe3063a8 to 0x9007fae 2024-05-11T23:03:11.109 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:11.109 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:11.111 INFO:teuthology.orchestra.run.smithi005.stdout:5235 write 0xc340751 thru 0xc34d6e0 (0xcf90 bytes) 2024-05-11T23:03:11.114 INFO:teuthology.orchestra.run.smithi005.stdout:5237 writesame 0x478d925 thru 0x4799fa7 data_size 0x1(0xc683 bytes) 2024-05-11T23:03:11.119 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:03:11.136 INFO:teuthology.orchestra.run.smithi005.stdout:5238 read 0x65db163 thru 0x65ea8b7 (0xf755 bytes) 2024-05-11T23:03:11.136 INFO:teuthology.orchestra.run.smithi005.stdout:5240 read 0x2df4684 thru 0x2df9c2d (0x55aa bytes) 2024-05-11T23:03:11.137 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:11.137 INFO:teuthology.orchestra.run.smithi005.stdout:5246 write 0xb82f0db thru 0xb837e15 (0x8d3b bytes) 2024-05-11T23:03:11.140 INFO:teuthology.orchestra.run.smithi005.stdout:5248 write 0xaf8043 thru 0xb00ae8 (0x8aa6 bytes) 2024-05-11T23:03:11.142 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:11.142 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:11.142 INFO:teuthology.orchestra.run.smithi005.stdout:5251 read 0x83bf48e thru 0x83c7608 (0x817b bytes) 2024-05-11T23:03:11.142 INFO:teuthology.orchestra.run.smithi005.stdout:5252 write 0xc098b1c thru 0xc09a218 (0x16fd bytes) 2024-05-11T23:03:11.145 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:11.145 INFO:teuthology.orchestra.run.smithi005.stdout:5255 writesame 0xc315e60 thru 0xc31bec2 data_size 0x4b(0x6063 bytes) 2024-05-11T23:03:11.147 INFO:teuthology.orchestra.run.smithi005.stdout:5257 write 0x891e9b8 thru 0x892deb0 (0xf4f9 bytes) 2024-05-11T23:03:11.152 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:11.152 INFO:teuthology.orchestra.run.smithi005.stdout:5261 read 0x28cdf69 thru 0x28ce00d (0xa5 bytes) 2024-05-11T23:03:11.153 INFO:teuthology.orchestra.run.smithi005.stdout:5262 read 0xac6f1e1 thru 0xac7e9ce (0xf7ee bytes) 2024-05-11T23:03:11.153 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:11.153 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:11.156 INFO:teuthology.orchestra.run.smithi005.stdout:5268 trunc from 0xc34d6e1 to 0xedf7ca6 2024-05-11T23:03:11.161 INFO:teuthology.orchestra.run.smithi005.stdout:5269 writesame 0x82acae0 thru 0x82b773f data_size 0x70(0xac60 bytes) 2024-05-11T23:03:11.163 INFO:teuthology.orchestra.run.smithi005.stdout:5270 write 0x9bb0f47 thru 0x9bbdda1 (0xce5b bytes) 2024-05-11T23:03:11.167 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:11.167+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:11.167 INFO:teuthology.orchestra.run.smithi005.stdout:5271 write 0x4f1b368 thru 0x4f1dabc (0x2755 bytes) 2024-05-11T23:03:11.170 INFO:teuthology.orchestra.run.smithi005.stdout:5272 writesame 0x2e3ca68 thru 0x2e4896c data_size 0x1(0xbf05 bytes) 2024-05-11T23:03:11.193 INFO:teuthology.orchestra.run.smithi005.stdout:5273 writesame 0xa937f54 thru 0xa945410 data_size 0x1(0xd4bd bytes) 2024-05-11T23:03:11.195 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-11T23:03:11.220 INFO:teuthology.orchestra.run.smithi005.stdout:5274 write 0xdb70154 thru 0xdb7ffe7 (0xfe94 bytes) 2024-05-11T23:03:11.223 INFO:teuthology.orchestra.run.smithi005.stdout:5276 writesame 0xc98fd86 thru 0xc993049 data_size 0x39(0x32c4 bytes) 2024-05-11T23:03:11.224 INFO:teuthology.orchestra.run.smithi005.stdout:5277 write 0x87d6b2f thru 0x87d9444 (0x2916 bytes) 2024-05-11T23:03:11.228 INFO:teuthology.orchestra.run.smithi005.stdout:5278 writesame 0xb816d5d thru 0xb81ef58 data_size 0x8d(0x81fc bytes) 2024-05-11T23:03:11.229 INFO:teuthology.orchestra.run.smithi005.stdout:5279 read 0x73a70a2 thru 0x73ae42e (0x738d bytes) 2024-05-11T23:03:11.229 INFO:teuthology.orchestra.run.smithi005.stdout:5280 read 0xc93f26f thru 0xc94293d (0x36cf bytes) 2024-05-11T23:03:11.230 INFO:teuthology.orchestra.run.smithi005.stdout:5281 write 0x26bbb50 thru 0x26bde0f (0x22c0 bytes) 2024-05-11T23:03:11.232 INFO:teuthology.orchestra.run.smithi005.stdout:5282 trunc from 0xedf7ca6 to 0xda91625 2024-05-11T23:03:11.233 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:11.233 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:11.239 INFO:teuthology.orchestra.run.smithi005.stdout:5283 read 0x2c4c04f thru 0x2c55740 (0x96f2 bytes) 2024-05-11T23:03:11.239 INFO:teuthology.orchestra.run.smithi005.stdout:5284 writesame 0x1cf0afd thru 0x1cff6fe data_size 0x11(0xec02 bytes) 2024-05-11T23:03:11.242 INFO:teuthology.orchestra.run.smithi005.stdout:5285 read 0x82b494e thru 0x82bd639 (0x8cec bytes) 2024-05-11T23:03:11.243 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:03:11.244 INFO:teuthology.orchestra.run.smithi005.stdout:5286 punch from 0x20654a7 to 0x206dbee, (0x8747 bytes) 2024-05-11T23:03:11.244 INFO:teuthology.orchestra.run.smithi005.stdout:5287 clone 49 order 20 su 4096 sc 9 2024-05-11T23:03:11.267 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:03:11.268 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:11.269+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:11.349 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:11.350 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:11.358 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:03:11.370 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:11.370+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:11.396 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:11.397 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:11.414 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:11.415 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:11.431 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:11.431 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:11.448 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:03:11.448 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:03:11.464 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:11.464 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:11.465 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:11.466 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:11.471 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:11.471+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:11.473 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:03:11.483 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:11.520 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:11.553 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:11.572 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:11.572+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:11.584 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:11.584 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:11.593 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:03:11.673 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:11.673+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:11.742 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:11.743 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:11.743 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:11.743 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:11.743 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:11.751 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:03:11.774 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:11.774+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226051 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6049792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5450263839999998, 2024-05-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050622418000000002, 2024-05-11T23:03:11.863 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226051 2024-05-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317156+0000", 2024-05-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317157+0000", 2024-05-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.144e-06 2024-05-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317160+0000", 2024-05-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1350000000000001e-06 2024-05-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317162+0000", 2024-05-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5050000000000002e-06 2024-05-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367689+0000", 2024-05-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526797999999998 2024-05-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367735+0000", 2024-05-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5698999999999997e-05 2024-05-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367778+0000", 2024-05-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2928e-05 2024-05-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.867 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226052 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6060032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5450163279999991, 2024-05-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050830110999999997, 2024-05-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226052 2024-05-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:11.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317166+0000", 2024-05-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317167+0000", 2024-05-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.155e-06 2024-05-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317170+0000", 2024-05-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.061e-06 2024-05-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317172+0000", 2024-05-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4420000000000001e-06 2024-05-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367816+0000", 2024-05-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050644013000000002 2024-05-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367906+0000", 2024-05-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9739000000000003e-05 2024-05-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367995+0000", 2024-05-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9499000000000005e-05 2024-05-11T23:03:11.871 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226053 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6070272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5450062889999998, 2024-05-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051106912999999997, 2024-05-11T23:03:11.872 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226053 2024-05-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:11.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317176+0000", 2024-05-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0100000000000001e-07 2024-05-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:11.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317179+0000", 2024-05-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-05-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317182+0000", 2024-05-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3589999999999999e-06 2024-05-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317184+0000", 2024-05-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4339999999999999e-06 2024-05-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:11.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368061+0000", 2024-05-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050876927000000002 2024-05-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368179+0000", 2024-05-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118001 2024-05-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368282+0000", 2024-05-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102942 2024-05-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226054 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6080512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5449939669999999, 2024-05-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051322, 2024-05-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226054 2024-05-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:11.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:11.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317189+0000", 2024-05-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.217e-06 2024-05-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317192+0000", 2024-05-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0699999999999998e-06 2024-05-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317195+0000", 2024-05-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3829999999999999e-06 2024-05-11T23:03:11.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368328+0000", 2024-05-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051132999999999998 2024-05-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368429+0000", 2024-05-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010168800000000001 2024-05-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368510+0000", 2024-05-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0420999999999995e-05 2024-05-11T23:03:11.880 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226055 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6090752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.544983899, 2024-05-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051562835000000001, 2024-05-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:11.881 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226055 2024-05-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317256+0000", 2024-05-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8047000000000002e-05 2024-05-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317259+0000", 2024-05-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2590000000000001e-06 2024-05-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317267+0000", 2024-05-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8769999999999993e-06 2024-05-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317274+0000", 2024-05-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.187e-06 2024-05-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368544+0000", 2024-05-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051270299999999998 2024-05-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368658+0000", 2024-05-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000113347 2024-05-11T23:03:11.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368761+0000", 2024-05-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102818 2024-05-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226056 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6100992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5448980530000007, 2024-05-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051615576000000003, 2024-05-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:11.885 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226056 2024-05-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:11.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318915+0000", 2024-05-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001631291 2024-05-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318918+0000", 2024-05-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.512e-06 2024-05-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318927+0000", 2024-05-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1960000000000008e-06 2024-05-11T23:03:11.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318932+0000", 2024-05-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368805+0000", 2024-05-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049872998000000002 2024-05-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368853+0000", 2024-05-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8041999999999998e-05 2024-05-11T23:03:11.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368899+0000", 2024-05-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5766000000000002e-05 2024-05-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226061 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6152192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5431069659999999, 2024-05-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050675395999999998, 2024-05-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:11.889 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226061 2024-05-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:11.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319078+0000", 2024-05-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5670000000000002e-06 2024-05-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319085+0000", 2024-05-11T23:03:11.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2570000000000001e-06 2024-05-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319100+0000", 2024-05-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.489e-05 2024-05-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369584+0000", 2024-05-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050484345999999999 2024-05-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:11.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369671+0000", 2024-05-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6654000000000001e-05 2024-05-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369750+0000", 2024-05-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9257000000000004e-05 2024-05-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226062 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6162432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5430734219999991, 2024-05-11T23:03:11.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050761158000000001, 2024-05-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226062 2024-05-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319109+0000", 2024-05-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2800000000000002e-07 2024-05-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319111+0000", 2024-05-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4980000000000001e-06 2024-05-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319120+0000", 2024-05-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3999999999999992e-06 2024-05-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319129+0000", 2024-05-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7580000000000004e-06 2024-05-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369775+0000", 2024-05-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050645580000000003 2024-05-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369824+0000", 2024-05-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9506999999999999e-05 2024-05-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369869+0000", 2024-05-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4987000000000001e-05 2024-05-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226063 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6172672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:11.897 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5430444649999995, 2024-05-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050939784000000002, 2024-05-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226063 2024-05-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:11.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319138+0000", 2024-05-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.39e-07 2024-05-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319140+0000", 2024-05-11T23:03:11.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4820000000000001e-06 2024-05-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319148+0000", 2024-05-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6599999999999995e-06 2024-05-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319154+0000", 2024-05-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4169999999999997e-06 2024-05-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369905+0000", 2024-05-11T23:03:11.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050751009999999999 2024-05-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369991+0000", 2024-05-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5909999999999996e-05 2024-05-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370077+0000", 2024-05-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5865999999999994e-05 2024-05-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226064 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6182912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5430182690000009, 2024-05-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051038241999999998, 2024-05-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226064 2024-05-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319164+0000", 2024-05-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9299999999999999e-07 2024-05-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319166+0000", 2024-05-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.48e-06 2024-05-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319176+0000", 2024-05-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9669999999999996e-06 2024-05-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319180+0000", 2024-05-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8020000000000002e-06 2024-05-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370100+0000", 2024-05-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050919845999999998 2024-05-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370155+0000", 2024-05-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5040000000000002e-05 2024-05-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370202+0000", 2024-05-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6714e-05 2024-05-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.905 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226065 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6193152~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5429956990000004, 2024-05-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051225087000000002, 2024-05-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226065 2024-05-11T23:03:11.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320913+0000", 2024-05-11T23:03:11.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001727492 2024-05-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320919+0000", 2024-05-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.964e-06 2024-05-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320927+0000", 2024-05-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2239999999999998e-06 2024-05-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320934+0000", 2024-05-11T23:03:11.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8469999999999998e-06 2024-05-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370240+0000", 2024-05-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049306562999999998 2024-05-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370328+0000", 2024-05-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8122999999999996e-05 2024-05-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370411+0000", 2024-05-11T23:03:11.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2874000000000006e-05 2024-05-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226073 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6275072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5411573119999993, 2024-05-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050730475999999997, 2024-05-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:11.911 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:11.911 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:11.911 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226073 2024-05-11T23:03:11.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.340 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:12.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:12.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322914+0000", 2024-05-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001889153 2024-05-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322917+0000", 2024-05-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2100000000000002e-06 2024-05-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322924+0000", 2024-05-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9929999999999998e-06 2024-05-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322928+0000", 2024-05-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5549999999999996e-06 2024-05-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371578+0000", 2024-05-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048650001999999998 2024-05-11T23:03:12.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371671+0000", 2024-05-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2230000000000003e-05 2024-05-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371755+0000", 2024-05-11T23:03:12.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4332999999999994e-05 2024-05-11T23:03:12.346 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.346 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:12.346 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.346 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226085 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 5338112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:12.347 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:12.347 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.5390662129999999, 2024-05-11T23:03:12.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050727622, 2024-05-11T23:03:12.347 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:12.347 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226085 2024-05-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:12.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324914+0000", 2024-05-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001798298 2024-05-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:12.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324918+0000", 2024-05-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3889999999999998e-06 2024-05-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324928+0000", 2024-05-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4339999999999998e-06 2024-05-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324936+0000", 2024-05-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2770000000000005e-06 2024-05-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373664+0000", 2024-05-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048728207000000003 2024-05-11T23:03:12.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373757+0000", 2024-05-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3332999999999996e-05 2024-05-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373843+0000", 2024-05-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5684000000000002e-05 2024-05-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:12.353 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232555 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7789568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.0620558879999997, 2024-05-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050583086999999999, 2024-05-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:12.354 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232555 2024-05-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800127+0000", 2024-05-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.207e-06 2024-05-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800132+0000", 2024-05-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:12.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800134+0000", 2024-05-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5459999999999998e-06 2024-05-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850548+0000", 2024-05-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050414182000000002 2024-05-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:12.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850635+0000", 2024-05-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6853000000000003e-05 2024-05-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850709+0000", 2024-05-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3585999999999996e-05 2024-05-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:12.359 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.360 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232556 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7799808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:12.360 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:12.360 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.062044234, 2024-05-11T23:03:12.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050749599999999999, 2024-05-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232556 2024-05-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:12.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801914+0000", 2024-05-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017766360000000001 2024-05-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801919+0000", 2024-05-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.42e-06 2024-05-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801927+0000", 2024-05-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1999999999999994e-06 2024-05-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801931+0000", 2024-05-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4279999999999998e-06 2024-05-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850740+0000", 2024-05-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048808658999999997 2024-05-11T23:03:12.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850819+0000", 2024-05-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9102000000000003e-05 2024-05-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850887+0000", 2024-05-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8155000000000001e-05 2024-05-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.365 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232578 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8025088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.0587509960000006, 2024-05-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050679473000000003, 2024-05-11T23:03:12.366 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232578 2024-05-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:12.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:12.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:12.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803432+0000", 2024-05-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.139e-06 2024-05-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:12.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803435+0000", 2024-05-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2440000000000002e-06 2024-05-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803439+0000", 2024-05-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3129999999999999e-06 2024-05-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:12.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.853953+0000", 2024-05-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050514846000000002 2024-05-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854035+0000", 2024-05-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1148000000000003e-05 2024-05-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854110+0000", 2024-05-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5562000000000006e-05 2024-05-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:12.372 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232579 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8035328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.0587383920000004, 2024-05-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050886656000000002, 2024-05-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:12.373 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232579 2024-05-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:12.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803914+0000", 2024-05-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00047026700000000002 2024-05-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:12.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803918+0000", 2024-05-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3460000000000004e-06 2024-05-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803927+0000", 2024-05-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5030000000000003e-06 2024-05-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803933+0000", 2024-05-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0310000000000004e-06 2024-05-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:12.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854153+0000", 2024-05-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050219321999999997 2024-05-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854242+0000", 2024-05-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9457999999999996e-05 2024-05-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854330+0000", 2024-05-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7729e-05 2024-05-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232581 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8055808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.0582149150000006, 2024-05-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050664439999999998, 2024-05-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232581 2024-05-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.378 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803969+0000", 2024-05-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0889999999999998e-06 2024-05-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803978+0000", 2024-05-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3070000000000003e-06 2024-05-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803983+0000", 2024-05-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1599999999999997e-06 2024-05-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854509+0000", 2024-05-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526339000000003 2024-05-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854576+0000", 2024-05-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7102999999999998e-05 2024-05-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854631+0000", 2024-05-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4997999999999998e-05 2024-05-11T23:03:12.381 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232582 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8066048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.0581921699999999, 2024-05-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050753983000000003, 2024-05-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:12.382 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232582 2024-05-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805917+0000", 2024-05-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001927432 2024-05-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805920+0000", 2024-05-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1219999999999999e-06 2024-05-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805933+0000", 2024-05-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2727e-05 2024-05-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:12.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805941+0000", 2024-05-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7090000000000002e-06 2024-05-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854657+0000", 2024-05-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048716164999999999 2024-05-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854699+0000", 2024-05-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2531999999999999e-05 2024-05-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:12.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854744+0000", 2024-05-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4295999999999997e-05 2024-05-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232593 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8178688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.0559927770000002, 2024-05-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050735178999999998, 2024-05-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:12.386 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232593 2024-05-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807913+0000", 2024-05-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017240319999999999 2024-05-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807916+0000", 2024-05-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4589999999999998e-06 2024-05-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807924+0000", 2024-05-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.272e-06 2024-05-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807928+0000", 2024-05-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5369999999999999e-06 2024-05-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856768+0000", 2024-05-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048839489999999999 2024-05-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856845+0000", 2024-05-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7746999999999999e-05 2024-05-11T23:03:12.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856924+0000", 2024-05-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8641999999999997e-05 2024-05-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:12.390 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:12.392 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:03:12.393 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-05-11T23:03:12.393 INFO:teuthology.orchestra.run.smithi005.stdout:truncating image image_client.0-clone48 from 0xda91625 (overlap 0x13996e4) to 0x1f5be3 2024-05-11T23:03:12.393 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #47, image image_client.0-clone47 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent48 2024-05-11T23:03:12.394 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:12.391+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:12.492 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:12.492+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:12.546 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:12.546 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:12.547 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:12.547 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:12.555 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:03:12.559 INFO:teuthology.orchestra.run.smithi005.stdout:5289 punch from 0xc1a0502 to 0xc1a050c, (0xa bytes) 2024-05-11T23:03:12.564 INFO:teuthology.orchestra.run.smithi005.stdout:5292 write 0xa21f335 thru 0xa21f718 (0x3e4 bytes) 2024-05-11T23:03:12.574 INFO:teuthology.orchestra.run.smithi005.stdout:5293 write 0xb433d5a thru 0xb43ee08 (0xb0af bytes) 2024-05-11T23:03:12.593 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:12.593+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:12.648 INFO:teuthology.orchestra.run.smithi005.stdout:5295 write 0xa2d387d thru 0xa2e3676 (0xfdfa bytes) 2024-05-11T23:03:12.694 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:12.693+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:12.710 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:12.711 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:12.711 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:12.711 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:12.711 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:12.711 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:12.724 INFO:teuthology.orchestra.run.smithi005.stdout:5297 trunc from 0xda91625 to 0x17a467d 2024-05-11T23:03:12.726 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:03:12.795 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:12.794+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:12.862 INFO:teuthology.orchestra.run.smithi005.stdout:5299 punch from 0xcd2201 to 0xcde025, (0xbe24 bytes) 2024-05-11T23:03:12.862 INFO:teuthology.orchestra.run.smithi005.stdout:5300 punch from 0xf1e08e to 0xf220cd, (0x403f bytes) 2024-05-11T23:03:12.862 INFO:teuthology.orchestra.run.smithi005.stdout:5301 punch from 0xd152f7 to 0xd20c0a, (0xb913 bytes) 2024-05-11T23:03:12.863 INFO:teuthology.orchestra.run.smithi005.stdout:5302 flatten 2024-05-11T23:03:12.897 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:12.896+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:12.947 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:12.948 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:12.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:12.948 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:12.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.948 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:64049 3.8 3:11d98727:::rbd_object_map.111e8974a681:head [read 0~38] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-05-11T23:03:12.948 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:12.949 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 72.164926609000005, 2024-05-11T23:03:12.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99884604399999999, 2024-05-11T23:03:12.949 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:12.949 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:12.949 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:12.949 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:12.950 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:12.950 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:12.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 64049 2024-05-11T23:03:12.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.950 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:12.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:12.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:12.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:12.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:12.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:12.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779883+0000", 2024-05-11T23:03:12.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.57e-06 2024-05-11T23:03:12.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:12.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779887+0000", 2024-05-11T23:03:12.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5709999999999999e-06 2024-05-11T23:03:12.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:12.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779898+0000", 2024-05-11T23:03:12.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.042e-05 2024-05-11T23:03:12.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:12.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779905+0000", 2024-05-11T23:03:12.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0110000000000003e-06 2024-05-11T23:03:12.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:12.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779962+0000", 2024-05-11T23:03:12.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7395999999999997e-05 2024-05-11T23:03:12.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:12.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779969+0000", 2024-05-11T23:03:12.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2459999999999997e-06 2024-05-11T23:03:12.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:12.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778379+0000", 2024-05-11T23:03:12.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99840986300000001 2024-05-11T23:03:12.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:12.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778564+0000", 2024-05-11T23:03:12.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018420900000000001 2024-05-11T23:03:12.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.956 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:12.956 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778727+0000", 2024-05-11T23:03:12.956 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016375800000000001 2024-05-11T23:03:12.956 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.956 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:12.956 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.956 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172773 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:12.956 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:12.957 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.340820191000001, 2024-05-11T23:03:12.957 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058721316000000003, 2024-05-11T23:03:12.957 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:12.957 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:12.957 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:12.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:12.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:12.957 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:12.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172773 2024-05-11T23:03:12.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.957 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:12.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.957 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:12.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:12.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:12.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:12.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.958 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:12.958 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:12.958 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3500000000000002e-07 2024-05-11T23:03:12.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:12.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603990+0000", 2024-05-11T23:03:12.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3549999999999999e-06 2024-05-11T23:03:12.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:12.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603997+0000", 2024-05-11T23:03:12.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3400000000000003e-06 2024-05-11T23:03:12.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:12.959 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604002+0000", 2024-05-11T23:03:12.959 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0509999999999996e-06 2024-05-11T23:03:12.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.959 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:12.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662697+0000", 2024-05-11T23:03:12.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058695487999999997 2024-05-11T23:03:12.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.960 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:12.960 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662709+0000", 2024-05-11T23:03:12.960 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1647e-05 2024-05-11T23:03:12.960 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.960 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:12.960 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.960 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172774 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:12.960 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:12.960 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.340798809000001, 2024-05-11T23:03:12.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058745040999999998, 2024-05-11T23:03:12.961 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:12.961 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:12.961 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:12.961 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:12.961 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:12.961 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:12.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172774 2024-05-11T23:03:12.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.961 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:12.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.961 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:12.961 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:12.961 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:12.962 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:12.962 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.962 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.962 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:12.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:12.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:03:12.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:12.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604012+0000", 2024-05-11T23:03:12.963 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2199999999999999e-06 2024-05-11T23:03:12.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.963 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.963 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:12.963 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604018+0000", 2024-05-11T23:03:12.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1770000000000003e-06 2024-05-11T23:03:12.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:12.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604023+0000", 2024-05-11T23:03:12.964 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0089999999999998e-06 2024-05-11T23:03:12.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.964 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.964 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:12.964 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662742+0000", 2024-05-11T23:03:12.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058719187999999999 2024-05-11T23:03:12.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.965 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:12.965 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662754+0000", 2024-05-11T23:03:12.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2095e-05 2024-05-11T23:03:12.965 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.965 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:12.965 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.965 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172775 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:12.965 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:12.965 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.340778597, 2024-05-11T23:03:12.965 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058764743000000001, 2024-05-11T23:03:12.966 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:12.966 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:12.966 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:12.966 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:12.966 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:12.967 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:12.967 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172775 2024-05-11T23:03:12.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.968 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:12.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.968 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:12.968 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:12.968 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:12.969 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:12.969 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.969 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:12.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604030+0000", 2024-05-11T23:03:12.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6699999999999999e-07 2024-05-11T23:03:12.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.970 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:12.970 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604032+0000", 2024-05-11T23:03:12.970 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9530000000000002e-06 2024-05-11T23:03:12.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.971 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:12.971 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604037+0000", 2024-05-11T23:03:12.971 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8590000000000001e-06 2024-05-11T23:03:12.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:12.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604042+0000", 2024-05-11T23:03:12.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8609999999999997e-06 2024-05-11T23:03:12.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.972 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:12.972 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662782+0000", 2024-05-11T23:03:12.972 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058739731000000003 2024-05-11T23:03:12.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.973 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:12.973 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662794+0000", 2024-05-11T23:03:12.973 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1972e-05 2024-05-11T23:03:12.973 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.973 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:12.973 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.974 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172776 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:12.974 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:12.974 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.340759307000001, 2024-05-11T23:03:12.974 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058778696999999998, 2024-05-11T23:03:12.974 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:12.974 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:12.974 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:12.974 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:12.974 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:12.974 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:12.974 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172776 2024-05-11T23:03:12.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.974 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:12.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:12.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:12.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.975 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.975 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:12.975 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:12.975 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:12.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:12.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2000000000000001e-07 2024-05-11T23:03:12.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.976 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:12.976 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604051+0000", 2024-05-11T23:03:12.976 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2879999999999999e-06 2024-05-11T23:03:12.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.976 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:12.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604057+0000", 2024-05-11T23:03:12.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:03:12.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:12.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604062+0000", 2024-05-11T23:03:12.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6800000000000001e-06 2024-05-11T23:03:12.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.977 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.977 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:12.977 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662820+0000", 2024-05-11T23:03:12.977 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058758444999999999 2024-05-11T23:03:12.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.978 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:12.978 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662827+0000", 2024-05-11T23:03:12.978 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1930000000000004e-06 2024-05-11T23:03:12.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.978 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:12.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.978 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172777 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:12.979 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:12.979 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.340740001, 2024-05-11T23:03:12.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058799918999999999, 2024-05-11T23:03:12.979 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:12.979 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:12.979 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:12.979 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:12.979 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:12.979 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:12.979 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172777 2024-05-11T23:03:12.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.979 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:12.979 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.979 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:12.979 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:12.979 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:12.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:12.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:12.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:12.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:03:12.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.980 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.980 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:12.980 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604075+0000", 2024-05-11T23:03:12.980 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.109e-06 2024-05-11T23:03:12.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:12.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604083+0000", 2024-05-11T23:03:12.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9170000000000006e-06 2024-05-11T23:03:12.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:12.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604089+0000", 2024-05-11T23:03:12.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3399999999999997e-06 2024-05-11T23:03:12.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.981 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:12.981 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662859+0000", 2024-05-11T23:03:12.981 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058770172000000002 2024-05-11T23:03:12.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.982 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:12.982 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662868+0000", 2024-05-11T23:03:12.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0289999999999994e-06 2024-05-11T23:03:12.982 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.982 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:12.982 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.982 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172778 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:12.982 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:12.982 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.340711911, 2024-05-11T23:03:12.982 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058845435000000001, 2024-05-11T23:03:12.982 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:12.982 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:12.983 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:12.983 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:12.983 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:12.983 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:12.983 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172778 2024-05-11T23:03:12.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.983 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:12.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:12.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:12.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.983 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:12.983 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:12.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:12.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:12.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5400000000000002e-07 2024-05-11T23:03:12.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:12.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604099+0000", 2024-05-11T23:03:12.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4650000000000001e-06 2024-05-11T23:03:12.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:12.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604105+0000", 2024-05-11T23:03:12.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.314e-06 2024-05-11T23:03:12.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:12.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604110+0000", 2024-05-11T23:03:12.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6310000000000002e-06 2024-05-11T23:03:12.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:12.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662931+0000", 2024-05-11T23:03:12.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058820960999999998 2024-05-11T23:03:12.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:12.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662941+0000", 2024-05-11T23:03:12.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.061e-05 2024-05-11T23:03:12.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.986 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:12.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.986 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172779 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:12.986 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:12.986 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.340691751, 2024-05-11T23:03:12.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058859163999999999, 2024-05-11T23:03:12.986 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:12.986 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:12.986 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:12.986 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:12.986 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:12.987 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:12.987 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172779 2024-05-11T23:03:12.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.987 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:12.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:12.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:12.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:12.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:12.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:12.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:12.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:03:12.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:12.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604119+0000", 2024-05-11T23:03:12.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2529999999999999e-06 2024-05-11T23:03:12.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:12.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604125+0000", 2024-05-11T23:03:12.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1670000000000004e-06 2024-05-11T23:03:12.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:12.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604130+0000", 2024-05-11T23:03:12.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.702e-06 2024-05-11T23:03:12.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:12.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662967+0000", 2024-05-11T23:03:12.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058837792999999999 2024-05-11T23:03:12.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:12.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662975+0000", 2024-05-11T23:03:12.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8490000000000006e-06 2024-05-11T23:03:12.990 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.990 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:12.990 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.990 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172780 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:12.990 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:12.990 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.340671601, 2024-05-11T23:03:12.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058872696000000002, 2024-05-11T23:03:12.990 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:12.990 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:12.990 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:12.991 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:12.991 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:12.991 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:12.991 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172780 2024-05-11T23:03:12.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.991 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:12.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:12.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:12.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:12.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:12.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:12.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604137+0000", 2024-05-11T23:03:12.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4400000000000001e-07 2024-05-11T23:03:12.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:12.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604139+0000", 2024-05-11T23:03:12.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3429999999999998e-06 2024-05-11T23:03:12.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:12.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604145+0000", 2024-05-11T23:03:12.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1940000000000003e-06 2024-05-11T23:03:12.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:12.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604150+0000", 2024-05-11T23:03:12.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:03:12.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:12.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663002+0000", 2024-05-11T23:03:12.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058851829000000001 2024-05-11T23:03:12.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:12.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663009+0000", 2024-05-11T23:03:12.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.199e-06 2024-05-11T23:03:12.993 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.993 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:12.994 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.994 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172781 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:12.994 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:12.994 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.340639761, 2024-05-11T23:03:12.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058875553999999997, 2024-05-11T23:03:12.994 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:12.994 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:12.994 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:12.994 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:12.994 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:12.994 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:12.994 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172781 2024-05-11T23:03:12.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.995 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:12.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:12.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:12.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:12.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:12.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:12.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:12.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:12.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:12.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:12.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604171+0000", 2024-05-11T23:03:12.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.21e-06 2024-05-11T23:03:12.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:12.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604176+0000", 2024-05-11T23:03:12.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2970000000000003e-06 2024-05-11T23:03:12.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:12.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604179+0000", 2024-05-11T23:03:12.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8459999999999999e-06 2024-05-11T23:03:12.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:12.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663033+0000", 2024-05-11T23:03:12.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058854565999999997 2024-05-11T23:03:12.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:12.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663044+0000", 2024-05-11T23:03:12.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0229e-05 2024-05-11T23:03:12.998 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.998 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:12.998 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:12.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:12.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:12.999 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172782 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:12.999 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:12.999 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.340625013, 2024-05-11T23:03:12.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058894375999999998, 2024-05-11T23:03:12.999 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:12.999 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:12.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:12.999 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.000 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.000 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.000 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172782 2024-05-11T23:03:13.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.000 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:13.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:13.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:13.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-05-11T23:03:13.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604184+0000", 2024-05-11T23:03:13.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:03:13.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604187+0000", 2024-05-11T23:03:13.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.044e-06 2024-05-11T23:03:13.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604190+0000", 2024-05-11T23:03:13.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6460000000000002e-06 2024-05-11T23:03:13.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663070+0000", 2024-05-11T23:03:13.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058879510000000003 2024-05-11T23:03:13.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663077+0000", 2024-05-11T23:03:13.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6669999999999996e-06 2024-05-11T23:03:13.006 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.006 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.006 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.007 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172783 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:13.007 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:13.008 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.340614693999999, 2024-05-11T23:03:13.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058913946000000002, 2024-05-11T23:03:13.008 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.008 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.008 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:13.008 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.008 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.009 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.009 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172783 2024-05-11T23:03:13.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.009 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:13.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:13.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:13.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-05-11T23:03:13.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604195+0000", 2024-05-11T23:03:13.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.19e-06 2024-05-11T23:03:13.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604198+0000", 2024-05-11T23:03:13.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:03:13.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604200+0000", 2024-05-11T23:03:13.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4719999999999998e-06 2024-05-11T23:03:13.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663100+0000", 2024-05-11T23:03:13.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058900300000000003 2024-05-11T23:03:13.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663107+0000", 2024-05-11T23:03:13.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7449999999999998e-06 2024-05-11T23:03:13.013 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.014 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.014 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.014 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172784 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:13.014 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:13.014 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.340604675, 2024-05-11T23:03:13.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933728999999997, 2024-05-11T23:03:13.014 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.015 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:13.015 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.015 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.015 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.015 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172784 2024-05-11T23:03:13.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.015 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:13.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:13.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:13.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.03e-07 2024-05-11T23:03:13.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604205+0000", 2024-05-11T23:03:13.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2300000000000001e-06 2024-05-11T23:03:13.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604207+0000", 2024-05-11T23:03:13.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8729999999999999e-06 2024-05-11T23:03:13.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604210+0000", 2024-05-11T23:03:13.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.858e-06 2024-05-11T23:03:13.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663130+0000", 2024-05-11T23:03:13.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058919314 2024-05-11T23:03:13.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663137+0000", 2024-05-11T23:03:13.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2509999999999997e-06 2024-05-11T23:03:13.018 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.018 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.018 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.018 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172785 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:13.018 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:13.019 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.340592405000001, 2024-05-11T23:03:13.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058951664000000001, 2024-05-11T23:03:13.019 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.019 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.019 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:13.019 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.019 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.019 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.019 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172785 2024-05-11T23:03:13.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.019 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:13.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:13.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604216+0000", 2024-05-11T23:03:13.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:03:13.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604218+0000", 2024-05-11T23:03:13.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7549999999999999e-06 2024-05-11T23:03:13.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604222+0000", 2024-05-11T23:03:13.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6700000000000002e-06 2024-05-11T23:03:13.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604227+0000", 2024-05-11T23:03:13.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4399999999999998e-06 2024-05-11T23:03:13.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663160+0000", 2024-05-11T23:03:13.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933119999999999 2024-05-11T23:03:13.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663167+0000", 2024-05-11T23:03:13.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1840000000000002e-06 2024-05-11T23:03:13.022 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.022 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.022 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.022 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172786 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:13.022 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:13.022 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.340575351, 2024-05-11T23:03:13.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058966991000000003, 2024-05-11T23:03:13.023 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.023 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.023 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:13.023 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.023 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.023 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.023 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172786 2024-05-11T23:03:13.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.023 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:13.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:13.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604233+0000", 2024-05-11T23:03:13.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6899999999999999e-07 2024-05-11T23:03:13.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604235+0000", 2024-05-11T23:03:13.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8369999999999999e-06 2024-05-11T23:03:13.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604240+0000", 2024-05-11T23:03:13.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0329999999999999e-06 2024-05-11T23:03:13.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604244+0000", 2024-05-11T23:03:13.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9859999999999996e-06 2024-05-11T23:03:13.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663192+0000", 2024-05-11T23:03:13.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058948852000000003 2024-05-11T23:03:13.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663199+0000", 2024-05-11T23:03:13.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0140000000000001e-06 2024-05-11T23:03:13.026 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.026 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.026 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.026 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172787 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:13.026 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:13.026 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.340559004999999, 2024-05-11T23:03:13.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058982018999999997, 2024-05-11T23:03:13.026 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.026 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.026 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:13.027 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.027 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.027 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.027 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172787 2024-05-11T23:03:13.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.027 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:13.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:13.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605916+0000", 2024-05-11T23:03:13.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00166752 2024-05-11T23:03:13.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605921+0000", 2024-05-11T23:03:13.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0669999999999999e-06 2024-05-11T23:03:13.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605934+0000", 2024-05-11T23:03:13.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2271e-05 2024-05-11T23:03:13.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605940+0000", 2024-05-11T23:03:13.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6880000000000002e-06 2024-05-11T23:03:13.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663224+0000", 2024-05-11T23:03:13.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057283251 2024-05-11T23:03:13.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663231+0000", 2024-05-11T23:03:13.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2219999999999996e-06 2024-05-11T23:03:13.030 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.030 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.030 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.030 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172793 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:13.030 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:13.030 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.338792158, 2024-05-11T23:03:13.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057402373999999999, 2024-05-11T23:03:13.030 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.031 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.031 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:13.031 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.031 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.031 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.031 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172793 2024-05-11T23:03:13.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.031 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:13.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:13.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:13.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3300000000000001e-07 2024-05-11T23:03:13.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606017+0000", 2024-05-11T23:03:13.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4920000000000001e-06 2024-05-11T23:03:13.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606020+0000", 2024-05-11T23:03:13.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:03:13.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606023+0000", 2024-05-11T23:03:13.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6680000000000001e-06 2024-05-11T23:03:13.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663411+0000", 2024-05-11T23:03:13.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057388206999999997 2024-05-11T23:03:13.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663418+0000", 2024-05-11T23:03:13.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.815e-06 2024-05-11T23:03:13.035 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.035 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.035 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.035 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172795 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:13.036 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:13.036 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.338750730999999, 2024-05-11T23:03:13.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057423015000000001, 2024-05-11T23:03:13.036 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.036 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.036 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:13.036 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.036 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.036 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.036 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172795 2024-05-11T23:03:13.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.037 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:13.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:13.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:13.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9499999999999998e-07 2024-05-11T23:03:13.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606059+0000", 2024-05-11T23:03:13.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-05-11T23:03:13.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606062+0000", 2024-05-11T23:03:13.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.977e-06 2024-05-11T23:03:13.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606064+0000", 2024-05-11T23:03:13.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7410000000000001e-06 2024-05-11T23:03:13.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663473+0000", 2024-05-11T23:03:13.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057408624999999998 2024-05-11T23:03:13.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663480+0000", 2024-05-11T23:03:13.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1049999999999997e-06 2024-05-11T23:03:13.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.041 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.041 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172796 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:13.041 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:13.041 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.338740167999999, 2024-05-11T23:03:13.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057449235000000001, 2024-05-11T23:03:13.041 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.042 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.042 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:13.042 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.042 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.042 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.042 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172796 2024-05-11T23:03:13.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.043 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:13.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:13.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:13.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:13.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606069+0000", 2024-05-11T23:03:13.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:03:13.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606072+0000", 2024-05-11T23:03:13.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8700000000000001e-06 2024-05-11T23:03:13.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606074+0000", 2024-05-11T23:03:13.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6230000000000001e-06 2024-05-11T23:03:13.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663510+0000", 2024-05-11T23:03:13.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05743529 2024-05-11T23:03:13.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663517+0000", 2024-05-11T23:03:13.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1230000000000002e-06 2024-05-11T23:03:13.047 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.047 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.047 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.047 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172797 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:13.047 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:13.047 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 14.338730136000001, 2024-05-11T23:03:13.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057470420000000001, 2024-05-11T23:03:13.047 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.048 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.048 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:13.048 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.048 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.048 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.048 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172797 2024-05-11T23:03:13.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.048 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:13.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:13.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607915+0000", 2024-05-11T23:03:13.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018375609999999999 2024-05-11T23:03:13.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607920+0000", 2024-05-11T23:03:13.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8899999999999998e-06 2024-05-11T23:03:13.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607930+0000", 2024-05-11T23:03:13.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4429999999999992e-06 2024-05-11T23:03:13.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607936+0000", 2024-05-11T23:03:13.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7089999999999996e-06 2024-05-11T23:03:13.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663541+0000", 2024-05-11T23:03:13.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055604537000000002 2024-05-11T23:03:13.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663548+0000", 2024-05-11T23:03:13.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2799999999999998e-06 2024-05-11T23:03:13.052 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.052 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.052 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.052 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.053 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.053 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:13.054 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:03:13.054 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:13.054+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:13.073 INFO:teuthology.orchestra.run.smithi005.stdout:5303 punch from 0x1c994c to 0x1cf798, (0x5e4c bytes) 2024-05-11T23:03:13.093 INFO:teuthology.orchestra.run.smithi005.stdout:5304 writesame 0xdfff8d3 thru 0xdfffe91 data_size 0x1(0x5bf bytes) 2024-05-11T23:03:13.093 INFO:teuthology.orchestra.run.smithi005.stdout:5306 read 0x9c4fd7 thru 0x9cecac (0x9cd6 bytes) 2024-05-11T23:03:13.094 INFO:teuthology.orchestra.run.smithi005.stdout:5308 punch from 0x4519e35 to 0x4523dda, (0x9fa5 bytes) 2024-05-11T23:03:13.094 INFO:teuthology.orchestra.run.smithi005.stdout:5312 read 0x33fc180 thru 0x3405ea0 (0x9d21 bytes) 2024-05-11T23:03:13.094 INFO:teuthology.orchestra.run.smithi005.stdout:5313 writesame 0xc03ead4 thru 0xc04336b data_size 0x2e(0x4898 bytes) 2024-05-11T23:03:13.100 INFO:teuthology.orchestra.run.smithi005.stdout:5314 write 0xddba996 thru 0xddbbc4d (0x12b8 bytes) 2024-05-11T23:03:13.105 INFO:teuthology.orchestra.run.smithi005.stdout:5315 read 0x201c9d thru 0x20293f (0xca3 bytes) 2024-05-11T23:03:13.105 INFO:teuthology.orchestra.run.smithi005.stdout:5317 punch from 0x45e4b7b to 0x45ec253, (0x76d8 bytes) 2024-05-11T23:03:13.105 INFO:teuthology.orchestra.run.smithi005.stdout:5319 punch from 0xbae4d5a to 0xbaeace8, (0x5f8e bytes) 2024-05-11T23:03:13.106 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:13.106 INFO:teuthology.orchestra.run.smithi005.stdout:5321 punch from 0x30bc40d to 0x30c8f6d, (0xcb60 bytes) 2024-05-11T23:03:13.106 INFO:teuthology.orchestra.run.smithi005.stdout:5322 writesame 0xcf0c72e thru 0xcf1853d data_size 0x8(0xbe10 bytes) 2024-05-11T23:03:13.116 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:13.116 INFO:teuthology.orchestra.run.smithi005.stdout:5327 punch from 0xaf81292 to 0xaf823ef, (0x115d bytes) 2024-05-11T23:03:13.116 INFO:teuthology.orchestra.run.smithi005.stdout:5328 read 0x4a32597 thru 0x4a387b7 (0x6221 bytes) 2024-05-11T23:03:13.118 INFO:teuthology.orchestra.run.smithi005.stdout:5330 trunc from 0xdfffe92 to 0xca41903 2024-05-11T23:03:13.130 INFO:teuthology.orchestra.run.smithi005.stdout:5333 punch from 0x65f282a to 0x6602661, (0xfe37 bytes) 2024-05-11T23:03:13.130 INFO:teuthology.orchestra.run.smithi005.stdout:5334 write 0xb0709bd thru 0xb07477e (0x3dc2 bytes) 2024-05-11T23:03:13.136 INFO:teuthology.orchestra.run.smithi005.stdout:5335 write 0x95e08aa thru 0x95e7a5a (0x71b1 bytes) 2024-05-11T23:03:13.146 INFO:teuthology.orchestra.run.smithi005.stdout:5336 read 0x8216d83 thru 0x821955d (0x27db bytes) 2024-05-11T23:03:13.146 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:13.147 INFO:teuthology.orchestra.run.smithi005.stdout:5338 write 0x5421286 thru 0x542ac5a (0x99d5 bytes) 2024-05-11T23:03:13.155 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:13.155+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:13.155 INFO:teuthology.orchestra.run.smithi005.stdout:5340 punch from 0x28ec975 to 0x28f2b86, (0x6211 bytes) 2024-05-11T23:03:13.158 INFO:teuthology.orchestra.run.smithi005.stdout:5342 trunc from 0xca41903 to 0x9faa5dc 2024-05-11T23:03:13.175 INFO:teuthology.orchestra.run.smithi005.stdout:5343 writesame 0x36195b3 thru 0x3624123 data_size 0x1(0xab71 bytes) 2024-05-11T23:03:13.191 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:13.192 INFO:teuthology.orchestra.run.smithi005.stdout:5345 writesame 0x8e4867e thru 0x8e4c87e data_size 0x1(0x4201 bytes) 2024-05-11T23:03:13.209 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:13.209 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:13.209 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:13.209 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:13.213 INFO:teuthology.orchestra.run.smithi005.stdout:5346 trunc from 0x9faa5dc to 0x1083997 2024-05-11T23:03:13.223 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:03:13.250 INFO:teuthology.orchestra.run.smithi005.stdout:5347 read 0xcf3050 thru 0xcf8c9d (0x5c4e bytes) 2024-05-11T23:03:13.251 INFO:teuthology.orchestra.run.smithi005.stdout:5348 punch from 0x58b51c to 0x59778c, (0xc270 bytes) 2024-05-11T23:03:13.255 INFO:teuthology.orchestra.run.smithi005.stdout:5349 trunc from 0x1083997 to 0x4c1c7d2 2024-05-11T23:03:13.256 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:13.255+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:13.271 INFO:teuthology.orchestra.run.smithi005.stdout:5350 trunc from 0x4c1c7d2 to 0xd3fa574 2024-05-11T23:03:13.277 INFO:teuthology.orchestra.run.smithi005.stdout:5351 writesame 0x810c6ae thru 0x8112f7e data_size 0x1(0x68d1 bytes) 2024-05-11T23:03:13.288 INFO:teuthology.orchestra.run.smithi005.stdout:5352 write 0x7431fb6 thru 0x7437ab2 (0x5afd bytes) 2024-05-11T23:03:13.296 INFO:teuthology.orchestra.run.smithi005.stdout:5354 write 0x3b26888 thru 0x3b28f97 (0x2710 bytes) 2024-05-11T23:03:13.300 INFO:teuthology.orchestra.run.smithi005.stdout:5356 write 0x898dd6a thru 0x8991145 (0x33dc bytes) 2024-05-11T23:03:13.307 INFO:teuthology.orchestra.run.smithi005.stdout:5358 punch from 0x46502bb to 0x465baa1, (0xb7e6 bytes) 2024-05-11T23:03:13.307 INFO:teuthology.orchestra.run.smithi005.stdout:5359 write 0xcc089d2 thru 0xcc14cd1 (0xc300 bytes) 2024-05-11T23:03:13.332 INFO:teuthology.orchestra.run.smithi005.stdout:5360 trunc from 0xd3fa574 to 0x62c5d6 2024-05-11T23:03:13.358 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:13.356+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:13.359 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:13.370 INFO:teuthology.orchestra.run.smithi005.stdout:5362 write 0x540209c thru 0x540cead (0xae12 bytes) 2024-05-11T23:03:13.384 INFO:teuthology.orchestra.run.smithi005.stdout:5364 read 0x182a62a thru 0x182f58d (0x4f64 bytes) 2024-05-11T23:03:13.384 INFO:teuthology.orchestra.run.smithi005.stdout:5365 read 0x523cef6 thru 0x523dce0 (0xdeb bytes) 2024-05-11T23:03:13.384 INFO:teuthology.orchestra.run.smithi005.stdout:5367 read 0x45e0b41 thru 0x45ea301 (0x97c1 bytes) 2024-05-11T23:03:13.385 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:13.385 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:13.385 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:13.385 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:13.385 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:13.398 INFO:teuthology.orchestra.run.smithi005.stdout:5369 write 0xb5f1053 thru 0xb5f761a (0x65c8 bytes) 2024-05-11T23:03:13.399 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:03:13.412 INFO:teuthology.orchestra.run.smithi005.stdout:5370 write 0x336ae61 thru 0x33790da (0xe27a bytes) 2024-05-11T23:03:13.427 INFO:teuthology.orchestra.run.smithi005.stdout:5371 trunc from 0xb5f761b to 0xe801651 2024-05-11T23:03:13.433 INFO:teuthology.orchestra.run.smithi005.stdout:5372 read 0x26da1fc thru 0x26e192a (0x772f bytes) 2024-05-11T23:03:13.433 INFO:teuthology.orchestra.run.smithi005.stdout:5373 writesame 0x5211dc5 thru 0x5212cee data_size 0x6(0xf2a bytes) 2024-05-11T23:03:13.435 INFO:teuthology.orchestra.run.smithi005.stdout:5374 writesame 0xe2b84d1 thru 0xe2c3962 data_size 0x2(0xb492 bytes) 2024-05-11T23:03:13.451 INFO:teuthology.orchestra.run.smithi005.stdout:5375 punch from 0x96f3d56 to 0x96faa4a, (0x6cf4 bytes) 2024-05-11T23:03:13.452 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:13.452 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:13.452 INFO:teuthology.orchestra.run.smithi005.stdout:5378 punch from 0x6ce5096 to 0x6cee52f, (0x9499 bytes) 2024-05-11T23:03:13.452 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:13.452 INFO:teuthology.orchestra.run.smithi005.stdout:5380 read 0x19a2e85 thru 0x19a574a (0x28c6 bytes) 2024-05-11T23:03:13.452 INFO:teuthology.orchestra.run.smithi005.stdout:5381 read 0x98a0a9 thru 0x9940dc (0xa034 bytes) 2024-05-11T23:03:13.453 INFO:teuthology.orchestra.run.smithi005.stdout:5383 punch from 0x2207467 to 0x2207b79, (0x712 bytes) 2024-05-11T23:03:13.453 INFO:teuthology.orchestra.run.smithi005.stdout:5386 read 0xbb2e692 thru 0xbb39f83 (0xb8f2 bytes) 2024-05-11T23:03:13.453 INFO:teuthology.orchestra.run.smithi005.stdout:5387 punch from 0x7486539 to 0x7488227, (0x1cee bytes) 2024-05-11T23:03:13.453 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:13.453 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:13.453 INFO:teuthology.orchestra.run.smithi005.stdout:5390 write 0x7a9124 thru 0x7b186d (0x874a bytes) 2024-05-11T23:03:13.454 INFO:teuthology.orchestra.run.smithi005.stdout:5394 write 0x5140356 thru 0x5141760 (0x140b bytes) 2024-05-11T23:03:13.456 INFO:teuthology.orchestra.run.smithi005.stdout:5397 punch from 0x37cd0e6 to 0x37cf0b6, (0x1fd0 bytes) 2024-05-11T23:03:13.457 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:13.457+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:13.466 INFO:teuthology.orchestra.run.smithi005.stdout:5398 trunc from 0xe801651 to 0x65a3a83 2024-05-11T23:03:13.500 INFO:teuthology.orchestra.run.smithi005.stdout:5399 write 0xe77ccf0 thru 0xe77d965 (0xc76 bytes) 2024-05-11T23:03:13.506 INFO:teuthology.orchestra.run.smithi005.stdout:5400 read 0x6af479b thru 0x6af55cc (0xe32 bytes) 2024-05-11T23:03:13.506 INFO:teuthology.orchestra.run.smithi005.stdout:5401 read 0x5c64db2 thru 0x5c669a0 (0x1bef bytes) 2024-05-11T23:03:13.506 INFO:teuthology.orchestra.run.smithi005.stdout:5402 read 0x19d57e1 thru 0x19d8295 (0x2ab5 bytes) 2024-05-11T23:03:13.506 INFO:teuthology.orchestra.run.smithi005.stdout:5403 write 0x351cb85 thru 0x3524ac4 (0x7f40 bytes) 2024-05-11T23:03:13.510 INFO:teuthology.orchestra.run.smithi005.stdout:5404 read 0x826f1ff thru 0x827519d (0x5f9f bytes) 2024-05-11T23:03:13.510 INFO:teuthology.orchestra.run.smithi005.stdout:5405 writesame 0x9751dd5 thru 0x975e4d5 data_size 0x17(0xc701 bytes) 2024-05-11T23:03:13.520 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:13.521 INFO:teuthology.orchestra.run.smithi005.stdout:5408 write 0xd329dc5 thru 0xd32bdd0 (0x200c bytes) 2024-05-11T23:03:13.527 INFO:teuthology.orchestra.run.smithi005.stdout:5412 write 0x63acf3c thru 0x63b2f48 (0x600d bytes) 2024-05-11T23:03:13.536 INFO:teuthology.orchestra.run.smithi005.stdout:5413 read 0x24a60b7 thru 0x24a9d38 (0x3c82 bytes) 2024-05-11T23:03:13.536 INFO:teuthology.orchestra.run.smithi005.stdout:5415 writesame 0xea9616 thru 0xeb4697 data_size 0x33(0xb082 bytes) 2024-05-11T23:03:13.549 INFO:teuthology.orchestra.run.smithi005.stdout:5416 read 0x7eda70d thru 0x7edd52b (0x2e1f bytes) 2024-05-11T23:03:13.549 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:13.549 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:13.558 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:13.558 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:13.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:77128 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_get in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-11T23:03:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.541670568000001, 2024-05-11T23:03:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91737855899999998, 2024-05-11T23:03:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 77128 2024-05-11T23:03:13.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016003+0000", 2024-05-11T23:03:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5379999999999999e-06 2024-05-11T23:03:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016011+0000", 2024-05-11T23:03:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6410000000000002e-06 2024-05-11T23:03:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016026+0000", 2024-05-11T23:03:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4779e-05 2024-05-11T23:03:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016032+0000", 2024-05-11T23:03:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.066e-06 2024-05-11T23:03:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016093+0000", 2024-05-11T23:03:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0878000000000002e-05 2024-05-11T23:03:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016100+0000", 2024-05-11T23:03:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6799999999999993e-06 2024-05-11T23:03:13.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933302+0000", 2024-05-11T23:03:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91720135300000005 2024-05-11T23:03:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933380+0000", 2024-05-11T23:03:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8623999999999998e-05 2024-05-11T23:03:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.563 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98950 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.425589678999998, 2024-05-11T23:03:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021419438999999998, 2024-05-11T23:03:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98950 2024-05-11T23:03:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.34e-07 2024-05-11T23:03:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132085+0000", 2024-05-11T23:03:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0049999999999999e-06 2024-05-11T23:03:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132093+0000", 2024-05-11T23:03:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6660000000000008e-06 2024-05-11T23:03:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132097+0000", 2024-05-11T23:03:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2590000000000002e-06 2024-05-11T23:03:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135114+0000", 2024-05-11T23:03:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003017573 2024-05-11T23:03:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153416+0000", 2024-05-11T23:03:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018301884000000001 2024-05-11T23:03:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153502+0000", 2024-05-11T23:03:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5717999999999995e-05 2024-05-11T23:03:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.567 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98959 3.1e 3:7807da99:::rbd_data.111ed2934718.000000000000006d:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.425096942000003, 2024-05-11T23:03:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020525928999999998, 2024-05-11T23:03:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.568 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98959 2024-05-11T23:03:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:13.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132576+0000", 2024-05-11T23:03:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4199999999999995e-07 2024-05-11T23:03:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132580+0000", 2024-05-11T23:03:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2910000000000001e-06 2024-05-11T23:03:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132588+0000", 2024-05-11T23:03:13.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7970000000000001e-06 2024-05-11T23:03:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132602+0000", 2024-05-11T23:03:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3743999999999999e-05 2024-05-11T23:03:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132655+0000", 2024-05-11T23:03:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2714999999999999e-05 2024-05-11T23:03:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132787+0000", 2024-05-11T23:03:13.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013273899999999999 2024-05-11T23:03:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153101+0000", 2024-05-11T23:03:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020313900999999999 2024-05-11T23:03:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98965 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:13.572 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.422573305999997, 2024-05-11T23:03:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018608983999999999, 2024-05-11T23:03:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98965 2024-05-11T23:03:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:13.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135100+0000", 2024-05-11T23:03:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.17e-06 2024-05-11T23:03:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135105+0000", 2024-05-11T23:03:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.899e-06 2024-05-11T23:03:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135122+0000", 2024-05-11T23:03:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7346999999999998e-05 2024-05-11T23:03:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135130+0000", 2024-05-11T23:03:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0639999999999994e-06 2024-05-11T23:03:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153533+0000", 2024-05-11T23:03:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018402753000000001 2024-05-11T23:03:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153615+0000", 2024-05-11T23:03:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1720999999999999e-05 2024-05-11T23:03:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153708+0000", 2024-05-11T23:03:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3029999999999995e-05 2024-05-11T23:03:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98966 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3760128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.422501179999998, 2024-05-11T23:03:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018716535999999999, 2024-05-11T23:03:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98966 2024-05-11T23:03:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.577 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135172+0000", 2024-05-11T23:03:13.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-05-11T23:03:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135175+0000", 2024-05-11T23:03:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-05-11T23:03:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135184+0000", 2024-05-11T23:03:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1840000000000005e-06 2024-05-11T23:03:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152903+0000", 2024-05-11T23:03:13.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017718357000000001 2024-05-11T23:03:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153736+0000", 2024-05-11T23:03:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00083324799999999997 2024-05-11T23:03:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153809+0000", 2024-05-11T23:03:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2995000000000002e-05 2024-05-11T23:03:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153888+0000", 2024-05-11T23:03:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8695999999999993e-05 2024-05-11T23:03:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "pg_update_log_missing_reply(3.1e epoch 72/53 rep_tid 104 lcod 72'363)", 2024-05-11T23:03:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 47.422438284000002, 2024-05-11T23:03:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017871951000000001, 2024-05-11T23:03:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135235+0000", 2024-05-11T23:03:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8899999999999999e-07 2024-05-11T23:03:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135238+0000", 2024-05-11T23:03:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.27e-06 2024-05-11T23:03:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152878+0000", 2024-05-11T23:03:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01763986 2024-05-11T23:03:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152896+0000", 2024-05-11T23:03:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8573e-05 2024-05-11T23:03:13.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152968+0000", 2024-05-11T23:03:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1780999999999996e-05 2024-05-11T23:03:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153106+0000", 2024-05-11T23:03:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013777799999999999 2024-05-11T23:03:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:239544 3.1a 3:5fe59cad:::rbd_data.111ebabb4604.000000000000016e:head [stat,delete] snapc 2f=[2f] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 9.9030815519999997, 2024-05-11T23:03:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.50514007500000002, 2024-05-11T23:03:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 239544 2024-05-11T23:03:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654592+0000", 2024-05-11T23:03:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.404e-06 2024-05-11T23:03:13.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654598+0000", 2024-05-11T23:03:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7660000000000004e-06 2024-05-11T23:03:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654611+0000", 2024-05-11T23:03:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2676e-05 2024-05-11T23:03:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654620+0000", 2024-05-11T23:03:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2990000000000002e-06 2024-05-11T23:03:13.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654698+0000", 2024-05-11T23:03:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7739999999999998e-05 2024-05-11T23:03:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654710+0000", 2024-05-11T23:03:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2648e-05 2024-05-11T23:03:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.157889+0000", 2024-05-11T23:03:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.50317850799999997 2024-05-11T23:03:13.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.158050+0000", 2024-05-11T23:03:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016084500000000001 2024-05-11T23:03:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.159731+0000", 2024-05-11T23:03:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001681189 2024-05-11T23:03:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262364 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2613248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.97120868699999996, 2024-05-11T23:03:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016312508, 2024-05-11T23:03:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262364 2024-05-11T23:03:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.13e-07 2024-05-11T23:03:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586467+0000", 2024-05-11T23:03:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-05-11T23:03:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586474+0000", 2024-05-11T23:03:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7739999999999999e-06 2024-05-11T23:03:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586483+0000", 2024-05-11T23:03:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0340000000000002e-06 2024-05-11T23:03:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602629+0000", 2024-05-11T23:03:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016145845999999998 2024-05-11T23:03:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602703+0000", 2024-05-11T23:03:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4671000000000004e-05 2024-05-11T23:03:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602776+0000", 2024-05-11T23:03:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2793000000000006e-05 2024-05-11T23:03:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.593 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262417 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.96946238799999995, 2024-05-11T23:03:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016403456, 2024-05-11T23:03:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262417 2024-05-11T23:03:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.594 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:13.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6199999999999999e-07 2024-05-11T23:03:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588212+0000", 2024-05-11T23:03:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.075e-06 2024-05-11T23:03:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588218+0000", 2024-05-11T23:03:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8150000000000002e-06 2024-05-11T23:03:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588223+0000", 2024-05-11T23:03:13.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7090000000000001e-06 2024-05-11T23:03:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604461+0000", 2024-05-11T23:03:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016237900999999999 2024-05-11T23:03:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604540+0000", 2024-05-11T23:03:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9376999999999996e-05 2024-05-11T23:03:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604613+0000", 2024-05-11T23:03:13.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3217000000000001e-05 2024-05-11T23:03:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262418 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2396160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.96944247900000002, 2024-05-11T23:03:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016591180000000001, 2024-05-11T23:03:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.598 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262418 2024-05-11T23:03:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:13.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6100000000000001e-07 2024-05-11T23:03:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588232+0000", 2024-05-11T23:03:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0370000000000001e-06 2024-05-11T23:03:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588238+0000", 2024-05-11T23:03:13.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9340000000000003e-06 2024-05-11T23:03:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588243+0000", 2024-05-11T23:03:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6179999999999997e-06 2024-05-11T23:03:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604642+0000", 2024-05-11T23:03:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0163991 2024-05-11T23:03:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604712+0000", 2024-05-11T23:03:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9935e-05 2024-05-11T23:03:13.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604821+0000", 2024-05-11T23:03:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000109095 2024-05-11T23:03:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262419 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2433024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.96942316100000003, 2024-05-11T23:03:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016799692000000001, 2024-05-11T23:03:13.602 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262419 2024-05-11T23:03:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588252+0000", 2024-05-11T23:03:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0990000000000001e-06 2024-05-11T23:03:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588257+0000", 2024-05-11T23:03:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8159999999999999e-06 2024-05-11T23:03:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588262+0000", 2024-05-11T23:03:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8069999999999999e-06 2024-05-11T23:03:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604868+0000", 2024-05-11T23:03:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016605880999999999 2024-05-11T23:03:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604966+0000", 2024-05-11T23:03:13.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7597999999999996e-05 2024-05-11T23:03:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605049+0000", 2024-05-11T23:03:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3141999999999998e-05 2024-05-11T23:03:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262420 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2469888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.96940305000000004, 2024-05-11T23:03:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016972514000000001, 2024-05-11T23:03:13.606 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262420 2024-05-11T23:03:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588271+0000", 2024-05-11T23:03:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6199999999999999e-06 2024-05-11T23:03:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588273+0000", 2024-05-11T23:03:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3209999999999999e-06 2024-05-11T23:03:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588279+0000", 2024-05-11T23:03:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2519999999999996e-06 2024-05-11T23:03:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588285+0000", 2024-05-11T23:03:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1440000000000002e-06 2024-05-11T23:03:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605083+0000", 2024-05-11T23:03:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016798418999999998 2024-05-11T23:03:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:13.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605168+0000", 2024-05-11T23:03:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4851000000000005e-05 2024-05-11T23:03:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605242+0000", 2024-05-11T23:03:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3906999999999996e-05 2024-05-11T23:03:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262421 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2506752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.96938090399999999, 2024-05-11T23:03:13.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017141427000000001, 2024-05-11T23:03:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262421 2024-05-11T23:03:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588292+0000", 2024-05-11T23:03:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588318+0000", 2024-05-11T23:03:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6376e-05 2024-05-11T23:03:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588326+0000", 2024-05-11T23:03:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6699999999999994e-06 2024-05-11T23:03:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588332+0000", 2024-05-11T23:03:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8540000000000002e-06 2024-05-11T23:03:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605274+0000", 2024-05-11T23:03:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016942024999999999 2024-05-11T23:03:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:13.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605360+0000", 2024-05-11T23:03:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6304999999999994e-05 2024-05-11T23:03:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605433+0000", 2024-05-11T23:03:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2772000000000001e-05 2024-05-11T23:03:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262422 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2543616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:13.614 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.96933168000000003, 2024-05-11T23:03:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017283705, 2024-05-11T23:03:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262422 2024-05-11T23:03:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:13.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:03:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588343+0000", 2024-05-11T23:03:13.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.34e-06 2024-05-11T23:03:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588350+0000", 2024-05-11T23:03:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1569999999999997e-06 2024-05-11T23:03:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588354+0000", 2024-05-11T23:03:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.865e-06 2024-05-11T23:03:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605466+0000", 2024-05-11T23:03:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017111498999999999 2024-05-11T23:03:13.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605551+0000", 2024-05-11T23:03:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5210999999999994e-05 2024-05-11T23:03:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605624+0000", 2024-05-11T23:03:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3153999999999998e-05 2024-05-11T23:03:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262423 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2580480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.96931143099999995, 2024-05-11T23:03:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017451194, 2024-05-11T23:03:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262423 2024-05-11T23:03:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.619 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588363+0000", 2024-05-11T23:03:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.855e-06 2024-05-11T23:03:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588369+0000", 2024-05-11T23:03:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1260000000000003e-06 2024-05-11T23:03:13.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588375+0000", 2024-05-11T23:03:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9630000000000003e-06 2024-05-11T23:03:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605656+0000", 2024-05-11T23:03:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017280865999999999 2024-05-11T23:03:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605737+0000", 2024-05-11T23:03:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0773999999999996e-05 2024-05-11T23:03:13.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605812+0000", 2024-05-11T23:03:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5203999999999994e-05 2024-05-11T23:03:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262424 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2617344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.969289913, 2024-05-11T23:03:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017617544999999998, 2024-05-11T23:03:13.623 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262424 2024-05-11T23:03:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588383+0000", 2024-05-11T23:03:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8299999999999997e-07 2024-05-11T23:03:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588385+0000", 2024-05-11T23:03:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2869999999999999e-06 2024-05-11T23:03:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588391+0000", 2024-05-11T23:03:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1689999999999997e-06 2024-05-11T23:03:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588396+0000", 2024-05-11T23:03:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5719999999999996e-06 2024-05-11T23:03:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605837+0000", 2024-05-11T23:03:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017440812999999999 2024-05-11T23:03:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:13.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605917+0000", 2024-05-11T23:03:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9924000000000003e-05 2024-05-11T23:03:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.606000+0000", 2024-05-11T23:03:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3296999999999999e-05 2024-05-11T23:03:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262896 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.89828527599999997, 2024-05-11T23:03:13.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016295738000000001, 2024-05-11T23:03:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262896 2024-05-11T23:03:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659390+0000", 2024-05-11T23:03:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1440000000000001e-06 2024-05-11T23:03:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659397+0000", 2024-05-11T23:03:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0010000000000004e-06 2024-05-11T23:03:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659402+0000", 2024-05-11T23:03:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2179999999999998e-06 2024-05-11T23:03:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675487+0000", 2024-05-11T23:03:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016085706000000002 2024-05-11T23:03:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675580+0000", 2024-05-11T23:03:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2884e-05 2024-05-11T23:03:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675683+0000", 2024-05-11T23:03:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010236 2024-05-11T23:03:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262897 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:13.631 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.89824993500000005, 2024-05-11T23:03:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016500932999999999, 2024-05-11T23:03:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262897 2024-05-11T23:03:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659423+0000", 2024-05-11T23:03:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9699999999999996e-07 2024-05-11T23:03:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659425+0000", 2024-05-11T23:03:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3199999999999998e-06 2024-05-11T23:03:13.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659433+0000", 2024-05-11T23:03:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3660000000000003e-06 2024-05-11T23:03:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659445+0000", 2024-05-11T23:03:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2102e-05 2024-05-11T23:03:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675727+0000", 2024-05-11T23:03:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016282331000000001 2024-05-11T23:03:13.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675815+0000", 2024-05-11T23:03:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7484999999999993e-05 2024-05-11T23:03:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675923+0000", 2024-05-11T23:03:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000108732 2024-05-11T23:03:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.635 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262898 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.89821881699999995, 2024-05-11T23:03:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016685855999999999, 2024-05-11T23:03:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262898 2024-05-11T23:03:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:13.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659454+0000", 2024-05-11T23:03:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2200000000000004e-07 2024-05-11T23:03:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659457+0000", 2024-05-11T23:03:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.779e-06 2024-05-11T23:03:13.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659463+0000", 2024-05-11T23:03:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6660000000000002e-06 2024-05-11T23:03:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659472+0000", 2024-05-11T23:03:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2139999999999996e-06 2024-05-11T23:03:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675956+0000", 2024-05-11T23:03:13.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016484431000000001 2024-05-11T23:03:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676039+0000", 2024-05-11T23:03:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3358999999999999e-05 2024-05-11T23:03:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676139+0000", 2024-05-11T23:03:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9784999999999994e-05 2024-05-11T23:03:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.639 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262899 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.89819298599999997, 2024-05-11T23:03:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017030087999999999, 2024-05-11T23:03:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262899 2024-05-11T23:03:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659480+0000", 2024-05-11T23:03:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7700000000000004e-07 2024-05-11T23:03:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659482+0000", 2024-05-11T23:03:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1390000000000002e-06 2024-05-11T23:03:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659489+0000", 2024-05-11T23:03:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8279999999999997e-06 2024-05-11T23:03:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659494+0000", 2024-05-11T23:03:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9019999999999998e-06 2024-05-11T23:03:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676167+0000", 2024-05-11T23:03:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016673621 2024-05-11T23:03:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676329+0000", 2024-05-11T23:03:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016206099999999999 2024-05-11T23:03:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676509+0000", 2024-05-11T23:03:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017996 2024-05-11T23:03:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.643 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.644 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.644 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.644 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:13.644 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:03:13.645 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:13.558+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:13.645 INFO:teuthology.orchestra.run.smithi005.stdout:5419 trunc from 0xe77d966 to 0x14d1c93 2024-05-11T23:03:13.646 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:03:13.647 INFO:teuthology.orchestra.run.smithi005.stdout:5420 trunc from 0x14d1c93 to 0x2f72922 2024-05-11T23:03:13.647 INFO:teuthology.orchestra.run.smithi005.stdout:5421 read 0x13d4cf5 thru 0x13dcecc (0x81d8 bytes) 2024-05-11T23:03:13.647 INFO:teuthology.orchestra.run.smithi005.stdout:5422 write 0x8263363 thru 0x826f373 (0xc011 bytes) 2024-05-11T23:03:13.647 INFO:teuthology.orchestra.run.smithi005.stdout:5423 write 0x9251270 thru 0x9254c2b (0x39bc bytes) 2024-05-11T23:03:13.647 INFO:teuthology.orchestra.run.smithi005.stdout:5424 write 0x263e3f7 thru 0x2649ec6 (0xbad0 bytes) 2024-05-11T23:03:13.648 INFO:teuthology.orchestra.run.smithi005.stdout:5425 read 0x591eb1c thru 0x59277ed (0x8cd2 bytes) 2024-05-11T23:03:13.648 INFO:teuthology.orchestra.run.smithi005.stdout:5426 writesame 0xd86786c thru 0xd8719b8 data_size 0x11(0xa14d bytes) 2024-05-11T23:03:13.648 INFO:teuthology.orchestra.run.smithi005.stdout:5427 punch from 0x7c8e614 to 0x7c9cc6c, (0xe658 bytes) 2024-05-11T23:03:13.651 INFO:teuthology.orchestra.run.smithi005.stdout:5428 write 0xde5d63c thru 0xde60a75 (0x343a bytes) 2024-05-11T23:03:13.656 INFO:teuthology.orchestra.run.smithi005.stdout:5429 read 0x3e557a7 thru 0x3e5ce9a (0x76f4 bytes) 2024-05-11T23:03:13.659 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:13.658+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:13.670 INFO:teuthology.orchestra.run.smithi005.stdout:5431 trunc from 0xde60a76 to 0x20d705f 2024-05-11T23:03:13.694 INFO:teuthology.orchestra.run.smithi005.stdout:5432 write 0x3695064 thru 0x369f975 (0xa912 bytes) 2024-05-11T23:03:13.704 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:13.718 INFO:teuthology.orchestra.run.smithi005.stdout:5435 write 0xa0be6fa thru 0xa0c6606 (0x7f0d bytes) 2024-05-11T23:03:13.734 INFO:teuthology.orchestra.run.smithi005.stdout:5436 writesame 0x3b829bb thru 0x3b8cf99 data_size 0x1(0xa5df bytes) 2024-05-11T23:03:13.749 INFO:teuthology.orchestra.run.smithi005.stdout:5438 trunc from 0xa0c6607 to 0x51caeb9 2024-05-11T23:03:13.759 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:13.759+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:13.761 INFO:teuthology.orchestra.run.smithi005.stdout:5439 writesame 0x12a023f thru 0x12a1d03 data_size 0xb(0x1ac5 bytes) 2024-05-11T23:03:13.766 INFO:teuthology.orchestra.run.smithi005.stdout:5445 writesame 0x29121bf thru 0x291af2e data_size 0x1f(0x8d70 bytes) 2024-05-11T23:03:13.786 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:13.786 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:13.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.786 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop(client.4382.0:270829 3.5 e110/76 3:a15040e3:::rbd_header.111e70ebcd3f:head v 110'541, mlcod=110'539)", 2024-05-11T23:03:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:13.786418+0000", 2024-05-11T23:03:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.00046295899999999998, 2024-05-11T23:03:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00050160199999999997, 2024-05-11T23:03:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:13.786418+0000", 2024-05-11T23:03:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:13.786418+0000", 2024-05-11T23:03:13.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:13.786419+0000", 2024-05-11T23:03:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2309999999999999e-06 2024-05-11T23:03:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:13.786429+0000", 2024-05-11T23:03:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.8530000000000004e-06 2024-05-11T23:03:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:13.786447+0000", 2024-05-11T23:03:13.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8414999999999999e-05 2024-05-11T23:03:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:13.786458+0000", 2024-05-11T23:03:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.062e-05 2024-05-11T23:03:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:13.786525+0000", 2024-05-11T23:03:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6867999999999997e-05 2024-05-11T23:03:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:13.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:13.786542+0000", 2024-05-11T23:03:13.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.749e-05 2024-05-11T23:03:13.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.790 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:13.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.790 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:13.790 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:03:13.790 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 1 2024-05-11T23:03:13.790 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:13.790 INFO:teuthology.orchestra.run.smithi005.stdout:5446 write 0x95d1a89 thru 0x95dcc55 (0xb1cd bytes) 2024-05-11T23:03:13.796 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:03:13.802 INFO:teuthology.orchestra.run.smithi005.stdout:5447 trunc from 0x95dcc56 to 0x744d9c5 2024-05-11T23:03:13.808 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:13.815 INFO:teuthology.orchestra.run.smithi005.stdout:5449 trunc from 0x744d9c5 to 0x7e1cbf0 2024-05-11T23:03:13.820 INFO:teuthology.orchestra.run.smithi005.stdout:5450 read 0x1a2d2b thru 0x1ab096 (0x836c bytes) 2024-05-11T23:03:13.829 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:13.830 INFO:teuthology.orchestra.run.smithi005.stdout:5452 write 0x9aa0312 thru 0x9aaa2a4 (0x9f93 bytes) 2024-05-11T23:03:13.844 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:13.845 INFO:teuthology.orchestra.run.smithi005.stdout:5457 writesame 0x502783d thru 0x502fd1f data_size 0x1(0x84e3 bytes) 2024-05-11T23:03:13.853 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:13.853 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:13.856 INFO:teuthology.orchestra.run.smithi005.stdout:5459 writesame 0x2d1d2ed thru 0x2d250e6 data_size 0xf(0x7dfa bytes) 2024-05-11T23:03:13.860 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:13.859+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:13.868 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:13.868 INFO:teuthology.orchestra.run.smithi005.stdout:5461 punch from 0x85b3970 to 0x85ba0af, (0x673f bytes) 2024-05-11T23:03:13.868 INFO:teuthology.orchestra.run.smithi005.stdout:5462 write 0x99a2d75 thru 0x99b1418 (0xe6a4 bytes) 2024-05-11T23:03:13.872 INFO:teuthology.orchestra.run.smithi005.stdout:5463 writesame 0x6672488 thru 0x667f24f data_size 0x3c(0xcdc8 bytes) 2024-05-11T23:03:13.877 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:03:13.877 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:03:13.886 INFO:teuthology.orchestra.run.smithi005.stdout:5464 writesame 0x475bb7a thru 0x475c102 data_size 0x1(0x589 bytes) 2024-05-11T23:03:13.891 INFO:teuthology.orchestra.run.smithi005.stdout:5465 writesame 0x6884f9 thru 0x6956c5 data_size 0x1(0xd1cd bytes) 2024-05-11T23:03:13.902 INFO:teuthology.orchestra.run.smithi005.stdout:5466 write 0x77286a7 thru 0x7730ec6 (0x8820 bytes) 2024-05-11T23:03:13.902 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:13.912 INFO:teuthology.orchestra.run.smithi005.stdout:5467 punch from 0x12d5efd to 0x12ddeb7, (0x7fba bytes) 2024-05-11T23:03:13.912 INFO:teuthology.orchestra.run.smithi005.stdout:5468 write 0x21a64c6 thru 0x21aabcb (0x4706 bytes) 2024-05-11T23:03:13.925 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:13.935 INFO:teuthology.orchestra.run.smithi005.stdout:5469 write 0xe2d684f thru 0xe2e1ebb (0xb66d bytes) 2024-05-11T23:03:13.941 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:13.941 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:13.948 INFO:teuthology.orchestra.run.smithi005.stdout:5470 write 0x749844c thru 0x74a025d (0x7e12 bytes) 2024-05-11T23:03:13.948 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:13.949 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:13.952 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:03:13.959 INFO:teuthology.orchestra.run.smithi005.stdout:5473 write 0x6345d34 thru 0x6350f1c (0xb1e9 bytes) 2024-05-11T23:03:13.961 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:13.961+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:13.963 INFO:teuthology.orchestra.run.smithi005.stdout:5474 write 0xd8fed57 thru 0xd902e0e (0x40b8 bytes) 2024-05-11T23:03:13.973 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:13.973 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:13.987 INFO:teuthology.orchestra.run.smithi005.stdout:5475 trunc from 0xe2e1ebc to 0x107d08d 2024-05-11T23:03:14.021 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:14.030 INFO:teuthology.orchestra.run.smithi005.stdout:5478 write 0x3065ddd thru 0x306ff6f (0xa193 bytes) 2024-05-11T23:03:14.044 INFO:teuthology.orchestra.run.smithi005.stdout:5479 trunc from 0x306ff70 to 0xf3537a 2024-05-11T23:03:14.062 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:14.062+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:14.068 INFO:teuthology.orchestra.run.smithi005.stdout:5481 writesame 0x2e72e4c thru 0x2e7a490 data_size 0x11(0x7645 bytes) 2024-05-11T23:03:14.068 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:14.069 INFO:teuthology.orchestra.run.smithi005.stdout:5484 trunc from 0x2e7a491 to 0x4888bcf 2024-05-11T23:03:14.075 INFO:teuthology.orchestra.run.smithi005.stdout:5485 punch from 0xfd002b to 0xfda9c7, (0xa99c bytes) 2024-05-11T23:03:14.075 INFO:teuthology.orchestra.run.smithi005.stdout:5486 punch from 0x34e315e to 0x34e4706, (0x15a8 bytes) 2024-05-11T23:03:14.075 INFO:teuthology.orchestra.run.smithi005.stdout:5487 write 0x2a759af thru 0x2a7acb8 (0x530a bytes) 2024-05-11T23:03:14.082 INFO:teuthology.orchestra.run.smithi005.stdout:5488 punch from 0x3042d1 to 0x308228, (0x3f57 bytes) 2024-05-11T23:03:14.096 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:14.097 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:14.099 INFO:teuthology.orchestra.run.smithi005.stdout:5489 write 0xcae7346 thru 0xcae9f58 (0x2c13 bytes) 2024-05-11T23:03:14.103 INFO:teuthology.orchestra.run.smithi005.stdout:5490 punch from 0x792c503 to 0x793a884, (0xe381 bytes) 2024-05-11T23:03:14.107 INFO:teuthology.orchestra.run.smithi005.stdout:5491 trunc from 0xcae9f59 to 0x997e453 2024-05-11T23:03:14.108 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:03:14.117 INFO:teuthology.orchestra.run.smithi005.stdout:5492 write 0x523d8b6 thru 0x524c886 (0xefd1 bytes) 2024-05-11T23:03:14.130 INFO:teuthology.orchestra.run.smithi005.stdout:5493 punch from 0x454f0f to 0x45cded, (0x7ede bytes) 2024-05-11T23:03:14.130 INFO:teuthology.orchestra.run.smithi005.stdout:5494 write 0x221f31b thru 0x2225342 (0x6028 bytes) 2024-05-11T23:03:14.141 INFO:teuthology.orchestra.run.smithi005.stdout:5497 write 0x6701404 thru 0x67095b7 (0x81b4 bytes) 2024-05-11T23:03:14.152 INFO:teuthology.orchestra.run.smithi005.stdout:5498 punch from 0x39acf51 to 0x39bc356, (0xf405 bytes) 2024-05-11T23:03:14.152 INFO:teuthology.orchestra.run.smithi005.stdout:5499 read 0x454b77e thru 0x454ea39 (0x32bc bytes) 2024-05-11T23:03:14.152 INFO:teuthology.orchestra.run.smithi005.stdout:5500 write 0x11e26ea thru 0x11ec74f (0xa066 bytes) 2024-05-11T23:03:14.155 INFO:teuthology.orchestra.run.smithi005.stdout:5501 read 0x264b6c7 thru 0x264d9f5 (0x232f bytes) 2024-05-11T23:03:14.158 INFO:teuthology.orchestra.run.smithi005.stdout:5502 trunc from 0x997e453 to 0x7883b11 2024-05-11T23:03:14.164 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:14.163+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:14.172 INFO:teuthology.orchestra.run.smithi005.stdout:5504 writesame 0x87f19b0 thru 0x87fd203 data_size 0x4(0xb854 bytes) 2024-05-11T23:03:14.173 INFO:teuthology.orchestra.run.smithi005.stdout:5505 clone 50 order 19 su 65536 sc 9 2024-05-11T23:03:14.197 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 5, 7, 3, 6] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:03:14.197 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:03:14.198 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-11T23:03:14.198 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.437347 2024-05-11T23:03:14.198 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 0.4373471900309288 2024-05-11T23:03:14.224 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:14.225 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:14.234 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:03:14.276 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:14.275+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:14.352 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #48, image image_client.0-clone48 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent49 2024-05-11T23:03:14.357 INFO:teuthology.orchestra.run.smithi005.stdout:5506 write 0x6a871d2 thru 0x6a90fce (0x9dfd bytes) 2024-05-11T23:03:14.366 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:14.365+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:14.374 INFO:teuthology.orchestra.run.smithi005.stdout:5507 read 0x460a4b5 thru 0x4616002 (0xbb4e bytes) 2024-05-11T23:03:14.374 INFO:teuthology.orchestra.run.smithi005.stdout:5508 write 0x14e754a thru 0x14f56a8 (0xe15f bytes) 2024-05-11T23:03:14.378 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:14.389 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:14.389 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:14.390 INFO:teuthology.orchestra.run.smithi005.stdout:5511 write 0xd0d0490 thru 0xd0d2fb3 (0x2b24 bytes) 2024-05-11T23:03:14.393 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-11T23:03:14.397 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:03:14.398 INFO:teuthology.orchestra.run.smithi005.stdout:5512 trunc from 0xd0d2fb4 to 0x87cea10 2024-05-11T23:03:14.422 INFO:teuthology.orchestra.run.smithi005.stdout:5513 write 0x5948a46 thru 0x594b748 (0x2d03 bytes) 2024-05-11T23:03:14.432 INFO:teuthology.orchestra.run.smithi005.stdout:5514 trunc from 0x87cea10 to 0x9e09ff6 2024-05-11T23:03:14.438 INFO:teuthology.orchestra.run.smithi005.stdout:5516 write 0x9cf26f7 thru 0x9cf8686 (0x5f90 bytes) 2024-05-11T23:03:14.447 INFO:teuthology.orchestra.run.smithi005.stdout:5517 trunc from 0x9e09ff6 to 0xd973ebb 2024-05-11T23:03:14.452 INFO:teuthology.orchestra.run.smithi005.stdout:5520 write 0x7463fbf thru 0x7465825 (0x1867 bytes) 2024-05-11T23:03:14.456 INFO:teuthology.orchestra.run.smithi005.stdout:5521 punch from 0xb79997a to 0xb7a7b33, (0xe1b9 bytes) 2024-05-11T23:03:14.457 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:14.457 INFO:teuthology.orchestra.run.smithi005.stdout:5523 write 0x3ee250b thru 0x3ee9438 (0x6f2e bytes) 2024-05-11T23:03:14.460 INFO:teuthology.orchestra.run.smithi005.stdout:5524 read 0x74bf1a9 thru 0x74ce443 (0xf29b bytes) 2024-05-11T23:03:14.460 INFO:teuthology.orchestra.run.smithi005.stdout:5525 write 0x2f77759 thru 0x2f7aeb2 (0x375a bytes) 2024-05-11T23:03:14.464 INFO:teuthology.orchestra.run.smithi005.stdout:5527 write 0x60b41d1 thru 0x60c08ac (0xc6dc bytes) 2024-05-11T23:03:14.470 INFO:teuthology.orchestra.run.smithi005.stdout:5528 trunc from 0xd973ebb to 0xd20fd36 2024-05-11T23:03:14.476 INFO:teuthology.orchestra.run.smithi005.stdout:5529 write 0xb03f3de thru 0xb045911 (0x6534 bytes) 2024-05-11T23:03:14.482 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:14.482+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:14.486 INFO:teuthology.orchestra.run.smithi005.stdout:5530 trunc from 0xd20fd36 to 0x8a8640f 2024-05-11T23:03:14.500 INFO:teuthology.orchestra.run.smithi005.stdout:5531 read 0x1d92056 thru 0x1da10c2 (0xf06d bytes) 2024-05-11T23:03:14.501 INFO:teuthology.orchestra.run.smithi005.stdout:5533 punch from 0x3878cba to 0x387da96, (0x4ddc bytes) 2024-05-11T23:03:14.501 INFO:teuthology.orchestra.run.smithi005.stdout:5535 punch from 0x661381d to 0x66189db, (0x51be bytes) 2024-05-11T23:03:14.501 INFO:teuthology.orchestra.run.smithi005.stdout:5536 write 0x2a0a3d3 thru 0x2a176d3 (0xd301 bytes) 2024-05-11T23:03:14.510 INFO:teuthology.orchestra.run.smithi005.stdout:5537 write 0x670f6b8 thru 0x671ae09 (0xb752 bytes) 2024-05-11T23:03:14.525 INFO:teuthology.orchestra.run.smithi005.stdout:5539 punch from 0x568201b to 0x568c4b4, (0xa499 bytes) 2024-05-11T23:03:14.525 INFO:teuthology.orchestra.run.smithi005.stdout:5540 punch from 0x7a92e39 to 0x7a9dbdf, (0xada6 bytes) 2024-05-11T23:03:14.532 INFO:teuthology.orchestra.run.smithi005.stdout:5541 write 0x9c61e8a thru 0x9c659c8 (0x3b3f bytes) 2024-05-11T23:03:14.542 INFO:teuthology.orchestra.run.smithi005.stdout:5542 trunc from 0x9c659c9 to 0x479db71 2024-05-11T23:03:14.555 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:14.555 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:14.566 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:03:14.567 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:14.567+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:14.572 INFO:teuthology.orchestra.run.smithi005.stdout:5543 write 0x54593c1 thru 0x5460e00 (0x7a40 bytes) 2024-05-11T23:03:14.590 INFO:teuthology.orchestra.run.smithi005.stdout:5544 writesame 0xb0ef95b thru 0xb0f6216 data_size 0x2(0x68bc bytes) 2024-05-11T23:03:14.590 INFO:teuthology.orchestra.run.smithi005.stdout:5545 write 0xaeec8e3 thru 0xaef34be (0x6bdc bytes) 2024-05-11T23:03:14.592 INFO:teuthology.orchestra.run.smithi005.stdout:5546 write 0x224895f thru 0x2251f11 (0x95b3 bytes) 2024-05-11T23:03:14.612 INFO:teuthology.orchestra.run.smithi005.stdout:5549 writesame 0xebe8426 thru 0xebef70d data_size 0x4(0x72e8 bytes) 2024-05-11T23:03:14.612 INFO:teuthology.orchestra.run.smithi005.stdout:5550 read 0x5f8768a thru 0x5f8f3c3 (0x7d3a bytes) 2024-05-11T23:03:14.625 INFO:teuthology.orchestra.run.smithi005.stdout:5551 trunc from 0xebef70e to 0x5bbd0e4 2024-05-11T23:03:14.646 INFO:teuthology.orchestra.run.smithi005.stdout:5552 trunc from 0x5bbd0e4 to 0xb715cfe 2024-05-11T23:03:14.652 INFO:teuthology.orchestra.run.smithi005.stdout:5554 read 0xb2c86ae thru 0xb2cb83e (0x3191 bytes) 2024-05-11T23:03:14.659 INFO:teuthology.orchestra.run.smithi005.stdout:5556 trunc from 0xb715cfe to 0x6d7f3fd 2024-05-11T23:03:14.665 INFO:teuthology.orchestra.run.smithi005.stdout:5558 write 0x16ad539 thru 0x16bd52c (0xfff4 bytes) 2024-05-11T23:03:14.668 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:14.668+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:14.674 INFO:teuthology.orchestra.run.smithi005.stdout:5560 trunc from 0x6d7f3fd to 0x3716dd5 2024-05-11T23:03:14.703 INFO:teuthology.orchestra.run.smithi005.stdout:5561 trunc from 0x3716dd5 to 0x695b8d6 2024-05-11T23:03:14.708 INFO:teuthology.orchestra.run.smithi005.stdout:5562 write 0x1582293 thru 0x1583140 (0xeae bytes) 2024-05-11T23:03:14.710 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:14.710 INFO:teuthology.orchestra.run.smithi005.stdout:5565 read 0x6079215 thru 0x607a617 (0x1403 bytes) 2024-05-11T23:03:14.714 INFO:teuthology.orchestra.run.smithi005.stdout:5566 trunc from 0x695b8d6 to 0xa339456 2024-05-11T23:03:14.719 INFO:teuthology.orchestra.run.smithi005.stdout:5567 punch from 0x63681e0 to 0x636d720, (0x5540 bytes) 2024-05-11T23:03:14.719 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:14.719 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:14.729 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:03:14.730 INFO:teuthology.orchestra.run.smithi005.stdout:5569 writesame 0xdf50922 thru 0xdf5a433 data_size 0x17(0x9b12 bytes) 2024-05-11T23:03:14.730 INFO:teuthology.orchestra.run.smithi005.stdout:5570 punch from 0xcd07496 to 0xcd0caa6, (0x5610 bytes) 2024-05-11T23:03:14.730 INFO:teuthology.orchestra.run.smithi005.stdout:5571 read 0xd286797 thru 0xd291639 (0xaea3 bytes) 2024-05-11T23:03:14.730 INFO:teuthology.orchestra.run.smithi005.stdout:5572 punch from 0x3ddbcdb to 0x3dde404, (0x2729 bytes) 2024-05-11T23:03:14.730 INFO:teuthology.orchestra.run.smithi005.stdout:5573 write 0x306901 thru 0x310f1f (0xa61f bytes) 2024-05-11T23:03:14.740 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:14.740 INFO:teuthology.orchestra.run.smithi005.stdout:5575 writesame 0x1a09546 thru 0x1a0983b data_size 0x1(0x2f6 bytes) 2024-05-11T23:03:14.742 INFO:teuthology.orchestra.run.smithi005.stdout:5576 writesame 0x624d5ad thru 0x624f90b data_size 0x1(0x235f bytes) 2024-05-11T23:03:14.748 INFO:teuthology.orchestra.run.smithi005.stdout:5578 write 0x2793e42 thru 0x279cce1 (0x8ea0 bytes) 2024-05-11T23:03:14.760 INFO:teuthology.orchestra.run.smithi005.stdout:5579 trunc from 0xdf5a434 to 0x955fe83 2024-05-11T23:03:14.766 INFO:teuthology.orchestra.run.smithi005.stdout:5580 read 0x66e15ac thru 0x66e17e0 (0x235 bytes) 2024-05-11T23:03:14.766 INFO:teuthology.orchestra.run.smithi005.stdout:5581 write 0x28a4905 thru 0x28ae3c8 (0x9ac4 bytes) 2024-05-11T23:03:14.769 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:14.769+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:14.772 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:14.776 INFO:teuthology.orchestra.run.smithi005.stdout:5585 trunc from 0x955fe83 to 0x5c44631 2024-05-11T23:03:14.786 INFO:teuthology.orchestra.run.smithi005.stdout:5586 read 0x304221d thru 0x304be02 (0x9be6 bytes) 2024-05-11T23:03:14.789 INFO:teuthology.orchestra.run.smithi005.stdout:5587 trunc from 0x5c44631 to 0x328e9b7 2024-05-11T23:03:14.822 INFO:teuthology.orchestra.run.smithi005.stdout:5590 write 0x789c472 thru 0x78aa7ac (0xe33b bytes) 2024-05-11T23:03:14.826 INFO:teuthology.orchestra.run.smithi005.stdout:5591 punch from 0x5cee805 to 0x5cfe0dd, (0xf8d8 bytes) 2024-05-11T23:03:14.826 INFO:teuthology.orchestra.run.smithi005.stdout:5592 read 0x5f4f0b8 thru 0x5f51102 (0x204b bytes) 2024-05-11T23:03:14.826 INFO:teuthology.orchestra.run.smithi005.stdout:5593 write 0x1875cc4 thru 0x1876685 (0x9c2 bytes) 2024-05-11T23:03:14.829 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:14.829 INFO:teuthology.orchestra.run.smithi005.stdout:5597 read 0x58cf694 thru 0x58d13ad (0x1d1a bytes) 2024-05-11T23:03:14.829 INFO:teuthology.orchestra.run.smithi005.stdout:5598 writesame 0x765d752 thru 0x7663552 data_size 0x1(0x5e01 bytes) 2024-05-11T23:03:14.836 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:14.836 INFO:teuthology.orchestra.run.smithi005.stdout:5602 read 0x5f0ccc7 thru 0x5f1a51f (0xd859 bytes) 2024-05-11T23:03:14.836 INFO:teuthology.orchestra.run.smithi005.stdout:5603 punch from 0x4b90320 to 0x4b94021, (0x3d01 bytes) 2024-05-11T23:03:14.836 INFO:teuthology.orchestra.run.smithi005.stdout:5604 read 0x17e2610 thru 0x17f21a2 (0xfb93 bytes) 2024-05-11T23:03:14.840 INFO:teuthology.orchestra.run.smithi005.stdout:5608 trunc from 0x78aa7ad to 0x3c9a130 2024-05-11T23:03:14.845 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:14.845 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:14.855 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:03:14.863 INFO:teuthology.orchestra.run.smithi005.stdout:5610 write 0xa14354e thru 0xa14e077 (0xab2a bytes) 2024-05-11T23:03:14.874 INFO:teuthology.orchestra.run.smithi005.stdout:5611 write 0xb9e27c0 thru 0xb9e411c (0x195d bytes) 2024-05-11T23:03:14.878 INFO:teuthology.orchestra.run.smithi005.stdout:5612 punch from 0xb8ab184 to 0xb8aef98, (0x3e14 bytes) 2024-05-11T23:03:14.878 INFO:teuthology.orchestra.run.smithi005.stdout:5613 write 0x82daa59 thru 0x82e275b (0x7d03 bytes) 2024-05-11T23:03:14.882 INFO:teuthology.orchestra.run.smithi005.stdout:5614 write 0x1335b00 thru 0x1343069 (0xd56a bytes) 2024-05-11T23:03:14.885 INFO:teuthology.orchestra.run.smithi005.stdout:5615 punch from 0x6316f16 to 0x631e742, (0x782c bytes) 2024-05-11T23:03:14.885 INFO:teuthology.orchestra.run.smithi005.stdout:5616 read 0x73d6b10 thru 0x73df5b5 (0x8aa6 bytes) 2024-05-11T23:03:14.885 INFO:teuthology.orchestra.run.smithi005.stdout:5617 trunc from 0xb9e411d to 0xc1b72d4 2024-05-11T23:03:14.890 INFO:teuthology.orchestra.run.smithi005.stdout:5618 write 0x673ba94 thru 0x674110e (0x567b bytes) 2024-05-11T23:03:14.895 INFO:teuthology.orchestra.run.smithi005.stdout:5619 write 0x94b0c3e thru 0x94b11e6 (0x5a9 bytes) 2024-05-11T23:03:14.897 INFO:teuthology.orchestra.run.smithi005.stdout:5620 punch from 0x2e41b19 to 0x2e43adc, (0x1fc3 bytes) 2024-05-11T23:03:14.900 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:14.900+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:14.903 INFO:teuthology.orchestra.run.smithi005.stdout:5622 trunc from 0xc1b72d4 to 0x6a3ee86 2024-05-11T23:03:14.913 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:14.913 INFO:teuthology.orchestra.run.smithi005.stdout:5624 punch from 0x64ceda2 to 0x64d1f01, (0x315f bytes) 2024-05-11T23:03:14.913 INFO:teuthology.orchestra.run.smithi005.stdout:5626 write 0x5e88805 thru 0x5e8b3e1 (0x2bdd bytes) 2024-05-11T23:03:14.916 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:14.925 INFO:teuthology.orchestra.run.smithi005.stdout:5628 trunc from 0x6a3ee86 to 0xe72499e 2024-05-11T23:03:14.930 INFO:teuthology.orchestra.run.smithi005.stdout:5629 writesame 0x10f2d33 thru 0x10fca06 data_size 0x2(0x9cd4 bytes) 2024-05-11T23:03:14.948 INFO:teuthology.orchestra.run.smithi005.stdout:5630 read 0x743d7da thru 0x74461f6 (0x8a1d bytes) 2024-05-11T23:03:14.949 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:14.949 INFO:teuthology.orchestra.run.smithi005.stdout:5632 writesame 0x8e640a3 thru 0x8e71578 data_size 0x1b(0xd4d6 bytes) 2024-05-11T23:03:14.952 INFO:teuthology.orchestra.run.smithi005.stdout:5633 write 0xc92359e thru 0xc9287c9 (0x522c bytes) 2024-05-11T23:03:14.956 INFO:teuthology.orchestra.run.smithi005.stdout:5637 punch from 0xd1fa757 to 0xd1faf5e, (0x807 bytes) 2024-05-11T23:03:14.956 INFO:teuthology.orchestra.run.smithi005.stdout:5639 read 0x3ac0722 thru 0x3ac23d0 (0x1caf bytes) 2024-05-11T23:03:14.956 INFO:teuthology.orchestra.run.smithi005.stdout:5641 punch from 0xd5f0726 to 0xd5fa8e5, (0xa1bf bytes) 2024-05-11T23:03:14.956 INFO:teuthology.orchestra.run.smithi005.stdout:5642 punch from 0xdbfc280 to 0xdbfe921, (0x26a1 bytes) 2024-05-11T23:03:14.956 INFO:teuthology.orchestra.run.smithi005.stdout:5643 punch from 0x4f6adce to 0x4f6b22b, (0x45d bytes) 2024-05-11T23:03:14.966 INFO:teuthology.orchestra.run.smithi005.stdout:5644 trunc from 0xe72499e to 0x4fe13ee 2024-05-11T23:03:14.967 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:14.967 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:14.972 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:14.972+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:14.976 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:03:14.977 INFO:teuthology.orchestra.run.smithi005.stdout:5645 write 0x39dfaa8 thru 0x39ed993 (0xdeec bytes) 2024-05-11T23:03:14.981 INFO:teuthology.orchestra.run.smithi005.stdout:5646 punch from 0x25c96c9 to 0x25d0415, (0x6d4c bytes) 2024-05-11T23:03:14.981 INFO:teuthology.orchestra.run.smithi005.stdout:5647 read 0x732cf6 thru 0x73abda (0x7ee5 bytes) 2024-05-11T23:03:14.982 INFO:teuthology.orchestra.run.smithi005.stdout:5648 punch from 0x47df9ab to 0x47e52ac, (0x5901 bytes) 2024-05-11T23:03:14.982 INFO:teuthology.orchestra.run.smithi005.stdout:5649 punch from 0x2415f12 to 0x2416dff, (0xeed bytes) 2024-05-11T23:03:14.982 INFO:teuthology.orchestra.run.smithi005.stdout:5651 read 0x38384bd thru 0x3840111 (0x7c55 bytes) 2024-05-11T23:03:14.982 INFO:teuthology.orchestra.run.smithi005.stdout:5653 punch from 0x3482f5e to 0x348f4d4, (0xc576 bytes) 2024-05-11T23:03:14.982 INFO:teuthology.orchestra.run.smithi005.stdout:5654 write 0x3d05e thru 0x4cc22 (0xfbc5 bytes) 2024-05-11T23:03:14.989 INFO:teuthology.orchestra.run.smithi005.stdout:5655 write 0x51610c1 thru 0x516fc95 (0xebd5 bytes) 2024-05-11T23:03:15.003 INFO:teuthology.orchestra.run.smithi005.stdout:5659 write 0x9abb21e thru 0x9ac7da0 (0xcb83 bytes) 2024-05-11T23:03:15.007 INFO:teuthology.orchestra.run.smithi005.stdout:5661 punch from 0x3e0a3f2 to 0x3e0af9f, (0xbad bytes) 2024-05-11T23:03:15.008 INFO:teuthology.orchestra.run.smithi005.stdout:5662 trunc from 0x9ac7da1 to 0x90fe2a4 2024-05-11T23:03:15.018 INFO:teuthology.orchestra.run.smithi005.stdout:5663 punch from 0x6a97954 to 0x6aa671b, (0xedc7 bytes) 2024-05-11T23:03:15.018 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:15.018 INFO:teuthology.orchestra.run.smithi005.stdout:5666 punch from 0x4bee8 to 0x591d7, (0xd2ef bytes) 2024-05-11T23:03:15.018 INFO:teuthology.orchestra.run.smithi005.stdout:5667 punch from 0x67b6e3c to 0x67bb4f6, (0x46ba bytes) 2024-05-11T23:03:15.018 INFO:teuthology.orchestra.run.smithi005.stdout:5669 write 0x2c514fa thru 0x2c5ce7a (0xb981 bytes) 2024-05-11T23:03:15.030 INFO:teuthology.orchestra.run.smithi005.stdout:5670 writesame 0xc8c4a08 thru 0xc8cae8d data_size 0x3(0x6486 bytes) 2024-05-11T23:03:15.031 INFO:teuthology.orchestra.run.smithi005.stdout:5673 read 0xa8f438a thru 0xa8f6f2d (0x2ba4 bytes) 2024-05-11T23:03:15.031 INFO:teuthology.orchestra.run.smithi005.stdout:5675 punch from 0x65b816f to 0x65c6eb5, (0xed46 bytes) 2024-05-11T23:03:15.031 INFO:teuthology.orchestra.run.smithi005.stdout:5676 read 0x96a457c thru 0x96b2e94 (0xe919 bytes) 2024-05-11T23:03:15.031 INFO:teuthology.orchestra.run.smithi005.stdout:5677 read 0x58a4dab thru 0x58ab0cf (0x6325 bytes) 2024-05-11T23:03:15.031 INFO:teuthology.orchestra.run.smithi005.stdout:5678 read 0x5f53f2e thru 0x5f63a39 (0xfb0c bytes) 2024-05-11T23:03:15.031 INFO:teuthology.orchestra.run.smithi005.stdout:5679 read 0x3f0c93c thru 0x3f14155 (0x781a bytes) 2024-05-11T23:03:15.032 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:15.032 INFO:teuthology.orchestra.run.smithi005.stdout:5683 write 0x1e48c87 thru 0x1e50cba (0x8034 bytes) 2024-05-11T23:03:15.037 INFO:teuthology.orchestra.run.smithi005.stdout:5684 punch from 0x9aae488 to 0x9aafc35, (0x17ad bytes) 2024-05-11T23:03:15.037 INFO:teuthology.orchestra.run.smithi005.stdout:5686 punch from 0xb3020b7 to 0xb306ffc, (0x4f45 bytes) 2024-05-11T23:03:15.037 INFO:teuthology.orchestra.run.smithi005.stdout:5688 read 0xad0efa9 thru 0xad193ff (0xa457 bytes) 2024-05-11T23:03:15.037 INFO:teuthology.orchestra.run.smithi005.stdout:5689 write 0x902dd90 thru 0x903cdf5 (0xf066 bytes) 2024-05-11T23:03:15.040 INFO:teuthology.orchestra.run.smithi005.stdout:5691 writesame 0x3b741af thru 0x3b7bdde data_size 0x8(0x7c30 bytes) 2024-05-11T23:03:15.045 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:15.045 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:15.045 INFO:teuthology.orchestra.run.smithi005.stdout:5694 read 0x8ce046b thru 0x8ce4e8d (0x4a23 bytes) 2024-05-11T23:03:15.045 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:15.045 INFO:teuthology.orchestra.run.smithi005.stdout:5701 read 0x27c1e91 thru 0x27c9fae (0x811e bytes) 2024-05-11T23:03:15.046 INFO:teuthology.orchestra.run.smithi005.stdout:5702 read 0x1a3705f thru 0x1a3cb0a (0x5aac bytes) 2024-05-11T23:03:15.046 INFO:teuthology.orchestra.run.smithi005.stdout:5703 read 0x909fff5 thru 0x90a4325 (0x4331 bytes) 2024-05-11T23:03:15.046 INFO:teuthology.orchestra.run.smithi005.stdout:5706 read 0x2f5c909 thru 0x2f66832 (0x9f2a bytes) 2024-05-11T23:03:15.046 INFO:teuthology.orchestra.run.smithi005.stdout:5707 write 0x41eef3a thru 0x41f8da2 (0x9e69 bytes) 2024-05-11T23:03:15.049 INFO:teuthology.orchestra.run.smithi005.stdout:5708 read 0x9dcb122 thru 0x9dd086c (0x574b bytes) 2024-05-11T23:03:15.057 INFO:teuthology.orchestra.run.smithi005.stdout:5710 write 0xe661874 thru 0xe670d01 (0xf48e bytes) 2024-05-11T23:03:15.061 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:15.061 INFO:teuthology.orchestra.run.smithi005.stdout:5712 read 0x7319b4b thru 0x731a70c (0xbc2 bytes) 2024-05-11T23:03:15.061 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:15.061 INFO:teuthology.orchestra.run.smithi005.stdout:5714 writesame 0xc1de775 thru 0xc1e8398 data_size 0x2(0x9c24 bytes) 2024-05-11T23:03:15.070 INFO:teuthology.orchestra.run.smithi005.stdout:5715 read 0xc5190fe thru 0xc526b3b (0xda3e bytes) 2024-05-11T23:03:15.070 INFO:teuthology.orchestra.run.smithi005.stdout:5717 punch from 0x8008104 to 0x8011d3a, (0x9c36 bytes) 2024-05-11T23:03:15.070 INFO:teuthology.orchestra.run.smithi005.stdout:5718 write 0x1175cb3 thru 0x1180b3b (0xae89 bytes) 2024-05-11T23:03:15.072 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:15.073+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:15.078 INFO:teuthology.orchestra.run.smithi005.stdout:5719 write 0xbec05f3 thru 0xbeccca1 (0xc6af bytes) 2024-05-11T23:03:15.081 INFO:teuthology.orchestra.run.smithi005.stdout:5720 write 0x8564572 thru 0x8567163 (0x2bf2 bytes) 2024-05-11T23:03:15.084 INFO:teuthology.orchestra.run.smithi005.stdout:5721 punch from 0x4b80812 to 0x4b86344, (0x5b32 bytes) 2024-05-11T23:03:15.084 INFO:teuthology.orchestra.run.smithi005.stdout:5724 read 0xcf5a2d thru 0xcf9f10 (0x44e4 bytes) 2024-05-11T23:03:15.085 INFO:teuthology.orchestra.run.smithi005.stdout:5725 punch from 0x753476d to 0x7542569, (0xddfc bytes) 2024-05-11T23:03:15.085 INFO:teuthology.orchestra.run.smithi005.stdout:5726 read 0x83212c1 thru 0x8327613 (0x6353 bytes) 2024-05-11T23:03:15.085 INFO:teuthology.orchestra.run.smithi005.stdout:5728 write 0x8f124a3 thru 0x8f13c56 (0x17b4 bytes) 2024-05-11T23:03:15.088 INFO:teuthology.orchestra.run.smithi005.stdout:5730 writesame 0x966ca9 thru 0x970415 data_size 0x1(0x976d bytes) 2024-05-11T23:03:15.114 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:15.114 INFO:teuthology.orchestra.run.smithi005.stdout:5734 punch from 0xa7bd55 to 0xa86a27, (0xacd2 bytes) 2024-05-11T23:03:15.119 INFO:teuthology.orchestra.run.smithi005.stdout:5735 trunc from 0xe670d02 to 0x942ba2a 2024-05-11T23:03:15.132 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:15.133 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:15.133 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:15.133 INFO:teuthology.orchestra.run.smithi005.stdout:5738 write 0x2138600 thru 0x2144308 (0xbd09 bytes) 2024-05-11T23:03:15.139 INFO:teuthology.orchestra.run.smithi005.stdout:5739 read 0x3b26cbd thru 0x3b2d765 (0x6aa9 bytes) 2024-05-11T23:03:15.139 INFO:teuthology.orchestra.run.smithi005.stdout:5742 writesame 0x8fd5aeb thru 0x8fd8c36 data_size 0xa(0x314c bytes) 2024-05-11T23:03:15.142 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:03:15.143 INFO:teuthology.orchestra.run.smithi005.stdout:5743 punch from 0x765af52 to 0x765e9c7, (0x3a75 bytes) 2024-05-11T23:03:15.151 INFO:teuthology.orchestra.run.smithi005.stdout:5744 write 0xb3a5249 thru 0xb3aa7ac (0x5564 bytes) 2024-05-11T23:03:15.154 INFO:teuthology.orchestra.run.smithi005.stdout:5747 read 0x441a7e4 thru 0x4421180 (0x699d bytes) 2024-05-11T23:03:15.154 INFO:teuthology.orchestra.run.smithi005.stdout:5748 read 0x75c26bb thru 0x75caff5 (0x893b bytes) 2024-05-11T23:03:15.154 INFO:teuthology.orchestra.run.smithi005.stdout:5749 write 0x18175b8 thru 0x181baa3 (0x44ec bytes) 2024-05-11T23:03:15.158 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:15.158 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:15.158 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:15.158 INFO:teuthology.orchestra.run.smithi005.stdout:5753 read 0x38f2fa8 thru 0x38f30c6 (0x11f bytes) 2024-05-11T23:03:15.158 INFO:teuthology.orchestra.run.smithi005.stdout:5754 read 0xa580d thru 0xad231 (0x7a25 bytes) 2024-05-11T23:03:15.167 INFO:teuthology.orchestra.run.smithi005.stdout:5755 writesame 0xd42c12c thru 0xd438312 data_size 0x1(0xc1e7 bytes) 2024-05-11T23:03:15.167 INFO:teuthology.orchestra.run.smithi005.stdout:5756 read 0x6b576 thru 0x6e876 (0x3301 bytes) 2024-05-11T23:03:15.167 INFO:teuthology.orchestra.run.smithi005.stdout:5757 read 0x40d7507 thru 0x40dafc2 (0x3abc bytes) 2024-05-11T23:03:15.168 INFO:teuthology.orchestra.run.smithi005.stdout:5758 read 0x4b4a6b4 thru 0x4b5817c (0xdac9 bytes) 2024-05-11T23:03:15.168 INFO:teuthology.orchestra.run.smithi005.stdout:5759 trunc from 0xd438313 to 0xcb6d6c4 2024-05-11T23:03:15.174 INFO:teuthology.orchestra.run.smithi005.stdout:5760 punch from 0x1aa1985 to 0x1aa46d0, (0x2d4b bytes) 2024-05-11T23:03:15.174 INFO:teuthology.orchestra.run.smithi005.stdout:5761 writesame 0x89250a thru 0x8a025e data_size 0x65(0xdd55 bytes) 2024-05-11T23:03:15.184 INFO:teuthology.orchestra.run.smithi005.stdout:5762 write 0x9d47c7f thru 0x9d4ac02 (0x2f84 bytes) 2024-05-11T23:03:15.186 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:15.186+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:15.187 INFO:teuthology.orchestra.run.smithi005.stdout:5763 punch from 0x3b0d148 to 0x3b16f4d, (0x9e05 bytes) 2024-05-11T23:03:15.187 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:15.188 INFO:teuthology.orchestra.run.smithi005.stdout:5765 write 0x4ccd468 thru 0x4cd2c73 (0x580c bytes) 2024-05-11T23:03:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.0 primary-affinity to 0.437347 (8286612) 2024-05-11T23:03:15.200 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-05-11T23:03:15.204 INFO:teuthology.orchestra.run.smithi005.stdout:5766 write 0xce1db1d thru 0xce26968 (0x8e4c bytes) 2024-05-11T23:03:15.253 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:15.253 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:15.261 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:03:15.302 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:15.302+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:15.375 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:15.375+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:15.407 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:15.408 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:15.416 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:03:15.476 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:15.476+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:15.522 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:15.522 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:15.532 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:03:15.577 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:15.577+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:15.637 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:15.638 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:15.647 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:03:15.678 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:15.678+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:15.751 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:15.752 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:15.760 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:03:15.806 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:15.806+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:15.880 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:15.880+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:15.883 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:15.883 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:15.892 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:03:15.981 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:15.981+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:15.992 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:03:16.046 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:16.046 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:16.056 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:03:16.083 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:16.083+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:16.120 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:16.120 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:16.138 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:16.138 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:16.155 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:16.156 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:16.171 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:16.171 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226051 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6049792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.852829624, 2024-05-11T23:03:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050622418000000002, 2024-05-11T23:03:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226051 2024-05-11T23:03:16.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317156+0000", 2024-05-11T23:03:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317157+0000", 2024-05-11T23:03:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.144e-06 2024-05-11T23:03:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317160+0000", 2024-05-11T23:03:16.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1350000000000001e-06 2024-05-11T23:03:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317162+0000", 2024-05-11T23:03:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5050000000000002e-06 2024-05-11T23:03:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367689+0000", 2024-05-11T23:03:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526797999999998 2024-05-11T23:03:16.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367735+0000", 2024-05-11T23:03:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5698999999999997e-05 2024-05-11T23:03:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367778+0000", 2024-05-11T23:03:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2928e-05 2024-05-11T23:03:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.176 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226052 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6060032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.852819567999999, 2024-05-11T23:03:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050830110999999997, 2024-05-11T23:03:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226052 2024-05-11T23:03:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317166+0000", 2024-05-11T23:03:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:03:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317167+0000", 2024-05-11T23:03:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.155e-06 2024-05-11T23:03:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317170+0000", 2024-05-11T23:03:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.061e-06 2024-05-11T23:03:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317172+0000", 2024-05-11T23:03:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4420000000000001e-06 2024-05-11T23:03:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367816+0000", 2024-05-11T23:03:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050644013000000002 2024-05-11T23:03:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367906+0000", 2024-05-11T23:03:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9739000000000003e-05 2024-05-11T23:03:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367995+0000", 2024-05-11T23:03:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9499000000000005e-05 2024-05-11T23:03:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226053 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6070272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.852809529, 2024-05-11T23:03:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051106912999999997, 2024-05-11T23:03:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.181 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226053 2024-05-11T23:03:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:16.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317176+0000", 2024-05-11T23:03:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0100000000000001e-07 2024-05-11T23:03:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317179+0000", 2024-05-11T23:03:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-05-11T23:03:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317182+0000", 2024-05-11T23:03:16.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3589999999999999e-06 2024-05-11T23:03:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317184+0000", 2024-05-11T23:03:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4339999999999999e-06 2024-05-11T23:03:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368061+0000", 2024-05-11T23:03:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050876927000000002 2024-05-11T23:03:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368179+0000", 2024-05-11T23:03:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118001 2024-05-11T23:03:16.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368282+0000", 2024-05-11T23:03:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102942 2024-05-11T23:03:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226054 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6080512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.852797207, 2024-05-11T23:03:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051322, 2024-05-11T23:03:16.185 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226054 2024-05-11T23:03:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317189+0000", 2024-05-11T23:03:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.217e-06 2024-05-11T23:03:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317192+0000", 2024-05-11T23:03:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0699999999999998e-06 2024-05-11T23:03:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317195+0000", 2024-05-11T23:03:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3829999999999999e-06 2024-05-11T23:03:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368328+0000", 2024-05-11T23:03:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051132999999999998 2024-05-11T23:03:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368429+0000", 2024-05-11T23:03:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010168800000000001 2024-05-11T23:03:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368510+0000", 2024-05-11T23:03:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0420999999999995e-05 2024-05-11T23:03:16.189 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226055 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6090752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.852787139, 2024-05-11T23:03:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051562835000000001, 2024-05-11T23:03:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.190 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226055 2024-05-11T23:03:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317256+0000", 2024-05-11T23:03:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8047000000000002e-05 2024-05-11T23:03:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317259+0000", 2024-05-11T23:03:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2590000000000001e-06 2024-05-11T23:03:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317267+0000", 2024-05-11T23:03:16.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8769999999999993e-06 2024-05-11T23:03:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317274+0000", 2024-05-11T23:03:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.187e-06 2024-05-11T23:03:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368544+0000", 2024-05-11T23:03:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051270299999999998 2024-05-11T23:03:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368658+0000", 2024-05-11T23:03:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000113347 2024-05-11T23:03:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368761+0000", 2024-05-11T23:03:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102818 2024-05-11T23:03:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226056 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6100992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.852701292999999, 2024-05-11T23:03:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051615576000000003, 2024-05-11T23:03:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226056 2024-05-11T23:03:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318915+0000", 2024-05-11T23:03:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001631291 2024-05-11T23:03:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318918+0000", 2024-05-11T23:03:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.512e-06 2024-05-11T23:03:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318927+0000", 2024-05-11T23:03:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1960000000000008e-06 2024-05-11T23:03:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318932+0000", 2024-05-11T23:03:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:03:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368805+0000", 2024-05-11T23:03:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049872998000000002 2024-05-11T23:03:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368853+0000", 2024-05-11T23:03:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8041999999999998e-05 2024-05-11T23:03:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368899+0000", 2024-05-11T23:03:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5766000000000002e-05 2024-05-11T23:03:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.198 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226061 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6152192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.850910206, 2024-05-11T23:03:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050675395999999998, 2024-05-11T23:03:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226061 2024-05-11T23:03:16.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319078+0000", 2024-05-11T23:03:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5670000000000002e-06 2024-05-11T23:03:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319085+0000", 2024-05-11T23:03:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2570000000000001e-06 2024-05-11T23:03:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319100+0000", 2024-05-11T23:03:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.489e-05 2024-05-11T23:03:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369584+0000", 2024-05-11T23:03:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050484345999999999 2024-05-11T23:03:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369671+0000", 2024-05-11T23:03:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6654000000000001e-05 2024-05-11T23:03:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369750+0000", 2024-05-11T23:03:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9257000000000004e-05 2024-05-11T23:03:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226062 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6162432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.850876661999999, 2024-05-11T23:03:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050761158000000001, 2024-05-11T23:03:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226062 2024-05-11T23:03:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319109+0000", 2024-05-11T23:03:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2800000000000002e-07 2024-05-11T23:03:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319111+0000", 2024-05-11T23:03:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4980000000000001e-06 2024-05-11T23:03:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319120+0000", 2024-05-11T23:03:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3999999999999992e-06 2024-05-11T23:03:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319129+0000", 2024-05-11T23:03:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7580000000000004e-06 2024-05-11T23:03:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369775+0000", 2024-05-11T23:03:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050645580000000003 2024-05-11T23:03:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369824+0000", 2024-05-11T23:03:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9506999999999999e-05 2024-05-11T23:03:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369869+0000", 2024-05-11T23:03:16.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4987000000000001e-05 2024-05-11T23:03:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226063 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6172672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.850847705, 2024-05-11T23:03:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050939784000000002, 2024-05-11T23:03:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.208 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226063 2024-05-11T23:03:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319138+0000", 2024-05-11T23:03:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.39e-07 2024-05-11T23:03:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319140+0000", 2024-05-11T23:03:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4820000000000001e-06 2024-05-11T23:03:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319148+0000", 2024-05-11T23:03:16.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6599999999999995e-06 2024-05-11T23:03:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319154+0000", 2024-05-11T23:03:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4169999999999997e-06 2024-05-11T23:03:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369905+0000", 2024-05-11T23:03:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050751009999999999 2024-05-11T23:03:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369991+0000", 2024-05-11T23:03:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5909999999999996e-05 2024-05-11T23:03:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370077+0000", 2024-05-11T23:03:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5865999999999994e-05 2024-05-11T23:03:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226064 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6182912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:16.212 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.850821508999999, 2024-05-11T23:03:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051038241999999998, 2024-05-11T23:03:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226064 2024-05-11T23:03:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319164+0000", 2024-05-11T23:03:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9299999999999999e-07 2024-05-11T23:03:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319166+0000", 2024-05-11T23:03:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.48e-06 2024-05-11T23:03:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319176+0000", 2024-05-11T23:03:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9669999999999996e-06 2024-05-11T23:03:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319180+0000", 2024-05-11T23:03:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8020000000000002e-06 2024-05-11T23:03:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370100+0000", 2024-05-11T23:03:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050919845999999998 2024-05-11T23:03:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370155+0000", 2024-05-11T23:03:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5040000000000002e-05 2024-05-11T23:03:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370202+0000", 2024-05-11T23:03:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6714e-05 2024-05-11T23:03:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.216 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226065 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6193152~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.850798939000001, 2024-05-11T23:03:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051225087000000002, 2024-05-11T23:03:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.218 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226065 2024-05-11T23:03:16.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.218 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320913+0000", 2024-05-11T23:03:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001727492 2024-05-11T23:03:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320919+0000", 2024-05-11T23:03:16.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.964e-06 2024-05-11T23:03:16.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320927+0000", 2024-05-11T23:03:16.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2239999999999998e-06 2024-05-11T23:03:16.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320934+0000", 2024-05-11T23:03:16.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8469999999999998e-06 2024-05-11T23:03:16.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370240+0000", 2024-05-11T23:03:16.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049306562999999998 2024-05-11T23:03:16.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370328+0000", 2024-05-11T23:03:16.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8122999999999996e-05 2024-05-11T23:03:16.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370411+0000", 2024-05-11T23:03:16.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2874000000000006e-05 2024-05-11T23:03:16.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.222 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.222 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.222 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226073 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6275072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:16.222 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:16.222 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.848960551999999, 2024-05-11T23:03:16.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050730475999999997, 2024-05-11T23:03:16.222 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.222 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226073 2024-05-11T23:03:16.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.223 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:16.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:16.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322914+0000", 2024-05-11T23:03:16.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001889153 2024-05-11T23:03:16.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322917+0000", 2024-05-11T23:03:16.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2100000000000002e-06 2024-05-11T23:03:16.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322924+0000", 2024-05-11T23:03:16.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9929999999999998e-06 2024-05-11T23:03:16.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322928+0000", 2024-05-11T23:03:16.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5549999999999996e-06 2024-05-11T23:03:16.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371578+0000", 2024-05-11T23:03:16.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048650001999999998 2024-05-11T23:03:16.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371671+0000", 2024-05-11T23:03:16.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2230000000000003e-05 2024-05-11T23:03:16.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371755+0000", 2024-05-11T23:03:16.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4332999999999994e-05 2024-05-11T23:03:16.226 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.226 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.226 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.226 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226085 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 5338112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:16.226 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:16.226 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.846869453, 2024-05-11T23:03:16.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050727622, 2024-05-11T23:03:16.226 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.226 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.226 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.226 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.227 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.227 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.227 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226085 2024-05-11T23:03:16.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.227 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:16.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:16.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324914+0000", 2024-05-11T23:03:16.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001798298 2024-05-11T23:03:16.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324918+0000", 2024-05-11T23:03:16.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3889999999999998e-06 2024-05-11T23:03:16.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324928+0000", 2024-05-11T23:03:16.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4339999999999998e-06 2024-05-11T23:03:16.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324936+0000", 2024-05-11T23:03:16.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2770000000000005e-06 2024-05-11T23:03:16.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373664+0000", 2024-05-11T23:03:16.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048728207000000003 2024-05-11T23:03:16.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373757+0000", 2024-05-11T23:03:16.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3332999999999996e-05 2024-05-11T23:03:16.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373843+0000", 2024-05-11T23:03:16.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5684000000000002e-05 2024-05-11T23:03:16.230 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.230 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.230 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.230 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232555 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7789568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:16.230 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:16.230 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.369859128, 2024-05-11T23:03:16.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050583086999999999, 2024-05-11T23:03:16.231 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.231 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.231 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.231 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.231 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232555 2024-05-11T23:03:16.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.231 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:16.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:16.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:16.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:03:16.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800127+0000", 2024-05-11T23:03:16.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.207e-06 2024-05-11T23:03:16.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800132+0000", 2024-05-11T23:03:16.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:03:16.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800134+0000", 2024-05-11T23:03:16.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5459999999999998e-06 2024-05-11T23:03:16.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850548+0000", 2024-05-11T23:03:16.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050414182000000002 2024-05-11T23:03:16.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850635+0000", 2024-05-11T23:03:16.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6853000000000003e-05 2024-05-11T23:03:16.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850709+0000", 2024-05-11T23:03:16.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3585999999999996e-05 2024-05-11T23:03:16.235 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.235 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.235 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.235 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232556 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7799808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:16.235 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:16.235 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.369847474, 2024-05-11T23:03:16.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050749599999999999, 2024-05-11T23:03:16.235 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.235 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.235 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.237 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.237 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232556 2024-05-11T23:03:16.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.237 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:16.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:16.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801914+0000", 2024-05-11T23:03:16.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017766360000000001 2024-05-11T23:03:16.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801919+0000", 2024-05-11T23:03:16.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.42e-06 2024-05-11T23:03:16.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801927+0000", 2024-05-11T23:03:16.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1999999999999994e-06 2024-05-11T23:03:16.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801931+0000", 2024-05-11T23:03:16.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4279999999999998e-06 2024-05-11T23:03:16.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850740+0000", 2024-05-11T23:03:16.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048808658999999997 2024-05-11T23:03:16.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850819+0000", 2024-05-11T23:03:16.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9102000000000003e-05 2024-05-11T23:03:16.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850887+0000", 2024-05-11T23:03:16.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8155000000000001e-05 2024-05-11T23:03:16.240 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.240 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.240 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.241 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232578 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8025088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:16.241 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:16.241 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.366554236000001, 2024-05-11T23:03:16.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050679473000000003, 2024-05-11T23:03:16.241 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.241 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.241 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.241 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.241 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232578 2024-05-11T23:03:16.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.241 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:16.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:16.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:16.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:16.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803432+0000", 2024-05-11T23:03:16.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.139e-06 2024-05-11T23:03:16.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803435+0000", 2024-05-11T23:03:16.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2440000000000002e-06 2024-05-11T23:03:16.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803439+0000", 2024-05-11T23:03:16.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3129999999999999e-06 2024-05-11T23:03:16.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.853953+0000", 2024-05-11T23:03:16.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050514846000000002 2024-05-11T23:03:16.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854035+0000", 2024-05-11T23:03:16.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1148000000000003e-05 2024-05-11T23:03:16.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854110+0000", 2024-05-11T23:03:16.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5562000000000006e-05 2024-05-11T23:03:16.245 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.245 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.245 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.245 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232579 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8035328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:16.245 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:16.245 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.366541632000001, 2024-05-11T23:03:16.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050886656000000002, 2024-05-11T23:03:16.245 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.245 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.245 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.245 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.246 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.246 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232579 2024-05-11T23:03:16.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.246 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:16.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:16.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803914+0000", 2024-05-11T23:03:16.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00047026700000000002 2024-05-11T23:03:16.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803918+0000", 2024-05-11T23:03:16.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3460000000000004e-06 2024-05-11T23:03:16.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803927+0000", 2024-05-11T23:03:16.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5030000000000003e-06 2024-05-11T23:03:16.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803933+0000", 2024-05-11T23:03:16.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0310000000000004e-06 2024-05-11T23:03:16.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854153+0000", 2024-05-11T23:03:16.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050219321999999997 2024-05-11T23:03:16.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854242+0000", 2024-05-11T23:03:16.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9457999999999996e-05 2024-05-11T23:03:16.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854330+0000", 2024-05-11T23:03:16.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7729e-05 2024-05-11T23:03:16.249 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.249 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.249 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.249 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232581 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8055808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:16.249 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:16.249 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.366018155000001, 2024-05-11T23:03:16.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050664439999999998, 2024-05-11T23:03:16.250 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.250 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.250 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.250 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.250 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.250 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.250 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232581 2024-05-11T23:03:16.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.250 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:16.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:16.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:16.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:03:16.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803969+0000", 2024-05-11T23:03:16.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0889999999999998e-06 2024-05-11T23:03:16.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803978+0000", 2024-05-11T23:03:16.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3070000000000003e-06 2024-05-11T23:03:16.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803983+0000", 2024-05-11T23:03:16.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1599999999999997e-06 2024-05-11T23:03:16.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854509+0000", 2024-05-11T23:03:16.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526339000000003 2024-05-11T23:03:16.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854576+0000", 2024-05-11T23:03:16.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7102999999999998e-05 2024-05-11T23:03:16.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.253 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.253 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854631+0000", 2024-05-11T23:03:16.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4997999999999998e-05 2024-05-11T23:03:16.253 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.253 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.253 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.254 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232582 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8066048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:16.254 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:16.254 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.36599541, 2024-05-11T23:03:16.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050753983000000003, 2024-05-11T23:03:16.254 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.254 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.254 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.254 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.254 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.254 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.254 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232582 2024-05-11T23:03:16.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.255 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:16.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:16.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805917+0000", 2024-05-11T23:03:16.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001927432 2024-05-11T23:03:16.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805920+0000", 2024-05-11T23:03:16.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1219999999999999e-06 2024-05-11T23:03:16.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805933+0000", 2024-05-11T23:03:16.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2727e-05 2024-05-11T23:03:16.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805941+0000", 2024-05-11T23:03:16.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7090000000000002e-06 2024-05-11T23:03:16.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854657+0000", 2024-05-11T23:03:16.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048716164999999999 2024-05-11T23:03:16.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854699+0000", 2024-05-11T23:03:16.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2531999999999999e-05 2024-05-11T23:03:16.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.257 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854744+0000", 2024-05-11T23:03:16.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4295999999999997e-05 2024-05-11T23:03:16.258 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.258 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.258 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.258 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232593 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8178688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:16.258 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:16.258 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.363796017, 2024-05-11T23:03:16.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050735178999999998, 2024-05-11T23:03:16.258 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.258 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.258 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.258 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.259 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.259 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.259 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232593 2024-05-11T23:03:16.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.259 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:16.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:16.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807913+0000", 2024-05-11T23:03:16.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017240319999999999 2024-05-11T23:03:16.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807916+0000", 2024-05-11T23:03:16.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4589999999999998e-06 2024-05-11T23:03:16.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807924+0000", 2024-05-11T23:03:16.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.272e-06 2024-05-11T23:03:16.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807928+0000", 2024-05-11T23:03:16.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5369999999999999e-06 2024-05-11T23:03:16.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856768+0000", 2024-05-11T23:03:16.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048839489999999999 2024-05-11T23:03:16.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856845+0000", 2024-05-11T23:03:16.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7746999999999999e-05 2024-05-11T23:03:16.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856924+0000", 2024-05-11T23:03:16.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8641999999999997e-05 2024-05-11T23:03:16.262 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.262 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.262 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.262 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.262 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.262 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:16.263 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:03:16.263 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:03:16.264 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:03:16.265 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:16.265 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:16.265 INFO:teuthology.orchestra.run.smithi005.stdout:5767 punch from 0x3e063a0 to 0x3e0f8c7, (0x9527 bytes) 2024-05-11T23:03:16.266 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:16.266 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:16.266 INFO:teuthology.orchestra.run.smithi005.stdout:5768 trunc from 0xce26969 to 0xbc18913 2024-05-11T23:03:16.266 INFO:teuthology.orchestra.run.smithi005.stdout:5770 write 0x5a40746 thru 0x5a43e6b (0x3726 bytes) 2024-05-11T23:03:16.267 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:16.267 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:16.267 INFO:teuthology.orchestra.run.smithi005.stdout:5771 trunc from 0xbc18913 to 0x21ea5d1 2024-05-11T23:03:16.268 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:16.268 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:16.268 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:16.263+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:16.275 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:16.282 INFO:teuthology.orchestra.run.smithi005.stdout:5773 write 0x3538463 thru 0x3539a2e (0x15cc bytes) 2024-05-11T23:03:16.285 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:16.296 INFO:teuthology.orchestra.run.smithi005.stdout:5776 write 0x7d2f6cb thru 0x7d37c47 (0x857d bytes) 2024-05-11T23:03:16.301 INFO:teuthology.orchestra.run.smithi005.stdout:5777 write 0x38158e1 thru 0x382521e (0xf93e bytes) 2024-05-11T23:03:16.308 INFO:teuthology.orchestra.run.smithi005.stdout:5779 trunc from 0x7d37c48 to 0xb10ed96 2024-05-11T23:03:16.314 INFO:teuthology.orchestra.run.smithi005.stdout:5780 punch from 0x9765d7d to 0x97715ec, (0xb86f bytes) 2024-05-11T23:03:16.314 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:16.314 INFO:teuthology.orchestra.run.smithi005.stdout:5782 read 0x7b7b234 thru 0x7b7b81a (0x5e7 bytes) 2024-05-11T23:03:16.316 INFO:teuthology.orchestra.run.smithi005.stdout:5783 trunc from 0xb10ed96 to 0x9b96368 2024-05-11T23:03:16.322 INFO:teuthology.orchestra.run.smithi005.stdout:5784 punch from 0x4e9bc58 to 0x4ea70ca, (0xb472 bytes) 2024-05-11T23:03:16.322 INFO:teuthology.orchestra.run.smithi005.stdout:5785 read 0x106b92 thru 0x10c199 (0x5608 bytes) 2024-05-11T23:03:16.323 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:16.323 INFO:teuthology.orchestra.run.smithi005.stdout:5787 read 0x5bd2cc9 thru 0x5bd9005 (0x633d bytes) 2024-05-11T23:03:16.323 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:16.330 INFO:teuthology.orchestra.run.smithi005.stdout:5790 write 0xacbc90d thru 0xacc68b5 (0x9fa9 bytes) 2024-05-11T23:03:16.334 INFO:teuthology.orchestra.run.smithi005.stdout:5791 writesame 0x232f111 thru 0x2331e17 data_size 0x1(0x2d07 bytes) 2024-05-11T23:03:16.338 INFO:teuthology.orchestra.run.smithi005.stdout:5792 punch from 0x3deb648 to 0x3def0ab, (0x3a63 bytes) 2024-05-11T23:03:16.338 INFO:teuthology.orchestra.run.smithi005.stdout:5793 punch from 0x400a865 to 0x40176ed, (0xce88 bytes) 2024-05-11T23:03:16.338 INFO:teuthology.orchestra.run.smithi005.stdout:5794 writesame 0x3afe7f0 thru 0x3b01a2b data_size 0xa(0x323c bytes) 2024-05-11T23:03:16.342 INFO:teuthology.orchestra.run.smithi005.stdout:5795 writesame 0xf72277 thru 0xf79708 data_size 0x2b(0x7492 bytes) 2024-05-11T23:03:16.361 INFO:teuthology.orchestra.run.smithi005.stdout:5796 write 0xe81a889 thru 0xe821fb5 (0x772d bytes) 2024-05-11T23:03:16.364 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:16.364+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:16.365 INFO:teuthology.orchestra.run.smithi005.stdout:5798 read 0x93b8e7a thru 0x93c6090 (0xd217 bytes) 2024-05-11T23:03:16.365 INFO:teuthology.orchestra.run.smithi005.stdout:5799 punch from 0xeb10fc to 0xeb5f0f, (0x4e13 bytes) 2024-05-11T23:03:16.365 INFO:teuthology.orchestra.run.smithi005.stdout:5801 write 0x1857b6b thru 0x185bf58 (0x43ee bytes) 2024-05-11T23:03:16.368 INFO:teuthology.orchestra.run.smithi005.stdout:5803 trunc from 0xe821fb6 to 0xd6522c6 2024-05-11T23:03:16.378 INFO:teuthology.orchestra.run.smithi005.stdout:5805 write 0x6a3cff0 thru 0x6a45a81 (0x8a92 bytes) 2024-05-11T23:03:16.383 INFO:teuthology.orchestra.run.smithi005.stdout:5807 read 0x6c647c8 thru 0x6c6e0e5 (0x991e bytes) 2024-05-11T23:03:16.383 INFO:teuthology.orchestra.run.smithi005.stdout:5809 write 0x8e94e01 thru 0x8e9ba7f (0x6c7f bytes) 2024-05-11T23:03:16.394 INFO:teuthology.orchestra.run.smithi005.stdout:5811 write 0xea3ec5a thru 0xea47a9a (0x8e41 bytes) 2024-05-11T23:03:16.398 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:16.398 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:16.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.398 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:274284 3.4 3:2181fb79:::rbd_data.111ecf9f3be0.00000000000001d8:head [set-alloc-hint object_size 524288 write_size 524288,write 0~31387 in=31387b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113)", 2024-05-11T23:03:16.398 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:16.397665+0000", 2024-05-11T23:03:16.398 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0014012989999999999, 2024-05-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001421262, 2024-05-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "waiting for sub ops", 2024-05-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 274284 2024-05-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:16.397665+0000", 2024-05-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:16.397665+0000", 2024-05-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:16.397666+0000", 2024-05-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8899999999999998e-07 2024-05-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:16.397679+0000", 2024-05-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3314e-05 2024-05-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:16.397690+0000", 2024-05-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0599e-05 2024-05-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:16.397697+0000", 2024-05-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4549999999999998e-06 2024-05-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:16.397746+0000", 2024-05-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9270000000000001e-05 2024-05-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:16.397850+0000", 2024-05-11T23:03:16.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010322 2024-05-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 2", 2024-05-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:16.397945+0000", 2024-05-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5505000000000001e-05 2024-05-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "op_commit", 2024-05-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:16.398718+0000", 2024-05-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00077313999999999996 2024-05-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_repop_reply(client.4382.0:274284 3.4 e113/19 ondisk, result = 0)", 2024-05-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:16.399059+0000", 2024-05-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 7.0090000000000001e-06, 2024-05-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6083999999999998e-05, 2024-05-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:16.399059+0000", 2024-05-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:16.399059+0000", 2024-05-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:16.399060+0000", 2024-05-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.5300000000000004e-07 2024-05-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:16.399063+0000", 2024-05-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2310000000000001e-06 2024-05-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:16.399073+0000", 2024-05-11T23:03:16.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5079999999999994e-06 2024-05-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:16.399079+0000", 2024-05-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6309999999999997e-06 2024-05-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:16.399122+0000", 2024-05-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2348999999999997e-05 2024-05-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:16.399128+0000", 2024-05-11T23:03:16.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1020000000000002e-06 2024-05-11T23:03:16.406 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.406 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.406 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.406 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.406 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:03:16.406 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 2 2024-05-11T23:03:16.406 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:16.411 INFO:teuthology.orchestra.run.smithi005.stdout:5812 trunc from 0xea47a9b to 0x33d8be0 2024-05-11T23:03:16.414 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:03:16.429 INFO:teuthology.orchestra.run.smithi005.stdout:5813 writesame 0x37aecd2 thru 0x37b6ba5 data_size 0x2(0x7ed4 bytes) 2024-05-11T23:03:16.429 INFO:teuthology.orchestra.run.smithi005.stdout:5814 flatten 2024-05-11T23:03:16.474 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:16.474+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:16.522 INFO:teuthology.orchestra.run.smithi005.stdout:5817 writesame 0x1d374c5 thru 0x1d432f8 data_size 0x4a(0xbe34 bytes) 2024-05-11T23:03:16.526 INFO:teuthology.orchestra.run.smithi005.stdout:5818 read 0x35f8a72 thru 0x35fcb49 (0x40d8 bytes) 2024-05-11T23:03:16.526 INFO:teuthology.orchestra.run.smithi005.stdout:5819 read 0x1a5061b thru 0x1a54dde (0x47c4 bytes) 2024-05-11T23:03:16.545 INFO:teuthology.orchestra.run.smithi005.stdout:5821 write 0xe653295 thru 0xe65837f (0x50eb bytes) 2024-05-11T23:03:16.549 INFO:teuthology.orchestra.run.smithi005.stdout:5822 read 0x13b1fa2 thru 0x13c0b5d (0xebbc bytes) 2024-05-11T23:03:16.549 INFO:teuthology.orchestra.run.smithi005.stdout:5823 write 0x82fcf65 thru 0x82feeee (0x1f8a bytes) 2024-05-11T23:03:16.553 INFO:teuthology.orchestra.run.smithi005.stdout:5826 read 0xa1315b4 thru 0xa133b9d (0x25ea bytes) 2024-05-11T23:03:16.553 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:16.553 INFO:teuthology.orchestra.run.smithi005.stdout:5828 writesame 0x9ed3d9f thru 0x9ee2d62 data_size 0x2d(0xefc4 bytes) 2024-05-11T23:03:16.557 INFO:teuthology.orchestra.run.smithi005.stdout:5829 read 0xcfe25ae thru 0xcfe2c48 (0x69b bytes) 2024-05-11T23:03:16.558 INFO:teuthology.orchestra.run.smithi005.stdout:5831 writesame 0x717c081 thru 0x718751c data_size 0x6(0xb49c bytes) 2024-05-11T23:03:16.564 INFO:teuthology.orchestra.run.smithi005.stdout:5832 read 0x83feee4 thru 0x8403218 (0x4335 bytes) 2024-05-11T23:03:16.564 INFO:teuthology.orchestra.run.smithi005.stdout:5834 punch from 0x7e41eae to 0x7e49037, (0x7189 bytes) 2024-05-11T23:03:16.564 INFO:teuthology.orchestra.run.smithi005.stdout:5835 writesame 0x8cd3a07 thru 0x8ce2623 data_size 0x4d(0xec1d bytes) 2024-05-11T23:03:16.566 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:16.566+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:16.569 INFO:teuthology.orchestra.run.smithi005.stdout:5836 write 0x4910662 thru 0x491f90f (0xf2ae bytes) 2024-05-11T23:03:16.583 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:16.583 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:16.583 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:16.586 INFO:teuthology.orchestra.run.smithi005.stdout:5837 trunc from 0xe658380 to 0x211140e 2024-05-11T23:03:16.597 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:03:16.616 INFO:teuthology.orchestra.run.smithi005.stdout:5838 write 0x9a10e01 thru 0x9a1a9e7 (0x9be7 bytes) 2024-05-11T23:03:16.626 INFO:teuthology.orchestra.run.smithi005.stdout:5840 writesame 0xa9f97ff thru 0xaa024b1 data_size 0x1(0x8cb3 bytes) 2024-05-11T23:03:16.626 INFO:teuthology.orchestra.run.smithi005.stdout:5841 read 0x743478b thru 0x743ce67 (0x86dd bytes) 2024-05-11T23:03:16.626 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:16.626 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:16.626 INFO:teuthology.orchestra.run.smithi005.stdout:5846 read 0x7201b thru 0x77637 (0x561d bytes) 2024-05-11T23:03:16.627 INFO:teuthology.orchestra.run.smithi005.stdout:5847 writesame 0x46f195d thru 0x47008d5 data_size 0xb7(0xef79 bytes) 2024-05-11T23:03:16.640 INFO:teuthology.orchestra.run.smithi005.stdout:5848 trunc from 0xaa024b2 to 0x25ff163 2024-05-11T23:03:16.653 INFO:teuthology.orchestra.run.smithi005.stdout:5850 trunc from 0x25ff163 to 0x90b2bc 2024-05-11T23:03:16.668 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:16.667+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:16.679 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:16.680 INFO:teuthology.orchestra.run.smithi005.stdout:5852 writesame 0x13bac5 thru 0x14b14f data_size 0xd(0xf68b bytes) 2024-05-11T23:03:16.691 INFO:teuthology.orchestra.run.smithi005.stdout:5853 trunc from 0x90b2bc to 0x7ae12d4 2024-05-11T23:03:16.705 INFO:teuthology.orchestra.run.smithi005.stdout:5854 writesame 0x95f1231 thru 0x95f7d7f data_size 0x1(0x6b4f bytes) 2024-05-11T23:03:16.705 INFO:teuthology.orchestra.run.smithi005.stdout:5855 read 0x7395d64 thru 0x73a231a (0xc5b7 bytes) 2024-05-11T23:03:16.714 INFO:teuthology.orchestra.run.smithi005.stdout:5856 writesame 0xcae4fd6 thru 0xcaf32d6 data_size 0x3(0xe301 bytes) 2024-05-11T23:03:16.714 INFO:teuthology.orchestra.run.smithi005.stdout:5859 write 0xc69720a thru 0xc6a4bfe (0xd9f5 bytes) 2024-05-11T23:03:16.718 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:16.718 INFO:teuthology.orchestra.run.smithi005.stdout:5861 punch from 0xa93ab75 to 0xa94a2a2, (0xf72d bytes) 2024-05-11T23:03:16.718 INFO:teuthology.orchestra.run.smithi005.stdout:5862 write 0x21a0866 thru 0x21a5e05 (0x55a0 bytes) 2024-05-11T23:03:16.721 INFO:teuthology.orchestra.run.smithi005.stdout:5863 punch from 0x6885193 to 0x688c39d, (0x720a bytes) 2024-05-11T23:03:16.722 INFO:teuthology.orchestra.run.smithi005.stdout:5864 write 0x506123c thru 0x5064a55 (0x381a bytes) 2024-05-11T23:03:16.724 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:16.724 INFO:teuthology.orchestra.run.smithi005.stdout:5866 write 0x109909e thru 0x109b4a3 (0x2406 bytes) 2024-05-11T23:03:16.727 INFO:teuthology.orchestra.run.smithi005.stdout:5869 write 0x973f313 thru 0x9741a83 (0x2771 bytes) 2024-05-11T23:03:16.732 INFO:teuthology.orchestra.run.smithi005.stdout:5870 write 0xd86e2a thru 0xd89839 (0x2a10 bytes) 2024-05-11T23:03:16.735 INFO:teuthology.orchestra.run.smithi005.stdout:5871 read 0x2edb495 thru 0x2ee2679 (0x71e5 bytes) 2024-05-11T23:03:16.742 INFO:teuthology.orchestra.run.smithi005.stdout:5872 trunc from 0xcaf32d7 to 0x61f0666 2024-05-11T23:03:16.750 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:64049 3.8 3:11d98727:::rbd_object_map.111e8974a681:head [read 0~38] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-05-11T23:03:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 75.969846670999999, 2024-05-11T23:03:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99884604399999999, 2024-05-11T23:03:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 64049 2024-05-11T23:03:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779883+0000", 2024-05-11T23:03:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.57e-06 2024-05-11T23:03:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779887+0000", 2024-05-11T23:03:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5709999999999999e-06 2024-05-11T23:03:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779898+0000", 2024-05-11T23:03:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.042e-05 2024-05-11T23:03:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779905+0000", 2024-05-11T23:03:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0110000000000003e-06 2024-05-11T23:03:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779962+0000", 2024-05-11T23:03:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7395999999999997e-05 2024-05-11T23:03:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779969+0000", 2024-05-11T23:03:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2459999999999997e-06 2024-05-11T23:03:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778379+0000", 2024-05-11T23:03:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99840986300000001 2024-05-11T23:03:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778564+0000", 2024-05-11T23:03:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018420900000000001 2024-05-11T23:03:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778727+0000", 2024-05-11T23:03:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016375800000000001 2024-05-11T23:03:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172773 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.145740253, 2024-05-11T23:03:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058721316000000003, 2024-05-11T23:03:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172773 2024-05-11T23:03:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3500000000000002e-07 2024-05-11T23:03:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603990+0000", 2024-05-11T23:03:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3549999999999999e-06 2024-05-11T23:03:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603997+0000", 2024-05-11T23:03:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3400000000000003e-06 2024-05-11T23:03:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604002+0000", 2024-05-11T23:03:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0509999999999996e-06 2024-05-11T23:03:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.758 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662697+0000", 2024-05-11T23:03:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058695487999999997 2024-05-11T23:03:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662709+0000", 2024-05-11T23:03:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1647e-05 2024-05-11T23:03:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172774 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:16.759 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.145718871, 2024-05-11T23:03:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058745040999999998, 2024-05-11T23:03:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172774 2024-05-11T23:03:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.760 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:03:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.761 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604012+0000", 2024-05-11T23:03:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2199999999999999e-06 2024-05-11T23:03:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604018+0000", 2024-05-11T23:03:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1770000000000003e-06 2024-05-11T23:03:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604023+0000", 2024-05-11T23:03:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0089999999999998e-06 2024-05-11T23:03:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.762 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662742+0000", 2024-05-11T23:03:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058719187999999999 2024-05-11T23:03:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662754+0000", 2024-05-11T23:03:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2095e-05 2024-05-11T23:03:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172775 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.145698659000001, 2024-05-11T23:03:16.763 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058764743000000001, 2024-05-11T23:03:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172775 2024-05-11T23:03:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604030+0000", 2024-05-11T23:03:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6699999999999999e-07 2024-05-11T23:03:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604032+0000", 2024-05-11T23:03:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9530000000000002e-06 2024-05-11T23:03:16.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604037+0000", 2024-05-11T23:03:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8590000000000001e-06 2024-05-11T23:03:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604042+0000", 2024-05-11T23:03:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8609999999999997e-06 2024-05-11T23:03:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662782+0000", 2024-05-11T23:03:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058739731000000003 2024-05-11T23:03:16.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662794+0000", 2024-05-11T23:03:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1972e-05 2024-05-11T23:03:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172776 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.145679369, 2024-05-11T23:03:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058778696999999998, 2024-05-11T23:03:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.767 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172776 2024-05-11T23:03:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.768 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2000000000000001e-07 2024-05-11T23:03:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604051+0000", 2024-05-11T23:03:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2879999999999999e-06 2024-05-11T23:03:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604057+0000", 2024-05-11T23:03:16.769 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:03:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604062+0000", 2024-05-11T23:03:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6800000000000001e-06 2024-05-11T23:03:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662820+0000", 2024-05-11T23:03:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058758444999999999 2024-05-11T23:03:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662827+0000", 2024-05-11T23:03:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1930000000000004e-06 2024-05-11T23:03:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172777 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.145660063000001, 2024-05-11T23:03:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058799918999999999, 2024-05-11T23:03:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.771 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172777 2024-05-11T23:03:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:16.772 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:03:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604075+0000", 2024-05-11T23:03:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.109e-06 2024-05-11T23:03:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604083+0000", 2024-05-11T23:03:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9170000000000006e-06 2024-05-11T23:03:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604089+0000", 2024-05-11T23:03:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3399999999999997e-06 2024-05-11T23:03:16.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662859+0000", 2024-05-11T23:03:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058770172000000002 2024-05-11T23:03:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662868+0000", 2024-05-11T23:03:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0289999999999994e-06 2024-05-11T23:03:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.774 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172778 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.145631973, 2024-05-11T23:03:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058845435000000001, 2024-05-11T23:03:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172778 2024-05-11T23:03:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.775 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5400000000000002e-07 2024-05-11T23:03:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604099+0000", 2024-05-11T23:03:16.776 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4650000000000001e-06 2024-05-11T23:03:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604105+0000", 2024-05-11T23:03:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.314e-06 2024-05-11T23:03:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604110+0000", 2024-05-11T23:03:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6310000000000002e-06 2024-05-11T23:03:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662931+0000", 2024-05-11T23:03:16.777 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058820960999999998 2024-05-11T23:03:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662941+0000", 2024-05-11T23:03:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.061e-05 2024-05-11T23:03:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172779 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.145611812999999, 2024-05-11T23:03:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058859163999999999, 2024-05-11T23:03:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172779 2024-05-11T23:03:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.779 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:03:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604119+0000", 2024-05-11T23:03:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2529999999999999e-06 2024-05-11T23:03:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.780 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604125+0000", 2024-05-11T23:03:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1670000000000004e-06 2024-05-11T23:03:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604130+0000", 2024-05-11T23:03:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.702e-06 2024-05-11T23:03:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662967+0000", 2024-05-11T23:03:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058837792999999999 2024-05-11T23:03:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.781 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662975+0000", 2024-05-11T23:03:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8490000000000006e-06 2024-05-11T23:03:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172780 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.145591663000001, 2024-05-11T23:03:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058872696000000002, 2024-05-11T23:03:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.782 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172780 2024-05-11T23:03:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.783 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604137+0000", 2024-05-11T23:03:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4400000000000001e-07 2024-05-11T23:03:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604139+0000", 2024-05-11T23:03:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3429999999999998e-06 2024-05-11T23:03:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604145+0000", 2024-05-11T23:03:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1940000000000003e-06 2024-05-11T23:03:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604150+0000", 2024-05-11T23:03:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:03:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663002+0000", 2024-05-11T23:03:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058851829000000001 2024-05-11T23:03:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663009+0000", 2024-05-11T23:03:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.199e-06 2024-05-11T23:03:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172781 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.145559822999999, 2024-05-11T23:03:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058875553999999997, 2024-05-11T23:03:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172781 2024-05-11T23:03:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604171+0000", 2024-05-11T23:03:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.21e-06 2024-05-11T23:03:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604176+0000", 2024-05-11T23:03:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2970000000000003e-06 2024-05-11T23:03:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604179+0000", 2024-05-11T23:03:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8459999999999999e-06 2024-05-11T23:03:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663033+0000", 2024-05-11T23:03:16.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058854565999999997 2024-05-11T23:03:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663044+0000", 2024-05-11T23:03:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0229e-05 2024-05-11T23:03:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172782 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.145545075000001, 2024-05-11T23:03:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058894375999999998, 2024-05-11T23:03:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.789 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172782 2024-05-11T23:03:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:16.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-05-11T23:03:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604184+0000", 2024-05-11T23:03:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:03:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604187+0000", 2024-05-11T23:03:16.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.044e-06 2024-05-11T23:03:16.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604190+0000", 2024-05-11T23:03:16.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6460000000000002e-06 2024-05-11T23:03:16.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663070+0000", 2024-05-11T23:03:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058879510000000003 2024-05-11T23:03:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663077+0000", 2024-05-11T23:03:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6669999999999996e-06 2024-05-11T23:03:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172783 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.145534756, 2024-05-11T23:03:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058913946000000002, 2024-05-11T23:03:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172783 2024-05-11T23:03:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-05-11T23:03:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604195+0000", 2024-05-11T23:03:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.19e-06 2024-05-11T23:03:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604198+0000", 2024-05-11T23:03:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:03:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604200+0000", 2024-05-11T23:03:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4719999999999998e-06 2024-05-11T23:03:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663100+0000", 2024-05-11T23:03:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058900300000000003 2024-05-11T23:03:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663107+0000", 2024-05-11T23:03:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7449999999999998e-06 2024-05-11T23:03:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172784 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:16.797 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.145524736999999, 2024-05-11T23:03:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933728999999997, 2024-05-11T23:03:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172784 2024-05-11T23:03:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.03e-07 2024-05-11T23:03:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604205+0000", 2024-05-11T23:03:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2300000000000001e-06 2024-05-11T23:03:16.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604207+0000", 2024-05-11T23:03:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8729999999999999e-06 2024-05-11T23:03:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604210+0000", 2024-05-11T23:03:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.858e-06 2024-05-11T23:03:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663130+0000", 2024-05-11T23:03:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058919314 2024-05-11T23:03:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663137+0000", 2024-05-11T23:03:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2509999999999997e-06 2024-05-11T23:03:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172785 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.145512467, 2024-05-11T23:03:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058951664000000001, 2024-05-11T23:03:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172785 2024-05-11T23:03:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604216+0000", 2024-05-11T23:03:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:03:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604218+0000", 2024-05-11T23:03:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7549999999999999e-06 2024-05-11T23:03:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604222+0000", 2024-05-11T23:03:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6700000000000002e-06 2024-05-11T23:03:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604227+0000", 2024-05-11T23:03:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4399999999999998e-06 2024-05-11T23:03:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663160+0000", 2024-05-11T23:03:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933119999999999 2024-05-11T23:03:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663167+0000", 2024-05-11T23:03:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1840000000000002e-06 2024-05-11T23:03:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.805 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172786 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.145495412999999, 2024-05-11T23:03:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058966991000000003, 2024-05-11T23:03:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172786 2024-05-11T23:03:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604233+0000", 2024-05-11T23:03:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6899999999999999e-07 2024-05-11T23:03:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604235+0000", 2024-05-11T23:03:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8369999999999999e-06 2024-05-11T23:03:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604240+0000", 2024-05-11T23:03:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0329999999999999e-06 2024-05-11T23:03:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604244+0000", 2024-05-11T23:03:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9859999999999996e-06 2024-05-11T23:03:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663192+0000", 2024-05-11T23:03:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058948852000000003 2024-05-11T23:03:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663199+0000", 2024-05-11T23:03:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0140000000000001e-06 2024-05-11T23:03:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172787 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:16.809 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.145479067, 2024-05-11T23:03:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058982018999999997, 2024-05-11T23:03:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172787 2024-05-11T23:03:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605916+0000", 2024-05-11T23:03:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00166752 2024-05-11T23:03:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605921+0000", 2024-05-11T23:03:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0669999999999999e-06 2024-05-11T23:03:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605934+0000", 2024-05-11T23:03:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2271e-05 2024-05-11T23:03:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605940+0000", 2024-05-11T23:03:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6880000000000002e-06 2024-05-11T23:03:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663224+0000", 2024-05-11T23:03:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057283251 2024-05-11T23:03:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663231+0000", 2024-05-11T23:03:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2219999999999996e-06 2024-05-11T23:03:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172793 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.143712220000001, 2024-05-11T23:03:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057402373999999999, 2024-05-11T23:03:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.813 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172793 2024-05-11T23:03:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3300000000000001e-07 2024-05-11T23:03:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606017+0000", 2024-05-11T23:03:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4920000000000001e-06 2024-05-11T23:03:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606020+0000", 2024-05-11T23:03:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:03:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606023+0000", 2024-05-11T23:03:16.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6680000000000001e-06 2024-05-11T23:03:16.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663411+0000", 2024-05-11T23:03:16.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057388206999999997 2024-05-11T23:03:16.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663418+0000", 2024-05-11T23:03:16.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.815e-06 2024-05-11T23:03:16.816 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.816 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.816 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.817 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172795 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:16.817 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:16.817 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.143670792999998, 2024-05-11T23:03:16.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057423015000000001, 2024-05-11T23:03:16.817 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.817 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.817 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:16.817 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.817 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.817 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.817 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172795 2024-05-11T23:03:16.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.817 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:16.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:16.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:16.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9499999999999998e-07 2024-05-11T23:03:16.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606059+0000", 2024-05-11T23:03:16.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-05-11T23:03:16.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606062+0000", 2024-05-11T23:03:16.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.977e-06 2024-05-11T23:03:16.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606064+0000", 2024-05-11T23:03:16.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7410000000000001e-06 2024-05-11T23:03:16.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663473+0000", 2024-05-11T23:03:16.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057408624999999998 2024-05-11T23:03:16.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663480+0000", 2024-05-11T23:03:16.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1049999999999997e-06 2024-05-11T23:03:16.820 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.820 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.820 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.820 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172796 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:16.820 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:16.820 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.143660229999998, 2024-05-11T23:03:16.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057449235000000001, 2024-05-11T23:03:16.821 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.821 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.821 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:16.821 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.821 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.821 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.821 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172796 2024-05-11T23:03:16.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.821 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:16.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:16.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:16.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:16.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606069+0000", 2024-05-11T23:03:16.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:03:16.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606072+0000", 2024-05-11T23:03:16.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8700000000000001e-06 2024-05-11T23:03:16.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606074+0000", 2024-05-11T23:03:16.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6230000000000001e-06 2024-05-11T23:03:16.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663510+0000", 2024-05-11T23:03:16.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05743529 2024-05-11T23:03:16.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663517+0000", 2024-05-11T23:03:16.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1230000000000002e-06 2024-05-11T23:03:16.824 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.824 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.824 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.824 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172797 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:16.824 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:16.824 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 18.143650198, 2024-05-11T23:03:16.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057470420000000001, 2024-05-11T23:03:16.824 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:16.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:16.824 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:16.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:16.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:16.825 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:16.825 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172797 2024-05-11T23:03:16.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.825 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:16.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:16.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:16.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:16.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:16.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:16.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:16.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607915+0000", 2024-05-11T23:03:16.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018375609999999999 2024-05-11T23:03:16.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:16.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607920+0000", 2024-05-11T23:03:16.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8899999999999998e-06 2024-05-11T23:03:16.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:16.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607930+0000", 2024-05-11T23:03:16.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4429999999999992e-06 2024-05-11T23:03:16.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:16.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607936+0000", 2024-05-11T23:03:16.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7089999999999996e-06 2024-05-11T23:03:16.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:16.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663541+0000", 2024-05-11T23:03:16.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055604537000000002 2024-05-11T23:03:16.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:16.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:16.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:16.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663548+0000", 2024-05-11T23:03:16.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2799999999999998e-06 2024-05-11T23:03:16.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.827 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:16.828 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:16.828 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:16.829 INFO:teuthology.orchestra.run.smithi005.stdout:5873 read 0x162622f thru 0x1636178 (0xff4a bytes) 2024-05-11T23:03:16.829 INFO:teuthology.orchestra.run.smithi005.stdout:5875 punch from 0x44b085b to 0x44bfec4, (0xf669 bytes) 2024-05-11T23:03:16.829 INFO:teuthology.orchestra.run.smithi005.stdout:5876 read 0x3daa737 thru 0x3dab49b (0xd65 bytes) 2024-05-11T23:03:16.829 INFO:teuthology.orchestra.run.smithi005.stdout:5879 punch from 0x192acf4 to 0x19338a0, (0x8bac bytes) 2024-05-11T23:03:16.830 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:03:16.832 INFO:teuthology.orchestra.run.smithi005.stdout:5880 writesame 0xb8d02ca thru 0xb8debfc data_size 0x1(0xe933 bytes) 2024-05-11T23:03:16.832 INFO:teuthology.orchestra.run.smithi005.stdout:5882 punch from 0x6d153b1 to 0x6d1ceef, (0x7b3e bytes) 2024-05-11T23:03:16.832 INFO:teuthology.orchestra.run.smithi005.stdout:5883 write 0xd7a44ac thru 0xd7a59ab (0x1500 bytes) 2024-05-11T23:03:16.832 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:16.832 INFO:teuthology.orchestra.run.smithi005.stdout:5886 read 0x20b9d27 thru 0x20c6027 (0xc301 bytes) 2024-05-11T23:03:16.832 INFO:teuthology.orchestra.run.smithi005.stdout:5888 read 0x4d6e87e thru 0x4d71db2 (0x3535 bytes) 2024-05-11T23:03:16.832 INFO:teuthology.orchestra.run.smithi005.stdout:5889 read 0x43198d6 thru 0x431ee44 (0x556f bytes) 2024-05-11T23:03:16.832 INFO:teuthology.orchestra.run.smithi005.stdout:5890 writesame 0x6647059 thru 0x66554d4 data_size 0xe(0xe47c bytes) 2024-05-11T23:03:16.832 INFO:teuthology.orchestra.run.smithi005.stdout:5891 writesame 0xd511e0c thru 0xd51e1d0 data_size 0x1(0xc3c5 bytes) 2024-05-11T23:03:16.832 INFO:teuthology.orchestra.run.smithi005.stdout:5892 punch from 0xae151f5 to 0xae1574b, (0x556 bytes) 2024-05-11T23:03:16.832 INFO:teuthology.orchestra.run.smithi005.stdout:5893 writesame 0x5702a7d thru 0x57091ac data_size 0x34(0x6730 bytes) 2024-05-11T23:03:16.832 INFO:teuthology.orchestra.run.smithi005.stdout:5894 read 0x853a5e4 thru 0x8547b2d (0xd54a bytes) 2024-05-11T23:03:16.832 INFO:teuthology.orchestra.run.smithi005.stdout:5897 punch from 0xa34f616 to 0xa3584f6, (0x8ee0 bytes) 2024-05-11T23:03:16.832 INFO:teuthology.orchestra.run.smithi005.stdout:5898 read 0x9e2a530 thru 0x9e2d699 (0x316a bytes) 2024-05-11T23:03:16.832 INFO:teuthology.orchestra.run.smithi005.stdout:5901 write 0x1cc5baa thru 0x1cce6bd (0x8b14 bytes) 2024-05-11T23:03:16.833 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:16.833 INFO:teuthology.orchestra.run.smithi005.stdout:5903 writesame 0x724a28f thru 0x724a381 data_size 0x1(0xf3 bytes) 2024-05-11T23:03:16.833 INFO:teuthology.orchestra.run.smithi005.stdout:5904 trunc from 0xd7a59ac to 0x10f8a99 2024-05-11T23:03:16.833 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:16.829+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:16.847 INFO:teuthology.orchestra.run.smithi005.stdout:5905 punch from 0xd43849 to 0xd523dc, (0xeb93 bytes) 2024-05-11T23:03:16.858 INFO:teuthology.orchestra.run.smithi005.stdout:5906 writesame 0x55d2e4d thru 0x55d31e3 data_size 0x1(0x397 bytes) 2024-05-11T23:03:16.858 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:16.858 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:16.858 INFO:teuthology.orchestra.run.smithi005.stdout:5909 read 0x7f301c thru 0x7f4476 (0x145b bytes) 2024-05-11T23:03:16.869 INFO:teuthology.orchestra.run.smithi005.stdout:5910 trunc from 0x55d31e4 to 0xed434ba 2024-05-11T23:03:16.875 INFO:teuthology.orchestra.run.smithi005.stdout:5911 write 0xae1aba4 thru 0xae242a9 (0x9706 bytes) 2024-05-11T23:03:16.880 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:16.880 INFO:teuthology.orchestra.run.smithi005.stdout:5913 write 0xfc4df4 thru 0xfd477e (0xf98b bytes) 2024-05-11T23:03:16.883 INFO:teuthology.orchestra.run.smithi005.stdout:5914 write 0x4dffe01 thru 0x4e0e41a (0xe61a bytes) 2024-05-11T23:03:16.887 INFO:teuthology.orchestra.run.smithi005.stdout:5916 writesame 0xb80982f thru 0xb80a461 data_size 0x3(0xc33 bytes) 2024-05-11T23:03:16.890 INFO:teuthology.orchestra.run.smithi005.stdout:5917 write 0xd0ce33e thru 0xd0cfdad (0x1a70 bytes) 2024-05-11T23:03:16.893 INFO:teuthology.orchestra.run.smithi005.stdout:5918 read 0x99e56a6 thru 0x99f09cb (0xb326 bytes) 2024-05-11T23:03:16.894 INFO:teuthology.orchestra.run.smithi005.stdout:5920 read 0xd5a6226 thru 0xd5aae7b (0x4c56 bytes) 2024-05-11T23:03:16.894 INFO:teuthology.orchestra.run.smithi005.stdout:5922 read 0x7f1ab5c thru 0x7f26e77 (0xc31c bytes) 2024-05-11T23:03:16.894 INFO:teuthology.orchestra.run.smithi005.stdout:5923 read 0x230e385 thru 0x2316c85 (0x8901 bytes) 2024-05-11T23:03:16.894 INFO:teuthology.orchestra.run.smithi005.stdout:5925 write 0x13e2945 thru 0x13ed8d6 (0xaf92 bytes) 2024-05-11T23:03:16.897 INFO:teuthology.orchestra.run.smithi005.stdout:5926 read 0x4ee8b99 thru 0x4ef77b3 (0xec1b bytes) 2024-05-11T23:03:16.897 INFO:teuthology.orchestra.run.smithi005.stdout:5927 punch from 0xc9ee02e to 0xc9f9a80, (0xba52 bytes) 2024-05-11T23:03:16.897 INFO:teuthology.orchestra.run.smithi005.stdout:5928 read 0x262f6b6 thru 0x262faff (0x44a bytes) 2024-05-11T23:03:16.897 INFO:teuthology.orchestra.run.smithi005.stdout:5929 punch from 0x9dd187b to 0x9dd6cc2, (0x5447 bytes) 2024-05-11T23:03:16.900 INFO:teuthology.orchestra.run.smithi005.stdout:5932 trunc from 0xed434ba to 0xc5c46a0 2024-05-11T23:03:16.911 INFO:teuthology.orchestra.run.smithi005.stdout:5933 read 0x858e77e thru 0x8592f88 (0x480b bytes) 2024-05-11T23:03:16.911 INFO:teuthology.orchestra.run.smithi005.stdout:5935 punch from 0x6ff8ff3 to 0x7003a2a, (0xaa37 bytes) 2024-05-11T23:03:16.912 INFO:teuthology.orchestra.run.smithi005.stdout:5936 read 0x62dc084 thru 0x62e3d2f (0x7cac bytes) 2024-05-11T23:03:16.912 INFO:teuthology.orchestra.run.smithi005.stdout:5937 trunc from 0xc5c46a0 to 0xd51c259 2024-05-11T23:03:16.930 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:16.930+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:16.938 INFO:teuthology.orchestra.run.smithi005.stdout:5938 trunc from 0xd51c259 to 0x182167a 2024-05-11T23:03:16.966 INFO:teuthology.orchestra.run.smithi005.stdout:5939 write 0xa572854 thru 0xa57d4ed (0xac9a bytes) 2024-05-11T23:03:16.969 INFO:teuthology.orchestra.run.smithi005.stdout:5940 clone 51 order 25 su 262144 sc 9 2024-05-11T23:03:16.977 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:16.977 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:16.977 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:16.977 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:16.987 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:03:17.030 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:17.030+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:17.131 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:17.131+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:17.136 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:17.136 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:17.136 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:17.136 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:17.136 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:17.145 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:03:17.217 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-11T23:03:17.231 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:17.231+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:17.321 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.321 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:77128 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_get in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-11T23:03:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 62.304181319000001, 2024-05-11T23:03:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91737855899999998, 2024-05-11T23:03:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 77128 2024-05-11T23:03:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.322 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016003+0000", 2024-05-11T23:03:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5379999999999999e-06 2024-05-11T23:03:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016011+0000", 2024-05-11T23:03:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6410000000000002e-06 2024-05-11T23:03:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016026+0000", 2024-05-11T23:03:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4779e-05 2024-05-11T23:03:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016032+0000", 2024-05-11T23:03:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.066e-06 2024-05-11T23:03:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016093+0000", 2024-05-11T23:03:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0878000000000002e-05 2024-05-11T23:03:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016100+0000", 2024-05-11T23:03:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6799999999999993e-06 2024-05-11T23:03:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933302+0000", 2024-05-11T23:03:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91720135300000005 2024-05-11T23:03:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933380+0000", 2024-05-11T23:03:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8623999999999998e-05 2024-05-11T23:03:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98950 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.188100429999999, 2024-05-11T23:03:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021419438999999998, 2024-05-11T23:03:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98950 2024-05-11T23:03:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.34e-07 2024-05-11T23:03:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132085+0000", 2024-05-11T23:03:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0049999999999999e-06 2024-05-11T23:03:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132093+0000", 2024-05-11T23:03:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6660000000000008e-06 2024-05-11T23:03:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132097+0000", 2024-05-11T23:03:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2590000000000002e-06 2024-05-11T23:03:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135114+0000", 2024-05-11T23:03:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003017573 2024-05-11T23:03:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153416+0000", 2024-05-11T23:03:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018301884000000001 2024-05-11T23:03:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153502+0000", 2024-05-11T23:03:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5717999999999995e-05 2024-05-11T23:03:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98959 3.1e 3:7807da99:::rbd_data.111ed2934718.000000000000006d:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.187607692999997, 2024-05-11T23:03:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020525928999999998, 2024-05-11T23:03:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98959 2024-05-11T23:03:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132576+0000", 2024-05-11T23:03:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4199999999999995e-07 2024-05-11T23:03:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132580+0000", 2024-05-11T23:03:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2910000000000001e-06 2024-05-11T23:03:17.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132588+0000", 2024-05-11T23:03:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7970000000000001e-06 2024-05-11T23:03:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132602+0000", 2024-05-11T23:03:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3743999999999999e-05 2024-05-11T23:03:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132655+0000", 2024-05-11T23:03:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2714999999999999e-05 2024-05-11T23:03:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132787+0000", 2024-05-11T23:03:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013273899999999999 2024-05-11T23:03:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153101+0000", 2024-05-11T23:03:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020313900999999999 2024-05-11T23:03:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.335 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98965 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.185084056999997, 2024-05-11T23:03:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018608983999999999, 2024-05-11T23:03:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.336 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98965 2024-05-11T23:03:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135100+0000", 2024-05-11T23:03:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.17e-06 2024-05-11T23:03:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135105+0000", 2024-05-11T23:03:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.899e-06 2024-05-11T23:03:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135122+0000", 2024-05-11T23:03:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7346999999999998e-05 2024-05-11T23:03:17.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135130+0000", 2024-05-11T23:03:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0639999999999994e-06 2024-05-11T23:03:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153533+0000", 2024-05-11T23:03:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018402753000000001 2024-05-11T23:03:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153615+0000", 2024-05-11T23:03:17.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1720999999999999e-05 2024-05-11T23:03:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153708+0000", 2024-05-11T23:03:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3029999999999995e-05 2024-05-11T23:03:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98966 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3760128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.185011930999998, 2024-05-11T23:03:17.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018716535999999999, 2024-05-11T23:03:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98966 2024-05-11T23:03:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135172+0000", 2024-05-11T23:03:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-05-11T23:03:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135175+0000", 2024-05-11T23:03:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-05-11T23:03:17.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135184+0000", 2024-05-11T23:03:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1840000000000005e-06 2024-05-11T23:03:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152903+0000", 2024-05-11T23:03:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017718357000000001 2024-05-11T23:03:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153736+0000", 2024-05-11T23:03:17.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00083324799999999997 2024-05-11T23:03:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153809+0000", 2024-05-11T23:03:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2995000000000002e-05 2024-05-11T23:03:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153888+0000", 2024-05-11T23:03:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8695999999999993e-05 2024-05-11T23:03:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "pg_update_log_missing_reply(3.1e epoch 72/53 rep_tid 104 lcod 72'363)", 2024-05-11T23:03:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.184949035000002, 2024-05-11T23:03:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017871951000000001, 2024-05-11T23:03:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135235+0000", 2024-05-11T23:03:17.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8899999999999999e-07 2024-05-11T23:03:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135238+0000", 2024-05-11T23:03:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.27e-06 2024-05-11T23:03:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152878+0000", 2024-05-11T23:03:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01763986 2024-05-11T23:03:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152896+0000", 2024-05-11T23:03:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8573e-05 2024-05-11T23:03:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152968+0000", 2024-05-11T23:03:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1780999999999996e-05 2024-05-11T23:03:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153106+0000", 2024-05-11T23:03:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013777799999999999 2024-05-11T23:03:17.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:239544 3.1a 3:5fe59cad:::rbd_data.111ebabb4604.000000000000016e:head [stat,delete] snapc 2f=[2f] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 13.665592303, 2024-05-11T23:03:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.50514007500000002, 2024-05-11T23:03:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 239544 2024-05-11T23:03:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654592+0000", 2024-05-11T23:03:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.404e-06 2024-05-11T23:03:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654598+0000", 2024-05-11T23:03:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7660000000000004e-06 2024-05-11T23:03:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654611+0000", 2024-05-11T23:03:17.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2676e-05 2024-05-11T23:03:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654620+0000", 2024-05-11T23:03:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2990000000000002e-06 2024-05-11T23:03:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654698+0000", 2024-05-11T23:03:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7739999999999998e-05 2024-05-11T23:03:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654710+0000", 2024-05-11T23:03:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2648e-05 2024-05-11T23:03:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.157889+0000", 2024-05-11T23:03:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.50317850799999997 2024-05-11T23:03:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.158050+0000", 2024-05-11T23:03:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016084500000000001 2024-05-11T23:03:17.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.159731+0000", 2024-05-11T23:03:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001681189 2024-05-11T23:03:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262364 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2613248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7337194379999996, 2024-05-11T23:03:17.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016312508, 2024-05-11T23:03:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262364 2024-05-11T23:03:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.13e-07 2024-05-11T23:03:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586467+0000", 2024-05-11T23:03:17.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-05-11T23:03:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586474+0000", 2024-05-11T23:03:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7739999999999999e-06 2024-05-11T23:03:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586483+0000", 2024-05-11T23:03:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0340000000000002e-06 2024-05-11T23:03:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602629+0000", 2024-05-11T23:03:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016145845999999998 2024-05-11T23:03:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602703+0000", 2024-05-11T23:03:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4671000000000004e-05 2024-05-11T23:03:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602776+0000", 2024-05-11T23:03:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2793000000000006e-05 2024-05-11T23:03:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.358 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262417 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7319731389999999, 2024-05-11T23:03:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016403456, 2024-05-11T23:03:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.359 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262417 2024-05-11T23:03:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6199999999999999e-07 2024-05-11T23:03:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588212+0000", 2024-05-11T23:03:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.075e-06 2024-05-11T23:03:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588218+0000", 2024-05-11T23:03:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8150000000000002e-06 2024-05-11T23:03:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588223+0000", 2024-05-11T23:03:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7090000000000001e-06 2024-05-11T23:03:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604461+0000", 2024-05-11T23:03:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016237900999999999 2024-05-11T23:03:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604540+0000", 2024-05-11T23:03:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9376999999999996e-05 2024-05-11T23:03:17.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604613+0000", 2024-05-11T23:03:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3217000000000001e-05 2024-05-11T23:03:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262418 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2396160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7319532300000002, 2024-05-11T23:03:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016591180000000001, 2024-05-11T23:03:17.363 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262418 2024-05-11T23:03:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6100000000000001e-07 2024-05-11T23:03:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588232+0000", 2024-05-11T23:03:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0370000000000001e-06 2024-05-11T23:03:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588238+0000", 2024-05-11T23:03:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9340000000000003e-06 2024-05-11T23:03:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588243+0000", 2024-05-11T23:03:17.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6179999999999997e-06 2024-05-11T23:03:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604642+0000", 2024-05-11T23:03:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0163991 2024-05-11T23:03:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604712+0000", 2024-05-11T23:03:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9935e-05 2024-05-11T23:03:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604821+0000", 2024-05-11T23:03:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000109095 2024-05-11T23:03:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262419 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2433024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7319339119999997, 2024-05-11T23:03:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016799692000000001, 2024-05-11T23:03:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.368 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262419 2024-05-11T23:03:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588252+0000", 2024-05-11T23:03:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0990000000000001e-06 2024-05-11T23:03:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588257+0000", 2024-05-11T23:03:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8159999999999999e-06 2024-05-11T23:03:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588262+0000", 2024-05-11T23:03:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8069999999999999e-06 2024-05-11T23:03:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604868+0000", 2024-05-11T23:03:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016605880999999999 2024-05-11T23:03:17.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604966+0000", 2024-05-11T23:03:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7597999999999996e-05 2024-05-11T23:03:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605049+0000", 2024-05-11T23:03:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3141999999999998e-05 2024-05-11T23:03:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262420 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2469888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7319138010000001, 2024-05-11T23:03:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016972514000000001, 2024-05-11T23:03:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262420 2024-05-11T23:03:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.373 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588271+0000", 2024-05-11T23:03:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6199999999999999e-06 2024-05-11T23:03:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588273+0000", 2024-05-11T23:03:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3209999999999999e-06 2024-05-11T23:03:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588279+0000", 2024-05-11T23:03:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2519999999999996e-06 2024-05-11T23:03:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588285+0000", 2024-05-11T23:03:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1440000000000002e-06 2024-05-11T23:03:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605083+0000", 2024-05-11T23:03:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016798418999999998 2024-05-11T23:03:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605168+0000", 2024-05-11T23:03:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4851000000000005e-05 2024-05-11T23:03:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605242+0000", 2024-05-11T23:03:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3906999999999996e-05 2024-05-11T23:03:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262421 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2506752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7318916550000001, 2024-05-11T23:03:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017141427000000001, 2024-05-11T23:03:17.377 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262421 2024-05-11T23:03:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588292+0000", 2024-05-11T23:03:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588318+0000", 2024-05-11T23:03:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6376e-05 2024-05-11T23:03:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588326+0000", 2024-05-11T23:03:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6699999999999994e-06 2024-05-11T23:03:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588332+0000", 2024-05-11T23:03:17.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8540000000000002e-06 2024-05-11T23:03:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605274+0000", 2024-05-11T23:03:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016942024999999999 2024-05-11T23:03:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605360+0000", 2024-05-11T23:03:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6304999999999994e-05 2024-05-11T23:03:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605433+0000", 2024-05-11T23:03:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2772000000000001e-05 2024-05-11T23:03:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262422 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2543616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7318424310000005, 2024-05-11T23:03:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017283705, 2024-05-11T23:03:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.382 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262422 2024-05-11T23:03:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:03:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588343+0000", 2024-05-11T23:03:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.34e-06 2024-05-11T23:03:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588350+0000", 2024-05-11T23:03:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1569999999999997e-06 2024-05-11T23:03:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588354+0000", 2024-05-11T23:03:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.865e-06 2024-05-11T23:03:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605466+0000", 2024-05-11T23:03:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017111498999999999 2024-05-11T23:03:17.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605551+0000", 2024-05-11T23:03:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5210999999999994e-05 2024-05-11T23:03:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605624+0000", 2024-05-11T23:03:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3153999999999998e-05 2024-05-11T23:03:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262423 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2580480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7318221820000002, 2024-05-11T23:03:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017451194, 2024-05-11T23:03:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262423 2024-05-11T23:03:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:17.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588363+0000", 2024-05-11T23:03:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.855e-06 2024-05-11T23:03:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588369+0000", 2024-05-11T23:03:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1260000000000003e-06 2024-05-11T23:03:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588375+0000", 2024-05-11T23:03:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9630000000000003e-06 2024-05-11T23:03:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605656+0000", 2024-05-11T23:03:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017280865999999999 2024-05-11T23:03:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605737+0000", 2024-05-11T23:03:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0773999999999996e-05 2024-05-11T23:03:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605812+0000", 2024-05-11T23:03:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5203999999999994e-05 2024-05-11T23:03:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262424 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2617344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.7318006639999997, 2024-05-11T23:03:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017617544999999998, 2024-05-11T23:03:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.391 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262424 2024-05-11T23:03:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588383+0000", 2024-05-11T23:03:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8299999999999997e-07 2024-05-11T23:03:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588385+0000", 2024-05-11T23:03:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2869999999999999e-06 2024-05-11T23:03:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588391+0000", 2024-05-11T23:03:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1689999999999997e-06 2024-05-11T23:03:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588396+0000", 2024-05-11T23:03:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5719999999999996e-06 2024-05-11T23:03:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605837+0000", 2024-05-11T23:03:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017440812999999999 2024-05-11T23:03:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605917+0000", 2024-05-11T23:03:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9924000000000003e-05 2024-05-11T23:03:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.606000+0000", 2024-05-11T23:03:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3296999999999999e-05 2024-05-11T23:03:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262896 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:17.395 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.660796027, 2024-05-11T23:03:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016295738000000001, 2024-05-11T23:03:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262896 2024-05-11T23:03:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659390+0000", 2024-05-11T23:03:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1440000000000001e-06 2024-05-11T23:03:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659397+0000", 2024-05-11T23:03:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0010000000000004e-06 2024-05-11T23:03:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659402+0000", 2024-05-11T23:03:17.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2179999999999998e-06 2024-05-11T23:03:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675487+0000", 2024-05-11T23:03:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016085706000000002 2024-05-11T23:03:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675580+0000", 2024-05-11T23:03:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2884e-05 2024-05-11T23:03:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675683+0000", 2024-05-11T23:03:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010236 2024-05-11T23:03:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262897 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.6607606859999997, 2024-05-11T23:03:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016500932999999999, 2024-05-11T23:03:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.400 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262897 2024-05-11T23:03:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659423+0000", 2024-05-11T23:03:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9699999999999996e-07 2024-05-11T23:03:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659425+0000", 2024-05-11T23:03:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3199999999999998e-06 2024-05-11T23:03:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659433+0000", 2024-05-11T23:03:17.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3660000000000003e-06 2024-05-11T23:03:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659445+0000", 2024-05-11T23:03:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2102e-05 2024-05-11T23:03:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675727+0000", 2024-05-11T23:03:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016282331000000001 2024-05-11T23:03:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675815+0000", 2024-05-11T23:03:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7484999999999993e-05 2024-05-11T23:03:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675923+0000", 2024-05-11T23:03:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000108732 2024-05-11T23:03:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262898 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:17.404 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.6607295679999998, 2024-05-11T23:03:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016685855999999999, 2024-05-11T23:03:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262898 2024-05-11T23:03:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659454+0000", 2024-05-11T23:03:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2200000000000004e-07 2024-05-11T23:03:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659457+0000", 2024-05-11T23:03:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.779e-06 2024-05-11T23:03:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659463+0000", 2024-05-11T23:03:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6660000000000002e-06 2024-05-11T23:03:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659472+0000", 2024-05-11T23:03:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2139999999999996e-06 2024-05-11T23:03:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675956+0000", 2024-05-11T23:03:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016484431000000001 2024-05-11T23:03:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676039+0000", 2024-05-11T23:03:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3358999999999999e-05 2024-05-11T23:03:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676139+0000", 2024-05-11T23:03:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9784999999999994e-05 2024-05-11T23:03:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262899 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 4.6607037370000004, 2024-05-11T23:03:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017030087999999999, 2024-05-11T23:03:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262899 2024-05-11T23:03:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.409 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659480+0000", 2024-05-11T23:03:17.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7700000000000004e-07 2024-05-11T23:03:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659482+0000", 2024-05-11T23:03:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1390000000000002e-06 2024-05-11T23:03:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659489+0000", 2024-05-11T23:03:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8279999999999997e-06 2024-05-11T23:03:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659494+0000", 2024-05-11T23:03:17.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9019999999999998e-06 2024-05-11T23:03:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676167+0000", 2024-05-11T23:03:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016673621 2024-05-11T23:03:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676329+0000", 2024-05-11T23:03:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016206099999999999 2024-05-11T23:03:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676509+0000", 2024-05-11T23:03:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017996 2024-05-11T23:03:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.413 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.413 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:17.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:03:17.415 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #49, image image_client.0-clone49 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent50 2024-05-11T23:03:17.416 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:17.414+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:17.516 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:17.515+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:17.537 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:17.537 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:17.537 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:17.537 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:17.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:03:17.572 INFO:teuthology.orchestra.run.smithi005.stdout:5942 trunc from 0xa57d4ee to 0x1008477 2024-05-11T23:03:17.600 INFO:teuthology.orchestra.run.smithi005.stdout:5943 punch from 0x3f3452 to 0x3f4249, (0xdf7 bytes) 2024-05-11T23:03:17.617 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:17.616+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:17.620 INFO:teuthology.orchestra.run.smithi005.stdout:5949 write 0xd76b6e1 thru 0xd777788 (0xc0a8 bytes) 2024-05-11T23:03:17.621 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:17.631 INFO:teuthology.orchestra.run.smithi005.stdout:5951 trunc from 0xd777789 to 0x47d7231 2024-05-11T23:03:17.651 INFO:teuthology.orchestra.run.smithi005.stdout:5953 trunc from 0x47d7231 to 0xe3be5b3 2024-05-11T23:03:17.656 INFO:teuthology.orchestra.run.smithi005.stdout:5954 read 0xc486206 thru 0xc486a1d (0x818 bytes) 2024-05-11T23:03:17.656 INFO:teuthology.orchestra.run.smithi005.stdout:5955 read 0xc388c8 thru 0xc417bc (0x8ef5 bytes) 2024-05-11T23:03:17.656 INFO:teuthology.orchestra.run.smithi005.stdout:5956 read 0x9020f3a thru 0x902b86d (0xa934 bytes) 2024-05-11T23:03:17.657 INFO:teuthology.orchestra.run.smithi005.stdout:5957 read 0x5b20ebf thru 0x5b24d21 (0x3e63 bytes) 2024-05-11T23:03:17.657 INFO:teuthology.orchestra.run.smithi005.stdout:5958 write 0x2c82bf8 thru 0x2c87622 (0x4a2b bytes) 2024-05-11T23:03:17.659 INFO:teuthology.orchestra.run.smithi005.stdout:5961 read 0x46ebd61 thru 0x46ec2b8 (0x558 bytes) 2024-05-11T23:03:17.659 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:17.659 INFO:teuthology.orchestra.run.smithi005.stdout:5964 read 0x63bc1f3 thru 0x63c5e1f (0x9c2d bytes) 2024-05-11T23:03:17.660 INFO:teuthology.orchestra.run.smithi005.stdout:5965 write 0x31afd50 thru 0x31b0d3e (0xfef bytes) 2024-05-11T23:03:17.661 INFO:teuthology.orchestra.run.smithi005.stdout:5966 punch from 0xa25bf62 to 0xa26797f, (0xba1d bytes) 2024-05-11T23:03:17.661 INFO:teuthology.orchestra.run.smithi005.stdout:5968 read 0x96da4f thru 0x979fbe (0xc570 bytes) 2024-05-11T23:03:17.662 INFO:teuthology.orchestra.run.smithi005.stdout:5969 write 0x610824d thru 0x6113881 (0xb635 bytes) 2024-05-11T23:03:17.664 INFO:teuthology.orchestra.run.smithi005.stdout:5970 write 0x734bafb thru 0x73577f7 (0xbcfd bytes) 2024-05-11T23:03:17.666 INFO:teuthology.orchestra.run.smithi005.stdout:5971 punch from 0x809b44c to 0x80a94a4, (0xe058 bytes) 2024-05-11T23:03:17.666 INFO:teuthology.orchestra.run.smithi005.stdout:5972 write 0x3bf4553 thru 0x3bfe5b1 (0xa05f bytes) 2024-05-11T23:03:17.669 INFO:teuthology.orchestra.run.smithi005.stdout:5973 punch from 0x6090ae2 to 0x609dfbf, (0xd4dd bytes) 2024-05-11T23:03:17.669 INFO:teuthology.orchestra.run.smithi005.stdout:5975 write 0xb66030a thru 0xb665352 (0x5049 bytes) 2024-05-11T23:03:17.670 INFO:teuthology.orchestra.run.smithi005.stdout:5976 read 0x7bf61b3 thru 0x7c056fe (0xf54c bytes) 2024-05-11T23:03:17.670 INFO:teuthology.orchestra.run.smithi005.stdout:5977 writesame 0x6511cdd thru 0x6515f82 data_size 0x13(0x42a6 bytes) 2024-05-11T23:03:17.671 INFO:teuthology.orchestra.run.smithi005.stdout:5978 punch from 0x92bf031 to 0x92c1098, (0x2067 bytes) 2024-05-11T23:03:17.672 INFO:teuthology.orchestra.run.smithi005.stdout:5979 write 0x15d5f8a thru 0x15dd53f (0x75b6 bytes) 2024-05-11T23:03:17.673 INFO:teuthology.orchestra.run.smithi005.stdout:5980 read 0x79757b0 thru 0x7983386 (0xdbd7 bytes) 2024-05-11T23:03:17.673 INFO:teuthology.orchestra.run.smithi005.stdout:5981 punch from 0x79e3e75 to 0x79ea784, (0x690f bytes) 2024-05-11T23:03:17.674 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:17.674 INFO:teuthology.orchestra.run.smithi005.stdout:5983 writesame 0xa9bde45 thru 0xa9c60b4 data_size 0x8(0x8270 bytes) 2024-05-11T23:03:17.676 INFO:teuthology.orchestra.run.smithi005.stdout:5984 read 0x92122ba thru 0x921ba64 (0x97ab bytes) 2024-05-11T23:03:17.676 INFO:teuthology.orchestra.run.smithi005.stdout:5987 write 0x290753 thru 0x298663 (0x7f11 bytes) 2024-05-11T23:03:17.679 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:17.679 INFO:teuthology.orchestra.run.smithi005.stdout:5991 punch from 0xc09ccb1 to 0xc09edc6, (0x2115 bytes) 2024-05-11T23:03:17.679 INFO:teuthology.orchestra.run.smithi005.stdout:compare and write disabled 2024-05-11T23:03:17.679 INFO:teuthology.orchestra.run.smithi005.stdout:5996 read 0x1de3610 thru 0x1de68c8 (0x32b9 bytes) 2024-05-11T23:03:17.679 INFO:teuthology.orchestra.run.smithi005.stdout:5997 punch from 0xcbabe59 to 0xcbb0896, (0x4a3d bytes) 2024-05-11T23:03:17.679 INFO:teuthology.orchestra.run.smithi005.stdout:5998 writesame 0x804a106 thru 0x804eec1 data_size 0x32(0x4dbc bytes) 2024-05-11T23:03:17.680 INFO:teuthology.orchestra.run.smithi005.stdout:5999 punch from 0x5ddb79b to 0x5ddbe4a, (0x6af bytes) 2024-05-11T23:03:17.693 INFO:teuthology.orchestra.run.smithi005.stdout:checking clone #50, image image_client.0-clone50 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent51 2024-05-11T23:03:17.702 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:17.702 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:17.702 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:17.702 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:17.702 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:17.715 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:03:17.757 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:17.757+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:17.820 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:17.820+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:17.872 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:17.872 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:17.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:17.872 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:17.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.872 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36828 3.b 3:d385bc8e:::rbd_data.111e5e4d8275.0000000000000020:11 [read 512000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:17.872 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:03:17.872 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 84.937801809999996, 2024-05-11T23:03:17.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051481218000000002, 2024-05-11T23:03:17.872 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.872 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.872 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.872 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.873 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.873 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.873 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36828 2024-05-11T23:03:17.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.873 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:03:17.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:03:17.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934915+0000", 2024-05-11T23:03:17.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00181814 2024-05-11T23:03:17.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934919+0000", 2024-05-11T23:03:17.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6739999999999996e-06 2024-05-11T23:03:17.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934934+0000", 2024-05-11T23:03:17.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4415e-05 2024-05-11T23:03:17.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934942+0000", 2024-05-11T23:03:17.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8619999999999992e-06 2024-05-11T23:03:17.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984433+0000", 2024-05-11T23:03:17.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049490658999999999 2024-05-11T23:03:17.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984508+0000", 2024-05-11T23:03:17.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4394999999999995e-05 2024-05-11T23:03:17.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984578+0000", 2024-05-11T23:03:17.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0073000000000005e-05 2024-05-11T23:03:17.876 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.876 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.876 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.876 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36950 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 520192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:17.876 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:03:17.876 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 84.933496043999995, 2024-05-11T23:03:17.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051450856000000003, 2024-05-11T23:03:17.876 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.876 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.876 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.877 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.877 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.877 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.877 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36950 2024-05-11T23:03:17.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.877 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:03:17.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:03:17.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938914+0000", 2024-05-11T23:03:17.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015121480000000001 2024-05-11T23:03:17.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938918+0000", 2024-05-11T23:03:17.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4149999999999999e-06 2024-05-11T23:03:17.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938925+0000", 2024-05-11T23:03:17.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0779999999999998e-06 2024-05-11T23:03:17.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938929+0000", 2024-05-11T23:03:17.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2930000000000002e-06 2024-05-11T23:03:17.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988675+0000", 2024-05-11T23:03:17.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049746394999999999 2024-05-11T23:03:17.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988756+0000", 2024-05-11T23:03:17.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0499000000000003e-05 2024-05-11T23:03:17.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988853+0000", 2024-05-11T23:03:17.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7027999999999993e-05 2024-05-11T23:03:17.880 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.880 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.880 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.880 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36957 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 778240~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:17.880 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:03:17.880 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 84.931874254999997, 2024-05-11T23:03:17.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051328273000000001, 2024-05-11T23:03:17.880 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.880 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.880 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.881 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.881 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.881 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.881 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36957 2024-05-11T23:03:17.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.881 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:03:17.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:03:17.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:03:17.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2099999999999998e-07 2024-05-11T23:03:17.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939026+0000", 2024-05-11T23:03:17.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5069999999999999e-06 2024-05-11T23:03:17.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939030+0000", 2024-05-11T23:03:17.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7919999999999999e-06 2024-05-11T23:03:17.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939032+0000", 2024-05-11T23:03:17.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9129999999999999e-06 2024-05-11T23:03:17.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990178+0000", 2024-05-11T23:03:17.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051145373000000001 2024-05-11T23:03:17.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990255+0000", 2024-05-11T23:03:17.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7195999999999997e-05 2024-05-11T23:03:17.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990352+0000", 2024-05-11T23:03:17.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7170999999999995e-05 2024-05-11T23:03:17.884 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.884 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.884 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.884 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36958 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 815104~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:17.884 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:03:17.884 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 84.931861936000004, 2024-05-11T23:03:17.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051531625999999997, 2024-05-11T23:03:17.884 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.884 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.885 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.885 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.885 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.885 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.885 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36958 2024-05-11T23:03:17.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.885 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:03:17.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:03:17.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939037+0000", 2024-05-11T23:03:17.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0400000000000002e-07 2024-05-11T23:03:17.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939038+0000", 2024-05-11T23:03:17.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.164e-06 2024-05-11T23:03:17.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939041+0000", 2024-05-11T23:03:17.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1109999999999999e-06 2024-05-11T23:03:17.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939043+0000", 2024-05-11T23:03:17.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.554e-06 2024-05-11T23:03:17.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990382+0000", 2024-05-11T23:03:17.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051338953999999999 2024-05-11T23:03:17.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990465+0000", 2024-05-11T23:03:17.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2461999999999998e-05 2024-05-11T23:03:17.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990568+0000", 2024-05-11T23:03:17.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000103077 2024-05-11T23:03:17.888 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.888 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.888 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.888 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36959 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 851968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:17.888 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:03:17.888 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 84.93185004, 2024-05-11T23:03:17.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051758936999999998, 2024-05-11T23:03:17.889 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.889 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.889 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.889 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.889 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.889 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.889 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36959 2024-05-11T23:03:17.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.889 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:03:17.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:03:17.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:03:17.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.16e-07 2024-05-11T23:03:17.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939050+0000", 2024-05-11T23:03:17.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.176e-06 2024-05-11T23:03:17.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939053+0000", 2024-05-11T23:03:17.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1140000000000002e-06 2024-05-11T23:03:17.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939055+0000", 2024-05-11T23:03:17.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.621e-06 2024-05-11T23:03:17.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990610+0000", 2024-05-11T23:03:17.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051554244999999999 2024-05-11T23:03:17.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990700+0000", 2024-05-11T23:03:17.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0284999999999994e-05 2024-05-11T23:03:17.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990807+0000", 2024-05-11T23:03:17.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010728 2024-05-11T23:03:17.892 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.892 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.892 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.893 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36960 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 888832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:17.893 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:03:17.893 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 84.931839804999996, 2024-05-11T23:03:17.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051886831000000001, 2024-05-11T23:03:17.893 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.893 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.893 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.893 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.893 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.893 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.893 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36960 2024-05-11T23:03:17.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.893 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:03:17.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:03:17.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940913+0000", 2024-05-11T23:03:17.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001855104 2024-05-11T23:03:17.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940916+0000", 2024-05-11T23:03:17.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0000000000000001e-06 2024-05-11T23:03:17.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940924+0000", 2024-05-11T23:03:17.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0999999999999998e-06 2024-05-11T23:03:17.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940927+0000", 2024-05-11T23:03:17.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7160000000000001e-06 2024-05-11T23:03:17.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990826+0000", 2024-05-11T23:03:17.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049898832999999997 2024-05-11T23:03:17.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990872+0000", 2024-05-11T23:03:17.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5717000000000003e-05 2024-05-11T23:03:17.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990945+0000", 2024-05-11T23:03:17.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3361000000000005e-05 2024-05-11T23:03:17.896 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.896 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.896 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.897 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36986 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 528384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:17.897 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:03:17.897 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 84.929833587000005, 2024-05-11T23:03:17.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051341053999999997, 2024-05-11T23:03:17.897 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.897 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.897 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.897 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.897 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.897 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36986 2024-05-11T23:03:17.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.897 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:03:17.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:03:17.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:03:17.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:17.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941066+0000", 2024-05-11T23:03:17.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1629999999999999e-06 2024-05-11T23:03:17.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941069+0000", 2024-05-11T23:03:17.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.118e-06 2024-05-11T23:03:17.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941072+0000", 2024-05-11T23:03:17.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5720000000000001e-06 2024-05-11T23:03:17.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992292+0000", 2024-05-11T23:03:17.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051220620000000001 2024-05-11T23:03:17.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992344+0000", 2024-05-11T23:03:17.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1737000000000003e-05 2024-05-11T23:03:17.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992406+0000", 2024-05-11T23:03:17.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1623000000000003e-05 2024-05-11T23:03:17.900 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.900 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.901 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.901 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36987 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 565248~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:17.901 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:03:17.901 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 84.929823466000002, 2024-05-11T23:03:17.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051568725000000003, 2024-05-11T23:03:17.901 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.901 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.901 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.901 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.901 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.901 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.901 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36987 2024-05-11T23:03:17.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.902 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:03:17.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:03:17.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:03:17.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.91e-07 2024-05-11T23:03:17.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941076+0000", 2024-05-11T23:03:17.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1379999999999999e-06 2024-05-11T23:03:17.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941079+0000", 2024-05-11T23:03:17.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9739999999999998e-06 2024-05-11T23:03:17.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941082+0000", 2024-05-11T23:03:17.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-05-11T23:03:17.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992446+0000", 2024-05-11T23:03:17.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051364058999999997 2024-05-11T23:03:17.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992536+0000", 2024-05-11T23:03:17.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0085000000000002e-05 2024-05-11T23:03:17.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992643+0000", 2024-05-11T23:03:17.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107377 2024-05-11T23:03:17.904 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.905 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.905 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.905 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36988 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 602112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:17.905 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:03:17.905 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 84.929813096999993, 2024-05-11T23:03:17.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051700007999999999, 2024-05-11T23:03:17.905 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.905 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.905 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.905 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.905 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.905 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.905 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36988 2024-05-11T23:03:17.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.906 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:03:17.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:03:17.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:03:17.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.05e-07 2024-05-11T23:03:17.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941086+0000", 2024-05-11T23:03:17.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.125e-06 2024-05-11T23:03:17.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941089+0000", 2024-05-11T23:03:17.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0599999999999999e-06 2024-05-11T23:03:17.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941092+0000", 2024-05-11T23:03:17.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6529999999999998e-06 2024-05-11T23:03:17.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992667+0000", 2024-05-11T23:03:17.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051574587999999998 2024-05-11T23:03:17.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992723+0000", 2024-05-11T23:03:17.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6682999999999999e-05 2024-05-11T23:03:17.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992785+0000", 2024-05-11T23:03:17.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1693999999999996e-05 2024-05-11T23:03:17.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.909 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.909 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36989 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 638976~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:17.909 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:03:17.909 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 84.929802907999999, 2024-05-11T23:03:17.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05192811, 2024-05-11T23:03:17.909 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.909 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.909 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.910 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.910 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.910 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36989 2024-05-11T23:03:17.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.910 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:03:17.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:03:17.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942916+0000", 2024-05-11T23:03:17.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001820835 2024-05-11T23:03:17.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942920+0000", 2024-05-11T23:03:17.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5310000000000002e-06 2024-05-11T23:03:17.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942931+0000", 2024-05-11T23:03:17.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1839e-05 2024-05-11T23:03:17.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942940+0000", 2024-05-11T23:03:17.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2309999999999996e-06 2024-05-11T23:03:17.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992820+0000", 2024-05-11T23:03:17.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049880751000000001 2024-05-11T23:03:17.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992918+0000", 2024-05-11T23:03:17.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7051000000000003e-05 2024-05-11T23:03:17.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.993023+0000", 2024-05-11T23:03:17.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010587200000000001 2024-05-11T23:03:17.913 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.913 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.913 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.913 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36997 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 933888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:17.913 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:03:17.913 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 84.927769072999993, 2024-05-11T23:03:17.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051390602000000001, 2024-05-11T23:03:17.913 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.913 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.913 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.913 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.914 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.914 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.914 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36997 2024-05-11T23:03:17.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.914 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:03:17.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:03:17.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943130+0000", 2024-05-11T23:03:17.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0200000000000003e-07 2024-05-11T23:03:17.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943132+0000", 2024-05-11T23:03:17.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.305e-06 2024-05-11T23:03:17.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943138+0000", 2024-05-11T23:03:17.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4219999999999997e-06 2024-05-11T23:03:17.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943143+0000", 2024-05-11T23:03:17.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0610000000000003e-06 2024-05-11T23:03:17.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994323+0000", 2024-05-11T23:03:17.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051179592000000003 2024-05-11T23:03:17.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994412+0000", 2024-05-11T23:03:17.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8709999999999996e-05 2024-05-11T23:03:17.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994520+0000", 2024-05-11T23:03:17.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010811000000000001 2024-05-11T23:03:17.917 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.917 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.917 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.917 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36998 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 970752~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:17.917 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:03:17.917 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 84.927748194000003, 2024-05-11T23:03:17.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051517669000000002, 2024-05-11T23:03:17.917 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.917 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.917 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.917 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.918 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.918 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.918 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36998 2024-05-11T23:03:17.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.918 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:03:17.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:03:17.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:03:17.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:03:17.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943153+0000", 2024-05-11T23:03:17.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.306e-06 2024-05-11T23:03:17.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943160+0000", 2024-05-11T23:03:17.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.452e-06 2024-05-11T23:03:17.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943165+0000", 2024-05-11T23:03:17.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.152e-06 2024-05-11T23:03:17.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994561+0000", 2024-05-11T23:03:17.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051395765000000003 2024-05-11T23:03:17.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994607+0000", 2024-05-11T23:03:17.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6415000000000002e-05 2024-05-11T23:03:17.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994668+0000", 2024-05-11T23:03:17.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0179000000000001e-05 2024-05-11T23:03:17.921 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.921 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.921 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.921 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36999 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 1007616~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:17.921 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:03:17.921 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 84.927703312999995, 2024-05-11T23:03:17.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051696919000000001, 2024-05-11T23:03:17.921 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.921 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.921 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.921 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.922 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.922 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.922 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36999 2024-05-11T23:03:17.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.922 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:03:17.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:03:17.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:03:17.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:03:17.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943197+0000", 2024-05-11T23:03:17.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1280000000000002e-06 2024-05-11T23:03:17.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943202+0000", 2024-05-11T23:03:17.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8910000000000003e-06 2024-05-11T23:03:17.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943206+0000", 2024-05-11T23:03:17.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3550000000000001e-06 2024-05-11T23:03:17.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994703+0000", 2024-05-11T23:03:17.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051497045999999998 2024-05-11T23:03:17.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994789+0000", 2024-05-11T23:03:17.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6458000000000005e-05 2024-05-11T23:03:17.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994892+0000", 2024-05-11T23:03:17.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102705 2024-05-11T23:03:17.925 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.925 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.925 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.925 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37000 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 1044480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:17.925 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:03:17.925 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 84.927687367999994, 2024-05-11T23:03:17.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051842371999999998, 2024-05-11T23:03:17.925 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.926 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.926 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.926 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.926 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.926 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37000 2024-05-11T23:03:17.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.926 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:03:17.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:03:17.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944916+0000", 2024-05-11T23:03:17.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001705097 2024-05-11T23:03:17.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944921+0000", 2024-05-11T23:03:17.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5809999999999998e-06 2024-05-11T23:03:17.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944930+0000", 2024-05-11T23:03:17.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5529999999999998e-06 2024-05-11T23:03:17.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944938+0000", 2024-05-11T23:03:17.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2519999999999999e-06 2024-05-11T23:03:17.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994911+0000", 2024-05-11T23:03:17.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049973113999999999 2024-05-11T23:03:17.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994982+0000", 2024-05-11T23:03:17.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0544999999999996e-05 2024-05-11T23:03:17.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.995053+0000", 2024-05-11T23:03:17.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1229999999999994e-05 2024-05-11T23:03:17.929 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.929 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.929 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.929 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37041 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 618496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:17.929 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:03:17.929 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 84.925843388999994, 2024-05-11T23:03:17.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051352768999999999, 2024-05-11T23:03:17.930 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.930 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.930 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.930 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.930 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.930 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.930 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37041 2024-05-11T23:03:17.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.930 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:03:17.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:03:17.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:03:17.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:03:17.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945056+0000", 2024-05-11T23:03:17.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:03:17.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.932 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.932 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945059+0000", 2024-05-11T23:03:17.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.965e-06 2024-05-11T23:03:17.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945062+0000", 2024-05-11T23:03:17.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:03:17.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996213+0000", 2024-05-11T23:03:17.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051151532999999999 2024-05-11T23:03:17.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996301+0000", 2024-05-11T23:03:17.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8143e-05 2024-05-11T23:03:17.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996408+0000", 2024-05-11T23:03:17.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010623700000000001 2024-05-11T23:03:17.934 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.934 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.934 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.934 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37042 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 655360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:17.934 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:03:17.934 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 84.925833404000002, 2024-05-11T23:03:17.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051462075000000003, 2024-05-11T23:03:17.935 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.935 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.935 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.935 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.935 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.935 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.935 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37042 2024-05-11T23:03:17.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.935 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:03:17.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:03:17.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:03:17.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0599999999999999e-07 2024-05-11T23:03:17.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945066+0000", 2024-05-11T23:03:17.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-05-11T23:03:17.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945069+0000", 2024-05-11T23:03:17.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0079999999999998e-06 2024-05-11T23:03:17.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945072+0000", 2024-05-11T23:03:17.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6740000000000001e-06 2024-05-11T23:03:17.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996422+0000", 2024-05-11T23:03:17.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051349999 2024-05-11T23:03:17.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996467+0000", 2024-05-11T23:03:17.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.528e-05 2024-05-11T23:03:17.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996527+0000", 2024-05-11T23:03:17.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9759999999999997e-05 2024-05-11T23:03:17.938 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.938 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.938 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.938 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37043 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 692224~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:17.939 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:03:17.939 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 84.925821330999995, 2024-05-11T23:03:17.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051679535999999998, 2024-05-11T23:03:17.939 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.939 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.939 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.939 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.939 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.939 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37043 2024-05-11T23:03:17.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.939 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:03:17.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:03:17.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:03:17.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8999999999999997e-07 2024-05-11T23:03:17.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945079+0000", 2024-05-11T23:03:17.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.001e-06 2024-05-11T23:03:17.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945087+0000", 2024-05-11T23:03:17.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1749999999999999e-06 2024-05-11T23:03:17.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945093+0000", 2024-05-11T23:03:17.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9639999999999997e-06 2024-05-11T23:03:17.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.941 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.941 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996563+0000", 2024-05-11T23:03:17.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051469050000000002 2024-05-11T23:03:17.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996651+0000", 2024-05-11T23:03:17.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8356000000000006e-05 2024-05-11T23:03:17.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996756+0000", 2024-05-11T23:03:17.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0001055 2024-05-11T23:03:17.942 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.942 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.942 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.943 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37044 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 729088~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:17.943 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:03:17.943 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 84.925787419000002, 2024-05-11T23:03:17.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051763445999999998, 2024-05-11T23:03:17.943 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.943 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.943 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.943 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.943 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.943 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.943 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37044 2024-05-11T23:03:17.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.943 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:03:17.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:03:17.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946915+0000", 2024-05-11T23:03:17.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018042760000000001 2024-05-11T23:03:17.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946918+0000", 2024-05-11T23:03:17.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.36e-06 2024-05-11T23:03:17.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946926+0000", 2024-05-11T23:03:17.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3130000000000001e-06 2024-05-11T23:03:17.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.945 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946930+0000", 2024-05-11T23:03:17.945 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5249999999999999e-06 2024-05-11T23:03:17.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.945 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.945 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996772+0000", 2024-05-11T23:03:17.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049841868999999997 2024-05-11T23:03:17.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996817+0000", 2024-05-11T23:03:17.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5351e-05 2024-05-11T23:03:17.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.946 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.946 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.946 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996874+0000", 2024-05-11T23:03:17.946 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6752000000000001e-05 2024-05-11T23:03:17.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.946 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.947 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.947 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.947 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37052 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 1024000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:17.947 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:03:17.947 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 84.923880568000001, 2024-05-11T23:03:17.947 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051326140999999999, 2024-05-11T23:03:17.947 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.947 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.947 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.947 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.948 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.948 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.948 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37052 2024-05-11T23:03:17.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.948 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:03:17.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.948 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.948 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:03:17.948 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:03:17.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:17.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947019+0000", 2024-05-11T23:03:17.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.218e-06 2024-05-11T23:03:17.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.949 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.949 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.949 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947022+0000", 2024-05-11T23:03:17.949 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1889999999999999e-06 2024-05-11T23:03:17.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947025+0000", 2024-05-11T23:03:17.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.464e-06 2024-05-11T23:03:17.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998237+0000", 2024-05-11T23:03:17.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051212494999999997 2024-05-11T23:03:17.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.950 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.950 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998284+0000", 2024-05-11T23:03:17.950 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7246000000000001e-05 2024-05-11T23:03:17.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.951 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.951 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998344+0000", 2024-05-11T23:03:17.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9320000000000001e-05 2024-05-11T23:03:17.951 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.951 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.951 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.951 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.951 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37071 3.b 3:d385bc8e:::rbd_data.111e5e4d8275.0000000000000020:11 [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:17.951 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:03:17.951 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 84.923870359000006, 2024-05-11T23:03:17.951 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051528796000000002, 2024-05-11T23:03:17.951 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:17.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:17.952 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:17.952 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:17.952 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:17.952 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:17.952 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37071 2024-05-11T23:03:17.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.952 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:17.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:17.952 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:03:17.952 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.952 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.952 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:17.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:03:17.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:17.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:17.953 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948913+0000", 2024-05-11T23:03:17.953 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018854970000000001 2024-05-11T23:03:17.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.953 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:17.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948917+0000", 2024-05-11T23:03:17.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8089999999999999e-06 2024-05-11T23:03:17.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:17.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948926+0000", 2024-05-11T23:03:17.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8389999999999996e-06 2024-05-11T23:03:17.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.954 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.954 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:17.954 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948930+0000", 2024-05-11T23:03:17.954 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4519999999999999e-06 2024-05-11T23:03:17.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:17.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998379+0000", 2024-05-11T23:03:17.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049449011000000001 2024-05-11T23:03:17.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:17.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998462+0000", 2024-05-11T23:03:17.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2739999999999997e-05 2024-05-11T23:03:17.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:17.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:17.955 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:17.955 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998557+0000", 2024-05-11T23:03:17.955 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4448000000000001e-05 2024-05-11T23:03:17.955 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.956 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.956 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.956 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:17.956 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:17.956 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:17.957 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:03:17.958 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:17.957+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:18.058 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:18.058+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:18.064 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:18.064 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:18.064 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:18.064 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:18.072 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:03:18.159 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:18.159+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:18.217 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-05-11T23:03:18.218 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:18.218 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:18.218 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:18.218 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:18.219 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:18.230 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:03:18.275 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:18.275+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:18.281 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:03:18.343 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116446 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8273920~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.966939296, 2024-05-11T23:03:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045535299000000001, 2024-05-11T23:03:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116446 2024-05-11T23:03:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375547+0000", 2024-05-11T23:03:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4000000000000001e-07 2024-05-11T23:03:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375549+0000", 2024-05-11T23:03:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.368e-06 2024-05-11T23:03:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375556+0000", 2024-05-11T23:03:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1840000000000004e-06 2024-05-11T23:03:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375560+0000", 2024-05-11T23:03:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.741e-06 2024-05-11T23:03:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420896+0000", 2024-05-11T23:03:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045335435 2024-05-11T23:03:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420987+0000", 2024-05-11T23:03:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1033999999999996e-05 2024-05-11T23:03:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421082+0000", 2024-05-11T23:03:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4896999999999996e-05 2024-05-11T23:03:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116447 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6316032~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.966917786000003, 2024-05-11T23:03:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045745830000000001, 2024-05-11T23:03:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116447 2024-05-11T23:03:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.58e-07 2024-05-11T23:03:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375570+0000", 2024-05-11T23:03:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:03:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375576+0000", 2024-05-11T23:03:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7939999999999999e-06 2024-05-11T23:03:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375581+0000", 2024-05-11T23:03:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9280000000000001e-06 2024-05-11T23:03:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421127+0000", 2024-05-11T23:03:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045545785999999998 2024-05-11T23:03:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421214+0000", 2024-05-11T23:03:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7039000000000005e-05 2024-05-11T23:03:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421314+0000", 2024-05-11T23:03:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9772999999999995e-05 2024-05-11T23:03:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116448 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8314880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.966898712000003, 2024-05-11T23:03:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045941753000000002, 2024-05-11T23:03:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116448 2024-05-11T23:03:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:18.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:18.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.77e-07 2024-05-11T23:03:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375589+0000", 2024-05-11T23:03:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.091e-06 2024-05-11T23:03:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375596+0000", 2024-05-11T23:03:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9179999999999997e-06 2024-05-11T23:03:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375601+0000", 2024-05-11T23:03:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0030000000000002e-06 2024-05-11T23:03:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421363+0000", 2024-05-11T23:03:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045761352999999998 2024-05-11T23:03:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421440+0000", 2024-05-11T23:03:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7585999999999999e-05 2024-05-11T23:03:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421529+0000", 2024-05-11T23:03:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8424999999999994e-05 2024-05-11T23:03:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116449 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6438912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.966878276999999, 2024-05-11T23:03:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046082208999999999, 2024-05-11T23:03:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116449 2024-05-11T23:03:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377225+0000", 2024-05-11T23:03:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016177030000000001 2024-05-11T23:03:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377229+0000", 2024-05-11T23:03:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8e-06 2024-05-11T23:03:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377238+0000", 2024-05-11T23:03:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2229999999999996e-06 2024-05-11T23:03:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377244+0000", 2024-05-11T23:03:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4060000000000002e-06 2024-05-11T23:03:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421520+0000", 2024-05-11T23:03:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044275222000000003 2024-05-11T23:03:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421595+0000", 2024-05-11T23:03:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4838000000000004e-05 2024-05-11T23:03:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421690+0000", 2024-05-11T23:03:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5017000000000002e-05 2024-05-11T23:03:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116458 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7094272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.965064060000003, 2024-05-11T23:03:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045706877999999999, 2024-05-11T23:03:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116458 2024-05-11T23:03:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9400000000000005e-07 2024-05-11T23:03:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377424+0000", 2024-05-11T23:03:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1859999999999999e-06 2024-05-11T23:03:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377431+0000", 2024-05-11T23:03:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4049999999999997e-06 2024-05-11T23:03:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377436+0000", 2024-05-11T23:03:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1399999999999999e-06 2024-05-11T23:03:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.422945+0000", 2024-05-11T23:03:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045509326000000003 2024-05-11T23:03:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423036+0000", 2024-05-11T23:03:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0649999999999994e-05 2024-05-11T23:03:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423128+0000", 2024-05-11T23:03:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2577000000000005e-05 2024-05-11T23:03:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116459 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7258112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.965043328, 2024-05-11T23:03:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045831945999999998, 2024-05-11T23:03:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116459 2024-05-11T23:03:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377443+0000", 2024-05-11T23:03:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999999e-07 2024-05-11T23:03:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377445+0000", 2024-05-11T23:03:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0209999999999998e-06 2024-05-11T23:03:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377451+0000", 2024-05-11T23:03:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.071e-06 2024-05-11T23:03:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377456+0000", 2024-05-11T23:03:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8289999999999998e-06 2024-05-11T23:03:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423155+0000", 2024-05-11T23:03:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045699447999999997 2024-05-11T23:03:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423200+0000", 2024-05-11T23:03:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5083000000000002e-05 2024-05-11T23:03:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423274+0000", 2024-05-11T23:03:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4046000000000002e-05 2024-05-11T23:03:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116460 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7299072~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.965024266, 2024-05-11T23:03:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046010987000000003, 2024-05-11T23:03:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116460 2024-05-11T23:03:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377462+0000", 2024-05-11T23:03:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:03:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377464+0000", 2024-05-11T23:03:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1600000000000001e-06 2024-05-11T23:03:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377469+0000", 2024-05-11T23:03:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5430000000000001e-06 2024-05-11T23:03:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377474+0000", 2024-05-11T23:03:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8509999999999998e-06 2024-05-11T23:03:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423312+0000", 2024-05-11T23:03:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045838080000000003 2024-05-11T23:03:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423401+0000", 2024-05-11T23:03:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8170000000000005e-05 2024-05-11T23:03:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423472+0000", 2024-05-11T23:03:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1847000000000005e-05 2024-05-11T23:03:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116461 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7421952~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.965005496000003, 2024-05-11T23:03:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046130749999999998, 2024-05-11T23:03:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116461 2024-05-11T23:03:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4700000000000002e-07 2024-05-11T23:03:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377483+0000", 2024-05-11T23:03:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0219999999999999e-06 2024-05-11T23:03:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377489+0000", 2024-05-11T23:03:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4420000000000003e-06 2024-05-11T23:03:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377494+0000", 2024-05-11T23:03:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9239999999999998e-06 2024-05-11T23:03:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423500+0000", 2024-05-11T23:03:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046006054999999997 2024-05-11T23:03:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423557+0000", 2024-05-11T23:03:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7169000000000001e-05 2024-05-11T23:03:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423611+0000", 2024-05-11T23:03:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3791e-05 2024-05-11T23:03:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116462 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7462912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.964985878, 2024-05-11T23:03:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046254666, 2024-05-11T23:03:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116462 2024-05-11T23:03:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377502+0000", 2024-05-11T23:03:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-05-11T23:03:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377508+0000", 2024-05-11T23:03:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4500000000000003e-06 2024-05-11T23:03:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377512+0000", 2024-05-11T23:03:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7700000000000001e-06 2024-05-11T23:03:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423649+0000", 2024-05-11T23:03:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046136859000000002 2024-05-11T23:03:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423704+0000", 2024-05-11T23:03:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4938000000000002e-05 2024-05-11T23:03:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423754+0000", 2024-05-11T23:03:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0145999999999997e-05 2024-05-11T23:03:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116463 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7585792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.964967452000003, 2024-05-11T23:03:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361763, 2024-05-11T23:03:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116463 2024-05-11T23:03:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377519+0000", 2024-05-11T23:03:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2600000000000002e-07 2024-05-11T23:03:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377521+0000", 2024-05-11T23:03:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1710000000000001e-06 2024-05-11T23:03:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377527+0000", 2024-05-11T23:03:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9649999999999996e-06 2024-05-11T23:03:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377532+0000", 2024-05-11T23:03:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.78e-06 2024-05-11T23:03:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423779+0000", 2024-05-11T23:03:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046247706 2024-05-11T23:03:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423829+0000", 2024-05-11T23:03:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9188999999999999e-05 2024-05-11T23:03:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423880+0000", 2024-05-11T23:03:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1425999999999999e-05 2024-05-11T23:03:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116464 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7749632~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.964948216000003, 2024-05-11T23:03:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046491719000000001, 2024-05-11T23:03:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116464 2024-05-11T23:03:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377538+0000", 2024-05-11T23:03:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4499999999999998e-07 2024-05-11T23:03:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377540+0000", 2024-05-11T23:03:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2340000000000001e-06 2024-05-11T23:03:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377546+0000", 2024-05-11T23:03:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8289999999999996e-06 2024-05-11T23:03:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377551+0000", 2024-05-11T23:03:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8559999999999998e-06 2024-05-11T23:03:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423913+0000", 2024-05-11T23:03:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361885999999998 2024-05-11T23:03:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423958+0000", 2024-05-11T23:03:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5513999999999998e-05 2024-05-11T23:03:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424029+0000", 2024-05-11T23:03:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1055000000000003e-05 2024-05-11T23:03:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116465 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7626752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.964929068000004, 2024-05-11T23:03:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046600057, 2024-05-11T23:03:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116465 2024-05-11T23:03:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3599999999999999e-07 2024-05-11T23:03:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377559+0000", 2024-05-11T23:03:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9640000000000002e-06 2024-05-11T23:03:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377572+0000", 2024-05-11T23:03:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3188e-05 2024-05-11T23:03:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377578+0000", 2024-05-11T23:03:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.942e-06 2024-05-11T23:03:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424058+0000", 2024-05-11T23:03:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046480021000000003 2024-05-11T23:03:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424103+0000", 2024-05-11T23:03:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5021999999999997e-05 2024-05-11T23:03:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424157+0000", 2024-05-11T23:03:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3483999999999998e-05 2024-05-11T23:03:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116466 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7790592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.964900485999998, 2024-05-11T23:03:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046743741999999998, 2024-05-11T23:03:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116466 2024-05-11T23:03:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377586+0000", 2024-05-11T23:03:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:03:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377588+0000", 2024-05-11T23:03:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3039999999999999e-06 2024-05-11T23:03:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377594+0000", 2024-05-11T23:03:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-05-11T23:03:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377600+0000", 2024-05-11T23:03:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7699999999999998e-06 2024-05-11T23:03:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424185+0000", 2024-05-11T23:03:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046585891999999997 2024-05-11T23:03:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424235+0000", 2024-05-11T23:03:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9356e-05 2024-05-11T23:03:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424329+0000", 2024-05-11T23:03:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4036999999999994e-05 2024-05-11T23:03:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116467 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7954432~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.964880199, 2024-05-11T23:03:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046930134999999998, 2024-05-11T23:03:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116467 2024-05-11T23:03:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379161+0000", 2024-05-11T23:03:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015555009999999999 2024-05-11T23:03:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379166+0000", 2024-05-11T23:03:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6689999999999997e-06 2024-05-11T23:03:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379176+0000", 2024-05-11T23:03:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0549e-05 2024-05-11T23:03:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379183+0000", 2024-05-11T23:03:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1199999999999996e-06 2024-05-11T23:03:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424359+0000", 2024-05-11T23:03:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045175949999999999 2024-05-11T23:03:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424433+0000", 2024-05-11T23:03:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3824000000000003e-05 2024-05-11T23:03:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424536+0000", 2024-05-11T23:03:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010252200000000001 2024-05-11T23:03:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116468 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7913472~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.963289979999999, 2024-05-11T23:03:18.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045525364999999998, 2024-05-11T23:03:18.406 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:18.406 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:18.406 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:18.406 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:18.406 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:18.406 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:18.406 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116468 2024-05-11T23:03:18.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.406 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:18.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:18.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:18.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:18.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:18.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:18.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:18.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3300000000000002e-07 2024-05-11T23:03:18.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:18.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379198+0000", 2024-05-11T23:03:18.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2670000000000001e-06 2024-05-11T23:03:18.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:18.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379206+0000", 2024-05-11T23:03:18.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7149999999999998e-06 2024-05-11T23:03:18.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:18.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379228+0000", 2024-05-11T23:03:18.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1429000000000001e-05 2024-05-11T23:03:18.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:18.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424574+0000", 2024-05-11T23:03:18.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045346209999999998 2024-05-11T23:03:18.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:18.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424627+0000", 2024-05-11T23:03:18.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3352e-05 2024-05-11T23:03:18.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:18.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424721+0000", 2024-05-11T23:03:18.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3858999999999997e-05 2024-05-11T23:03:18.409 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.409 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.409 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.410 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116469 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8077312~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:18.410 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:18.410 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.963240159999998, 2024-05-11T23:03:18.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045636687000000002, 2024-05-11T23:03:18.410 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:18.410 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:18.410 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:18.410 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:18.410 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:18.410 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:18.410 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116469 2024-05-11T23:03:18.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.411 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:18.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:18.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:18.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:18.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:18.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:18.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379246+0000", 2024-05-11T23:03:18.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.32e-07 2024-05-11T23:03:18.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:18.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379254+0000", 2024-05-11T23:03:18.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.807e-06 2024-05-11T23:03:18.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:18.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379261+0000", 2024-05-11T23:03:18.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.985e-06 2024-05-11T23:03:18.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:18.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379264+0000", 2024-05-11T23:03:18.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6480000000000001e-06 2024-05-11T23:03:18.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:18.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424744+0000", 2024-05-11T23:03:18.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045479638000000003 2024-05-11T23:03:18.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:18.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424797+0000", 2024-05-11T23:03:18.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2725000000000001e-05 2024-05-11T23:03:18.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:18.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424882+0000", 2024-05-11T23:03:18.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5451999999999994e-05 2024-05-11T23:03:18.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.414 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.414 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116470 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8118272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:18.414 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:18.414 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.963214948000001, 2024-05-11T23:03:18.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045755745, 2024-05-11T23:03:18.414 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:18.414 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:18.415 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:18.415 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:18.415 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:18.415 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:18.415 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116470 2024-05-11T23:03:18.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.415 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:18.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:18.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:18.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:18.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:18.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:18.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:18.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3999999999999997e-07 2024-05-11T23:03:18.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:18.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379273+0000", 2024-05-11T23:03:18.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-05-11T23:03:18.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:18.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379277+0000", 2024-05-11T23:03:18.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4020000000000004e-06 2024-05-11T23:03:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379280+0000", 2024-05-11T23:03:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-05-11T23:03:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424922+0000", 2024-05-11T23:03:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045642598999999999 2024-05-11T23:03:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424973+0000", 2024-05-11T23:03:18.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0674999999999998e-05 2024-05-11T23:03:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425026+0000", 2024-05-11T23:03:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3492000000000002e-05 2024-05-11T23:03:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116471 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8241152~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.963202164999998, 2024-05-11T23:03:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045867374000000002, 2024-05-11T23:03:18.418 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116471 2024-05-11T23:03:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379284+0000", 2024-05-11T23:03:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.53e-07 2024-05-11T23:03:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379285+0000", 2024-05-11T23:03:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.223e-06 2024-05-11T23:03:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:18.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379288+0000", 2024-05-11T23:03:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5379999999999998e-06 2024-05-11T23:03:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379291+0000", 2024-05-11T23:03:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6699999999999998e-06 2024-05-11T23:03:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425055+0000", 2024-05-11T23:03:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045763497 2024-05-11T23:03:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425099+0000", 2024-05-11T23:03:18.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3899000000000001e-05 2024-05-11T23:03:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425151+0000", 2024-05-11T23:03:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2293999999999998e-05 2024-05-11T23:03:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116472 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8282112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.963191166000001, 2024-05-11T23:03:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045973778999999999, 2024-05-11T23:03:18.422 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116472 2024-05-11T23:03:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381226+0000", 2024-05-11T23:03:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001931627 2024-05-11T23:03:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381231+0000", 2024-05-11T23:03:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8679999999999998e-06 2024-05-11T23:03:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:18.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381242+0000", 2024-05-11T23:03:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0702e-05 2024-05-11T23:03:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381249+0000", 2024-05-11T23:03:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9800000000000001e-06 2024-05-11T23:03:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425175+0000", 2024-05-11T23:03:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043926805999999999 2024-05-11T23:03:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:18.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425215+0000", 2024-05-11T23:03:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9737999999999998e-05 2024-05-11T23:03:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425268+0000", 2024-05-11T23:03:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3057999999999999e-05 2024-05-11T23:03:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116482 3.1b 3:db6e8843:::rbd_object_map.111e67f27da3.0000000000000028:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 42.961222622999998, 2024-05-11T23:03:18.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046526076, 2024-05-11T23:03:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116482 2024-05-11T23:03:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381264+0000", 2024-05-11T23:03:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5300000000000004e-07 2024-05-11T23:03:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381267+0000", 2024-05-11T23:03:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1999999999999999e-06 2024-05-11T23:03:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381274+0000", 2024-05-11T23:03:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7109999999999995e-06 2024-05-11T23:03:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381285+0000", 2024-05-11T23:03:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0856000000000001e-05 2024-05-11T23:03:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425292+0000", 2024-05-11T23:03:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044006376 2024-05-11T23:03:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425337+0000", 2024-05-11T23:03:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4907999999999998e-05 2024-05-11T23:03:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:03:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425507+0000", 2024-05-11T23:03:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017029999999999999 2024-05-11T23:03:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427668+0000", 2024-05-11T23:03:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021612430000000002 2024-05-11T23:03:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427683+0000", 2024-05-11T23:03:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5016e-05 2024-05-11T23:03:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427742+0000", 2024-05-11T23:03:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8811999999999999e-05 2024-05-11T23:03:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427789+0000", 2024-05-11T23:03:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7101000000000002e-05 2024-05-11T23:03:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.431 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:18.432 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:18.432 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:18.432 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:18.433 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:18.433 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:18.434 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:18.434 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:03:18.435 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:18.433+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:18.448 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:18.448 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:18.465 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:03:18.465 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:03:18.487 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:18.487 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:18.501 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:18.501 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:18.518 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:18.519 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:18.536 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:18.534+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:18.537 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:18.538 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:18.543 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:18.543 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:18.551 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:03:18.635 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:18.635+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:18.655 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:18.655 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:18.663 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:03:18.736 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:18.736+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:18.768 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:18.768 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:18.777 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:03:18.837 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:18.837+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:18.923 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:18.923 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:18.931 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:03:18.938 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:18.938+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:19.037 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:19.037 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:19.039 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:19.039+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:19.046 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:03:19.140 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:19.140+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:19.191 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:19.192 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:19.200 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:03:19.241 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:19.241+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:19.318 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:19.318 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:19.329 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:03:19.342 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:19.342+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:19.436 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:19.436 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:19.443 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:19.444+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:19.445 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:03:19.545 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:19.545+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:19.550 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:19.550 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:19.558 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:03:19.645 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:19.645+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:19.672 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:19.673 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:19.681 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:03:19.747 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:19.746+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:19.791 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:19.792 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:19.800 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:03:19.848 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:19.848+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:19.910 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:19.911 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:19.919 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:03:19.948 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:19.948+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:20.028 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:20.029 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:20.037 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:03:20.081 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:20.081+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:20.147 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:20.147 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:20.149 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:20.149+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:20.155 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:03:20.217 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 5, 7, 3, 6] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:03:20.217 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:03:20.217 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-05-11T23:03:20.217 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-11T23:03:20.233 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-11T23:03:20.284 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:20.284+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:20.323 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:20.323 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:20.334 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:03:20.375 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:20.375+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:20.452 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:20.452+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:20.486 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:20.486 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:20.495 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:03:20.549 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:03:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:03:20.557 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 117, 2024-05-11T23:03:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: "fsid": "34b20022-f11b-4169-a5f6-6386270a2f8f", 2024-05-11T23:03:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: "created": "2024-05-11T23:01:05.977693+0000", 2024-05-11T23:03:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: "modified": "2024-05-11T23:03:20.220269+0000", 2024-05-11T23:03:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: "last_up_change": "2024-05-11T23:01:17.148916+0000", 2024-05-11T23:03:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: "last_in_change": "2024-05-11T23:02:51.095898+0000", 2024-05-11T23:03:20.557 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-11T23:03:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_num": 5806080, 2024-05-11T23:03:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_set": [ 2024-05-11T23:03:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "nodeep-scrub", 2024-05-11T23:03:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "noscrub", 2024-05-11T23:03:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "pglog_hardlimit", 2024-05-11T23:03:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "purged_snapdirs", 2024-05-11T23:03:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_deletes", 2024-05-11T23:03:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "sortbitwise" 2024-05-11T23:03:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:03:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_version": 5, 2024-05-11T23:03:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "full_ratio": 0.94999998807907104, 2024-05-11T23:03:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-11T23:03:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-11T23:03:20.558 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_snapshot": "", 2024-05-11T23:03:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_max": 3, 2024-05-11T23:03:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd": 8, 2024-05-11T23:03:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "require_min_compat_client": "luminous", 2024-05-11T23:03:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "min_compat_client": "jewel", 2024-05-11T23:03:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "require_osd_release": "squid", 2024-05-11T23:03:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_crimson": false, 2024-05-11T23:03:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "pools": [ 2024-05-11T23:03:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 1, 2024-05-11T23:03:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": ".mgr", 2024-05-11T23:03:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-05-11T23:01:15.187614+0000", 2024-05-11T23:03:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-11T23:03:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool", 2024-05-11T23:03:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-05-11T23:03:20.559 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-05-11T23:03:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-05-11T23:03:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-05-11T23:03:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-05-11T23:03:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-05-11T23:03:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-05-11T23:03:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-11T23:03:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-05-11T23:03:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-05-11T23:03:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 1, 2024-05-11T23:03:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 1, 2024-05-11T23:03:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 1, 2024-05-11T23:03:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 1, 2024-05-11T23:03:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 1, 2024-05-11T23:03:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-05-11T23:03:20.560 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-05-11T23:03:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-05-11T23:03:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-05-11T23:03:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-05-11T23:03:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-05-11T23:03:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-05-11T23:03:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "17", 2024-05-11T23:03:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-05-11T23:03:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-11T23:03:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-05-11T23:03:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-05-11T23:03:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-05-11T23:03:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 0, 2024-05-11T23:03:20.561 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 0, 2024-05-11T23:03:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-05-11T23:03:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-05-11T23:03:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-05-11T23:03:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-05-11T23:03:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-05-11T23:03:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-05-11T23:03:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-05-11T23:03:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-05-11T23:03:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-05-11T23:03:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-05-11T23:03:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-05-11T23:03:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-11T23:03:20.562 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-11T23:03:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-11T23:03:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-05-11T23:03:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-05-11T23:03:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-05-11T23:03:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-05-11T23:03:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-05-11T23:03:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-05-11T23:03:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-05-11T23:03:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-05-11T23:03:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-05-11T23:03:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-05-11T23:03:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-05-11T23:03:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-05-11T23:03:20.563 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-05-11T23:03:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-05-11T23:03:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-05-11T23:03:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-05-11T23:03:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "options": { 2024-05-11T23:03:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_max": 32, 2024-05-11T23:03:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_min": 1 2024-05-11T23:03:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-05-11T23:03:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": {} 2024-05-11T23:03:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-05-11T23:03:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-05-11T23:03:20.564 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 8, 2024-05-11T23:03:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 8, 2024-05-11T23:03:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 0.25, 2024-05-11T23:03:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 2, 2024-05-11T23:03:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 2, 2024-05-11T23:03:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.89640998840332031, 2024-05-11T23:03:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.89999997615814209, 2024-05-11T23:03:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-05-11T23:03:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 2, 2024-05-11T23:03:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "rbd", 2024-05-11T23:03:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-05-11T23:01:17.689285+0000", 2024-05-11T23:03:20.565 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-05-11T23:03:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool", 2024-05-11T23:03:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-05-11T23:03:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-05-11T23:03:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-05-11T23:03:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-05-11T23:03:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-05-11T23:03:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-05-11T23:03:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-05-11T23:03:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-11T23:03:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-05-11T23:03:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-05-11T23:03:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 8, 2024-05-11T23:03:20.566 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 8, 2024-05-11T23:03:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 8, 2024-05-11T23:03:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 8, 2024-05-11T23:03:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 8, 2024-05-11T23:03:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-05-11T23:03:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-05-11T23:03:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-05-11T23:03:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-05-11T23:03:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-05-11T23:03:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-05-11T23:03:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-05-11T23:03:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "18", 2024-05-11T23:03:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-05-11T23:03:20.567 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-11T23:03:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-05-11T23:03:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-05-11T23:03:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-05-11T23:03:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 0, 2024-05-11T23:03:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 0, 2024-05-11T23:03:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-05-11T23:03:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-05-11T23:03:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-05-11T23:03:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-05-11T23:03:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-05-11T23:03:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-05-11T23:03:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-05-11T23:03:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-05-11T23:03:20.568 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-05-11T23:03:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-05-11T23:03:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-05-11T23:03:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-11T23:03:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-11T23:03:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-11T23:03:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-05-11T23:03:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-05-11T23:03:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-05-11T23:03:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-05-11T23:03:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-05-11T23:03:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-05-11T23:03:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-05-11T23:03:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-05-11T23:03:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-05-11T23:03:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-05-11T23:03:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-05-11T23:03:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-05-11T23:03:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-05-11T23:03:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-05-11T23:03:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-05-11T23:03:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-05-11T23:03:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-05-11T23:03:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-05-11T23:03:20.570 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-05-11T23:03:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-05-11T23:03:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-05-11T23:03:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 1.9900000095367432, 2024-05-11T23:03:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 1.9900000095367432, 2024-05-11T23:03:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 0.87999999523162842, 2024-05-11T23:03:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 1.75, 2024-05-11T23:03:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 1.75, 2024-05-11T23:03:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.89640998840332031, 2024-05-11T23:03:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.89999997615814209, 2024-05-11T23:03:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-05-11T23:03:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 3, 2024-05-11T23:03:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": "pool_client.0", 2024-05-11T23:03:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "create_time": "2024-05-11T23:01:25.743347+0000", 2024-05-11T23:03:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 8193, 2024-05-11T23:03:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-11T23:03:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "type": 1, 2024-05-11T23:03:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 2, 2024-05-11T23:03:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": 1, 2024-05-11T23:03:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_rule": 0, 2024-05-11T23:03:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_count": 0, 2024-05-11T23:03:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_target": 0, 2024-05-11T23:03:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_barrier": 0, 2024-05-11T23:03:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-11T23:03:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "object_hash": 2, 2024-05-11T23:03:20.572 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscale_mode": "off", 2024-05-11T23:03:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": 32, 2024-05-11T23:03:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num": 32, 2024-05-11T23:03:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_placement_num_target": 32, 2024-05-11T23:03:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_target": 32, 2024-05-11T23:03:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num_pending": 32, 2024-05-11T23:03:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "last_pg_merge_meta": { 2024-05-11T23:03:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "source_pgid": "0.0", 2024-05-11T23:03:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "ready_epoch": 0, 2024-05-11T23:03:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_started": 0, 2024-05-11T23:03:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "last_epoch_clean": 0, 2024-05-11T23:03:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "source_version": "0'0", 2024-05-11T23:03:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: "target_version": "0'0" 2024-05-11T23:03:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "last_change": "117", 2024-05-11T23:03:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend": "0", 2024-05-11T23:03:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-11T23:03:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "last_force_op_resend_preluminous": "0", 2024-05-11T23:03:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "auid": 0, 2024-05-11T23:03:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_mode": "selfmanaged", 2024-05-11T23:03:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_seq": 53, 2024-05-11T23:03:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_epoch": 117, 2024-05-11T23:03:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_snaps": [], 2024-05-11T23:03:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps": "[]", 2024-05-11T23:03:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_bytes": 0, 2024-05-11T23:03:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "quota_max_objects": 0, 2024-05-11T23:03:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "tiers": [], 2024-05-11T23:03:20.574 INFO:teuthology.orchestra.run.smithi005.stdout: "tier_of": -1, 2024-05-11T23:03:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "read_tier": -1, 2024-05-11T23:03:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "write_tier": -1, 2024-05-11T23:03:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_mode": "none", 2024-05-11T23:03:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_bytes": 0, 2024-05-11T23:03:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "target_max_objects": 0, 2024-05-11T23:03:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-11T23:03:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-11T23:03:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-11T23:03:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_flush_age": 0, 2024-05-11T23:03:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "cache_min_evict_age": 0, 2024-05-11T23:03:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profile": "", 2024-05-11T23:03:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_params": { 2024-05-11T23:03:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "none" 2024-05-11T23:03:20.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_period": 0, 2024-05-11T23:03:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_count": 0, 2024-05-11T23:03:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "use_gmt_hitset": true, 2024-05-11T23:03:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "min_read_recency_for_promote": 0, 2024-05-11T23:03:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "min_write_recency_for_promote": 0, 2024-05-11T23:03:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_grade_decay_rate": 0, 2024-05-11T23:03:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "hit_set_search_last_n": 0, 2024-05-11T23:03:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "grade_table": [], 2024-05-11T23:03:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "stripe_width": 0, 2024-05-11T23:03:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "expected_num_objects": 0, 2024-05-11T23:03:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "fast_read": false, 2024-05-11T23:03:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "options": {}, 2024-05-11T23:03:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "application_metadata": { 2024-05-11T23:03:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd": {} 2024-05-11T23:03:20.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "read_balance": { 2024-05-11T23:03:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "score_type": "Fair distribution", 2024-05-11T23:03:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "score_acting": 2.2400000095367432, 2024-05-11T23:03:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "score_stable": 2.2400000095367432, 2024-05-11T23:03:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "optimal_score": 0.87999999523162842, 2024-05-11T23:03:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_acting": 1.9700000286102295, 2024-05-11T23:03:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "raw_score_stable": 1.9700000286102295, 2024-05-11T23:03:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity_weighted": 0.89640998840332031, 2024-05-11T23:03:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity": 0.89999997615814209, 2024-05-11T23:03:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-05-11T23:03:20.577 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:03:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "osds": [ 2024-05-11T23:03:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-05-11T23:03:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "57ab1c70-9478-4a26-bb41-d753b4f3eb79", 2024-05-11T23:03:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:03:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:03:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 0.95001220703125, 2024-05-11T23:03:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 0.4373321533203125, 2024-05-11T23:03:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:03:20.578 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:03:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 13, 2024-05-11T23:03:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 97, 2024-05-11T23:03:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:03:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:03:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:03:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-05-11T23:03:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:03:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-05-11T23:03:20.579 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:03:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:03:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6802", 2024-05-11T23:03:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:03:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6803", 2024-05-11T23:03:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:03:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.580 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:03:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6806", 2024-05-11T23:03:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:03:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6808", 2024-05-11T23:03:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:03:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:03:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6804", 2024-05-11T23:03:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:03:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6805", 2024-05-11T23:03:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3452725643 2024-05-11T23:03:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.582 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6801/3452725643", 2024-05-11T23:03:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6803/3452725643", 2024-05-11T23:03:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6808/3452725643", 2024-05-11T23:03:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6805/3452725643", 2024-05-11T23:03:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:03:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:03:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:03:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-05-11T23:03:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "05ece110-4a99-46f0-b061-5980f7ef2a55", 2024-05-11T23:03:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:03:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:03:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 0.95001220703125, 2024-05-11T23:03:20.583 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 0.935302734375, 2024-05-11T23:03:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:03:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:03:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 14, 2024-05-11T23:03:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 108, 2024-05-11T23:03:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:03:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:03:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:03:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6824", 2024-05-11T23:03:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:03:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6825", 2024-05-11T23:03:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:03:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:03:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6826", 2024-05-11T23:03:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:03:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.585 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6827", 2024-05-11T23:03:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:03:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:03:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6830", 2024-05-11T23:03:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:03:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6831", 2024-05-11T23:03:20.586 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:03:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:03:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6828", 2024-05-11T23:03:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:03:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6829", 2024-05-11T23:03:20.587 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4028727138 2024-05-11T23:03:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6825/4028727138", 2024-05-11T23:03:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6827/4028727138", 2024-05-11T23:03:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6831/4028727138", 2024-05-11T23:03:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6829/4028727138", 2024-05-11T23:03:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:03:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:03:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:03:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-05-11T23:03:20.588 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5", 2024-05-11T23:03:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:03:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:03:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:03:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 0.79864501953125, 2024-05-11T23:03:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:03:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:03:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 13, 2024-05-11T23:03:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 103, 2024-05-11T23:03:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:03:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:03:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:03:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6807", 2024-05-11T23:03:20.589 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:03:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6809", 2024-05-11T23:03:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:03:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:03:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.590 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6810", 2024-05-11T23:03:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:03:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6811", 2024-05-11T23:03:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:03:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:03:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.591 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6814", 2024-05-11T23:03:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:03:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6815", 2024-05-11T23:03:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:03:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:03:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6812", 2024-05-11T23:03:20.592 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:03:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6813", 2024-05-11T23:03:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2059959509 2024-05-11T23:03:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6809/2059959509", 2024-05-11T23:03:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6811/2059959509", 2024-05-11T23:03:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6815/2059959509", 2024-05-11T23:03:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6813/2059959509", 2024-05-11T23:03:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:03:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:03:20.593 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:03:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-05-11T23:03:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a", 2024-05-11T23:03:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:03:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:03:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 0.9000244140625, 2024-05-11T23:03:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:03:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:03:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:03:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 14, 2024-05-11T23:03:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 112, 2024-05-11T23:03:20.594 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:03:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:03:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:03:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6816", 2024-05-11T23:03:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:03:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6817", 2024-05-11T23:03:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:03:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:03:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6818", 2024-05-11T23:03:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:03:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6819", 2024-05-11T23:03:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:03:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.596 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:03:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6822", 2024-05-11T23:03:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:03:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6823", 2024-05-11T23:03:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:03:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.597 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:03:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6820", 2024-05-11T23:03:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:03:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6821", 2024-05-11T23:03:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 296930146 2024-05-11T23:03:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.598 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.5:6817/296930146", 2024-05-11T23:03:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.5:6819/296930146", 2024-05-11T23:03:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.5:6823/296930146", 2024-05-11T23:03:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.5:6821/296930146", 2024-05-11T23:03:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:03:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:03:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:03:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-05-11T23:03:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "4e87e4f0-288e-4ee6-919e-9337925a5d5e", 2024-05-11T23:03:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:03:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 0, 2024-05-11T23:03:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 0, 2024-05-11T23:03:20.599 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:03:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:03:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:03:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 15, 2024-05-11T23:03:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 75, 2024-05-11T23:03:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:03:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:03:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:03:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6800", 2024-05-11T23:03:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:03:20.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6801", 2024-05-11T23:03:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:03:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:03:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6802", 2024-05-11T23:03:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:03:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6803", 2024-05-11T23:03:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:03:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:03:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6806", 2024-05-11T23:03:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:03:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6807", 2024-05-11T23:03:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:03:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:03:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6804", 2024-05-11T23:03:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:03:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6805", 2024-05-11T23:03:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 495703077 2024-05-11T23:03:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.115:6801/495703077", 2024-05-11T23:03:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.115:6803/495703077", 2024-05-11T23:03:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.115:6807/495703077", 2024-05-11T23:03:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.115:6805/495703077", 2024-05-11T23:03:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:03:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:03:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:03:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-05-11T23:03:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5", 2024-05-11T23:03:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:03:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:03:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:03:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:03:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:03:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:03:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 15, 2024-05-11T23:03:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 112, 2024-05-11T23:03:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:03:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:03:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:03:20.605 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6811", 2024-05-11T23:03:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:03:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6813", 2024-05-11T23:03:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:03:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:03:20.606 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6815", 2024-05-11T23:03:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:03:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6817", 2024-05-11T23:03:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:03:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:03:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6822", 2024-05-11T23:03:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:03:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6823", 2024-05-11T23:03:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:03:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:03:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6819", 2024-05-11T23:03:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:03:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6821", 2024-05-11T23:03:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 902573489 2024-05-11T23:03:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.115:6813/902573489", 2024-05-11T23:03:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.115:6817/902573489", 2024-05-11T23:03:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.115:6823/902573489", 2024-05-11T23:03:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.115:6821/902573489", 2024-05-11T23:03:20.609 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:03:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:03:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:03:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 6, 2024-05-11T23:03:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "edb686ec-e1a2-40d6-975a-e32e0e7760ff", 2024-05-11T23:03:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:03:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:03:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:03:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:03:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:03:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:03:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 15, 2024-05-11T23:03:20.610 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 112, 2024-05-11T23:03:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:03:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:03:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:03:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6824", 2024-05-11T23:03:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:03:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6825", 2024-05-11T23:03:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:03:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:03:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6826", 2024-05-11T23:03:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:03:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6827", 2024-05-11T23:03:20.612 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:03:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:03:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6830", 2024-05-11T23:03:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:03:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6831", 2024-05-11T23:03:20.613 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:03:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:03:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6828", 2024-05-11T23:03:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:03:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6829", 2024-05-11T23:03:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2749332272 2024-05-11T23:03:20.614 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.115:6825/2749332272", 2024-05-11T23:03:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.115:6827/2749332272", 2024-05-11T23:03:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.115:6831/2749332272", 2024-05-11T23:03:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.115:6829/2749332272", 2024-05-11T23:03:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:03:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:03:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:03:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.615 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 7, 2024-05-11T23:03:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: "uuid": "7aa14dbb-c6cb-4540-b160-3fd346f6a046", 2024-05-11T23:03:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: "up": 1, 2024-05-11T23:03:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: "in": 1, 2024-05-11T23:03:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: "weight": 1, 2024-05-11T23:03:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_affinity": 1, 2024-05-11T23:03:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_begin": 0, 2024-05-11T23:03:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: "last_clean_end": 0, 2024-05-11T23:03:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: "up_from": 15, 2024-05-11T23:03:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: "up_thru": 112, 2024-05-11T23:03:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: "down_at": 0, 2024-05-11T23:03:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: "lost_at": 0, 2024-05-11T23:03:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addrs": { 2024-05-11T23:03:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6808", 2024-05-11T23:03:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:03:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6809", 2024-05-11T23:03:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:03:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addrs": { 2024-05-11T23:03:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6810", 2024-05-11T23:03:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:03:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6812", 2024-05-11T23:03:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:03:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.618 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addrs": { 2024-05-11T23:03:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6818", 2024-05-11T23:03:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:03:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6820", 2024-05-11T23:03:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:03:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addrs": { 2024-05-11T23:03:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-05-11T23:03:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-05-11T23:03:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6814", 2024-05-11T23:03:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:03:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-05-11T23:03:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.115:6816", 2024-05-11T23:03:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1043826337 2024-05-11T23:03:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.620 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: "public_addr": "172.21.15.115:6809/1043826337", 2024-05-11T23:03:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_addr": "172.21.15.115:6812/1043826337", 2024-05-11T23:03:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_back_addr": "172.21.15.115:6820/1043826337", 2024-05-11T23:03:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: "heartbeat_front_addr": "172.21.15.115:6816/1043826337", 2024-05-11T23:03:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: "state": [ 2024-05-11T23:03:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: "exists", 2024-05-11T23:03:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: "up" 2024-05-11T23:03:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.621 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:03:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_xinfo": [ 2024-05-11T23:03:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 0, 2024-05-11T23:03:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:03:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:03:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:03:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:03:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:03:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.113232+0000", 2024-05-11T23:03:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:03:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 1, 2024-05-11T23:03:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:03:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:03:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:03:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:03:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:03:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.174679+0000", 2024-05-11T23:03:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:03:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 2, 2024-05-11T23:03:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:03:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:03:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:03:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:03:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:03:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.131118+0000", 2024-05-11T23:03:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:03:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 3, 2024-05-11T23:03:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:03:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:03:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:03:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:03:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:03:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.207668+0000", 2024-05-11T23:03:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:03:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 4, 2024-05-11T23:03:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:03:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:03:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:03:20.625 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:03:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 65536, 2024-05-11T23:03:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.124542+0000", 2024-05-11T23:03:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:03:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 5, 2024-05-11T23:03:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:03:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:03:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:03:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:03:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:03:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.200832+0000", 2024-05-11T23:03:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:03:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 6, 2024-05-11T23:03:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:03:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:03:20.627 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:03:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:03:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:03:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.169560+0000", 2024-05-11T23:03:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:03:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": 7, 2024-05-11T23:03:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: "down_stamp": "0.000000", 2024-05-11T23:03:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_probability": 0, 2024-05-11T23:03:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: "laggy_interval": 0, 2024-05-11T23:03:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: "features": 4540701547738038271, 2024-05-11T23:03:20.628 INFO:teuthology.orchestra.run.smithi005.stdout: "old_weight": 0, 2024-05-11T23:03:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: "last_purged_snaps_scrub": "2024-05-11T23:01:14.182465+0000", 2024-05-11T23:03:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: "dead_epoch": 0 2024-05-11T23:03:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:03:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap": [], 2024-05-11T23:03:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_items": [], 2024-05-11T23:03:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_upmap_primaries": [], 2024-05-11T23:03:20.629 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_temp": [], 2024-05-11T23:03:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "primary_temp": [], 2024-05-11T23:03:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklist": {}, 2024-05-11T23:03:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "range_blocklist": {}, 2024-05-11T23:03:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "erasure_code_profiles": { 2024-05-11T23:03:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "default": { 2024-05-11T23:03:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-failure-domain": "osd", 2024-05-11T23:03:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "k": "2", 2024-05-11T23:03:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "m": "1", 2024-05-11T23:03:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "plugin": "jerasure", 2024-05-11T23:03:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "technique": "reed_sol_van" 2024-05-11T23:03:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.630 INFO:teuthology.orchestra.run.smithi005.stdout: "removed_snaps_queue": [ 2024-05-11T23:03:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 3, 2024-05-11T23:03:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "snaps": [ 2024-05-11T23:03:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "begin": 51, 2024-05-11T23:03:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "length": 3 2024-05-11T23:03:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:03:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "new_removed_snaps": [ 2024-05-11T23:03:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "pool": 3, 2024-05-11T23:03:20.631 INFO:teuthology.orchestra.run.smithi005.stdout: "snaps": [ 2024-05-11T23:03:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "begin": 51, 2024-05-11T23:03:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "length": 1 2024-05-11T23:03:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:03:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "new_purged_snaps": [], 2024-05-11T23:03:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_node_flags": {}, 2024-05-11T23:03:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "device_class_flags": {}, 2024-05-11T23:03:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode": { 2024-05-11T23:03:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_enabled": false, 2024-05-11T23:03:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_bucket_count": 0, 2024-05-11T23:03:20.632 INFO:teuthology.orchestra.run.smithi005.stdout: "degraded_stretch_mode": 0, 2024-05-11T23:03:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: "recovering_stretch_mode": 0, 2024-05-11T23:03:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: "stretch_mode_bucket": 0 2024-05-11T23:03:20.633 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.633 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:20.633 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:20.556+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:20.635 INFO:tasks.thrashosds.thrasher:j is {'epoch': 117, 'fsid': '34b20022-f11b-4169-a5f6-6386270a2f8f', 'created': '2024-05-11T23:01:05.977693+0000', 'modified': '2024-05-11T23:03:20.220269+0000', 'last_up_change': '2024-05-11T23:01:17.148916+0000', 'last_in_change': '2024-05-11T23:02:51.095898+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-11T23:01:15.187614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8964099884033203, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.8999999761581421}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-05-11T23:01:17.689285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8964099884033203, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.8999999761581421}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-11T23:01:25.743347+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 53, 'snap_epoch': 117, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.240000009536743, 'score_stable': 2.240000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.9700000286102295, 'raw_score_stable': 1.9700000286102295, 'primary_affinity_weighted': 0.8964099884033203, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.8999999761581421}}], 'osds': [{'osd': 0, 'uuid': '57ab1c70-9478-4a26-bb41-d753b4f3eb79', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 0.4373321533203125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 97, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6800', 'nonce': 3452725643}, {'type': 'v1', 'addr': '172.21.15.5:6801', 'nonce': 3452725643}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6802', 'nonce': 3452725643}, {'type': 'v1', 'addr': '172.21.15.5:6803', 'nonce': 3452725643}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6806', 'nonce': 3452725643}, {'type': 'v1', 'addr': '172.21.15.5:6808', 'nonce': 3452725643}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6804', 'nonce': 3452725643}, {'type': 'v1', 'addr': '172.21.15.5:6805', 'nonce': 3452725643}]}, 'public_addr': '172.21.15.5:6801/3452725643', 'cluster_addr': '172.21.15.5:6803/3452725643', 'heartbeat_back_addr': '172.21.15.5:6808/3452725643', 'heartbeat_front_addr': '172.21.15.5:6805/3452725643', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '05ece110-4a99-46f0-b061-5980f7ef2a55', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 0.935302734375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 108, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6824', 'nonce': 4028727138}, {'type': 'v1', 'addr': '172.21.15.5:6825', 'nonce': 4028727138}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6826', 'nonce': 4028727138}, {'type': 'v1', 'addr': '172.21.15.5:6827', 'nonce': 4028727138}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6830', 'nonce': 4028727138}, {'type': 'v1', 'addr': '172.21.15.5:6831', 'nonce': 4028727138}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6828', 'nonce': 4028727138}, {'type': 'v1', 'addr': '172.21.15.5:6829', 'nonce': 4028727138}]}, 'public_addr': '172.21.15.5:6825/4028727138', 'cluster_addr': '172.21.15.5:6827/4028727138', 'heartbeat_back_addr': '172.21.15.5:6831/4028727138', 'heartbeat_front_addr': '172.21.15.5:6829/4028727138', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.79864501953125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 103, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6807', 'nonce': 2059959509}, {'type': 'v1', 'addr': '172.21.15.5:6809', 'nonce': 2059959509}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6810', 'nonce': 2059959509}, {'type': 'v1', 'addr': '172.21.15.5:6811', 'nonce': 2059959509}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6814', 'nonce': 2059959509}, {'type': 'v1', 'addr': '172.21.15.5:6815', 'nonce': 2059959509}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6812', 'nonce': 2059959509}, {'type': 'v1', 'addr': '172.21.15.5:6813', 'nonce': 2059959509}]}, 'public_addr': '172.21.15.5:6809/2059959509', 'cluster_addr': '172.21.15.5:6811/2059959509', 'heartbeat_back_addr': '172.21.15.5:6815/2059959509', 'heartbeat_front_addr': '172.21.15.5:6813/2059959509', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a', 'up': 1, 'in': 1, 'weight': 0.9000244140625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 112, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6816', 'nonce': 296930146}, {'type': 'v1', 'addr': '172.21.15.5:6817', 'nonce': 296930146}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6818', 'nonce': 296930146}, {'type': 'v1', 'addr': '172.21.15.5:6819', 'nonce': 296930146}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6822', 'nonce': 296930146}, {'type': 'v1', 'addr': '172.21.15.5:6823', 'nonce': 296930146}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6820', 'nonce': 296930146}, {'type': 'v1', 'addr': '172.21.15.5:6821', 'nonce': 296930146}]}, 'public_addr': '172.21.15.5:6817/296930146', 'cluster_addr': '172.21.15.5:6819/296930146', 'heartbeat_back_addr': '172.21.15.5:6823/296930146', 'heartbeat_front_addr': '172.21.15.5:6821/296930146', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '4e87e4f0-288e-4ee6-919e-9337925a5d5e', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6800', 'nonce': 495703077}, {'type': 'v1', 'addr': '172.21.15.115:6801', 'nonce': 495703077}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6802', 'nonce': 495703077}, {'type': 'v1', 'addr': '172.21.15.115:6803', 'nonce': 495703077}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6806', 'nonce': 495703077}, {'type': 'v1', 'addr': '172.21.15.115:6807', 'nonce': 495703077}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6804', 'nonce': 495703077}, {'type': 'v1', 'addr': '172.21.15.115:6805', 'nonce': 495703077}]}, 'public_addr': '172.21.15.115:6801/495703077', 'cluster_addr': '172.21.15.115:6803/495703077', 'heartbeat_back_addr': '172.21.15.115:6807/495703077', 'heartbeat_front_addr': '172.21.15.115:6805/495703077', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 112, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6811', 'nonce': 902573489}, {'type': 'v1', 'addr': '172.21.15.115:6813', 'nonce': 902573489}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6815', 'nonce': 902573489}, {'type': 'v1', 'addr': '172.21.15.115:6817', 'nonce': 902573489}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6822', 'nonce': 902573489}, {'type': 'v1', 'addr': '172.21.15.115:6823', 'nonce': 902573489}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6819', 'nonce': 902573489}, {'type': 'v1', 'addr': '172.21.15.115:6821', 'nonce': 902573489}]}, 'public_addr': '172.21.15.115:6813/902573489', 'cluster_addr': '172.21.15.115:6817/902573489', 'heartbeat_back_addr': '172.21.15.115:6823/902573489', 'heartbeat_front_addr': '172.21.15.115:6821/902573489', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'edb686ec-e1a2-40d6-975a-e32e0e7760ff', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 112, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6824', 'nonce': 2749332272}, {'type': 'v1', 'addr': '172.21.15.115:6825', 'nonce': 2749332272}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6826', 'nonce': 2749332272}, {'type': 'v1', 'addr': '172.21.15.115:6827', 'nonce': 2749332272}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6830', 'nonce': 2749332272}, {'type': 'v1', 'addr': '172.21.15.115:6831', 'nonce': 2749332272}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6828', 'nonce': 2749332272}, {'type': 'v1', 'addr': '172.21.15.115:6829', 'nonce': 2749332272}]}, 'public_addr': '172.21.15.115:6825/2749332272', 'cluster_addr': '172.21.15.115:6827/2749332272', 'heartbeat_back_addr': '172.21.15.115:6831/2749332272', 'heartbeat_front_addr': '172.21.15.115:6829/2749332272', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '7aa14dbb-c6cb-4540-b160-3fd346f6a046', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 112, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6808', 'nonce': 1043826337}, {'type': 'v1', 'addr': '172.21.15.115:6809', 'nonce': 1043826337}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6810', 'nonce': 1043826337}, {'type': 'v1', 'addr': '172.21.15.115:6812', 'nonce': 1043826337}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6818', 'nonce': 1043826337}, {'type': 'v1', 'addr': '172.21.15.115:6820', 'nonce': 1043826337}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.115:6814', 'nonce': 1043826337}, {'type': 'v1', 'addr': '172.21.15.115:6816', 'nonce': 1043826337}]}, 'public_addr': '172.21.15.115:6809/1043826337', 'cluster_addr': '172.21.15.115:6812/1043826337', 'heartbeat_back_addr': '172.21.15.115:6820/1043826337', 'heartbeat_front_addr': '172.21.15.115:6816/1043826337', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.113232+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.174679+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.131118+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.207668+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.124542+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.200832+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.169560+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-11T23:01:14.182465+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 51, 'length': 3}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 51, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-11T23:03:20.635 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T23:03:20.646 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:20.647 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:20.656 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:03:20.656 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:20.656+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:20.754 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:20.754+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:20.764 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:20.764 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:20.782 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:20.783 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:20.800 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:20.800 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:20.818 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:03:20.818 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:03:20.831 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:20.831 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:20.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:20.831 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:20.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.831 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:77128 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_get in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-11T23:03:20.831 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:20.832 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.813991506999997, 2024-05-11T23:03:20.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91737855899999998, 2024-05-11T23:03:20.832 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:20.832 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:20.832 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:20.832 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:20.832 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:20.832 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:20.832 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 77128 2024-05-11T23:03:20.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.832 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:20.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:20.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:20.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:20.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:20.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:20.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016003+0000", 2024-05-11T23:03:20.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5379999999999999e-06 2024-05-11T23:03:20.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:20.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016011+0000", 2024-05-11T23:03:20.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6410000000000002e-06 2024-05-11T23:03:20.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:20.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016026+0000", 2024-05-11T23:03:20.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4779e-05 2024-05-11T23:03:20.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:20.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016032+0000", 2024-05-11T23:03:20.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.066e-06 2024-05-11T23:03:20.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016093+0000", 2024-05-11T23:03:20.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0878000000000002e-05 2024-05-11T23:03:20.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016100+0000", 2024-05-11T23:03:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6799999999999993e-06 2024-05-11T23:03:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933302+0000", 2024-05-11T23:03:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91720135300000005 2024-05-11T23:03:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933380+0000", 2024-05-11T23:03:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8623999999999998e-05 2024-05-11T23:03:20.836 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.837 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.837 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.837 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98950 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:20.837 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:20.837 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 54.697910618000002, 2024-05-11T23:03:20.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021419438999999998, 2024-05-11T23:03:20.837 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:20.837 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:20.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:20.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:20.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:20.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:20.838 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98950 2024-05-11T23:03:20.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.838 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:20.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:20.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:20.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:20.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:20.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:20.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:20.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.34e-07 2024-05-11T23:03:20.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:20.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132085+0000", 2024-05-11T23:03:20.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0049999999999999e-06 2024-05-11T23:03:20.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:20.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132093+0000", 2024-05-11T23:03:20.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6660000000000008e-06 2024-05-11T23:03:20.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:20.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132097+0000", 2024-05-11T23:03:20.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2590000000000002e-06 2024-05-11T23:03:20.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135114+0000", 2024-05-11T23:03:20.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003017573 2024-05-11T23:03:20.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:20.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153416+0000", 2024-05-11T23:03:20.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018301884000000001 2024-05-11T23:03:20.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:20.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153502+0000", 2024-05-11T23:03:20.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5717999999999995e-05 2024-05-11T23:03:20.841 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.841 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.841 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.841 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98959 3.1e 3:7807da99:::rbd_data.111ed2934718.000000000000006d:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:20.841 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:20.841 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 54.697417881, 2024-05-11T23:03:20.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020525928999999998, 2024-05-11T23:03:20.841 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:20.841 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:20.841 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:20.841 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:20.842 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:20.842 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:20.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98959 2024-05-11T23:03:20.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.842 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:20.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:20.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:20.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:20.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:20.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:20.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132576+0000", 2024-05-11T23:03:20.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4199999999999995e-07 2024-05-11T23:03:20.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:20.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132580+0000", 2024-05-11T23:03:20.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2910000000000001e-06 2024-05-11T23:03:20.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:20.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132588+0000", 2024-05-11T23:03:20.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7970000000000001e-06 2024-05-11T23:03:20.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:20.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132602+0000", 2024-05-11T23:03:20.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3743999999999999e-05 2024-05-11T23:03:20.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132655+0000", 2024-05-11T23:03:20.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2714999999999999e-05 2024-05-11T23:03:20.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:20.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132787+0000", 2024-05-11T23:03:20.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013273899999999999 2024-05-11T23:03:20.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:20.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153101+0000", 2024-05-11T23:03:20.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020313900999999999 2024-05-11T23:03:20.845 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.845 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.845 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.845 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98965 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:20.845 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:20.845 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 54.694894245, 2024-05-11T23:03:20.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018608983999999999, 2024-05-11T23:03:20.845 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:20.845 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:20.846 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:20.846 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:20.846 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:20.846 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:20.846 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98965 2024-05-11T23:03:20.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.846 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:20.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:20.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:20.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:20.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:20.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:20.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135100+0000", 2024-05-11T23:03:20.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.17e-06 2024-05-11T23:03:20.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:20.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135105+0000", 2024-05-11T23:03:20.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.899e-06 2024-05-11T23:03:20.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:20.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135122+0000", 2024-05-11T23:03:20.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7346999999999998e-05 2024-05-11T23:03:20.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:20.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135130+0000", 2024-05-11T23:03:20.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0639999999999994e-06 2024-05-11T23:03:20.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153533+0000", 2024-05-11T23:03:20.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018402753000000001 2024-05-11T23:03:20.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:20.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153615+0000", 2024-05-11T23:03:20.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1720999999999999e-05 2024-05-11T23:03:20.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:20.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153708+0000", 2024-05-11T23:03:20.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3029999999999995e-05 2024-05-11T23:03:20.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.850 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.850 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98966 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3760128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:20.850 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:20.850 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 54.694822119000001, 2024-05-11T23:03:20.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018716535999999999, 2024-05-11T23:03:20.850 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:20.850 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:20.850 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:20.851 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:20.851 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:20.851 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:20.851 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98966 2024-05-11T23:03:20.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.851 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:20.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:20.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:20.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:20.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:20.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:20.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135172+0000", 2024-05-11T23:03:20.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-05-11T23:03:20.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:20.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135175+0000", 2024-05-11T23:03:20.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-05-11T23:03:20.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:20.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135184+0000", 2024-05-11T23:03:20.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1840000000000005e-06 2024-05-11T23:03:20.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:20.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152903+0000", 2024-05-11T23:03:20.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017718357000000001 2024-05-11T23:03:20.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153736+0000", 2024-05-11T23:03:20.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00083324799999999997 2024-05-11T23:03:20.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:20.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153809+0000", 2024-05-11T23:03:20.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2995000000000002e-05 2024-05-11T23:03:20.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:20.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153888+0000", 2024-05-11T23:03:20.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8695999999999993e-05 2024-05-11T23:03:20.855 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.855 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.855 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.855 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "pg_update_log_missing_reply(3.1e epoch 72/53 rep_tid 104 lcod 72'363)", 2024-05-11T23:03:20.855 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:20.855 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 54.694759222999998, 2024-05-11T23:03:20.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017871951000000001, 2024-05-11T23:03:20.855 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:20.855 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:20.855 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:20.856 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:20.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:20.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:20.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:20.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:20.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:20.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135235+0000", 2024-05-11T23:03:20.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8899999999999999e-07 2024-05-11T23:03:20.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:20.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135238+0000", 2024-05-11T23:03:20.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.27e-06 2024-05-11T23:03:20.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:20.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152878+0000", 2024-05-11T23:03:20.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01763986 2024-05-11T23:03:20.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:20.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152896+0000", 2024-05-11T23:03:20.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8573e-05 2024-05-11T23:03:20.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152968+0000", 2024-05-11T23:03:20.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1780999999999996e-05 2024-05-11T23:03:20.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:20.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153106+0000", 2024-05-11T23:03:20.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013777799999999999 2024-05-11T23:03:20.859 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.859 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.859 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.859 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:239544 3.1a 3:5fe59cad:::rbd_data.111ebabb4604.000000000000016e:head [stat,delete] snapc 2f=[2f] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:20.859 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:20.859 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 17.175402491, 2024-05-11T23:03:20.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.50514007500000002, 2024-05-11T23:03:20.859 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:20.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:20.860 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:20.860 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:20.860 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:20.860 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:20.860 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 239544 2024-05-11T23:03:20.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.860 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:20.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:20.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:20.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:20.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:20.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:20.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654592+0000", 2024-05-11T23:03:20.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.404e-06 2024-05-11T23:03:20.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:20.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654598+0000", 2024-05-11T23:03:20.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7660000000000004e-06 2024-05-11T23:03:20.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:20.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654611+0000", 2024-05-11T23:03:20.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2676e-05 2024-05-11T23:03:20.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:20.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654620+0000", 2024-05-11T23:03:20.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2990000000000002e-06 2024-05-11T23:03:20.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654698+0000", 2024-05-11T23:03:20.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7739999999999998e-05 2024-05-11T23:03:20.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:20.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654710+0000", 2024-05-11T23:03:20.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2648e-05 2024-05-11T23:03:20.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.157889+0000", 2024-05-11T23:03:20.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.50317850799999997 2024-05-11T23:03:20.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:20.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.158050+0000", 2024-05-11T23:03:20.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016084500000000001 2024-05-11T23:03:20.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:20.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.159731+0000", 2024-05-11T23:03:20.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001681189 2024-05-11T23:03:20.865 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.865 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.865 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.865 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262364 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2613248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:20.865 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:20.866 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.2435296260000008, 2024-05-11T23:03:20.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016312508, 2024-05-11T23:03:20.866 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:20.866 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:20.866 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:20.866 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:20.866 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:20.866 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:20.866 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262364 2024-05-11T23:03:20.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.866 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:20.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:20.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:20.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:20.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:20.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:20.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:20.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.13e-07 2024-05-11T23:03:20.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:20.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586467+0000", 2024-05-11T23:03:20.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-05-11T23:03:20.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:20.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586474+0000", 2024-05-11T23:03:20.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7739999999999999e-06 2024-05-11T23:03:20.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:20.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586483+0000", 2024-05-11T23:03:20.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0340000000000002e-06 2024-05-11T23:03:20.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602629+0000", 2024-05-11T23:03:20.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016145845999999998 2024-05-11T23:03:20.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:20.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602703+0000", 2024-05-11T23:03:20.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4671000000000004e-05 2024-05-11T23:03:20.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:20.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602776+0000", 2024-05-11T23:03:20.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2793000000000006e-05 2024-05-11T23:03:20.870 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.870 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.870 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.870 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262417 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:20.870 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:20.870 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.2417833270000003, 2024-05-11T23:03:20.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016403456, 2024-05-11T23:03:20.870 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:20.870 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:20.870 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:20.870 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:20.871 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:20.871 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:20.871 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262417 2024-05-11T23:03:20.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.871 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:20.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:20.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:20.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:20.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:20.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:20.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:20.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6199999999999999e-07 2024-05-11T23:03:20.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:20.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588212+0000", 2024-05-11T23:03:20.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.075e-06 2024-05-11T23:03:20.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:20.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588218+0000", 2024-05-11T23:03:20.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8150000000000002e-06 2024-05-11T23:03:20.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:20.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588223+0000", 2024-05-11T23:03:20.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7090000000000001e-06 2024-05-11T23:03:20.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604461+0000", 2024-05-11T23:03:20.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016237900999999999 2024-05-11T23:03:20.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:20.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604540+0000", 2024-05-11T23:03:20.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9376999999999996e-05 2024-05-11T23:03:20.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:20.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604613+0000", 2024-05-11T23:03:20.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3217000000000001e-05 2024-05-11T23:03:20.874 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.874 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.874 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.874 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262418 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2396160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:20.874 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:20.874 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.2417634179999997, 2024-05-11T23:03:20.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016591180000000001, 2024-05-11T23:03:20.875 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:20.875 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:20.875 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:20.875 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:20.875 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:20.875 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:20.875 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262418 2024-05-11T23:03:20.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.875 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:20.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:20.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:20.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:20.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:20.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:20.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:20.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6100000000000001e-07 2024-05-11T23:03:20.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:20.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588232+0000", 2024-05-11T23:03:20.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0370000000000001e-06 2024-05-11T23:03:20.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:20.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588238+0000", 2024-05-11T23:03:20.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9340000000000003e-06 2024-05-11T23:03:20.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:20.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588243+0000", 2024-05-11T23:03:20.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6179999999999997e-06 2024-05-11T23:03:20.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604642+0000", 2024-05-11T23:03:20.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0163991 2024-05-11T23:03:20.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:20.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604712+0000", 2024-05-11T23:03:20.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9935e-05 2024-05-11T23:03:20.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:20.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604821+0000", 2024-05-11T23:03:20.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000109095 2024-05-11T23:03:20.878 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.878 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.878 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.878 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262419 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2433024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:20.878 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:20.878 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.2417441, 2024-05-11T23:03:20.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016799692000000001, 2024-05-11T23:03:20.879 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:20.879 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:20.879 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:20.879 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:20.879 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:20.879 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:20.879 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262419 2024-05-11T23:03:20.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.879 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:20.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:20.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:20.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:20.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:20.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:20.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:20.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:20.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:20.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588252+0000", 2024-05-11T23:03:20.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0990000000000001e-06 2024-05-11T23:03:20.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:20.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588257+0000", 2024-05-11T23:03:20.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8159999999999999e-06 2024-05-11T23:03:20.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:20.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588262+0000", 2024-05-11T23:03:20.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8069999999999999e-06 2024-05-11T23:03:20.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604868+0000", 2024-05-11T23:03:20.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016605880999999999 2024-05-11T23:03:20.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:20.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604966+0000", 2024-05-11T23:03:20.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7597999999999996e-05 2024-05-11T23:03:20.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:20.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605049+0000", 2024-05-11T23:03:20.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3141999999999998e-05 2024-05-11T23:03:20.883 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.883 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.883 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.883 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262420 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2469888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:20.883 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:20.883 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.2417239890000005, 2024-05-11T23:03:20.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016972514000000001, 2024-05-11T23:03:20.883 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:20.883 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:20.883 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:20.884 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:20.884 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:20.884 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:20.884 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262420 2024-05-11T23:03:20.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.884 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:20.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:20.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:20.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:20.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:20.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:20.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588271+0000", 2024-05-11T23:03:20.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6199999999999999e-06 2024-05-11T23:03:20.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:20.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588273+0000", 2024-05-11T23:03:20.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3209999999999999e-06 2024-05-11T23:03:20.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:20.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588279+0000", 2024-05-11T23:03:20.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2519999999999996e-06 2024-05-11T23:03:20.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:20.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588285+0000", 2024-05-11T23:03:20.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1440000000000002e-06 2024-05-11T23:03:20.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605083+0000", 2024-05-11T23:03:20.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016798418999999998 2024-05-11T23:03:20.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:20.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605168+0000", 2024-05-11T23:03:20.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4851000000000005e-05 2024-05-11T23:03:20.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:20.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605242+0000", 2024-05-11T23:03:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3906999999999996e-05 2024-05-11T23:03:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262421 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2506752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.2417018429999995, 2024-05-11T23:03:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017141427000000001, 2024-05-11T23:03:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:20.889 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:20.889 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:20.889 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:20.889 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:20.889 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262421 2024-05-11T23:03:20.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.889 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:20.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:20.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:20.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588292+0000", 2024-05-11T23:03:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588318+0000", 2024-05-11T23:03:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6376e-05 2024-05-11T23:03:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588326+0000", 2024-05-11T23:03:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6699999999999994e-06 2024-05-11T23:03:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:20.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588332+0000", 2024-05-11T23:03:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8540000000000002e-06 2024-05-11T23:03:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605274+0000", 2024-05-11T23:03:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016942024999999999 2024-05-11T23:03:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605360+0000", 2024-05-11T23:03:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6304999999999994e-05 2024-05-11T23:03:20.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605433+0000", 2024-05-11T23:03:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2772000000000001e-05 2024-05-11T23:03:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262422 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2543616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.2416526189999999, 2024-05-11T23:03:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017283705, 2024-05-11T23:03:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262422 2024-05-11T23:03:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:20.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:03:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588343+0000", 2024-05-11T23:03:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.34e-06 2024-05-11T23:03:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588350+0000", 2024-05-11T23:03:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1569999999999997e-06 2024-05-11T23:03:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588354+0000", 2024-05-11T23:03:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.865e-06 2024-05-11T23:03:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605466+0000", 2024-05-11T23:03:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017111498999999999 2024-05-11T23:03:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:20.897 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605551+0000", 2024-05-11T23:03:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5210999999999994e-05 2024-05-11T23:03:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605624+0000", 2024-05-11T23:03:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3153999999999998e-05 2024-05-11T23:03:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262423 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2580480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:20.898 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.2416323699999996, 2024-05-11T23:03:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017451194, 2024-05-11T23:03:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262423 2024-05-11T23:03:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:20.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588363+0000", 2024-05-11T23:03:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.855e-06 2024-05-11T23:03:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588369+0000", 2024-05-11T23:03:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1260000000000003e-06 2024-05-11T23:03:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588375+0000", 2024-05-11T23:03:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9630000000000003e-06 2024-05-11T23:03:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605656+0000", 2024-05-11T23:03:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017280865999999999 2024-05-11T23:03:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605737+0000", 2024-05-11T23:03:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0773999999999996e-05 2024-05-11T23:03:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605812+0000", 2024-05-11T23:03:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5203999999999994e-05 2024-05-11T23:03:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.902 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262424 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2617344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.2416108520000009, 2024-05-11T23:03:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017617544999999998, 2024-05-11T23:03:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262424 2024-05-11T23:03:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.903 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588383+0000", 2024-05-11T23:03:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8299999999999997e-07 2024-05-11T23:03:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588385+0000", 2024-05-11T23:03:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2869999999999999e-06 2024-05-11T23:03:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588391+0000", 2024-05-11T23:03:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1689999999999997e-06 2024-05-11T23:03:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588396+0000", 2024-05-11T23:03:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5719999999999996e-06 2024-05-11T23:03:20.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605837+0000", 2024-05-11T23:03:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017440812999999999 2024-05-11T23:03:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605917+0000", 2024-05-11T23:03:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9924000000000003e-05 2024-05-11T23:03:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.606000+0000", 2024-05-11T23:03:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3296999999999999e-05 2024-05-11T23:03:20.906 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262896 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.1706062149999994, 2024-05-11T23:03:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016295738000000001, 2024-05-11T23:03:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:20.907 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262896 2024-05-11T23:03:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659390+0000", 2024-05-11T23:03:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1440000000000001e-06 2024-05-11T23:03:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659397+0000", 2024-05-11T23:03:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0010000000000004e-06 2024-05-11T23:03:20.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659402+0000", 2024-05-11T23:03:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2179999999999998e-06 2024-05-11T23:03:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675487+0000", 2024-05-11T23:03:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016085706000000002 2024-05-11T23:03:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675580+0000", 2024-05-11T23:03:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2884e-05 2024-05-11T23:03:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675683+0000", 2024-05-11T23:03:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010236 2024-05-11T23:03:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262897 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.1705708739999992, 2024-05-11T23:03:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016500932999999999, 2024-05-11T23:03:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262897 2024-05-11T23:03:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:20.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:20.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659423+0000", 2024-05-11T23:03:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9699999999999996e-07 2024-05-11T23:03:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659425+0000", 2024-05-11T23:03:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3199999999999998e-06 2024-05-11T23:03:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659433+0000", 2024-05-11T23:03:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3660000000000003e-06 2024-05-11T23:03:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659445+0000", 2024-05-11T23:03:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2102e-05 2024-05-11T23:03:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675727+0000", 2024-05-11T23:03:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016282331000000001 2024-05-11T23:03:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675815+0000", 2024-05-11T23:03:20.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7484999999999993e-05 2024-05-11T23:03:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675923+0000", 2024-05-11T23:03:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000108732 2024-05-11T23:03:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262898 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:20.916 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.1705397560000002, 2024-05-11T23:03:20.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016685855999999999, 2024-05-11T23:03:20.917 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:20.917 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:20.917 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:20.917 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:20.917 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:20.917 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:20.917 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262898 2024-05-11T23:03:20.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.917 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:20.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:20.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:20.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:20.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:20.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:20.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659454+0000", 2024-05-11T23:03:20.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2200000000000004e-07 2024-05-11T23:03:20.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:20.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659457+0000", 2024-05-11T23:03:20.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.779e-06 2024-05-11T23:03:20.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:20.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659463+0000", 2024-05-11T23:03:20.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6660000000000002e-06 2024-05-11T23:03:20.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:20.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659472+0000", 2024-05-11T23:03:20.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2139999999999996e-06 2024-05-11T23:03:20.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675956+0000", 2024-05-11T23:03:20.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016484431000000001 2024-05-11T23:03:20.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:20.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676039+0000", 2024-05-11T23:03:20.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3358999999999999e-05 2024-05-11T23:03:20.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:20.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676139+0000", 2024-05-11T23:03:20.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9784999999999994e-05 2024-05-11T23:03:20.921 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.921 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.921 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.921 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262899 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:20.921 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:20.922 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 8.1705139249999998, 2024-05-11T23:03:20.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017030087999999999, 2024-05-11T23:03:20.922 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:20.922 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:20.922 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:20.922 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:20.922 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:20.922 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:20.922 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262899 2024-05-11T23:03:20.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.922 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:20.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:20.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:20.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:20.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:20.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:20.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:20.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659480+0000", 2024-05-11T23:03:20.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7700000000000004e-07 2024-05-11T23:03:20.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:20.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659482+0000", 2024-05-11T23:03:20.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1390000000000002e-06 2024-05-11T23:03:20.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:20.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659489+0000", 2024-05-11T23:03:20.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8279999999999997e-06 2024-05-11T23:03:20.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:20.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659494+0000", 2024-05-11T23:03:20.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9019999999999998e-06 2024-05-11T23:03:20.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:20.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676167+0000", 2024-05-11T23:03:20.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016673621 2024-05-11T23:03:20.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:20.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676329+0000", 2024-05-11T23:03:20.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016206099999999999 2024-05-11T23:03:20.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:20.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:20.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:20.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676509+0000", 2024-05-11T23:03:20.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017996 2024-05-11T23:03:20.925 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.925 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.925 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.925 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:20.925 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:20.925 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:20.926 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:20.927 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:20.928 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:03:20.928 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:20.928 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:20.929 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:20.929 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:20.929 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:20.930 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:20.931 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:20.927+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:20.977 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:03:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-11T23:03:20.989 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-05-11T23:03:20.222261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1573878744,"num_objects":3679,"num_object_clones":789,"num_object_copies":7358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3679,"num_whiteouts":2112,"num_read":323752,"num_read_kb":1001439,"num_write":25838,"num_write_kb":497613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4013,"num_bytes_recovered":297258457,"num_keys_recovered":695,"num_objects_omap":53,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22177,"ondisk_log_size":22177,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":439840,"kb_used_data":82764,"kb_used_omap":10,"kb_used_meta":356981,"kb_avail":660163040,"statfs":{"total":676457349120,"available":676006952960,"internally_reserved":0,"allocated":84750336,"data_stored":792081262,"data_compressed":43902087,"data_compressed_allocated":48824320,"data_compressed_original":764271267,"omap_allocated":11157,"internal_metadata":365548651},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2562015,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":8226,"num_read_kb":32113,"num_write":1458,"num_write_kb":10961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.015357"},"pg_stats":[{"pgid":"3.1f","version":"116'461","reported_seq":4665,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-11T23:03:19.320458+0000","last_change":"2024-05-11T23:03:19.223970+0000","last_active":"2024-05-11T23:03:19.320458+0000","last_peered":"2024-05-11T23:03:19.320458+0000","last_clean":"2024-05-11T23:03:19.320458+0000","last_became_active":"2024-05-11T23:01:27.459623+0000","last_became_peered":"2024-05-11T23:01:27.459623+0000","last_unstale":"2024-05-11T23:03:19.320458+0000","last_undegraded":"2024-05-11T23:03:19.320458+0000","last_fullsized":"2024-05-11T23:03:19.320458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T09:11:49.765023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043262500000000001,"stat_sum":{"num_bytes":24770326,"num_objects":93,"num_object_clones":17,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":61,"num_read":5093,"num_read_kb":14467,"num_write":449,"num_write_kb":18075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"34","length":"2"}]},{"pgid":"3.1e","version":"111'777","reported_seq":6010,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-11T23:03:15.204210+0000","last_change":"2024-05-11T23:02:43.062858+0000","last_active":"2024-05-11T23:03:15.204210+0000","last_peered":"2024-05-11T23:03:15.204210+0000","last_clean":"2024-05-11T23:03:15.204210+0000","last_became_active":"2024-05-11T23:02:43.060463+0000","last_became_peered":"2024-05-11T23:02:43.060463+0000","last_unstale":"2024-05-11T23:03:15.204210+0000","last_undegraded":"2024-05-11T23:03:15.204210+0000","last_fullsized":"2024-05-11T23:03:15.204210+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:19:22.604277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062673000000000004,"stat_sum":{"num_bytes":68301543,"num_objects":131,"num_object_clones":29,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":73,"num_read":6386,"num_read_kb":48378,"num_write":879,"num_write_kb":22610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":53290749,"num_keys_recovered":96,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"113'443","reported_seq":3624,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-11T23:03:17.216295+0000","last_change":"2024-05-11T23:02:55.124789+0000","last_active":"2024-05-11T23:03:17.216295+0000","last_peered":"2024-05-11T23:03:17.216295+0000","last_clean":"2024-05-11T23:03:17.216295+0000","last_became_active":"2024-05-11T23:02:55.123539+0000","last_became_peered":"2024-05-11T23:02:55.123539+0000","last_unstale":"2024-05-11T23:03:17.216295+0000","last_undegraded":"2024-05-11T23:03:17.216295+0000","last_fullsized":"2024-05-11T23:03:17.216295+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:45:45.708869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001000658,"stat_sum":{"num_bytes":31058284,"num_objects":91,"num_object_clones":15,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":62,"num_read":2987,"num_read_kb":16270,"num_write":376,"num_write_kb":9623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":6729031,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"114'602","reported_seq":17746,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-11T23:03:17.920617+0000","last_change":"2024-05-11T23:02:55.124001+0000","last_active":"2024-05-11T23:03:17.920617+0000","last_peered":"2024-05-11T23:03:17.920617+0000","last_clean":"2024-05-11T23:03:17.920617+0000","last_became_active":"2024-05-11T23:02:55.123016+0000","last_became_peered":"2024-05-11T23:02:55.123016+0000","last_unstale":"2024-05-11T23:03:17.920617+0000","last_undegraded":"2024-05-11T23:03:17.920617+0000","last_fullsized":"2024-05-11T23:03:17.920617+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:35:38.186812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091161300000000003,"stat_sum":{"num_bytes":53139115,"num_objects":138,"num_object_clones":26,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":86,"num_read":17657,"num_read_kb":39071,"num_write":503,"num_write_kb":10494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":5203606,"num_keys_recovered":112,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"113'515","reported_seq":14627,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-11T23:03:17.432728+0000","last_change":"2024-05-11T23:02:56.128308+0000","last_active":"2024-05-11T23:03:17.432728+0000","last_peered":"2024-05-11T23:03:17.432728+0000","last_clean":"2024-05-11T23:03:17.432728+0000","last_became_active":"2024-05-11T23:02:56.127543+0000","last_became_peered":"2024-05-11T23:02:56.127543+0000","last_unstale":"2024-05-11T23:03:17.432728+0000","last_undegraded":"2024-05-11T23:03:17.432728+0000","last_fullsized":"2024-05-11T23:03:17.432728+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:02:31.388990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065062200000000001,"stat_sum":{"num_bytes":100212041,"num_objects":111,"num_object_clones":22,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":65,"num_read":13937,"num_read_kb":68580,"num_write":561,"num_write_kb":35788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":158,"num_bytes_recovered":13394965,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1a","version":"111'1550","reported_seq":19965,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-11T23:03:15.201676+0000","last_change":"2024-05-11T23:03:10.184766+0000","last_active":"2024-05-11T23:03:15.201676+0000","last_peered":"2024-05-11T23:03:15.201676+0000","last_clean":"2024-05-11T23:03:15.201676+0000","last_became_active":"2024-05-11T23:03:10.183172+0000","last_became_peered":"2024-05-11T23:03:10.183172+0000","last_unstale":"2024-05-11T23:03:15.201676+0000","last_undegraded":"2024-05-11T23:03:15.201676+0000","last_fullsized":"2024-05-11T23:03:15.201676+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1550,"log_dups_size":0,"ondisk_log_size":1550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:33:10.921761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087678099999999998,"stat_sum":{"num_bytes":83111179,"num_objects":120,"num_object_clones":25,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":60,"num_read":24722,"num_read_kb":46226,"num_write":2654,"num_write_kb":22908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":5021141,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"111'926","reported_seq":14680,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-11T23:03:15.201081+0000","last_change":"2024-05-11T23:02:31.781713+0000","last_active":"2024-05-11T23:03:15.201081+0000","last_peered":"2024-05-11T23:03:15.201081+0000","last_clean":"2024-05-11T23:03:15.201081+0000","last_became_active":"2024-05-11T23:02:31.780769+0000","last_became_peered":"2024-05-11T23:02:31.780769+0000","last_unstale":"2024-05-11T23:03:15.201081+0000","last_undegraded":"2024-05-11T23:03:15.201081+0000","last_fullsized":"2024-05-11T23:03:15.201081+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:36:37.188293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000433275,"stat_sum":{"num_bytes":75801939,"num_objects":115,"num_object_clones":21,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":64,"num_read":15988,"num_read_kb":42914,"num_write":1291,"num_write_kb":14628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":13589947,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"112'718","reported_seq":3376,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-11T23:03:15.208196+0000","last_change":"2024-05-11T23:02:31.781325+0000","last_active":"2024-05-11T23:03:15.208196+0000","last_peered":"2024-05-11T23:03:15.208196+0000","last_clean":"2024-05-11T23:03:15.208196+0000","last_became_active":"2024-05-11T23:02:31.779496+0000","last_became_peered":"2024-05-11T23:02:31.779496+0000","last_unstale":"2024-05-11T23:03:15.208196+0000","last_undegraded":"2024-05-11T23:03:15.208196+0000","last_fullsized":"2024-05-11T23:03:15.208196+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":718,"log_dups_size":0,"ondisk_log_size":718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:50:41.650832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079167699999999996,"stat_sum":{"num_bytes":27817497,"num_objects":105,"num_object_clones":22,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":61,"num_read":4585,"num_read_kb":8219,"num_write":825,"num_write_kb":16243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":4822628,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"114'597","reported_seq":3395,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-11T23:03:17.957932+0000","last_change":"2024-05-11T23:03:16.211336+0000","last_active":"2024-05-11T23:03:17.957932+0000","last_peered":"2024-05-11T23:03:17.957932+0000","last_clean":"2024-05-11T23:03:17.957932+0000","last_became_active":"2024-05-11T23:03:16.210318+0000","last_became_peered":"2024-05-11T23:03:16.210318+0000","last_unstale":"2024-05-11T23:03:17.957932+0000","last_undegraded":"2024-05-11T23:03:17.957932+0000","last_fullsized":"2024-05-11T23:03:17.957932+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:28:53.251773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001045912,"stat_sum":{"num_bytes":18165946,"num_objects":88,"num_object_clones":13,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":53,"num_read":3903,"num_read_kb":15195,"num_write":761,"num_write_kb":6413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"114'488","reported_seq":15198,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-11T23:03:17.968511+0000","last_change":"2024-05-11T23:01:29.474960+0000","last_active":"2024-05-11T23:03:17.968511+0000","last_peered":"2024-05-11T23:03:17.968511+0000","last_clean":"2024-05-11T23:03:17.968511+0000","last_became_active":"2024-05-11T23:01:27.463623+0000","last_became_peered":"2024-05-11T23:01:27.463623+0000","last_unstale":"2024-05-11T23:03:17.968511+0000","last_undegraded":"2024-05-11T23:03:17.968511+0000","last_fullsized":"2024-05-11T23:03:17.968511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:59:13.617932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001047748,"stat_sum":{"num_bytes":59219777,"num_objects":104,"num_object_clones":20,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":64,"num_read":14978,"num_read_kb":54756,"num_write":449,"num_write_kb":18598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"113'516","reported_seq":15779,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-11T23:03:17.222399+0000","last_change":"2024-05-11T23:02:30.784538+0000","last_active":"2024-05-11T23:03:17.222399+0000","last_peered":"2024-05-11T23:03:17.222399+0000","last_clean":"2024-05-11T23:03:17.222399+0000","last_became_active":"2024-05-11T23:02:30.783475+0000","last_became_peered":"2024-05-11T23:02:30.783475+0000","last_unstale":"2024-05-11T23:03:17.222399+0000","last_undegraded":"2024-05-11T23:03:17.222399+0000","last_fullsized":"2024-05-11T23:03:17.222399+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T09:09:43.696908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039810900000000001,"stat_sum":{"num_bytes":64859819,"num_objects":134,"num_object_clones":33,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":82,"num_read":14050,"num_read_kb":25117,"num_write":511,"num_write_kb":13148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":12363304,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"113'549","reported_seq":18928,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-11T23:03:17.584869+0000","last_change":"2024-05-11T23:02:55.125262+0000","last_active":"2024-05-11T23:03:17.584869+0000","last_peered":"2024-05-11T23:03:17.584869+0000","last_clean":"2024-05-11T23:03:17.584869+0000","last_became_active":"2024-05-11T23:02:55.124111+0000","last_became_peered":"2024-05-11T23:02:55.124111+0000","last_unstale":"2024-05-11T23:03:17.584869+0000","last_undegraded":"2024-05-11T23:03:17.584869+0000","last_fullsized":"2024-05-11T23:03:17.584869+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:26:15.345268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066166100000000002,"stat_sum":{"num_bytes":35635685,"num_objects":106,"num_object_clones":24,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":60,"num_read":18969,"num_read_kb":41754,"num_write":515,"num_write_kb":6412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":2593983,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":68,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-11T23:03:16.209931+0000","last_change":"2024-05-11T23:02:39.044686+0000","last_active":"2024-05-11T23:03:16.209931+0000","last_peered":"2024-05-11T23:03:16.209931+0000","last_clean":"2024-05-11T23:03:16.209931+0000","last_became_active":"2024-05-11T23:02:39.044216+0000","last_became_peered":"2024-05-11T23:02:39.044216+0000","last_unstale":"2024-05-11T23:03:16.209931+0000","last_undegraded":"2024-05-11T23:03:16.209931+0000","last_fullsized":"2024-05-11T23:03:16.209931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:31:03.826529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"116'842","reported_seq":6144,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-11T23:03:19.320050+0000","last_change":"2024-05-11T23:03:19.223722+0000","last_active":"2024-05-11T23:03:19.320050+0000","last_peered":"2024-05-11T23:03:19.320050+0000","last_clean":"2024-05-11T23:03:19.320050+0000","last_became_active":"2024-05-11T23:01:27.458177+0000","last_became_peered":"2024-05-11T23:01:27.458177+0000","last_unstale":"2024-05-11T23:03:19.320050+0000","last_undegraded":"2024-05-11T23:03:19.320050+0000","last_fullsized":"2024-05-11T23:03:19.320050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":842,"log_dups_size":0,"ondisk_log_size":842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:01:47.780301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049631100000000004,"stat_sum":{"num_bytes":29045242,"num_objects":109,"num_object_clones":15,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":73,"num_read":8823,"num_read_kb":22349,"num_write":1045,"num_write_kb":12586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"34","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":210,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-11T23:03:15.201554+0000","last_change":"2024-05-11T23:02:39.047593+0000","last_active":"2024-05-11T23:03:15.201554+0000","last_peered":"2024-05-11T23:03:15.201554+0000","last_clean":"2024-05-11T23:03:15.201554+0000","last_became_active":"2024-05-11T23:02:39.047225+0000","last_became_peered":"2024-05-11T23:02:39.047225+0000","last_unstale":"2024-05-11T23:03:15.201554+0000","last_undegraded":"2024-05-11T23:03:15.201554+0000","last_fullsized":"2024-05-11T23:03:15.201554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:34:13.738178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"111'807","reported_seq":5230,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-11T23:03:15.204245+0000","last_change":"2024-05-11T23:02:31.782882+0000","last_active":"2024-05-11T23:03:15.204245+0000","last_peered":"2024-05-11T23:03:15.204245+0000","last_clean":"2024-05-11T23:03:15.204245+0000","last_became_active":"2024-05-11T23:02:31.781966+0000","last_became_peered":"2024-05-11T23:02:31.781966+0000","last_unstale":"2024-05-11T23:03:15.204245+0000","last_undegraded":"2024-05-11T23:03:15.204245+0000","last_fullsized":"2024-05-11T23:03:15.204245+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:27:14.161195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058786099999999996,"stat_sum":{"num_bytes":32625242,"num_objects":106,"num_object_clones":19,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":62,"num_read":6784,"num_read_kb":17599,"num_write":1007,"num_write_kb":7143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":114,"num_bytes_recovered":4415701,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":204,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-11T23:03:17.222316+0000","last_change":"2024-05-11T23:01:19.216700+0000","last_active":"2024-05-11T23:03:17.222316+0000","last_peered":"2024-05-11T23:03:17.222316+0000","last_clean":"2024-05-11T23:03:17.222316+0000","last_became_active":"2024-05-11T23:01:19.216413+0000","last_became_peered":"2024-05-11T23:01:19.216413+0000","last_unstale":"2024-05-11T23:03:17.222316+0000","last_undegraded":"2024-05-11T23:03:17.222316+0000","last_fullsized":"2024-05-11T23:03:17.222316+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:07:38.639409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"111'448","reported_seq":14364,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-11T23:03:15.200904+0000","last_change":"2024-05-11T23:01:29.468176+0000","last_active":"2024-05-11T23:03:15.200904+0000","last_peered":"2024-05-11T23:03:15.200904+0000","last_clean":"2024-05-11T23:03:15.200904+0000","last_became_active":"2024-05-11T23:01:27.460866+0000","last_became_peered":"2024-05-11T23:01:27.460866+0000","last_unstale":"2024-05-11T23:03:15.200904+0000","last_undegraded":"2024-05-11T23:03:15.200904+0000","last_fullsized":"2024-05-11T23:03:15.200904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:54:57.619901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050028300000000002,"stat_sum":{"num_bytes":74718994,"num_objects":112,"num_object_clones":28,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":62,"num_read":14073,"num_read_kb":53612,"num_write":468,"num_write_kb":33610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":200,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-11T23:03:15.201026+0000","last_change":"2024-05-11T23:01:19.220232+0000","last_active":"2024-05-11T23:03:15.201026+0000","last_peered":"2024-05-11T23:03:15.201026+0000","last_clean":"2024-05-11T23:03:15.201026+0000","last_became_active":"2024-05-11T23:01:19.219917+0000","last_became_peered":"2024-05-11T23:01:19.219917+0000","last_unstale":"2024-05-11T23:03:15.201026+0000","last_undegraded":"2024-05-11T23:03:15.201026+0000","last_fullsized":"2024-05-11T23:03:15.201026+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:44:02.868252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"114'591","reported_seq":5454,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-11T23:03:17.957651+0000","last_change":"2024-05-11T23:02:30.785107+0000","last_active":"2024-05-11T23:03:17.957651+0000","last_peered":"2024-05-11T23:03:17.957651+0000","last_clean":"2024-05-11T23:03:17.957651+0000","last_became_active":"2024-05-11T23:02:30.784580+0000","last_became_peered":"2024-05-11T23:02:30.784580+0000","last_unstale":"2024-05-11T23:03:17.957651+0000","last_undegraded":"2024-05-11T23:03:17.957651+0000","last_fullsized":"2024-05-11T23:03:17.957651+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:55:04.421488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012391380000000001,"stat_sum":{"num_bytes":50696121,"num_objects":119,"num_object_clones":30,"num_object_copies":238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":65,"num_read":6103,"num_read_kb":25167,"num_write":560,"num_write_kb":13099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":125,"num_bytes_recovered":13712581,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"113'655","reported_seq":9162,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-11T23:03:16.898025+0000","last_change":"2024-05-11T23:03:16.213177+0000","last_active":"2024-05-11T23:03:16.898025+0000","last_peered":"2024-05-11T23:03:16.898025+0000","last_clean":"2024-05-11T23:03:16.898025+0000","last_became_active":"2024-05-11T23:03:16.211268+0000","last_became_peered":"2024-05-11T23:03:16.211268+0000","last_unstale":"2024-05-11T23:03:16.898025+0000","last_undegraded":"2024-05-11T23:03:16.898025+0000","last_fullsized":"2024-05-11T23:03:16.898025+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:50:02.724717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038468999999999998,"stat_sum":{"num_bytes":72690090,"num_objects":139,"num_object_clones":30,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":83,"num_read":9817,"num_read_kb":33059,"num_write":675,"num_write_kb":17078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":11848011,"num_keys_recovered":36,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":204,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-11T23:03:17.216137+0000","last_change":"2024-05-11T23:01:19.221118+0000","last_active":"2024-05-11T23:03:17.216137+0000","last_peered":"2024-05-11T23:03:17.216137+0000","last_clean":"2024-05-11T23:03:17.216137+0000","last_became_active":"2024-05-11T23:01:19.220855+0000","last_became_peered":"2024-05-11T23:01:19.220855+0000","last_unstale":"2024-05-11T23:03:17.216137+0000","last_undegraded":"2024-05-11T23:03:17.216137+0000","last_fullsized":"2024-05-11T23:03:17.216137+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:01:56.076963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"111'964","reported_seq":5381,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-11T23:03:15.201037+0000","last_change":"2024-05-11T23:01:29.467921+0000","last_active":"2024-05-11T23:03:15.201037+0000","last_peered":"2024-05-11T23:03:15.201037+0000","last_clean":"2024-05-11T23:03:15.201037+0000","last_became_active":"2024-05-11T23:01:27.459593+0000","last_became_peered":"2024-05-11T23:01:27.459593+0000","last_unstale":"2024-05-11T23:03:15.201037+0000","last_undegraded":"2024-05-11T23:03:15.201037+0000","last_fullsized":"2024-05-11T23:03:15.201037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":964,"log_dups_size":0,"ondisk_log_size":964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:01:24.615157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047314299999999998,"stat_sum":{"num_bytes":20626654,"num_objects":106,"num_object_clones":22,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":63,"num_read":9027,"num_read_kb":17622,"num_write":1333,"num_write_kb":3988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":218,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-11T23:03:19.222450+0000","last_change":"2024-05-11T23:02:27.995538+0000","last_active":"2024-05-11T23:03:19.222450+0000","last_peered":"2024-05-11T23:03:19.222450+0000","last_clean":"2024-05-11T23:03:19.222450+0000","last_became_active":"2024-05-11T23:02:27.995242+0000","last_became_peered":"2024-05-11T23:02:27.995242+0000","last_unstale":"2024-05-11T23:03:19.222450+0000","last_undegraded":"2024-05-11T23:03:19.222450+0000","last_fullsized":"2024-05-11T23:03:19.222450+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T09:00:41.429605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"111'540","reported_seq":14668,"reported_epoch":112,"state":"peering","last_fresh":"2024-05-11T23:03:15.204024+0000","last_change":"2024-05-11T23:03:15.204024+0000","last_active":"2024-05-11T23:03:15.204024+0000","last_peered":"2024-05-11T23:03:14.993301+0000","last_clean":"2024-05-11T23:03:14.993301+0000","last_became_active":"2024-05-11T23:02:55.123484+0000","last_became_peered":"2024-05-11T23:02:55.123484+0000","last_unstale":"2024-05-11T23:03:15.204024+0000","last_undegraded":"2024-05-11T23:03:15.204024+0000","last_fullsized":"2024-05-11T23:03:15.204024+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062926200000000001,"stat_sum":{"num_bytes":29387810,"num_objects":107,"num_object_clones":22,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":62,"num_read":14955,"num_read_kb":23512,"num_write":551,"num_write_kb":5901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":4252353,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":204,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-11T23:03:17.222005+0000","last_change":"2024-05-11T23:01:19.219755+0000","last_active":"2024-05-11T23:03:17.222005+0000","last_peered":"2024-05-11T23:03:17.222005+0000","last_clean":"2024-05-11T23:03:17.222005+0000","last_became_active":"2024-05-11T23:01:19.219471+0000","last_became_peered":"2024-05-11T23:01:19.219471+0000","last_unstale":"2024-05-11T23:03:17.222005+0000","last_undegraded":"2024-05-11T23:03:17.222005+0000","last_fullsized":"2024-05-11T23:03:17.222005+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:39:05.284213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"111'581","reported_seq":3642,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-11T23:03:15.203998+0000","last_change":"2024-05-11T23:02:30.783741+0000","last_active":"2024-05-11T23:03:15.203998+0000","last_peered":"2024-05-11T23:03:15.203998+0000","last_clean":"2024-05-11T23:03:15.203998+0000","last_became_active":"2024-05-11T23:02:30.782707+0000","last_became_peered":"2024-05-11T23:02:30.782707+0000","last_unstale":"2024-05-11T23:03:15.203998+0000","last_undegraded":"2024-05-11T23:03:15.203998+0000","last_fullsized":"2024-05-11T23:03:15.203998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:36:54.124156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000407852,"stat_sum":{"num_bytes":44856738,"num_objects":131,"num_object_clones":30,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":74,"num_read":3302,"num_read_kb":14614,"num_write":588,"num_write_kb":10400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":10719266,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":204,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-11T23:03:17.216137+0000","last_change":"2024-05-11T23:01:19.219128+0000","last_active":"2024-05-11T23:03:17.216137+0000","last_peered":"2024-05-11T23:03:17.216137+0000","last_clean":"2024-05-11T23:03:17.216137+0000","last_became_active":"2024-05-11T23:01:19.218846+0000","last_became_peered":"2024-05-11T23:01:19.218846+0000","last_unstale":"2024-05-11T23:03:17.216137+0000","last_undegraded":"2024-05-11T23:03:17.216137+0000","last_fullsized":"2024-05-11T23:03:17.216137+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:41:09.111000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":357,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-11T23:03:17.222116+0000","last_change":"2024-05-11T23:01:18.164452+0000","last_active":"2024-05-11T23:03:17.222116+0000","last_peered":"2024-05-11T23:03:17.222116+0000","last_clean":"2024-05-11T23:03:17.222116+0000","last_became_active":"2024-05-11T23:01:18.163431+0000","last_became_peered":"2024-05-11T23:01:18.163431+0000","last_unstale":"2024-05-11T23:03:17.222116+0000","last_undegraded":"2024-05-11T23:03:17.222116+0000","last_fullsized":"2024-05-11T23:03:17.222116+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_clean_scrub_stamp":"2024-05-11T23:01:16.147564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:40:59.974639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"111'842","reported_seq":5042,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-11T23:03:15.203614+0000","last_change":"2024-05-11T23:02:31.782579+0000","last_active":"2024-05-11T23:03:15.203614+0000","last_peered":"2024-05-11T23:03:15.203614+0000","last_clean":"2024-05-11T23:03:15.203614+0000","last_became_active":"2024-05-11T23:02:31.781467+0000","last_became_peered":"2024-05-11T23:02:31.781467+0000","last_unstale":"2024-05-11T23:03:15.203614+0000","last_undegraded":"2024-05-11T23:03:15.203614+0000","last_fullsized":"2024-05-11T23:03:15.203614+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":842,"log_dups_size":0,"ondisk_log_size":842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:18:58.144841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061400699999999996,"stat_sum":{"num_bytes":50194212,"num_objects":119,"num_object_clones":26,"num_object_copies":238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":67,"num_read":6081,"num_read_kb":18247,"num_write":1071,"num_write_kb":10259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":4220183,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"116'540","reported_seq":8492,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-05-11T23:03:19.222962+0000","last_change":"2024-05-11T23:03:19.222962+0000","last_active":"2024-05-11T23:03:19.222962+0000","last_peered":"2024-05-11T23:03:19.222962+0000","last_clean":"2024-05-11T23:03:19.222962+0000","last_became_active":"2024-05-11T23:02:55.123796+0000","last_became_peered":"2024-05-11T23:02:55.123796+0000","last_unstale":"2024-05-11T23:03:19.222962+0000","last_undegraded":"2024-05-11T23:03:19.222962+0000","last_fullsized":"2024-05-11T23:03:19.222962+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T09:01:30.000752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052279599999999996,"stat_sum":{"num_bytes":83892308,"num_objects":134,"num_object_clones":32,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":73,"num_read":8575,"num_read_kb":47151,"num_write":480,"num_write_kb":29793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":20983320,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"34","length":"2"}]},{"pgid":"3.a","version":"113'604","reported_seq":5406,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-11T23:03:16.834368+0000","last_change":"2024-05-11T23:02:55.126633+0000","last_active":"2024-05-11T23:03:16.834368+0000","last_peered":"2024-05-11T23:03:16.834368+0000","last_clean":"2024-05-11T23:03:16.834368+0000","last_became_active":"2024-05-11T23:02:55.125456+0000","last_became_peered":"2024-05-11T23:02:55.125456+0000","last_unstale":"2024-05-11T23:03:16.834368+0000","last_undegraded":"2024-05-11T23:03:16.834368+0000","last_fullsized":"2024-05-11T23:03:16.834368+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:35:03.226203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030522600000000002,"stat_sum":{"num_bytes":35550895,"num_objects":118,"num_object_clones":26,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":66,"num_read":5144,"num_read_kb":21260,"num_write":568,"num_write_kb":9010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":12891554,"num_keys_recovered":125,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"111'498","reported_seq":9607,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-05-11T23:03:15.204145+0000","last_change":"2024-05-11T23:02:30.783606+0000","last_active":"2024-05-11T23:03:15.204145+0000","last_peered":"2024-05-11T23:03:15.204145+0000","last_clean":"2024-05-11T23:03:15.204145+0000","last_became_active":"2024-05-11T23:02:30.779706+0000","last_became_peered":"2024-05-11T23:02:30.779706+0000","last_unstale":"2024-05-11T23:03:15.204145+0000","last_undegraded":"2024-05-11T23:03:15.204145+0000","last_fullsized":"2024-05-11T23:03:15.204145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:34:16.979260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042056999999999999,"stat_sum":{"num_bytes":83233063,"num_objects":118,"num_object_clones":32,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":57,"num_read":9160,"num_read_kb":51661,"num_write":519,"num_write_kb":42627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":155,"num_bytes_recovered":30909049,"num_keys_recovered":11,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"114'574","reported_seq":4408,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-11T23:03:17.995274+0000","last_change":"2024-05-11T23:02:31.781981+0000","last_active":"2024-05-11T23:03:17.995274+0000","last_peered":"2024-05-11T23:03:17.995274+0000","last_clean":"2024-05-11T23:03:17.995274+0000","last_became_active":"2024-05-11T23:02:31.781419+0000","last_became_peered":"2024-05-11T23:02:31.781419+0000","last_unstale":"2024-05-11T23:03:17.995274+0000","last_undegraded":"2024-05-11T23:03:17.995274+0000","last_fullsized":"2024-05-11T23:03:17.995274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:44:28.047457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084388499999999995,"stat_sum":{"num_bytes":21078257,"num_objects":92,"num_object_clones":15,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":55,"num_read":5716,"num_read_kb":24696,"num_write":587,"num_write_kb":11696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":10615766,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"114'729","reported_seq":20733,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-11T23:03:17.729640+0000","last_change":"2024-05-11T23:01:29.864324+0000","last_active":"2024-05-11T23:03:17.729640+0000","last_peered":"2024-05-11T23:03:17.729640+0000","last_clean":"2024-05-11T23:03:17.729640+0000","last_became_active":"2024-05-11T23:01:27.465570+0000","last_became_peered":"2024-05-11T23:01:27.465570+0000","last_unstale":"2024-05-11T23:03:17.729640+0000","last_undegraded":"2024-05-11T23:03:17.729640+0000","last_fullsized":"2024-05-11T23:03:17.729640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:23:04.976475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062217299999999995,"stat_sum":{"num_bytes":48408059,"num_objects":125,"num_object_clones":32,"num_object_copies":250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":125,"num_whiteouts":64,"num_read":21710,"num_read_kb":40406,"num_write":777,"num_write_kb":12329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"113'1114","reported_seq":13122,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-11T23:03:17.585328+0000","last_change":"2024-05-11T23:02:56.142024+0000","last_active":"2024-05-11T23:03:17.585328+0000","last_peered":"2024-05-11T23:03:17.585328+0000","last_clean":"2024-05-11T23:03:17.585328+0000","last_became_active":"2024-05-11T23:02:56.127434+0000","last_became_peered":"2024-05-11T23:02:56.127434+0000","last_unstale":"2024-05-11T23:03:17.585328+0000","last_undegraded":"2024-05-11T23:03:17.585328+0000","last_fullsized":"2024-05-11T23:03:17.585328+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:16:45.651654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087175200000000005,"stat_sum":{"num_bytes":45974914,"num_objects":148,"num_object_clones":38,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":76,"num_read":16353,"num_read_kb":45013,"num_write":1452,"num_write_kb":23743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":12589433,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"114'1000","reported_seq":8469,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-11T23:03:17.995800+0000","last_change":"2024-05-11T23:02:56.126336+0000","last_active":"2024-05-11T23:03:17.995800+0000","last_peered":"2024-05-11T23:03:17.995800+0000","last_clean":"2024-05-11T23:03:17.995800+0000","last_became_active":"2024-05-11T23:02:56.125669+0000","last_became_peered":"2024-05-11T23:02:56.125669+0000","last_unstale":"2024-05-11T23:03:17.995800+0000","last_undegraded":"2024-05-11T23:03:17.995800+0000","last_fullsized":"2024-05-11T23:03:17.995800+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1000,"log_dups_size":0,"ondisk_log_size":1000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:45:39.438961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080844100000000002,"stat_sum":{"num_bytes":36954004,"num_objects":132,"num_object_clones":32,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":82,"num_read":10840,"num_read_kb":37609,"num_write":1343,"num_write_kb":15144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":11135010,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"113'540","reported_seq":5539,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-05-11T23:03:16.209946+0000","last_change":"2024-05-11T23:02:42.060650+0000","last_active":"2024-05-11T23:03:16.209946+0000","last_peered":"2024-05-11T23:03:16.209946+0000","last_clean":"2024-05-11T23:03:16.209946+0000","last_became_active":"2024-05-11T23:02:42.059228+0000","last_became_peered":"2024-05-11T23:02:42.059228+0000","last_unstale":"2024-05-11T23:03:16.209946+0000","last_undegraded":"2024-05-11T23:03:16.209946+0000","last_fullsized":"2024-05-11T23:03:16.209946+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:47:33.882668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033171200000000001,"stat_sum":{"num_bytes":47816892,"num_objects":114,"num_object_clones":29,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":59,"num_read":6473,"num_read_kb":24131,"num_write":563,"num_write_kb":13614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":14890358,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"113'681","reported_seq":5060,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-11T23:03:17.222169+0000","last_change":"2024-05-11T23:02:55.124578+0000","last_active":"2024-05-11T23:03:17.222169+0000","last_peered":"2024-05-11T23:03:17.222169+0000","last_clean":"2024-05-11T23:03:17.222169+0000","last_became_active":"2024-05-11T23:02:55.123458+0000","last_became_peered":"2024-05-11T23:02:55.123458+0000","last_unstale":"2024-05-11T23:03:17.222169+0000","last_undegraded":"2024-05-11T23:03:17.222169+0000","last_fullsized":"2024-05-11T23:03:17.222169+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:20:41.559748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044654299999999998,"stat_sum":{"num_bytes":27330968,"num_objects":101,"num_object_clones":22,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":58,"num_read":6776,"num_read_kb":21021,"num_write":744,"num_write_kb":7538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":13792898,"num_keys_recovered":67,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"113'854","reported_seq":4680,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-11T23:03:17.429981+0000","last_change":"2024-05-11T23:03:16.212663+0000","last_active":"2024-05-11T23:03:17.429981+0000","last_peered":"2024-05-11T23:03:17.429981+0000","last_clean":"2024-05-11T23:03:17.429981+0000","last_became_active":"2024-05-11T23:03:16.211633+0000","last_became_peered":"2024-05-11T23:03:16.211633+0000","last_unstale":"2024-05-11T23:03:17.429981+0000","last_undegraded":"2024-05-11T23:03:17.429981+0000","last_fullsized":"2024-05-11T23:03:17.429981+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T09:58:18.365771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010326510000000001,"stat_sum":{"num_bytes":74295136,"num_objects":112,"num_object_clones":25,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":57,"num_read":6025,"num_read_kb":23814,"num_write":1068,"num_write_kb":13291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"114'514","reported_seq":4601,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-05-11T23:03:17.886268+0000","last_change":"2024-05-11T23:02:56.126245+0000","last_active":"2024-05-11T23:03:17.886268+0000","last_peered":"2024-05-11T23:03:17.886268+0000","last_clean":"2024-05-11T23:03:17.886268+0000","last_became_active":"2024-05-11T23:02:56.125661+0000","last_became_peered":"2024-05-11T23:02:56.125661+0000","last_unstale":"2024-05-11T23:03:17.886268+0000","last_undegraded":"2024-05-11T23:03:17.886268+0000","last_fullsized":"2024-05-11T23:03:17.886268+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:49:22.764559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065432399999999996,"stat_sum":{"num_bytes":21819626,"num_objects":99,"num_object_clones":17,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":63,"num_read":4644,"num_read_kb":17849,"num_write":506,"num_write_kb":7408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":128,"num_bytes_recovered":3273620,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1573288376,"num_objects":3677,"num_object_clones":789,"num_object_copies":7354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3677,"num_whiteouts":2112,"num_read":323636,"num_read_kb":1001339,"num_write":25679,"num_write_kb":495197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4013,"num_bytes_recovered":297258457,"num_keys_recovered":695,"num_objects_omap":53,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86306816,"data_stored":907740680,"data_compressed":50330619,"data_compressed_allocated":55939072,"data_compressed_original":880899310,"omap_allocated":0,"internal_metadata":0},"log_size":22060,"ondisk_log_size":22060,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65020,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509466,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96488,"kb_used_data":16736,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94275352,"statfs":{"total":96636764160,"available":96537960448,"internally_reserved":0,"allocated":17137664,"data_stored":165393907,"data_compressed":9583256,"data_compressed_allocated":10276864,"data_compressed_original":159820490,"omap_allocated":1590,"internal_metadata":81656266},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509467,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41512,"kb_used_data":10336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330328,"statfs":{"total":96636764160,"available":96594255872,"internally_reserved":0,"allocated":10584064,"data_stored":91274762,"data_compressed":5414385,"data_compressed_allocated":6049792,"data_compressed_original":87950302,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":0,"seq":64424509467,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509466,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48300,"kb_used_data":8804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323540,"statfs":{"total":96636764160,"available":96587304960,"internally_reserved":0,"allocated":9015296,"data_stored":80436266,"data_compressed":4399017,"data_compressed_allocated":4939776,"data_compressed_original":77312000,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542170,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61132,"kb_used_data":16064,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94310708,"statfs":{"total":96636764160,"available":96574164992,"internally_reserved":0,"allocated":16449536,"data_stored":156719469,"data_compressed":8479736,"data_compressed_allocated":9605120,"data_compressed_original":151321224,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542170,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79036,"kb_used_data":11756,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94292804,"statfs":{"total":96636764160,"available":96555831296,"internally_reserved":0,"allocated":12038144,"data_stored":104279588,"data_compressed":5502184,"data_compressed_allocated":6660096,"data_compressed_original":99952753,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38888,"kb_used_data":7700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332952,"statfs":{"total":96636764160,"available":96596942848,"internally_reserved":0,"allocated":7884800,"data_stored":71785137,"data_compressed":3700257,"data_compressed_allocated":4239360,"data_compressed_original":68992064,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74484,"kb_used_data":11368,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94297356,"statfs":{"total":96636764160,"available":96560492544,"internally_reserved":0,"allocated":11640832,"data_stored":122192133,"data_compressed":6823252,"data_compressed_allocated":7053312,"data_compressed_original":118922434,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10199040,"data_stored":120952131,"data_compressed":6790742,"data_compressed_allocated":7008256,"data_compressed_original":118348994,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10711040,"data_stored":103654388,"data_compressed":5502184,"data_compressed_allocated":6660096,"data_compressed_original":99952753,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6508544,"data_stored":71135503,"data_compressed":3700257,"data_compressed_allocated":4239360,"data_compressed_original":68992064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":15122432,"data_stored":156094269,"data_compressed":8479736,"data_compressed_allocated":9605120,"data_compressed_original":151321224,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11431936,"data_stored":122861335,"data_compressed":6586813,"data_compressed_allocated":7327744,"data_compressed_original":119196474,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8855552,"data_stored":88958792,"data_compressed":5319187,"data_compressed_allocated":5922816,"data_compressed_original":86487791,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7761920,"data_stored":79918021,"data_compressed":4400954,"data_compressed_allocated":4943872,"data_compressed_original":77352960,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":15716352,"data_stored":164166241,"data_compressed":9550746,"data_compressed_allocated":10231808,"data_compressed_original":159247050,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T23:03:20.992 INFO:tasks.thrashosds.thrasher:Setting 3.4 to [3, 6, 4, 2] 2024-05-11T23:03:20.992 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.4', '3', '6', '4', '2'] 2024-05-11T23:03:20.992 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.4 3 6 4 2 2024-05-11T23:03:21.032 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:21.031+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:21.082 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:21.082 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:21.082 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:21.092 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:03:21.129 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:21.129+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:21.208 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:21.208 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:21.208 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:21.208 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:21.208 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:21.217 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:03:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-05-11T23:03:21.237 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:21.237+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:21.330 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:21.330+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:21.350 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:21.351 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:21.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:21.351 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:21.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.351 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36828 3.b 3:d385bc8e:::rbd_data.111e5e4d8275.0000000000000020:11 [read 512000~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:21.351 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:03:21.351 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.416015040999994, 2024-05-11T23:03:21.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051481218000000002, 2024-05-11T23:03:21.351 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:21.351 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:21.351 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:21.352 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:21.352 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:21.352 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36828 2024-05-11T23:03:21.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.352 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:21.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:21.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:03:21.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:21.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.933096+0000", 2024-05-11T23:03:21.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:21.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934915+0000", 2024-05-11T23:03:21.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00181814 2024-05-11T23:03:21.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:21.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934919+0000", 2024-05-11T23:03:21.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6739999999999996e-06 2024-05-11T23:03:21.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:21.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934934+0000", 2024-05-11T23:03:21.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4415e-05 2024-05-11T23:03:21.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.934942+0000", 2024-05-11T23:03:21.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8619999999999992e-06 2024-05-11T23:03:21.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:21.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984433+0000", 2024-05-11T23:03:21.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049490658999999999 2024-05-11T23:03:21.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:21.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984508+0000", 2024-05-11T23:03:21.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4394999999999995e-05 2024-05-11T23:03:21.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:21.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.984578+0000", 2024-05-11T23:03:21.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0073000000000005e-05 2024-05-11T23:03:21.355 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.355 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.355 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.355 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36950 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 520192~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:21.355 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:03:21.355 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.411709275000007, 2024-05-11T23:03:21.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051450856000000003, 2024-05-11T23:03:21.356 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:21.356 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:21.356 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:21.356 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:21.356 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:21.356 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36950 2024-05-11T23:03:21.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.356 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:21.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:21.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:03:21.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:21.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.937402+0000", 2024-05-11T23:03:21.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:21.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938914+0000", 2024-05-11T23:03:21.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0015121480000000001 2024-05-11T23:03:21.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:21.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938918+0000", 2024-05-11T23:03:21.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4149999999999999e-06 2024-05-11T23:03:21.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:21.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938925+0000", 2024-05-11T23:03:21.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0779999999999998e-06 2024-05-11T23:03:21.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.938929+0000", 2024-05-11T23:03:21.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2930000000000002e-06 2024-05-11T23:03:21.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:21.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988675+0000", 2024-05-11T23:03:21.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049746394999999999 2024-05-11T23:03:21.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:21.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988756+0000", 2024-05-11T23:03:21.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0499000000000003e-05 2024-05-11T23:03:21.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:21.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.988853+0000", 2024-05-11T23:03:21.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7027999999999993e-05 2024-05-11T23:03:21.359 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.359 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.359 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.359 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36957 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 778240~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:21.359 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:03:21.359 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.410087485999995, 2024-05-11T23:03:21.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051328273000000001, 2024-05-11T23:03:21.360 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:21.360 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:21.360 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:21.360 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:21.360 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:21.360 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.360 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36957 2024-05-11T23:03:21.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.360 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:21.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:21.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:03:21.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:21.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:03:21.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:21.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939024+0000", 2024-05-11T23:03:21.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2099999999999998e-07 2024-05-11T23:03:21.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:21.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939026+0000", 2024-05-11T23:03:21.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5069999999999999e-06 2024-05-11T23:03:21.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:21.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939030+0000", 2024-05-11T23:03:21.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7919999999999999e-06 2024-05-11T23:03:21.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939032+0000", 2024-05-11T23:03:21.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9129999999999999e-06 2024-05-11T23:03:21.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:21.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990178+0000", 2024-05-11T23:03:21.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051145373000000001 2024-05-11T23:03:21.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:21.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990255+0000", 2024-05-11T23:03:21.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7195999999999997e-05 2024-05-11T23:03:21.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:21.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990352+0000", 2024-05-11T23:03:21.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7170999999999995e-05 2024-05-11T23:03:21.363 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.363 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.363 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.364 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36958 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 815104~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:21.364 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:03:21.364 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.410075167000002, 2024-05-11T23:03:21.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051531625999999997, 2024-05-11T23:03:21.364 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:21.364 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:21.364 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:21.364 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:21.364 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:21.364 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.364 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36958 2024-05-11T23:03:21.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.365 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:21.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:21.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:03:21.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:21.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939036+0000", 2024-05-11T23:03:21.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:21.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939037+0000", 2024-05-11T23:03:21.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0400000000000002e-07 2024-05-11T23:03:21.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:21.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939038+0000", 2024-05-11T23:03:21.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.164e-06 2024-05-11T23:03:21.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:21.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939041+0000", 2024-05-11T23:03:21.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1109999999999999e-06 2024-05-11T23:03:21.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939043+0000", 2024-05-11T23:03:21.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.554e-06 2024-05-11T23:03:21.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:21.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990382+0000", 2024-05-11T23:03:21.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051338953999999999 2024-05-11T23:03:21.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:21.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990465+0000", 2024-05-11T23:03:21.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2461999999999998e-05 2024-05-11T23:03:21.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:21.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990568+0000", 2024-05-11T23:03:21.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000103077 2024-05-11T23:03:21.368 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.368 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.368 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.368 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36959 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 851968~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:21.368 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:03:21.368 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.410063270999999, 2024-05-11T23:03:21.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051758936999999998, 2024-05-11T23:03:21.368 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:21.368 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:21.368 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:21.368 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:21.368 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:21.369 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.369 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36959 2024-05-11T23:03:21.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.369 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:21.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:21.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:03:21.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:21.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:03:21.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:21.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939048+0000", 2024-05-11T23:03:21.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.16e-07 2024-05-11T23:03:21.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:21.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939050+0000", 2024-05-11T23:03:21.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.176e-06 2024-05-11T23:03:21.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:21.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939053+0000", 2024-05-11T23:03:21.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1140000000000002e-06 2024-05-11T23:03:21.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939055+0000", 2024-05-11T23:03:21.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.621e-06 2024-05-11T23:03:21.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:21.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990610+0000", 2024-05-11T23:03:21.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051554244999999999 2024-05-11T23:03:21.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:21.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990700+0000", 2024-05-11T23:03:21.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0284999999999994e-05 2024-05-11T23:03:21.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:21.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990807+0000", 2024-05-11T23:03:21.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010728 2024-05-11T23:03:21.372 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.372 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.372 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.372 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36960 3.b 3:d468b622:::rbd_data.111e5e4d8275.0000000000000019:11 [read 888832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:21.372 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:03:21.372 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.410053035999994, 2024-05-11T23:03:21.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051886831000000001, 2024-05-11T23:03:21.373 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:21.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:21.373 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:21.373 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:21.373 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:21.373 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.373 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36960 2024-05-11T23:03:21.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.373 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:21.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:21.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:03:21.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:21.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.939058+0000", 2024-05-11T23:03:21.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:21.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940913+0000", 2024-05-11T23:03:21.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001855104 2024-05-11T23:03:21.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:21.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940916+0000", 2024-05-11T23:03:21.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0000000000000001e-06 2024-05-11T23:03:21.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:21.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940924+0000", 2024-05-11T23:03:21.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0999999999999998e-06 2024-05-11T23:03:21.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.940927+0000", 2024-05-11T23:03:21.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.7160000000000001e-06 2024-05-11T23:03:21.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:21.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990826+0000", 2024-05-11T23:03:21.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049898832999999997 2024-05-11T23:03:21.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:21.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990872+0000", 2024-05-11T23:03:21.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5717000000000003e-05 2024-05-11T23:03:21.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:21.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.990945+0000", 2024-05-11T23:03:21.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3361000000000005e-05 2024-05-11T23:03:21.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.378 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.378 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36986 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 528384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:21.378 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:03:21.378 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.408046818000003, 2024-05-11T23:03:21.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051341053999999997, 2024-05-11T23:03:21.378 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:21.379 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:21.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:21.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:21.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:21.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.379 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36986 2024-05-11T23:03:21.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.379 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:21.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:21.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:03:21.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:21.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:03:21.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:21.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941065+0000", 2024-05-11T23:03:21.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:21.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:21.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941066+0000", 2024-05-11T23:03:21.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1629999999999999e-06 2024-05-11T23:03:21.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941069+0000", 2024-05-11T23:03:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.118e-06 2024-05-11T23:03:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941072+0000", 2024-05-11T23:03:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5720000000000001e-06 2024-05-11T23:03:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992292+0000", 2024-05-11T23:03:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051220620000000001 2024-05-11T23:03:21.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:21.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992344+0000", 2024-05-11T23:03:21.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1737000000000003e-05 2024-05-11T23:03:21.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:21.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992406+0000", 2024-05-11T23:03:21.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1623000000000003e-05 2024-05-11T23:03:21.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.383 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.383 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.383 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36987 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 565248~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:21.383 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:03:21.383 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.408036697, 2024-05-11T23:03:21.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051568725000000003, 2024-05-11T23:03:21.383 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:21.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:21.383 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:21.383 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:21.383 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:21.383 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.384 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36987 2024-05-11T23:03:21.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.384 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:21.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:21.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:03:21.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:21.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:03:21.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:21.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941075+0000", 2024-05-11T23:03:21.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.91e-07 2024-05-11T23:03:21.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:21.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941076+0000", 2024-05-11T23:03:21.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1379999999999999e-06 2024-05-11T23:03:21.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:21.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941079+0000", 2024-05-11T23:03:21.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9739999999999998e-06 2024-05-11T23:03:21.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941082+0000", 2024-05-11T23:03:21.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8009999999999999e-06 2024-05-11T23:03:21.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:21.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992446+0000", 2024-05-11T23:03:21.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051364058999999997 2024-05-11T23:03:21.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:21.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992536+0000", 2024-05-11T23:03:21.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0085000000000002e-05 2024-05-11T23:03:21.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:21.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992643+0000", 2024-05-11T23:03:21.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000107377 2024-05-11T23:03:21.387 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.387 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.387 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.388 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36988 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 602112~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:21.388 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:03:21.388 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.408026328000005, 2024-05-11T23:03:21.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051700007999999999, 2024-05-11T23:03:21.388 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:21.388 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:21.388 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:21.388 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:21.388 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:21.388 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36988 2024-05-11T23:03:21.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.388 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:21.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:21.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:03:21.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:21.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:03:21.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:21.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941085+0000", 2024-05-11T23:03:21.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.05e-07 2024-05-11T23:03:21.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:21.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941086+0000", 2024-05-11T23:03:21.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.125e-06 2024-05-11T23:03:21.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:21.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941089+0000", 2024-05-11T23:03:21.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0599999999999999e-06 2024-05-11T23:03:21.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941092+0000", 2024-05-11T23:03:21.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6529999999999998e-06 2024-05-11T23:03:21.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:21.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992667+0000", 2024-05-11T23:03:21.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051574587999999998 2024-05-11T23:03:21.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:21.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992723+0000", 2024-05-11T23:03:21.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6682999999999999e-05 2024-05-11T23:03:21.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:21.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992785+0000", 2024-05-11T23:03:21.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1693999999999996e-05 2024-05-11T23:03:21.392 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.392 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.392 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.393 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36989 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 638976~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:21.393 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:03:21.393 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.408016138999997, 2024-05-11T23:03:21.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05192811, 2024-05-11T23:03:21.393 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:21.393 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:21.393 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:21.393 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:21.393 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:21.393 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.393 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36989 2024-05-11T23:03:21.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.394 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:21.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:21.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:03:21.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:21.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.941095+0000", 2024-05-11T23:03:21.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:21.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942916+0000", 2024-05-11T23:03:21.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001820835 2024-05-11T23:03:21.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:21.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942920+0000", 2024-05-11T23:03:21.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5310000000000002e-06 2024-05-11T23:03:21.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:21.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942931+0000", 2024-05-11T23:03:21.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1839e-05 2024-05-11T23:03:21.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.942940+0000", 2024-05-11T23:03:21.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2309999999999996e-06 2024-05-11T23:03:21.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:21.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992820+0000", 2024-05-11T23:03:21.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049880751000000001 2024-05-11T23:03:21.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:21.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.992918+0000", 2024-05-11T23:03:21.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7051000000000003e-05 2024-05-11T23:03:21.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:21.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.993023+0000", 2024-05-11T23:03:21.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010587200000000001 2024-05-11T23:03:21.399 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.399 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.399 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.399 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36997 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 933888~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:21.399 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:03:21.400 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.405982304000005, 2024-05-11T23:03:21.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051390602000000001, 2024-05-11T23:03:21.400 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:21.400 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:21.400 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:21.400 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:21.400 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:21.400 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36997 2024-05-11T23:03:21.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.400 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:21.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:21.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:03:21.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:21.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943129+0000", 2024-05-11T23:03:21.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:21.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943130+0000", 2024-05-11T23:03:21.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0200000000000003e-07 2024-05-11T23:03:21.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.401 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:21.401 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943132+0000", 2024-05-11T23:03:21.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.305e-06 2024-05-11T23:03:21.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:21.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943138+0000", 2024-05-11T23:03:21.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4219999999999997e-06 2024-05-11T23:03:21.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.402 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.402 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943143+0000", 2024-05-11T23:03:21.402 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0610000000000003e-06 2024-05-11T23:03:21.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:21.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994323+0000", 2024-05-11T23:03:21.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051179592000000003 2024-05-11T23:03:21.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:21.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994412+0000", 2024-05-11T23:03:21.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8709999999999996e-05 2024-05-11T23:03:21.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.404 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:21.404 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994520+0000", 2024-05-11T23:03:21.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010811000000000001 2024-05-11T23:03:21.404 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.404 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.404 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.404 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36998 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 970752~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:21.404 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:03:21.404 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.405961425000001, 2024-05-11T23:03:21.404 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051517669000000002, 2024-05-11T23:03:21.404 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:21.404 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:21.405 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:21.405 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:21.405 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:21.405 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.405 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36998 2024-05-11T23:03:21.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.405 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:21.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:21.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:03:21.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:21.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:03:21.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:21.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943150+0000", 2024-05-11T23:03:21.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:03:21.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:21.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943153+0000", 2024-05-11T23:03:21.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.306e-06 2024-05-11T23:03:21.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:21.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943160+0000", 2024-05-11T23:03:21.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.452e-06 2024-05-11T23:03:21.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943165+0000", 2024-05-11T23:03:21.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.152e-06 2024-05-11T23:03:21.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:21.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994561+0000", 2024-05-11T23:03:21.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051395765000000003 2024-05-11T23:03:21.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:21.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994607+0000", 2024-05-11T23:03:21.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6415000000000002e-05 2024-05-11T23:03:21.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.408 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:21.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994668+0000", 2024-05-11T23:03:21.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0179000000000001e-05 2024-05-11T23:03:21.408 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.408 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.408 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.409 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:36999 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 1007616~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:21.409 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:03:21.409 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.405916543999993, 2024-05-11T23:03:21.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051696919000000001, 2024-05-11T23:03:21.409 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:21.409 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:21.409 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:21.409 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:21.409 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:21.409 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.409 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 36999 2024-05-11T23:03:21.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.410 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:21.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:21.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:03:21.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:21.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:03:21.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:21.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943195+0000", 2024-05-11T23:03:21.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:03:21.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:21.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943197+0000", 2024-05-11T23:03:21.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1280000000000002e-06 2024-05-11T23:03:21.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:21.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943202+0000", 2024-05-11T23:03:21.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8910000000000003e-06 2024-05-11T23:03:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943206+0000", 2024-05-11T23:03:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3550000000000001e-06 2024-05-11T23:03:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994703+0000", 2024-05-11T23:03:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051497045999999998 2024-05-11T23:03:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994789+0000", 2024-05-11T23:03:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6458000000000005e-05 2024-05-11T23:03:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994892+0000", 2024-05-11T23:03:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102705 2024-05-11T23:03:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37000 3.b 3:d7dad6df:::rbd_data.111e5e4d8275.000000000000001b:11 [read 1044480~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:03:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.405900599000006, 2024-05-11T23:03:21.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051842371999999998, 2024-05-11T23:03:21.414 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:21.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:21.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:21.414 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:21.414 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:21.414 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.414 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37000 2024-05-11T23:03:21.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.414 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:21.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:21.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:03:21.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:21.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.943211+0000", 2024-05-11T23:03:21.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:21.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944916+0000", 2024-05-11T23:03:21.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001705097 2024-05-11T23:03:21.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:21.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944921+0000", 2024-05-11T23:03:21.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5809999999999998e-06 2024-05-11T23:03:21.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:21.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944930+0000", 2024-05-11T23:03:21.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5529999999999998e-06 2024-05-11T23:03:21.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.944938+0000", 2024-05-11T23:03:21.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2519999999999999e-06 2024-05-11T23:03:21.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:21.416 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994911+0000", 2024-05-11T23:03:21.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049973113999999999 2024-05-11T23:03:21.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:21.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.994982+0000", 2024-05-11T23:03:21.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0544999999999996e-05 2024-05-11T23:03:21.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:21.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.995053+0000", 2024-05-11T23:03:21.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1229999999999994e-05 2024-05-11T23:03:21.417 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.417 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.417 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.417 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37041 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 618496~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:21.418 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:03:21.418 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.404056620000006, 2024-05-11T23:03:21.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051352768999999999, 2024-05-11T23:03:21.418 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:21.418 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:21.418 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:21.418 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:21.418 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:21.418 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.418 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37041 2024-05-11T23:03:21.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.419 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:21.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:21.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:03:21.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:21.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:03:21.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:21.420 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945055+0000", 2024-05-11T23:03:21.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:03:21.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:21.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945056+0000", 2024-05-11T23:03:21.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:03:21.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:21.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945059+0000", 2024-05-11T23:03:21.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.965e-06 2024-05-11T23:03:21.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945062+0000", 2024-05-11T23:03:21.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5399999999999998e-06 2024-05-11T23:03:21.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:21.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996213+0000", 2024-05-11T23:03:21.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051151532999999999 2024-05-11T23:03:21.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:21.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996301+0000", 2024-05-11T23:03:21.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8143e-05 2024-05-11T23:03:21.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:21.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996408+0000", 2024-05-11T23:03:21.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010623700000000001 2024-05-11T23:03:21.423 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.423 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.423 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.423 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37042 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 655360~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:21.423 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:03:21.423 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.404046635, 2024-05-11T23:03:21.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051462075000000003, 2024-05-11T23:03:21.424 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:21.424 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:21.424 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:21.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:21.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:21.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.424 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37042 2024-05-11T23:03:21.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.424 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:21.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.424 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:21.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:03:21.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:21.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:03:21.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:21.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945065+0000", 2024-05-11T23:03:21.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0599999999999999e-07 2024-05-11T23:03:21.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:21.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945066+0000", 2024-05-11T23:03:21.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1480000000000001e-06 2024-05-11T23:03:21.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:21.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945069+0000", 2024-05-11T23:03:21.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0079999999999998e-06 2024-05-11T23:03:21.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945072+0000", 2024-05-11T23:03:21.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6740000000000001e-06 2024-05-11T23:03:21.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:21.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996422+0000", 2024-05-11T23:03:21.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051349999 2024-05-11T23:03:21.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:21.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996467+0000", 2024-05-11T23:03:21.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.528e-05 2024-05-11T23:03:21.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.428 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:21.428 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996527+0000", 2024-05-11T23:03:21.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9759999999999997e-05 2024-05-11T23:03:21.428 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.428 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.428 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.429 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37043 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 692224~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:21.429 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:03:21.429 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.404034562000007, 2024-05-11T23:03:21.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051679535999999998, 2024-05-11T23:03:21.429 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:21.430 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:21.430 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:21.430 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:21.430 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:21.430 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.430 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37043 2024-05-11T23:03:21.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.430 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:21.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.431 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:21.431 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:03:21.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.431 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:21.431 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:03:21.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:21.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945077+0000", 2024-05-11T23:03:21.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8999999999999997e-07 2024-05-11T23:03:21.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.432 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:21.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945079+0000", 2024-05-11T23:03:21.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.001e-06 2024-05-11T23:03:21.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:21.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945087+0000", 2024-05-11T23:03:21.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1749999999999999e-06 2024-05-11T23:03:21.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945093+0000", 2024-05-11T23:03:21.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9639999999999997e-06 2024-05-11T23:03:21.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:21.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996563+0000", 2024-05-11T23:03:21.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051469050000000002 2024-05-11T23:03:21.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:21.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996651+0000", 2024-05-11T23:03:21.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8356000000000006e-05 2024-05-11T23:03:21.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:21.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996756+0000", 2024-05-11T23:03:21.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0001055 2024-05-11T23:03:21.434 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.435 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.435 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.435 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37044 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 729088~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:21.435 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:03:21.435 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.40400065, 2024-05-11T23:03:21.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051763445999999998, 2024-05-11T23:03:21.435 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:21.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:21.435 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:21.435 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:21.435 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:21.436 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.436 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37044 2024-05-11T23:03:21.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.436 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:21.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:21.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:03:21.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:21.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.945111+0000", 2024-05-11T23:03:21.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:21.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946915+0000", 2024-05-11T23:03:21.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018042760000000001 2024-05-11T23:03:21.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:21.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946918+0000", 2024-05-11T23:03:21.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.36e-06 2024-05-11T23:03:21.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.437 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:21.437 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946926+0000", 2024-05-11T23:03:21.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3130000000000001e-06 2024-05-11T23:03:21.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.946930+0000", 2024-05-11T23:03:21.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5249999999999999e-06 2024-05-11T23:03:21.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:21.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996772+0000", 2024-05-11T23:03:21.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049841868999999997 2024-05-11T23:03:21.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:21.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996817+0000", 2024-05-11T23:03:21.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5351e-05 2024-05-11T23:03:21.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:21.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.996874+0000", 2024-05-11T23:03:21.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.6752000000000001e-05 2024-05-11T23:03:21.439 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.439 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.439 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.439 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37052 3.b 3:d1b08ddc:::rbd_data.111e5e4d8275.000000000000001e:11 [read 1024000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:21.440 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:03:21.440 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.402093798999999, 2024-05-11T23:03:21.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051326140999999999, 2024-05-11T23:03:21.440 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:21.440 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:21.440 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:21.440 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:21.440 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:21.440 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.440 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37052 2024-05-11T23:03:21.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.441 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:21.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:21.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:03:21.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.441 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:21.441 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:03:21.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:21.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947018+0000", 2024-05-11T23:03:21.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:21.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:21.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947019+0000", 2024-05-11T23:03:21.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.218e-06 2024-05-11T23:03:21.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:21.443 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947022+0000", 2024-05-11T23:03:21.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1889999999999999e-06 2024-05-11T23:03:21.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.443 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.443 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947025+0000", 2024-05-11T23:03:21.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.464e-06 2024-05-11T23:03:21.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.443 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:21.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998237+0000", 2024-05-11T23:03:21.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051212494999999997 2024-05-11T23:03:21.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:21.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998284+0000", 2024-05-11T23:03:21.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7246000000000001e-05 2024-05-11T23:03:21.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:21.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998344+0000", 2024-05-11T23:03:21.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9320000000000001e-05 2024-05-11T23:03:21.444 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.445 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.445 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.445 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:37071 3.b 3:d385bc8e:::rbd_data.111e5e4d8275.0000000000000020:11 [read 24576~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:21.445 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:03:21.445 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 88.402083590000004, 2024-05-11T23:03:21.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051528796000000002, 2024-05-11T23:03:21.445 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:21.445 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:21.445 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:21.445 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:21.445 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:21.445 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.445 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 37071 2024-05-11T23:03:21.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.446 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:21.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:21.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:03:21.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:21.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.947028+0000", 2024-05-11T23:03:21.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:21.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:21.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948913+0000", 2024-05-11T23:03:21.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018854970000000001 2024-05-11T23:03:21.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:21.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948917+0000", 2024-05-11T23:03:21.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8089999999999999e-06 2024-05-11T23:03:21.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:21.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948926+0000", 2024-05-11T23:03:21.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8389999999999996e-06 2024-05-11T23:03:21.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.948930+0000", 2024-05-11T23:03:21.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4519999999999999e-06 2024-05-11T23:03:21.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:21.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998379+0000", 2024-05-11T23:03:21.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049449011000000001 2024-05-11T23:03:21.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:21.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998462+0000", 2024-05-11T23:03:21.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2739999999999997e-05 2024-05-11T23:03:21.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:21.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:21.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:21.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:01:52.998557+0000", 2024-05-11T23:03:21.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4448000000000001e-05 2024-05-11T23:03:21.449 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.449 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.449 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.449 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:21.449 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:21.449 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:21.451 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:03:21.451 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:21.450+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:21.551 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:21.551+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:21.556 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:21.556 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:21.556 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:21.557 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:21.565 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:03:21.652 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:21.652+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:21.711 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:21.711 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:21.711 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:21.719 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:03:21.753 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:21.753+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:21.854 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:21.854+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:21.865 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116446 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8273920~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.489429495000003, 2024-05-11T23:03:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045535299000000001, 2024-05-11T23:03:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:21.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116446 2024-05-11T23:03:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:21.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375547+0000", 2024-05-11T23:03:21.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4000000000000001e-07 2024-05-11T23:03:21.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:21.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375549+0000", 2024-05-11T23:03:21.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.368e-06 2024-05-11T23:03:21.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:21.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375556+0000", 2024-05-11T23:03:21.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1840000000000004e-06 2024-05-11T23:03:21.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375560+0000", 2024-05-11T23:03:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.741e-06 2024-05-11T23:03:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420896+0000", 2024-05-11T23:03:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045335435 2024-05-11T23:03:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420987+0000", 2024-05-11T23:03:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1033999999999996e-05 2024-05-11T23:03:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:21.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421082+0000", 2024-05-11T23:03:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4896999999999996e-05 2024-05-11T23:03:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116447 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6316032~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.489407985, 2024-05-11T23:03:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045745830000000001, 2024-05-11T23:03:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:21.870 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116447 2024-05-11T23:03:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.58e-07 2024-05-11T23:03:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375570+0000", 2024-05-11T23:03:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:03:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375576+0000", 2024-05-11T23:03:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7939999999999999e-06 2024-05-11T23:03:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375581+0000", 2024-05-11T23:03:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9280000000000001e-06 2024-05-11T23:03:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421127+0000", 2024-05-11T23:03:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045545785999999998 2024-05-11T23:03:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421214+0000", 2024-05-11T23:03:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7039000000000005e-05 2024-05-11T23:03:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:21.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421314+0000", 2024-05-11T23:03:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9772999999999995e-05 2024-05-11T23:03:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116448 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8314880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.489388910999999, 2024-05-11T23:03:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045941753000000002, 2024-05-11T23:03:21.874 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116448 2024-05-11T23:03:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.77e-07 2024-05-11T23:03:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375589+0000", 2024-05-11T23:03:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.091e-06 2024-05-11T23:03:21.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375596+0000", 2024-05-11T23:03:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9179999999999997e-06 2024-05-11T23:03:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375601+0000", 2024-05-11T23:03:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0030000000000002e-06 2024-05-11T23:03:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421363+0000", 2024-05-11T23:03:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045761352999999998 2024-05-11T23:03:21.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421440+0000", 2024-05-11T23:03:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7585999999999999e-05 2024-05-11T23:03:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421529+0000", 2024-05-11T23:03:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8424999999999994e-05 2024-05-11T23:03:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.878 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116449 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6438912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.489368476000003, 2024-05-11T23:03:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046082208999999999, 2024-05-11T23:03:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116449 2024-05-11T23:03:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:21.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377225+0000", 2024-05-11T23:03:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016177030000000001 2024-05-11T23:03:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377229+0000", 2024-05-11T23:03:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8e-06 2024-05-11T23:03:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377238+0000", 2024-05-11T23:03:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2229999999999996e-06 2024-05-11T23:03:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377244+0000", 2024-05-11T23:03:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4060000000000002e-06 2024-05-11T23:03:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421520+0000", 2024-05-11T23:03:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044275222000000003 2024-05-11T23:03:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421595+0000", 2024-05-11T23:03:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4838000000000004e-05 2024-05-11T23:03:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421690+0000", 2024-05-11T23:03:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5017000000000002e-05 2024-05-11T23:03:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.882 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116458 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7094272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.487554258999999, 2024-05-11T23:03:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045706877999999999, 2024-05-11T23:03:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116458 2024-05-11T23:03:21.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9400000000000005e-07 2024-05-11T23:03:21.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377424+0000", 2024-05-11T23:03:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1859999999999999e-06 2024-05-11T23:03:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377431+0000", 2024-05-11T23:03:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4049999999999997e-06 2024-05-11T23:03:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377436+0000", 2024-05-11T23:03:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1399999999999999e-06 2024-05-11T23:03:21.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.422945+0000", 2024-05-11T23:03:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045509326000000003 2024-05-11T23:03:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423036+0000", 2024-05-11T23:03:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0649999999999994e-05 2024-05-11T23:03:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423128+0000", 2024-05-11T23:03:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2577000000000005e-05 2024-05-11T23:03:21.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116459 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7258112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.487533526999997, 2024-05-11T23:03:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045831945999999998, 2024-05-11T23:03:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116459 2024-05-11T23:03:21.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:21.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377443+0000", 2024-05-11T23:03:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999999e-07 2024-05-11T23:03:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377445+0000", 2024-05-11T23:03:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0209999999999998e-06 2024-05-11T23:03:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377451+0000", 2024-05-11T23:03:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.071e-06 2024-05-11T23:03:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377456+0000", 2024-05-11T23:03:21.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8289999999999998e-06 2024-05-11T23:03:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423155+0000", 2024-05-11T23:03:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045699447999999997 2024-05-11T23:03:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423200+0000", 2024-05-11T23:03:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5083000000000002e-05 2024-05-11T23:03:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423274+0000", 2024-05-11T23:03:21.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4046000000000002e-05 2024-05-11T23:03:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116460 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7299072~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.487514464999997, 2024-05-11T23:03:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046010987000000003, 2024-05-11T23:03:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:21.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116460 2024-05-11T23:03:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377462+0000", 2024-05-11T23:03:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:03:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377464+0000", 2024-05-11T23:03:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1600000000000001e-06 2024-05-11T23:03:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377469+0000", 2024-05-11T23:03:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5430000000000001e-06 2024-05-11T23:03:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377474+0000", 2024-05-11T23:03:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8509999999999998e-06 2024-05-11T23:03:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423312+0000", 2024-05-11T23:03:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045838080000000003 2024-05-11T23:03:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423401+0000", 2024-05-11T23:03:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8170000000000005e-05 2024-05-11T23:03:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:21.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423472+0000", 2024-05-11T23:03:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1847000000000005e-05 2024-05-11T23:03:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116461 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7421952~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.487495695, 2024-05-11T23:03:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046130749999999998, 2024-05-11T23:03:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:21.895 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116461 2024-05-11T23:03:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4700000000000002e-07 2024-05-11T23:03:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377483+0000", 2024-05-11T23:03:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0219999999999999e-06 2024-05-11T23:03:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377489+0000", 2024-05-11T23:03:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4420000000000003e-06 2024-05-11T23:03:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377494+0000", 2024-05-11T23:03:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9239999999999998e-06 2024-05-11T23:03:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423500+0000", 2024-05-11T23:03:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046006054999999997 2024-05-11T23:03:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423557+0000", 2024-05-11T23:03:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7169000000000001e-05 2024-05-11T23:03:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:21.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423611+0000", 2024-05-11T23:03:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3791e-05 2024-05-11T23:03:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116462 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7462912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.487476076999997, 2024-05-11T23:03:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046254666, 2024-05-11T23:03:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:21.899 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116462 2024-05-11T23:03:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377502+0000", 2024-05-11T23:03:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-05-11T23:03:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377508+0000", 2024-05-11T23:03:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4500000000000003e-06 2024-05-11T23:03:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377512+0000", 2024-05-11T23:03:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7700000000000001e-06 2024-05-11T23:03:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423649+0000", 2024-05-11T23:03:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046136859000000002 2024-05-11T23:03:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423704+0000", 2024-05-11T23:03:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4938000000000002e-05 2024-05-11T23:03:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423754+0000", 2024-05-11T23:03:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0145999999999997e-05 2024-05-11T23:03:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116463 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7585792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.487457651, 2024-05-11T23:03:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361763, 2024-05-11T23:03:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:21.903 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116463 2024-05-11T23:03:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377519+0000", 2024-05-11T23:03:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2600000000000002e-07 2024-05-11T23:03:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377521+0000", 2024-05-11T23:03:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1710000000000001e-06 2024-05-11T23:03:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377527+0000", 2024-05-11T23:03:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9649999999999996e-06 2024-05-11T23:03:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377532+0000", 2024-05-11T23:03:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.78e-06 2024-05-11T23:03:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423779+0000", 2024-05-11T23:03:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046247706 2024-05-11T23:03:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423829+0000", 2024-05-11T23:03:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9188999999999999e-05 2024-05-11T23:03:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423880+0000", 2024-05-11T23:03:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1425999999999999e-05 2024-05-11T23:03:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116464 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7749632~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.487438415, 2024-05-11T23:03:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046491719000000001, 2024-05-11T23:03:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:21.907 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116464 2024-05-11T23:03:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377538+0000", 2024-05-11T23:03:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4499999999999998e-07 2024-05-11T23:03:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377540+0000", 2024-05-11T23:03:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2340000000000001e-06 2024-05-11T23:03:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377546+0000", 2024-05-11T23:03:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8289999999999996e-06 2024-05-11T23:03:21.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377551+0000", 2024-05-11T23:03:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8559999999999998e-06 2024-05-11T23:03:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423913+0000", 2024-05-11T23:03:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361885999999998 2024-05-11T23:03:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423958+0000", 2024-05-11T23:03:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5513999999999998e-05 2024-05-11T23:03:21.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424029+0000", 2024-05-11T23:03:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1055000000000003e-05 2024-05-11T23:03:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116465 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7626752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.487419267, 2024-05-11T23:03:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046600057, 2024-05-11T23:03:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:21.911 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116465 2024-05-11T23:03:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:21.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3599999999999999e-07 2024-05-11T23:03:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377559+0000", 2024-05-11T23:03:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9640000000000002e-06 2024-05-11T23:03:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377572+0000", 2024-05-11T23:03:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3188e-05 2024-05-11T23:03:21.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377578+0000", 2024-05-11T23:03:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.942e-06 2024-05-11T23:03:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424058+0000", 2024-05-11T23:03:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046480021000000003 2024-05-11T23:03:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424103+0000", 2024-05-11T23:03:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5021999999999997e-05 2024-05-11T23:03:21.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424157+0000", 2024-05-11T23:03:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3483999999999998e-05 2024-05-11T23:03:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116466 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7790592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.487390685000001, 2024-05-11T23:03:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046743741999999998, 2024-05-11T23:03:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:21.915 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116466 2024-05-11T23:03:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:21.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377586+0000", 2024-05-11T23:03:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:03:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377588+0000", 2024-05-11T23:03:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3039999999999999e-06 2024-05-11T23:03:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377594+0000", 2024-05-11T23:03:21.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-05-11T23:03:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377600+0000", 2024-05-11T23:03:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7699999999999998e-06 2024-05-11T23:03:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424185+0000", 2024-05-11T23:03:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046585891999999997 2024-05-11T23:03:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424235+0000", 2024-05-11T23:03:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9356e-05 2024-05-11T23:03:21.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424329+0000", 2024-05-11T23:03:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4036999999999994e-05 2024-05-11T23:03:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116467 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7954432~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.487370398000003, 2024-05-11T23:03:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046930134999999998, 2024-05-11T23:03:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116467 2024-05-11T23:03:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:21.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379161+0000", 2024-05-11T23:03:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015555009999999999 2024-05-11T23:03:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379166+0000", 2024-05-11T23:03:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6689999999999997e-06 2024-05-11T23:03:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379176+0000", 2024-05-11T23:03:21.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0549e-05 2024-05-11T23:03:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379183+0000", 2024-05-11T23:03:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1199999999999996e-06 2024-05-11T23:03:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424359+0000", 2024-05-11T23:03:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045175949999999999 2024-05-11T23:03:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424433+0000", 2024-05-11T23:03:21.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3824000000000003e-05 2024-05-11T23:03:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424536+0000", 2024-05-11T23:03:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010252200000000001 2024-05-11T23:03:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116468 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7913472~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.485780179000002, 2024-05-11T23:03:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045525364999999998, 2024-05-11T23:03:21.923 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116468 2024-05-11T23:03:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3300000000000002e-07 2024-05-11T23:03:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379198+0000", 2024-05-11T23:03:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2670000000000001e-06 2024-05-11T23:03:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:21.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379206+0000", 2024-05-11T23:03:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7149999999999998e-06 2024-05-11T23:03:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379228+0000", 2024-05-11T23:03:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1429000000000001e-05 2024-05-11T23:03:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424574+0000", 2024-05-11T23:03:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045346209999999998 2024-05-11T23:03:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:21.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424627+0000", 2024-05-11T23:03:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3352e-05 2024-05-11T23:03:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424721+0000", 2024-05-11T23:03:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3858999999999997e-05 2024-05-11T23:03:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116469 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8077312~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.485730359000001, 2024-05-11T23:03:21.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045636687000000002, 2024-05-11T23:03:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116469 2024-05-11T23:03:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379246+0000", 2024-05-11T23:03:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.32e-07 2024-05-11T23:03:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379254+0000", 2024-05-11T23:03:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.807e-06 2024-05-11T23:03:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379261+0000", 2024-05-11T23:03:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.985e-06 2024-05-11T23:03:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379264+0000", 2024-05-11T23:03:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6480000000000001e-06 2024-05-11T23:03:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424744+0000", 2024-05-11T23:03:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045479638000000003 2024-05-11T23:03:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424797+0000", 2024-05-11T23:03:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2725000000000001e-05 2024-05-11T23:03:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424882+0000", 2024-05-11T23:03:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5451999999999994e-05 2024-05-11T23:03:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116470 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8118272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:21.931 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.485705146999997, 2024-05-11T23:03:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045755745, 2024-05-11T23:03:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116470 2024-05-11T23:03:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3999999999999997e-07 2024-05-11T23:03:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379273+0000", 2024-05-11T23:03:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-05-11T23:03:21.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379277+0000", 2024-05-11T23:03:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4020000000000004e-06 2024-05-11T23:03:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379280+0000", 2024-05-11T23:03:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-05-11T23:03:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424922+0000", 2024-05-11T23:03:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045642598999999999 2024-05-11T23:03:21.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424973+0000", 2024-05-11T23:03:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0674999999999998e-05 2024-05-11T23:03:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425026+0000", 2024-05-11T23:03:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3492000000000002e-05 2024-05-11T23:03:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.935 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116471 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8241152~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.485692364000002, 2024-05-11T23:03:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045867374000000002, 2024-05-11T23:03:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116471 2024-05-11T23:03:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:21.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379284+0000", 2024-05-11T23:03:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.53e-07 2024-05-11T23:03:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379285+0000", 2024-05-11T23:03:21.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.223e-06 2024-05-11T23:03:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379288+0000", 2024-05-11T23:03:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5379999999999998e-06 2024-05-11T23:03:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379291+0000", 2024-05-11T23:03:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6699999999999998e-06 2024-05-11T23:03:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425055+0000", 2024-05-11T23:03:21.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045763497 2024-05-11T23:03:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425099+0000", 2024-05-11T23:03:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3899000000000001e-05 2024-05-11T23:03:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425151+0000", 2024-05-11T23:03:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2293999999999998e-05 2024-05-11T23:03:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116472 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8282112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.485681364999998, 2024-05-11T23:03:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045973778999999999, 2024-05-11T23:03:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116472 2024-05-11T23:03:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:21.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381226+0000", 2024-05-11T23:03:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001931627 2024-05-11T23:03:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381231+0000", 2024-05-11T23:03:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8679999999999998e-06 2024-05-11T23:03:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381242+0000", 2024-05-11T23:03:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0702e-05 2024-05-11T23:03:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381249+0000", 2024-05-11T23:03:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9800000000000001e-06 2024-05-11T23:03:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425175+0000", 2024-05-11T23:03:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043926805999999999 2024-05-11T23:03:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425215+0000", 2024-05-11T23:03:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9737999999999998e-05 2024-05-11T23:03:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425268+0000", 2024-05-11T23:03:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3057999999999999e-05 2024-05-11T23:03:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.943 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116482 3.1b 3:db6e8843:::rbd_object_map.111e67f27da3.0000000000000028:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.483712822000001, 2024-05-11T23:03:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046526076, 2024-05-11T23:03:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116482 2024-05-11T23:03:21.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381264+0000", 2024-05-11T23:03:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5300000000000004e-07 2024-05-11T23:03:21.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381267+0000", 2024-05-11T23:03:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1999999999999999e-06 2024-05-11T23:03:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381274+0000", 2024-05-11T23:03:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7109999999999995e-06 2024-05-11T23:03:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381285+0000", 2024-05-11T23:03:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0856000000000001e-05 2024-05-11T23:03:21.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425292+0000", 2024-05-11T23:03:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044006376 2024-05-11T23:03:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425337+0000", 2024-05-11T23:03:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4907999999999998e-05 2024-05-11T23:03:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:03:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425507+0000", 2024-05-11T23:03:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017029999999999999 2024-05-11T23:03:21.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427668+0000", 2024-05-11T23:03:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021612430000000002 2024-05-11T23:03:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427683+0000", 2024-05-11T23:03:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5016e-05 2024-05-11T23:03:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427742+0000", 2024-05-11T23:03:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8811999999999999e-05 2024-05-11T23:03:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:21.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427789+0000", 2024-05-11T23:03:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7101000000000002e-05 2024-05-11T23:03:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:21.949 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:21.949 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:21.950 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:03:21.994 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:21.994+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:22.056 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:22.056 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:22.056 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:22.056 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:22.056 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:22.056+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:22.064 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:03:22.157 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:22.157+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:22.209 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:22.209 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:22.209 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:22.210 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:22.210 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:22.218 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:03:22.233 INFO:teuthology.orchestra.run.smithi005.stderr:set 3.4 pg_upmap_items mapping to [3->6,4->2] 2024-05-11T23:03:22.261 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:22.261+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:22.333 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:22.333 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:22.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:22.333 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:22.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.333 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267820 3.16 3:6b963329:::rbd_data.111ec06c817a.000000000000000a:33 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:22.333 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:03:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3897871649999995, 2024-05-11T23:03:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050141218000000001, 2024-05-11T23:03:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267820 2024-05-11T23:03:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:03:22.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:03:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942754+0000", 2024-05-11T23:03:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8559000000000003e-05 2024-05-11T23:03:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942756+0000", 2024-05-11T23:03:22.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.604e-06 2024-05-11T23:03:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942763+0000", 2024-05-11T23:03:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7939999999999997e-06 2024-05-11T23:03:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942768+0000", 2024-05-11T23:03:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4410000000000001e-06 2024-05-11T23:03:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992601+0000", 2024-05-11T23:03:22.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049832754999999999 2024-05-11T23:03:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992687+0000", 2024-05-11T23:03:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5605000000000005e-05 2024-05-11T23:03:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992796+0000", 2024-05-11T23:03:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010946 2024-05-11T23:03:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.337 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267852 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:03:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3896659610000004, 2024-05-11T23:03:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050212600000000003, 2024-05-11T23:03:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267852 2024-05-11T23:03:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:03:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:03:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943228+0000", 2024-05-11T23:03:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00045180600000000002 2024-05-11T23:03:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943232+0000", 2024-05-11T23:03:22.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8399999999999997e-06 2024-05-11T23:03:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943240+0000", 2024-05-11T23:03:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8900000000000007e-06 2024-05-11T23:03:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943246+0000", 2024-05-11T23:03:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6239999999999996e-06 2024-05-11T23:03:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992828+0000", 2024-05-11T23:03:22.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049581247000000002 2024-05-11T23:03:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992905+0000", 2024-05-11T23:03:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7656000000000003e-05 2024-05-11T23:03:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992989+0000", 2024-05-11T23:03:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3536999999999997e-05 2024-05-11T23:03:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267854 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1134592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:03:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3891528199999996, 2024-05-11T23:03:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050077425000000002, 2024-05-11T23:03:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267854 2024-05-11T23:03:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:03:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:03:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943290+0000", 2024-05-11T23:03:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2300000000000002e-07 2024-05-11T23:03:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943291+0000", 2024-05-11T23:03:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6190000000000001e-06 2024-05-11T23:03:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943296+0000", 2024-05-11T23:03:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2889999999999999e-06 2024-05-11T23:03:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943299+0000", 2024-05-11T23:03:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1030000000000002e-06 2024-05-11T23:03:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993205+0000", 2024-05-11T23:03:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049906405000000001 2024-05-11T23:03:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993285+0000", 2024-05-11T23:03:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9752999999999993e-05 2024-05-11T23:03:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993367+0000", 2024-05-11T23:03:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1933000000000003e-05 2024-05-11T23:03:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267855 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1171456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:03:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3891358779999994, 2024-05-11T23:03:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050250346000000001, 2024-05-11T23:03:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267855 2024-05-11T23:03:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:03:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:03:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945226+0000", 2024-05-11T23:03:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001919235 2024-05-11T23:03:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945230+0000", 2024-05-11T23:03:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9580000000000001e-06 2024-05-11T23:03:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945238+0000", 2024-05-11T23:03:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1040000000000007e-06 2024-05-11T23:03:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945244+0000", 2024-05-11T23:03:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6919999999999996e-06 2024-05-11T23:03:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993399+0000", 2024-05-11T23:03:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048154651 2024-05-11T23:03:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993472+0000", 2024-05-11T23:03:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2780000000000005e-05 2024-05-11T23:03:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993557+0000", 2024-05-11T23:03:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4926000000000006e-05 2024-05-11T23:03:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267865 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1540096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3869842519999995, 2024-05-11T23:03:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050054409000000001, 2024-05-11T23:03:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267865 2024-05-11T23:03:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6600000000000002e-07 2024-05-11T23:03:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945461+0000", 2024-05-11T23:03:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2670000000000001e-06 2024-05-11T23:03:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945466+0000", 2024-05-11T23:03:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7200000000000003e-06 2024-05-11T23:03:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945471+0000", 2024-05-11T23:03:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9439999999999996e-06 2024-05-11T23:03:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995337+0000", 2024-05-11T23:03:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049865415000000003 2024-05-11T23:03:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995413+0000", 2024-05-11T23:03:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5989999999999996e-05 2024-05-11T23:03:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995512+0000", 2024-05-11T23:03:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9706999999999999e-05 2024-05-11T23:03:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268017 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947359+0000", 2024-05-11T23:03:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.385083367, 2024-05-11T23:03:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050017655000000001, 2024-05-11T23:03:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268017 2024-05-11T23:03:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947359+0000", 2024-05-11T23:03:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947359+0000", 2024-05-11T23:03:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947359+0000", 2024-05-11T23:03:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5000000000000003e-07 2024-05-11T23:03:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947411+0000", 2024-05-11T23:03:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1158e-05 2024-05-11T23:03:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947418+0000", 2024-05-11T23:03:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8269999999999998e-06 2024-05-11T23:03:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947424+0000", 2024-05-11T23:03:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6529999999999999e-06 2024-05-11T23:03:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997179+0000", 2024-05-11T23:03:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049754675999999998 2024-05-11T23:03:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997279+0000", 2024-05-11T23:03:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9789000000000003e-05 2024-05-11T23:03:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997377+0000", 2024-05-11T23:03:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8002000000000001e-05 2024-05-11T23:03:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268018 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 872448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:03:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3850081870000004, 2024-05-11T23:03:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050170927999999997, 2024-05-11T23:03:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268018 2024-05-11T23:03:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:03:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:03:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947435+0000", 2024-05-11T23:03:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3099999999999997e-07 2024-05-11T23:03:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947437+0000", 2024-05-11T23:03:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6340000000000001e-06 2024-05-11T23:03:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947443+0000", 2024-05-11T23:03:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7810000000000002e-06 2024-05-11T23:03:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947448+0000", 2024-05-11T23:03:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9679999999999997e-06 2024-05-11T23:03:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997418+0000", 2024-05-11T23:03:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049969587000000003 2024-05-11T23:03:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997504+0000", 2024-05-11T23:03:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6426000000000002e-05 2024-05-11T23:03:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997605+0000", 2024-05-11T23:03:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100901 2024-05-11T23:03:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268019 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3849876289999994, 2024-05-11T23:03:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050285074999999999, 2024-05-11T23:03:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268019 2024-05-11T23:03:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0999999999999999e-07 2024-05-11T23:03:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947457+0000", 2024-05-11T23:03:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-05-11T23:03:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947462+0000", 2024-05-11T23:03:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2410000000000003e-06 2024-05-11T23:03:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947467+0000", 2024-05-11T23:03:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8339999999999998e-06 2024-05-11T23:03:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997639+0000", 2024-05-11T23:03:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050171697000000001 2024-05-11T23:03:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997685+0000", 2024-05-11T23:03:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6146000000000002e-05 2024-05-11T23:03:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997740+0000", 2024-05-11T23:03:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4737000000000001e-05 2024-05-11T23:03:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.365 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268020 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:03:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3849693839999997, 2024-05-11T23:03:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050504089000000002, 2024-05-11T23:03:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268020 2024-05-11T23:03:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.366 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:03:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:03:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947591+0000", 2024-05-11T23:03:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011805 2024-05-11T23:03:22.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947594+0000", 2024-05-11T23:03:22.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1549999999999999e-06 2024-05-11T23:03:22.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947602+0000", 2024-05-11T23:03:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.174e-06 2024-05-11T23:03:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947608+0000", 2024-05-11T23:03:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.263e-06 2024-05-11T23:03:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997775+0000", 2024-05-11T23:03:22.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050166980999999999 2024-05-11T23:03:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997875+0000", 2024-05-11T23:03:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9844000000000002e-05 2024-05-11T23:03:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997977+0000", 2024-05-11T23:03:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101622 2024-05-11T23:03:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.371 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268021 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:03:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3848239239999991, 2024-05-11T23:03:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050498874999999999, 2024-05-11T23:03:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268021 2024-05-11T23:03:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:03:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:03:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947695+0000", 2024-05-11T23:03:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6247000000000004e-05 2024-05-11T23:03:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947698+0000", 2024-05-11T23:03:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9199999999999997e-06 2024-05-11T23:03:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947706+0000", 2024-05-11T23:03:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:03:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947711+0000", 2024-05-11T23:03:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6720000000000001e-06 2024-05-11T23:03:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998017+0000", 2024-05-11T23:03:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050305470999999997 2024-05-11T23:03:22.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998060+0000", 2024-05-11T23:03:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3374000000000002e-05 2024-05-11T23:03:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998117+0000", 2024-05-11T23:03:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6971000000000001e-05 2024-05-11T23:03:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268022 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:03:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3847208880000004, 2024-05-11T23:03:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050731361000000003, 2024-05-11T23:03:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.377 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268022 2024-05-11T23:03:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:03:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:03:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947722+0000", 2024-05-11T23:03:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8999999999999996e-07 2024-05-11T23:03:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947724+0000", 2024-05-11T23:03:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1639999999999999e-06 2024-05-11T23:03:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947730+0000", 2024-05-11T23:03:22.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6099999999999997e-06 2024-05-11T23:03:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947735+0000", 2024-05-11T23:03:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7189999999999998e-06 2024-05-11T23:03:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998140+0000", 2024-05-11T23:03:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050404254000000003 2024-05-11T23:03:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998239+0000", 2024-05-11T23:03:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9192999999999997e-05 2024-05-11T23:03:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998453+0000", 2024-05-11T23:03:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021383099999999999 2024-05-11T23:03:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.381 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268023 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1069056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:03:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3846986230000002, 2024-05-11T23:03:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050874509999999998, 2024-05-11T23:03:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268023 2024-05-11T23:03:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.382 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:03:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:03:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948934+0000", 2024-05-11T23:03:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011903339999999999 2024-05-11T23:03:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948938+0000", 2024-05-11T23:03:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1330000000000001e-06 2024-05-11T23:03:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948949+0000", 2024-05-11T23:03:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0872e-05 2024-05-11T23:03:22.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948956+0000", 2024-05-11T23:03:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.131e-06 2024-05-11T23:03:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998494+0000", 2024-05-11T23:03:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049537678000000002 2024-05-11T23:03:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998551+0000", 2024-05-11T23:03:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7256000000000002e-05 2024-05-11T23:03:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998618+0000", 2024-05-11T23:03:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7106000000000004e-05 2024-05-11T23:03:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.386 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268026 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1179648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:03:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3834215699999994, 2024-05-11T23:03:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050107977999999997, 2024-05-11T23:03:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268026 2024-05-11T23:03:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:03:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:03:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949029+0000", 2024-05-11T23:03:22.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2220000000000002e-06 2024-05-11T23:03:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949041+0000", 2024-05-11T23:03:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.241e-05 2024-05-11T23:03:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949049+0000", 2024-05-11T23:03:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.072e-06 2024-05-11T23:03:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949056+0000", 2024-05-11T23:03:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1380000000000001e-06 2024-05-11T23:03:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999016+0000", 2024-05-11T23:03:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049959212000000003 2024-05-11T23:03:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999067+0000", 2024-05-11T23:03:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1598000000000003e-05 2024-05-11T23:03:22.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999129+0000", 2024-05-11T23:03:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1326000000000002e-05 2024-05-11T23:03:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268027 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1216512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:22.391 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3833774200000004, 2024-05-11T23:03:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050189865, 2024-05-11T23:03:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268027 2024-05-11T23:03:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-05-11T23:03:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949068+0000", 2024-05-11T23:03:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.26e-06 2024-05-11T23:03:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949074+0000", 2024-05-11T23:03:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7050000000000002e-06 2024-05-11T23:03:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949080+0000", 2024-05-11T23:03:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3689999999999998e-06 2024-05-11T23:03:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999152+0000", 2024-05-11T23:03:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050072653000000002 2024-05-11T23:03:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999197+0000", 2024-05-11T23:03:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4972999999999998e-05 2024-05-11T23:03:22.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999255+0000", 2024-05-11T23:03:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7425e-05 2024-05-11T23:03:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268028 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1253376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:03:22.396 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3833557600000006, 2024-05-11T23:03:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050280017000000003, 2024-05-11T23:03:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268028 2024-05-11T23:03:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:03:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:03:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949087+0000", 2024-05-11T23:03:22.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:03:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949089+0000", 2024-05-11T23:03:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8950000000000001e-06 2024-05-11T23:03:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949095+0000", 2024-05-11T23:03:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7540000000000003e-06 2024-05-11T23:03:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949114+0000", 2024-05-11T23:03:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9738e-05 2024-05-11T23:03:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999278+0000", 2024-05-11T23:03:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050163368999999999 2024-05-11T23:03:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999319+0000", 2024-05-11T23:03:22.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0979000000000002e-05 2024-05-11T23:03:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999366+0000", 2024-05-11T23:03:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7879e-05 2024-05-11T23:03:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268029 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1290240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:22.401 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3833156520000003, 2024-05-11T23:03:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050344904000000003, 2024-05-11T23:03:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268029 2024-05-11T23:03:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:03:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949130+0000", 2024-05-11T23:03:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-05-11T23:03:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949139+0000", 2024-05-11T23:03:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9800000000000004e-06 2024-05-11T23:03:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949145+0000", 2024-05-11T23:03:22.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0510000000000002e-06 2024-05-11T23:03:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999387+0000", 2024-05-11T23:03:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050242167999999997 2024-05-11T23:03:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999427+0000", 2024-05-11T23:03:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9566e-05 2024-05-11T23:03:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999471+0000", 2024-05-11T23:03:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4914999999999999e-05 2024-05-11T23:03:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268030 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1327104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3832903680000008, 2024-05-11T23:03:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050447394, 2024-05-11T23:03:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.406 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268030 2024-05-11T23:03:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0500000000000004e-07 2024-05-11T23:03:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949155+0000", 2024-05-11T23:03:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3819999999999998e-06 2024-05-11T23:03:22.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949162+0000", 2024-05-11T23:03:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9609999999999999e-06 2024-05-11T23:03:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949188+0000", 2024-05-11T23:03:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5877e-05 2024-05-11T23:03:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999494+0000", 2024-05-11T23:03:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050306265000000003 2024-05-11T23:03:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999534+0000", 2024-05-11T23:03:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0447000000000001e-05 2024-05-11T23:03:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999599+0000", 2024-05-11T23:03:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4956999999999995e-05 2024-05-11T23:03:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.410 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268031 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1363968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:03:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3832452679999996, 2024-05-11T23:03:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050511525000000002, 2024-05-11T23:03:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268031 2024-05-11T23:03:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:03:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:03:22.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949198+0000", 2024-05-11T23:03:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-05-11T23:03:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949200+0000", 2024-05-11T23:03:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-05-11T23:03:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949208+0000", 2024-05-11T23:03:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2840000000000001e-06 2024-05-11T23:03:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949214+0000", 2024-05-11T23:03:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9869999999999996e-06 2024-05-11T23:03:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999619+0000", 2024-05-11T23:03:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050405278999999997 2024-05-11T23:03:22.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999660+0000", 2024-05-11T23:03:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1427999999999997e-05 2024-05-11T23:03:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999708+0000", 2024-05-11T23:03:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8229999999999997e-05 2024-05-11T23:03:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268032 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1400832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:03:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3832156480000002, 2024-05-11T23:03:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050611863999999999, 2024-05-11T23:03:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268032 2024-05-11T23:03:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.416 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:03:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:03:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949230+0000", 2024-05-11T23:03:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.371e-06 2024-05-11T23:03:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949233+0000", 2024-05-11T23:03:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6599999999999999e-06 2024-05-11T23:03:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949241+0000", 2024-05-11T23:03:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8939999999999993e-06 2024-05-11T23:03:22.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949246+0000", 2024-05-11T23:03:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6590000000000004e-06 2024-05-11T23:03:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999733+0000", 2024-05-11T23:03:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050487268000000002 2024-05-11T23:03:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999784+0000", 2024-05-11T23:03:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0562000000000001e-05 2024-05-11T23:03:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999838+0000", 2024-05-11T23:03:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000002e-05 2024-05-11T23:03:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268033 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1437696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:22.420 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:03:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.3831885209999992, 2024-05-11T23:03:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050697157999999999, 2024-05-11T23:03:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268033 2024-05-11T23:03:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:03:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:03:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950862+0000", 2024-05-11T23:03:22.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001607975 2024-05-11T23:03:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950866+0000", 2024-05-11T23:03:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7770000000000002e-06 2024-05-11T23:03:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950878+0000", 2024-05-11T23:03:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1421e-05 2024-05-11T23:03:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950885+0000", 2024-05-11T23:03:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-05-11T23:03:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999859+0000", 2024-05-11T23:03:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048973199000000002 2024-05-11T23:03:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999901+0000", 2024-05-11T23:03:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1943000000000001e-05 2024-05-11T23:03:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999951+0000", 2024-05-11T23:03:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0243e-05 2024-05-11T23:03:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.425 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.425 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:22.427 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:03:22.427 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:22.426+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:22.528 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:22.528+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:22.574 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:22.574 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:22.574 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:22.574 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:22.582 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:03:22.629 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:22.629+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:22.727 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:22.727 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:22.728 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:22.728 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:22.728 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:22.730 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:22.730+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:22.736 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:03:22.831 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:22.831+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:22.842 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:22.842 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.187820614000003, 2024-05-11T23:03:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:03:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.843 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:03:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:03:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:03:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:03:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:03:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:03:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:03:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:03:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:03:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:03:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:03:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:03:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:03:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:03:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:22.847 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.184439424999994, 2024-05-11T23:03:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:03:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:03:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:03:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:03:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:03:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:03:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:03:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:03:22.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:03:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:03:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:03:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:03:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:03:22.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:03:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.184420717999998, 2024-05-11T23:03:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:03:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.852 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:03:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:22.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:03:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:03:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:03:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:03:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:03:22.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:03:22.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:03:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:03:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:03:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:03:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:03:22.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:03:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:03:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:03:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.857 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.184402575000007, 2024-05-11T23:03:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:03:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:03:22.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:22.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:03:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:03:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:03:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:03:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:03:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:03:22.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:03:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:03:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:03:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:03:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:03:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:03:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:03:22.861 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.184375087999996, 2024-05-11T23:03:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:03:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.862 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:03:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:03:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:03:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:03:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:03:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:03:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:03:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:03:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:03:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:03:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:03:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:03:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:03:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:03:22.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:03:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.184354560000003, 2024-05-11T23:03:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:03:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.866 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:03:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:03:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:03:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:03:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:03:22.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:03:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:03:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:03:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:03:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:03:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:03:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:03:22.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:03:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:03:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.184334785000004, 2024-05-11T23:03:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:03:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:03:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:22.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:03:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:03:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:03:22.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:03:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:03:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:03:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:03:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:03:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:03:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:03:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:03:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:03:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:03:22.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:03:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.184315979000004, 2024-05-11T23:03:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:03:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.875 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:03:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:03:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:03:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:03:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:03:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:03:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:03:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:03:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:03:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:03:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:03:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:03:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:03:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:03:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:03:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.184297056000005, 2024-05-11T23:03:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:03:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.879 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:03:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:03:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:03:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:03:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:03:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:03:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:03:22.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:03:22.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:03:22.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:03:22.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:03:22.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:03:22.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:03:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:03:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.184278394000003, 2024-05-11T23:03:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:03:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.883 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:03:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:03:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:03:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:03:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:03:22.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:03:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:03:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:03:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:03:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:03:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:03:22.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:03:22.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:03:22.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:03:22.887 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.887 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.887 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.887 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:22.888 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:22.888 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.183933232000001, 2024-05-11T23:03:22.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:03:22.888 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.888 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.888 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.888 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.888 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:03:22.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.888 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:03:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:03:22.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:03:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:03:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:03:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:03:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:03:22.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:03:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:03:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:03:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:03:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:03:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:03:22.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:03:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.183907681999997, 2024-05-11T23:03:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:03:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:03:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:03:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:03:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:03:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:03:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:03:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:03:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:03:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:03:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:03:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:03:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:03:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:03:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:03:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.183825424000005, 2024-05-11T23:03:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:03:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:03:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:03:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:03:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:03:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:03:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:03:22.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:03:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:03:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:03:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:03:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:03:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:03:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:03:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:03:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.183802976999999, 2024-05-11T23:03:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:03:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:03:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:03:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:03:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:03:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:03:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:03:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:03:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:03:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:03:22.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:03:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:03:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:03:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:03:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:03:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:03:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.904 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.183783519000002, 2024-05-11T23:03:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:03:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:03:22.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:22.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:03:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:03:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:03:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:03:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:03:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:03:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:03:22.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:03:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:03:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:03:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:03:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:03:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:03:22.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.183763459000005, 2024-05-11T23:03:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:03:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:03:22.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:03:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:03:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:03:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:03:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:03:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:03:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:03:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:03:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:03:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:03:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:03:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:03:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.183744188999995, 2024-05-11T23:03:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:03:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.913 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:03:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:03:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:03:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:03:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:03:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:03:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:03:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:03:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:03:22.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:03:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:03:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:03:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:03:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:03:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.917 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.183198122999997, 2024-05-11T23:03:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:03:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.918 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:03:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:03:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:03:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:03:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:03:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:03:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:03:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:03:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:03:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:03:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:03:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:03:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:03:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:03:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:03:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 90.183166338999996, 2024-05-11T23:03:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:03:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.922 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:03:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:22.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:03:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:03:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:03:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:03:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:03:22.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:03:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:03:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:03:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:03:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:03:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:03:22.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:03:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:03:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:03:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 80.745920874999996, 2024-05-11T23:03:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:03:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:22.926 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:03:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:22.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:03:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:03:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:03:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:03:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:03:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:03:22.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:03:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:03:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:03:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:03:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:03:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:03:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:03:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:03:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:03:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:03:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:03:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:03:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:03:22.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:03:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:03:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:03:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:03:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:22.931 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:22.931 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:22.932 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:03:22.933 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:22.933+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:22.934 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:03:23.034 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:23.034+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:23.038 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:23.038 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:23.039 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:23.039 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:23.047 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:03:23.073 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:23.090 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:23.090 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:23.107 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:23.107 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:23.124 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:03:23.125 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:03:23.135 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:23.135+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:23.142 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:23.142 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:23.153 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:23.153 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:23.153 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:23.153 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:23.153 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:23.160 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:23.160 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:23.163 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:03:23.178 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:23.178 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:23.196 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:23.196 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:23.246 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:23.246+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:23.258 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-11T23:03:23.312 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:23.312 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:23.324 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:03:23.368 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:23.368+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:23.437 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:23.437+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:23.466 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:23.466 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:23.475 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:03:23.538 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:23.538+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:23.626 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:23.626 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:23.635 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:03:23.679 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:23.679+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:23.739 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:23.739+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:23.744 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:23.744 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:23.753 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:03:23.840 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:23.840+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:23.903 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:23.903 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:23.911 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:03:23.941 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:23.940+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:24.041 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:24.041+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:24.061 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:24.061 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:24.069 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:03:24.154 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:24.154+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:24.220 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:24.221 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:24.229 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:03:24.243 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:24.243+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:24.248 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-05-11T23:03:24.344 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:24.343+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:24.383 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:24.383 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:24.392 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:03:24.477 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:24.477+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:24.542 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:24.542 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:24.546 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:24.546+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:24.551 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:03:24.647 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:24.646+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:24.701 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:24.701 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:24.709 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:03:24.753 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:24.753+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:24.818 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:24.819 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:24.827 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:03:24.868 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:24.868+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:24.950 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:24.950+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:24.976 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:24.976 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:24.986 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:03:25.052 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:25.052+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:25.096 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:25.096 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:25.105 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:03:25.153 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:25.153+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:25.207 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:03:25.222 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:25.222 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:25.231 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:03:25.253 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:25.253+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:25.345 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:25.346 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:25.353 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:25.353 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:25.355 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:03:25.355 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:25.354+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:25.372 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:25.372 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:25.393 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:25.393 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:25.410 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:03:25.410 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:03:25.427 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:25.428 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:25.445 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:25.445 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:25.455 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:25.455+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:25.462 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:25.462 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:25.462 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:25.462 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:25.462 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.463 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116446 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8273920~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:25.463 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:25.463 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.085885685999997, 2024-05-11T23:03:25.463 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045535299000000001, 2024-05-11T23:03:25.463 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.463 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.463 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.463 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.463 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.463 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.463 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116446 2024-05-11T23:03:25.463 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.463 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.463 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.463 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.463 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:25.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:25.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375547+0000", 2024-05-11T23:03:25.464 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4000000000000001e-07 2024-05-11T23:03:25.464 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.464 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.464 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.464 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375549+0000", 2024-05-11T23:03:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.368e-06 2024-05-11T23:03:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375556+0000", 2024-05-11T23:03:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1840000000000004e-06 2024-05-11T23:03:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375560+0000", 2024-05-11T23:03:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.741e-06 2024-05-11T23:03:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420896+0000", 2024-05-11T23:03:25.465 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045335435 2024-05-11T23:03:25.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420987+0000", 2024-05-11T23:03:25.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1033999999999996e-05 2024-05-11T23:03:25.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.466 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.466 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.466 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421082+0000", 2024-05-11T23:03:25.466 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4896999999999996e-05 2024-05-11T23:03:25.466 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.466 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.466 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.466 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.467 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116447 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6316032~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:25.467 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:25.467 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.085864176000001, 2024-05-11T23:03:25.467 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045745830000000001, 2024-05-11T23:03:25.467 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.467 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.467 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.467 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.467 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.467 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.467 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116447 2024-05-11T23:03:25.467 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.467 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.467 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.467 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:25.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:25.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.468 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:25.468 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.58e-07 2024-05-11T23:03:25.468 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.468 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.468 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375570+0000", 2024-05-11T23:03:25.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:03:25.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375576+0000", 2024-05-11T23:03:25.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7939999999999999e-06 2024-05-11T23:03:25.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375581+0000", 2024-05-11T23:03:25.469 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9280000000000001e-06 2024-05-11T23:03:25.469 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.469 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.469 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.469 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421127+0000", 2024-05-11T23:03:25.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045545785999999998 2024-05-11T23:03:25.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421214+0000", 2024-05-11T23:03:25.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7039000000000005e-05 2024-05-11T23:03:25.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.470 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.470 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.470 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421314+0000", 2024-05-11T23:03:25.470 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9772999999999995e-05 2024-05-11T23:03:25.470 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.470 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.470 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.470 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.471 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116448 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8314880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:25.471 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:25.471 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.085845102, 2024-05-11T23:03:25.471 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045941753000000002, 2024-05-11T23:03:25.471 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.471 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.471 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.471 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.471 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.471 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.471 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116448 2024-05-11T23:03:25.471 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.471 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.471 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:25.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:25.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.472 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:25.472 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.77e-07 2024-05-11T23:03:25.472 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.472 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.472 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375589+0000", 2024-05-11T23:03:25.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.091e-06 2024-05-11T23:03:25.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375596+0000", 2024-05-11T23:03:25.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9179999999999997e-06 2024-05-11T23:03:25.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375601+0000", 2024-05-11T23:03:25.473 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0030000000000002e-06 2024-05-11T23:03:25.473 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.473 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.473 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.473 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421363+0000", 2024-05-11T23:03:25.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045761352999999998 2024-05-11T23:03:25.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421440+0000", 2024-05-11T23:03:25.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7585999999999999e-05 2024-05-11T23:03:25.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.474 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.474 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.474 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421529+0000", 2024-05-11T23:03:25.474 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8424999999999994e-05 2024-05-11T23:03:25.474 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.474 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.474 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.474 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.475 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116449 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6438912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:25.475 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:25.475 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.085824666999997, 2024-05-11T23:03:25.475 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046082208999999999, 2024-05-11T23:03:25.475 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.475 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.475 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.475 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.475 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.475 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.475 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116449 2024-05-11T23:03:25.475 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.475 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.475 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:25.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:25.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.476 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377225+0000", 2024-05-11T23:03:25.476 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016177030000000001 2024-05-11T23:03:25.476 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.476 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.476 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377229+0000", 2024-05-11T23:03:25.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8e-06 2024-05-11T23:03:25.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377238+0000", 2024-05-11T23:03:25.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2229999999999996e-06 2024-05-11T23:03:25.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.477 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377244+0000", 2024-05-11T23:03:25.477 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4060000000000002e-06 2024-05-11T23:03:25.477 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.477 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.477 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421520+0000", 2024-05-11T23:03:25.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044275222000000003 2024-05-11T23:03:25.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421595+0000", 2024-05-11T23:03:25.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4838000000000004e-05 2024-05-11T23:03:25.478 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.478 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.478 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.478 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421690+0000", 2024-05-11T23:03:25.478 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5017000000000002e-05 2024-05-11T23:03:25.478 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.478 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.478 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.479 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.479 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116458 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7094272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:25.479 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:25.479 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.084010450000001, 2024-05-11T23:03:25.479 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045706877999999999, 2024-05-11T23:03:25.479 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.479 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.479 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.479 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.479 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.479 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.479 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116458 2024-05-11T23:03:25.479 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.480 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:25.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:25.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.480 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.480 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.480 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.480 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:25.480 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9400000000000005e-07 2024-05-11T23:03:25.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377424+0000", 2024-05-11T23:03:25.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1859999999999999e-06 2024-05-11T23:03:25.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377431+0000", 2024-05-11T23:03:25.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4049999999999997e-06 2024-05-11T23:03:25.481 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.481 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.481 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.481 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377436+0000", 2024-05-11T23:03:25.481 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1399999999999999e-06 2024-05-11T23:03:25.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.422945+0000", 2024-05-11T23:03:25.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045509326000000003 2024-05-11T23:03:25.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423036+0000", 2024-05-11T23:03:25.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0649999999999994e-05 2024-05-11T23:03:25.482 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.482 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.482 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423128+0000", 2024-05-11T23:03:25.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2577000000000005e-05 2024-05-11T23:03:25.483 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.483 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.483 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.483 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116459 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7258112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:25.483 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:25.483 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.083989717999998, 2024-05-11T23:03:25.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045831945999999998, 2024-05-11T23:03:25.483 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.483 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.483 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.483 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.484 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.484 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.484 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116459 2024-05-11T23:03:25.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.484 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:25.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:25.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377443+0000", 2024-05-11T23:03:25.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999999e-07 2024-05-11T23:03:25.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377445+0000", 2024-05-11T23:03:25.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0209999999999998e-06 2024-05-11T23:03:25.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377451+0000", 2024-05-11T23:03:25.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.071e-06 2024-05-11T23:03:25.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377456+0000", 2024-05-11T23:03:25.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8289999999999998e-06 2024-05-11T23:03:25.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423155+0000", 2024-05-11T23:03:25.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045699447999999997 2024-05-11T23:03:25.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.486 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423200+0000", 2024-05-11T23:03:25.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5083000000000002e-05 2024-05-11T23:03:25.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423274+0000", 2024-05-11T23:03:25.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4046000000000002e-05 2024-05-11T23:03:25.487 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.487 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.487 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.487 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116460 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7299072~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:25.487 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:25.487 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.083970655999998, 2024-05-11T23:03:25.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046010987000000003, 2024-05-11T23:03:25.487 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.487 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.487 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.488 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.488 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.488 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.488 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116460 2024-05-11T23:03:25.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.488 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:25.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:25.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377462+0000", 2024-05-11T23:03:25.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:03:25.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377464+0000", 2024-05-11T23:03:25.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1600000000000001e-06 2024-05-11T23:03:25.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377469+0000", 2024-05-11T23:03:25.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5430000000000001e-06 2024-05-11T23:03:25.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377474+0000", 2024-05-11T23:03:25.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8509999999999998e-06 2024-05-11T23:03:25.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423312+0000", 2024-05-11T23:03:25.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045838080000000003 2024-05-11T23:03:25.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.490 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.490 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423401+0000", 2024-05-11T23:03:25.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8170000000000005e-05 2024-05-11T23:03:25.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423472+0000", 2024-05-11T23:03:25.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1847000000000005e-05 2024-05-11T23:03:25.491 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.491 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.491 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.491 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116461 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7421952~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:25.491 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:25.491 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.083951886000001, 2024-05-11T23:03:25.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046130749999999998, 2024-05-11T23:03:25.491 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.492 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.492 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.492 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.492 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.492 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.492 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116461 2024-05-11T23:03:25.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.492 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:25.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:25.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:25.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4700000000000002e-07 2024-05-11T23:03:25.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377483+0000", 2024-05-11T23:03:25.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0219999999999999e-06 2024-05-11T23:03:25.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377489+0000", 2024-05-11T23:03:25.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4420000000000003e-06 2024-05-11T23:03:25.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377494+0000", 2024-05-11T23:03:25.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9239999999999998e-06 2024-05-11T23:03:25.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423500+0000", 2024-05-11T23:03:25.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046006054999999997 2024-05-11T23:03:25.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.494 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.494 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423557+0000", 2024-05-11T23:03:25.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7169000000000001e-05 2024-05-11T23:03:25.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423611+0000", 2024-05-11T23:03:25.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3791e-05 2024-05-11T23:03:25.495 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.495 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.495 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.495 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116462 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7462912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:25.496 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:25.496 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.083932267999998, 2024-05-11T23:03:25.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046254666, 2024-05-11T23:03:25.496 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.496 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.496 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.496 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.496 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.496 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.496 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116462 2024-05-11T23:03:25.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.496 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:25.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:25.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:25.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:25.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377502+0000", 2024-05-11T23:03:25.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-05-11T23:03:25.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377508+0000", 2024-05-11T23:03:25.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4500000000000003e-06 2024-05-11T23:03:25.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377512+0000", 2024-05-11T23:03:25.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7700000000000001e-06 2024-05-11T23:03:25.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.498 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423649+0000", 2024-05-11T23:03:25.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046136859000000002 2024-05-11T23:03:25.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423704+0000", 2024-05-11T23:03:25.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4938000000000002e-05 2024-05-11T23:03:25.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423754+0000", 2024-05-11T23:03:25.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0145999999999997e-05 2024-05-11T23:03:25.499 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.499 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.499 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.499 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116463 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7585792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:25.500 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:25.500 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.083913842000001, 2024-05-11T23:03:25.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361763, 2024-05-11T23:03:25.500 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.500 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.500 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.500 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.500 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.500 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.500 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116463 2024-05-11T23:03:25.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.500 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:25.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:25.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377519+0000", 2024-05-11T23:03:25.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2600000000000002e-07 2024-05-11T23:03:25.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377521+0000", 2024-05-11T23:03:25.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1710000000000001e-06 2024-05-11T23:03:25.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377527+0000", 2024-05-11T23:03:25.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9649999999999996e-06 2024-05-11T23:03:25.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377532+0000", 2024-05-11T23:03:25.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.78e-06 2024-05-11T23:03:25.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.502 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.502 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423779+0000", 2024-05-11T23:03:25.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046247706 2024-05-11T23:03:25.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423829+0000", 2024-05-11T23:03:25.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9188999999999999e-05 2024-05-11T23:03:25.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423880+0000", 2024-05-11T23:03:25.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1425999999999999e-05 2024-05-11T23:03:25.503 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.503 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.503 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.503 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116464 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7749632~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:25.504 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:25.504 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.083894606000001, 2024-05-11T23:03:25.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046491719000000001, 2024-05-11T23:03:25.504 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.504 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.504 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.504 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.504 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.504 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.504 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116464 2024-05-11T23:03:25.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.504 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:25.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:25.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377538+0000", 2024-05-11T23:03:25.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4499999999999998e-07 2024-05-11T23:03:25.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377540+0000", 2024-05-11T23:03:25.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2340000000000001e-06 2024-05-11T23:03:25.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377546+0000", 2024-05-11T23:03:25.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8289999999999996e-06 2024-05-11T23:03:25.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377551+0000", 2024-05-11T23:03:25.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8559999999999998e-06 2024-05-11T23:03:25.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.506 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.506 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423913+0000", 2024-05-11T23:03:25.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361885999999998 2024-05-11T23:03:25.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423958+0000", 2024-05-11T23:03:25.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5513999999999998e-05 2024-05-11T23:03:25.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424029+0000", 2024-05-11T23:03:25.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1055000000000003e-05 2024-05-11T23:03:25.507 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.507 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.507 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.507 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116465 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7626752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:25.508 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:25.508 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.083875458000001, 2024-05-11T23:03:25.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046600057, 2024-05-11T23:03:25.508 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.508 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.508 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.508 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.508 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.508 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.508 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116465 2024-05-11T23:03:25.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.508 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.508 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.508 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.508 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:25.508 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:25.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:25.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3599999999999999e-07 2024-05-11T23:03:25.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377559+0000", 2024-05-11T23:03:25.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9640000000000002e-06 2024-05-11T23:03:25.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377572+0000", 2024-05-11T23:03:25.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3188e-05 2024-05-11T23:03:25.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377578+0000", 2024-05-11T23:03:25.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.942e-06 2024-05-11T23:03:25.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424058+0000", 2024-05-11T23:03:25.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046480021000000003 2024-05-11T23:03:25.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424103+0000", 2024-05-11T23:03:25.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5021999999999997e-05 2024-05-11T23:03:25.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424157+0000", 2024-05-11T23:03:25.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3483999999999998e-05 2024-05-11T23:03:25.511 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.511 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.511 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.511 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116466 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7790592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:25.512 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:25.512 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.083846876000003, 2024-05-11T23:03:25.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046743741999999998, 2024-05-11T23:03:25.512 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.512 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.512 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.512 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.512 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.512 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.512 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116466 2024-05-11T23:03:25.512 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.512 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.512 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.512 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.512 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:25.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:25.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377586+0000", 2024-05-11T23:03:25.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:03:25.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377588+0000", 2024-05-11T23:03:25.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3039999999999999e-06 2024-05-11T23:03:25.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377594+0000", 2024-05-11T23:03:25.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-05-11T23:03:25.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377600+0000", 2024-05-11T23:03:25.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7699999999999998e-06 2024-05-11T23:03:25.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424185+0000", 2024-05-11T23:03:25.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046585891999999997 2024-05-11T23:03:25.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424235+0000", 2024-05-11T23:03:25.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9356e-05 2024-05-11T23:03:25.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424329+0000", 2024-05-11T23:03:25.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4036999999999994e-05 2024-05-11T23:03:25.515 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.515 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.515 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.516 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116467 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7954432~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:25.516 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:25.516 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.083826588999997, 2024-05-11T23:03:25.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046930134999999998, 2024-05-11T23:03:25.516 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.516 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.516 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.516 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.516 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116467 2024-05-11T23:03:25.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.516 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:25.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:25.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379161+0000", 2024-05-11T23:03:25.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015555009999999999 2024-05-11T23:03:25.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379166+0000", 2024-05-11T23:03:25.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6689999999999997e-06 2024-05-11T23:03:25.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379176+0000", 2024-05-11T23:03:25.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0549e-05 2024-05-11T23:03:25.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379183+0000", 2024-05-11T23:03:25.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1199999999999996e-06 2024-05-11T23:03:25.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424359+0000", 2024-05-11T23:03:25.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045175949999999999 2024-05-11T23:03:25.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424433+0000", 2024-05-11T23:03:25.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3824000000000003e-05 2024-05-11T23:03:25.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424536+0000", 2024-05-11T23:03:25.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010252200000000001 2024-05-11T23:03:25.519 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.519 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.519 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.520 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116468 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7913472~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:25.520 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:25.520 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.082236369999997, 2024-05-11T23:03:25.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045525364999999998, 2024-05-11T23:03:25.520 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.520 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.520 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.520 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.520 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.520 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.520 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116468 2024-05-11T23:03:25.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.520 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:25.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:25.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:25.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3300000000000002e-07 2024-05-11T23:03:25.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379198+0000", 2024-05-11T23:03:25.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2670000000000001e-06 2024-05-11T23:03:25.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379206+0000", 2024-05-11T23:03:25.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7149999999999998e-06 2024-05-11T23:03:25.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379228+0000", 2024-05-11T23:03:25.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1429000000000001e-05 2024-05-11T23:03:25.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424574+0000", 2024-05-11T23:03:25.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045346209999999998 2024-05-11T23:03:25.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424627+0000", 2024-05-11T23:03:25.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3352e-05 2024-05-11T23:03:25.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424721+0000", 2024-05-11T23:03:25.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3858999999999997e-05 2024-05-11T23:03:25.523 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.523 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.523 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.524 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116469 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8077312~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:25.524 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:25.524 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.082186550000003, 2024-05-11T23:03:25.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045636687000000002, 2024-05-11T23:03:25.524 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.524 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.524 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.524 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.524 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.524 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.524 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116469 2024-05-11T23:03:25.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.524 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:25.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:25.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379246+0000", 2024-05-11T23:03:25.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.32e-07 2024-05-11T23:03:25.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379254+0000", 2024-05-11T23:03:25.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.807e-06 2024-05-11T23:03:25.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379261+0000", 2024-05-11T23:03:25.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.985e-06 2024-05-11T23:03:25.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379264+0000", 2024-05-11T23:03:25.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6480000000000001e-06 2024-05-11T23:03:25.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424744+0000", 2024-05-11T23:03:25.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045479638000000003 2024-05-11T23:03:25.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424797+0000", 2024-05-11T23:03:25.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2725000000000001e-05 2024-05-11T23:03:25.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424882+0000", 2024-05-11T23:03:25.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5451999999999994e-05 2024-05-11T23:03:25.527 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.527 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.528 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.528 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116470 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8118272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:25.528 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:25.528 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.082161337999999, 2024-05-11T23:03:25.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045755745, 2024-05-11T23:03:25.528 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.528 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.528 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.528 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.528 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.528 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.528 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116470 2024-05-11T23:03:25.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.528 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3999999999999997e-07 2024-05-11T23:03:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379273+0000", 2024-05-11T23:03:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-05-11T23:03:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379277+0000", 2024-05-11T23:03:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4020000000000004e-06 2024-05-11T23:03:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379280+0000", 2024-05-11T23:03:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-05-11T23:03:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424922+0000", 2024-05-11T23:03:25.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045642598999999999 2024-05-11T23:03:25.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424973+0000", 2024-05-11T23:03:25.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0674999999999998e-05 2024-05-11T23:03:25.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425026+0000", 2024-05-11T23:03:25.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3492000000000002e-05 2024-05-11T23:03:25.531 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.531 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.532 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.532 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116471 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8241152~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:25.532 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:25.532 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.082148555000003, 2024-05-11T23:03:25.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045867374000000002, 2024-05-11T23:03:25.532 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.532 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.532 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.532 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116471 2024-05-11T23:03:25.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.532 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:25.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:25.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379284+0000", 2024-05-11T23:03:25.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.53e-07 2024-05-11T23:03:25.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379285+0000", 2024-05-11T23:03:25.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.223e-06 2024-05-11T23:03:25.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379288+0000", 2024-05-11T23:03:25.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5379999999999998e-06 2024-05-11T23:03:25.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379291+0000", 2024-05-11T23:03:25.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6699999999999998e-06 2024-05-11T23:03:25.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425055+0000", 2024-05-11T23:03:25.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045763497 2024-05-11T23:03:25.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425099+0000", 2024-05-11T23:03:25.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3899000000000001e-05 2024-05-11T23:03:25.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425151+0000", 2024-05-11T23:03:25.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2293999999999998e-05 2024-05-11T23:03:25.535 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.535 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.536 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.536 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116472 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8282112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:25.536 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:25.536 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.082137555999999, 2024-05-11T23:03:25.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045973778999999999, 2024-05-11T23:03:25.536 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.536 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.536 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.536 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.536 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.536 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.536 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116472 2024-05-11T23:03:25.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.536 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:25.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:25.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381226+0000", 2024-05-11T23:03:25.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001931627 2024-05-11T23:03:25.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381231+0000", 2024-05-11T23:03:25.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8679999999999998e-06 2024-05-11T23:03:25.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381242+0000", 2024-05-11T23:03:25.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0702e-05 2024-05-11T23:03:25.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381249+0000", 2024-05-11T23:03:25.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9800000000000001e-06 2024-05-11T23:03:25.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425175+0000", 2024-05-11T23:03:25.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043926805999999999 2024-05-11T23:03:25.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425215+0000", 2024-05-11T23:03:25.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9737999999999998e-05 2024-05-11T23:03:25.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425268+0000", 2024-05-11T23:03:25.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3057999999999999e-05 2024-05-11T23:03:25.539 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.539 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.540 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.540 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116482 3.1b 3:db6e8843:::rbd_object_map.111e67f27da3.0000000000000028:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:25.540 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:25.540 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 50.080169013000003, 2024-05-11T23:03:25.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046526076, 2024-05-11T23:03:25.541 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.541 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.541 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:25.541 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.541 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.541 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.541 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116482 2024-05-11T23:03:25.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.541 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:25.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:25.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381264+0000", 2024-05-11T23:03:25.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5300000000000004e-07 2024-05-11T23:03:25.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381267+0000", 2024-05-11T23:03:25.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1999999999999999e-06 2024-05-11T23:03:25.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381274+0000", 2024-05-11T23:03:25.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7109999999999995e-06 2024-05-11T23:03:25.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381285+0000", 2024-05-11T23:03:25.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0856000000000001e-05 2024-05-11T23:03:25.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425292+0000", 2024-05-11T23:03:25.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044006376 2024-05-11T23:03:25.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425337+0000", 2024-05-11T23:03:25.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4907999999999998e-05 2024-05-11T23:03:25.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:03:25.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425507+0000", 2024-05-11T23:03:25.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017029999999999999 2024-05-11T23:03:25.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:25.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427668+0000", 2024-05-11T23:03:25.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021612430000000002 2024-05-11T23:03:25.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:25.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427683+0000", 2024-05-11T23:03:25.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5016e-05 2024-05-11T23:03:25.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:25.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427742+0000", 2024-05-11T23:03:25.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8811999999999999e-05 2024-05-11T23:03:25.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427789+0000", 2024-05-11T23:03:25.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7101000000000002e-05 2024-05-11T23:03:25.545 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.545 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.545 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.546 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.546 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.546 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:25.546 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:25.547 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:25.547 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:25.548 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:03:25.549 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:25.590 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:25.590+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:25.657 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:25.657+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:25.694 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:25.694 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:25.695 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:25.695 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:25.703 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:03:25.758 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:25.758+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:25.813 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:25.813 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:25.813 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:25.813 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:25.813 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:25.822 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:03:25.859 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:25.859+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:25.936 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:25.936 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:25.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:25.936 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:25.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.936 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267820 3.16 3:6b963329:::rbd_data.111ec06c817a.000000000000000a:33 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:25.936 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:03:25.936 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.992773929, 2024-05-11T23:03:25.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050141218000000001, 2024-05-11T23:03:25.937 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.937 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.937 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.937 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.937 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.937 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.937 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267820 2024-05-11T23:03:25.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.937 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:03:25.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:03:25.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942754+0000", 2024-05-11T23:03:25.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8559000000000003e-05 2024-05-11T23:03:25.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942756+0000", 2024-05-11T23:03:25.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.604e-06 2024-05-11T23:03:25.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942763+0000", 2024-05-11T23:03:25.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7939999999999997e-06 2024-05-11T23:03:25.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942768+0000", 2024-05-11T23:03:25.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4410000000000001e-06 2024-05-11T23:03:25.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992601+0000", 2024-05-11T23:03:25.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049832754999999999 2024-05-11T23:03:25.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992687+0000", 2024-05-11T23:03:25.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5605000000000005e-05 2024-05-11T23:03:25.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.940 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.940 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992796+0000", 2024-05-11T23:03:25.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010946 2024-05-11T23:03:25.940 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.940 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.940 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.940 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267852 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:25.940 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:03:25.941 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.992652724999999, 2024-05-11T23:03:25.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050212600000000003, 2024-05-11T23:03:25.941 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.941 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.941 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.941 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.941 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.941 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.941 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267852 2024-05-11T23:03:25.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.941 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:03:25.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:03:25.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943228+0000", 2024-05-11T23:03:25.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00045180600000000002 2024-05-11T23:03:25.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943232+0000", 2024-05-11T23:03:25.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8399999999999997e-06 2024-05-11T23:03:25.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943240+0000", 2024-05-11T23:03:25.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8900000000000007e-06 2024-05-11T23:03:25.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943246+0000", 2024-05-11T23:03:25.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6239999999999996e-06 2024-05-11T23:03:25.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992828+0000", 2024-05-11T23:03:25.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049581247000000002 2024-05-11T23:03:25.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.944 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.944 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992905+0000", 2024-05-11T23:03:25.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7656000000000003e-05 2024-05-11T23:03:25.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992989+0000", 2024-05-11T23:03:25.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3536999999999997e-05 2024-05-11T23:03:25.945 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.945 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.945 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.945 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267854 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1134592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:25.945 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:03:25.945 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.992139584, 2024-05-11T23:03:25.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050077425000000002, 2024-05-11T23:03:25.945 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.946 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.946 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.946 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.946 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.946 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.946 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267854 2024-05-11T23:03:25.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.946 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:03:25.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:03:25.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943290+0000", 2024-05-11T23:03:25.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2300000000000002e-07 2024-05-11T23:03:25.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943291+0000", 2024-05-11T23:03:25.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6190000000000001e-06 2024-05-11T23:03:25.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943296+0000", 2024-05-11T23:03:25.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2889999999999999e-06 2024-05-11T23:03:25.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943299+0000", 2024-05-11T23:03:25.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1030000000000002e-06 2024-05-11T23:03:25.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993205+0000", 2024-05-11T23:03:25.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049906405000000001 2024-05-11T23:03:25.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.948 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.948 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993285+0000", 2024-05-11T23:03:25.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9752999999999993e-05 2024-05-11T23:03:25.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993367+0000", 2024-05-11T23:03:25.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1933000000000003e-05 2024-05-11T23:03:25.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.949 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.949 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.949 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267855 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1171456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:25.949 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:03:25.949 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.992122642, 2024-05-11T23:03:25.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050250346000000001, 2024-05-11T23:03:25.950 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.950 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.950 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.950 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.950 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267855 2024-05-11T23:03:25.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.950 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:03:25.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:03:25.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945226+0000", 2024-05-11T23:03:25.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001919235 2024-05-11T23:03:25.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945230+0000", 2024-05-11T23:03:25.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9580000000000001e-06 2024-05-11T23:03:25.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945238+0000", 2024-05-11T23:03:25.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1040000000000007e-06 2024-05-11T23:03:25.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945244+0000", 2024-05-11T23:03:25.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6919999999999996e-06 2024-05-11T23:03:25.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993399+0000", 2024-05-11T23:03:25.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048154651 2024-05-11T23:03:25.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993472+0000", 2024-05-11T23:03:25.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2780000000000005e-05 2024-05-11T23:03:25.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993557+0000", 2024-05-11T23:03:25.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4926000000000006e-05 2024-05-11T23:03:25.954 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.954 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.954 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.954 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267865 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1540096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:25.954 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:25.954 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.989971016, 2024-05-11T23:03:25.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050054409000000001, 2024-05-11T23:03:25.954 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.954 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.954 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.954 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.954 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.954 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.955 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267865 2024-05-11T23:03:25.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.955 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:25.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:25.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:25.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6600000000000002e-07 2024-05-11T23:03:25.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945461+0000", 2024-05-11T23:03:25.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2670000000000001e-06 2024-05-11T23:03:25.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945466+0000", 2024-05-11T23:03:25.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7200000000000003e-06 2024-05-11T23:03:25.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945471+0000", 2024-05-11T23:03:25.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9439999999999996e-06 2024-05-11T23:03:25.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995337+0000", 2024-05-11T23:03:25.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049865415000000003 2024-05-11T23:03:25.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.957 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.957 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995413+0000", 2024-05-11T23:03:25.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5989999999999996e-05 2024-05-11T23:03:25.957 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995512+0000", 2024-05-11T23:03:25.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9706999999999999e-05 2024-05-11T23:03:25.958 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.958 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.958 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.958 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268017 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:25.958 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947359+0000", 2024-05-11T23:03:25.958 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.988070131000001, 2024-05-11T23:03:25.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050017655000000001, 2024-05-11T23:03:25.958 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.958 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.959 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.959 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.959 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268017 2024-05-11T23:03:25.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.959 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947359+0000", 2024-05-11T23:03:25.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947359+0000", 2024-05-11T23:03:25.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947359+0000", 2024-05-11T23:03:25.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5000000000000003e-07 2024-05-11T23:03:25.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947411+0000", 2024-05-11T23:03:25.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1158e-05 2024-05-11T23:03:25.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947418+0000", 2024-05-11T23:03:25.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8269999999999998e-06 2024-05-11T23:03:25.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947424+0000", 2024-05-11T23:03:25.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6529999999999999e-06 2024-05-11T23:03:25.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997179+0000", 2024-05-11T23:03:25.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049754675999999998 2024-05-11T23:03:25.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997279+0000", 2024-05-11T23:03:25.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9789000000000003e-05 2024-05-11T23:03:25.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997377+0000", 2024-05-11T23:03:25.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8002000000000001e-05 2024-05-11T23:03:25.962 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.962 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.962 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.962 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268018 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 872448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:25.962 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:03:25.963 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.987994950999999, 2024-05-11T23:03:25.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050170927999999997, 2024-05-11T23:03:25.963 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.963 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.963 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.963 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.963 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.963 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.963 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268018 2024-05-11T23:03:25.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.963 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:03:25.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:03:25.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947435+0000", 2024-05-11T23:03:25.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3099999999999997e-07 2024-05-11T23:03:25.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947437+0000", 2024-05-11T23:03:25.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6340000000000001e-06 2024-05-11T23:03:25.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947443+0000", 2024-05-11T23:03:25.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7810000000000002e-06 2024-05-11T23:03:25.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947448+0000", 2024-05-11T23:03:25.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9679999999999997e-06 2024-05-11T23:03:25.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997418+0000", 2024-05-11T23:03:25.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049969587000000003 2024-05-11T23:03:25.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997504+0000", 2024-05-11T23:03:25.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6426000000000002e-05 2024-05-11T23:03:25.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997605+0000", 2024-05-11T23:03:25.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100901 2024-05-11T23:03:25.966 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.966 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.966 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.967 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268019 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:25.967 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:25.967 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.987974393, 2024-05-11T23:03:25.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050285074999999999, 2024-05-11T23:03:25.967 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.967 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.967 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.967 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.967 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.967 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.967 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268019 2024-05-11T23:03:25.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.967 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:25.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:25.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:25.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0999999999999999e-07 2024-05-11T23:03:25.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947457+0000", 2024-05-11T23:03:25.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-05-11T23:03:25.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947462+0000", 2024-05-11T23:03:25.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2410000000000003e-06 2024-05-11T23:03:25.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947467+0000", 2024-05-11T23:03:25.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8339999999999998e-06 2024-05-11T23:03:25.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997639+0000", 2024-05-11T23:03:25.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050171697000000001 2024-05-11T23:03:25.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997685+0000", 2024-05-11T23:03:25.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6146000000000002e-05 2024-05-11T23:03:25.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997740+0000", 2024-05-11T23:03:25.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4737000000000001e-05 2024-05-11T23:03:25.970 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.970 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.971 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.971 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268020 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:25.971 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:03:25.971 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.987956148, 2024-05-11T23:03:25.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050504089000000002, 2024-05-11T23:03:25.971 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.971 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.971 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.971 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.971 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.971 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.971 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268020 2024-05-11T23:03:25.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.972 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:03:25.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:03:25.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947591+0000", 2024-05-11T23:03:25.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011805 2024-05-11T23:03:25.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947594+0000", 2024-05-11T23:03:25.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1549999999999999e-06 2024-05-11T23:03:25.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947602+0000", 2024-05-11T23:03:25.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.174e-06 2024-05-11T23:03:25.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947608+0000", 2024-05-11T23:03:25.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.263e-06 2024-05-11T23:03:25.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997775+0000", 2024-05-11T23:03:25.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050166980999999999 2024-05-11T23:03:25.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997875+0000", 2024-05-11T23:03:25.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9844000000000002e-05 2024-05-11T23:03:25.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997977+0000", 2024-05-11T23:03:25.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101622 2024-05-11T23:03:25.975 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.975 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.975 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.975 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268021 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:25.975 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:03:25.975 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.987810688, 2024-05-11T23:03:25.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050498874999999999, 2024-05-11T23:03:25.975 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.975 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.975 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.975 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.976 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268021 2024-05-11T23:03:25.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.976 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:03:25.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:03:25.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947695+0000", 2024-05-11T23:03:25.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6247000000000004e-05 2024-05-11T23:03:25.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947698+0000", 2024-05-11T23:03:25.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9199999999999997e-06 2024-05-11T23:03:25.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947706+0000", 2024-05-11T23:03:25.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:03:25.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947711+0000", 2024-05-11T23:03:25.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6720000000000001e-06 2024-05-11T23:03:25.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998017+0000", 2024-05-11T23:03:25.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050305470999999997 2024-05-11T23:03:25.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998060+0000", 2024-05-11T23:03:25.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3374000000000002e-05 2024-05-11T23:03:25.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998117+0000", 2024-05-11T23:03:25.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6971000000000001e-05 2024-05-11T23:03:25.979 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.979 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.979 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.979 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268022 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:25.979 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:03:25.979 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.987707651999999, 2024-05-11T23:03:25.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050731361000000003, 2024-05-11T23:03:25.979 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.979 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.979 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.979 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.980 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.980 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.980 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268022 2024-05-11T23:03:25.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.980 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:03:25.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:03:25.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947722+0000", 2024-05-11T23:03:25.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8999999999999996e-07 2024-05-11T23:03:25.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947724+0000", 2024-05-11T23:03:25.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1639999999999999e-06 2024-05-11T23:03:25.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947730+0000", 2024-05-11T23:03:25.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6099999999999997e-06 2024-05-11T23:03:25.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947735+0000", 2024-05-11T23:03:25.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7189999999999998e-06 2024-05-11T23:03:25.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998140+0000", 2024-05-11T23:03:25.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050404254000000003 2024-05-11T23:03:25.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998239+0000", 2024-05-11T23:03:25.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9192999999999997e-05 2024-05-11T23:03:25.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998453+0000", 2024-05-11T23:03:25.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021383099999999999 2024-05-11T23:03:25.983 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.983 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.983 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.983 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268023 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1069056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:25.983 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:03:25.983 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.987685386999999, 2024-05-11T23:03:25.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050874509999999998, 2024-05-11T23:03:25.983 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.984 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.984 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.984 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.984 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.984 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.984 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268023 2024-05-11T23:03:25.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.984 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:03:25.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:03:25.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948934+0000", 2024-05-11T23:03:25.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011903339999999999 2024-05-11T23:03:25.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.985 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948938+0000", 2024-05-11T23:03:25.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1330000000000001e-06 2024-05-11T23:03:25.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.985 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948949+0000", 2024-05-11T23:03:25.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0872e-05 2024-05-11T23:03:25.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948956+0000", 2024-05-11T23:03:25.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.131e-06 2024-05-11T23:03:25.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998494+0000", 2024-05-11T23:03:25.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049537678000000002 2024-05-11T23:03:25.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998551+0000", 2024-05-11T23:03:25.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7256000000000002e-05 2024-05-11T23:03:25.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998618+0000", 2024-05-11T23:03:25.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7106000000000004e-05 2024-05-11T23:03:25.987 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.987 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.987 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.987 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268026 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1179648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:25.987 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:03:25.988 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.986408334, 2024-05-11T23:03:25.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050107977999999997, 2024-05-11T23:03:25.988 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.988 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.988 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.988 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.988 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268026 2024-05-11T23:03:25.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.988 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:03:25.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:03:25.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949029+0000", 2024-05-11T23:03:25.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2220000000000002e-06 2024-05-11T23:03:25.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949041+0000", 2024-05-11T23:03:25.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.241e-05 2024-05-11T23:03:25.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949049+0000", 2024-05-11T23:03:25.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.072e-06 2024-05-11T23:03:25.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949056+0000", 2024-05-11T23:03:25.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1380000000000001e-06 2024-05-11T23:03:25.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999016+0000", 2024-05-11T23:03:25.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049959212000000003 2024-05-11T23:03:25.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999067+0000", 2024-05-11T23:03:25.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1598000000000003e-05 2024-05-11T23:03:25.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999129+0000", 2024-05-11T23:03:25.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1326000000000002e-05 2024-05-11T23:03:25.991 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.991 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.991 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.992 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268027 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1216512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:25.992 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:25.992 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.986364183999999, 2024-05-11T23:03:25.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050189865, 2024-05-11T23:03:25.992 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.992 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.992 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.992 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.992 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.992 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.992 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268027 2024-05-11T23:03:25.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.992 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:25.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:25.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:25.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-05-11T23:03:25.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949068+0000", 2024-05-11T23:03:25.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.26e-06 2024-05-11T23:03:25.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949074+0000", 2024-05-11T23:03:25.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7050000000000002e-06 2024-05-11T23:03:25.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.994 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.994 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.994 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949080+0000", 2024-05-11T23:03:25.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3689999999999998e-06 2024-05-11T23:03:25.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999152+0000", 2024-05-11T23:03:25.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050072653000000002 2024-05-11T23:03:25.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999197+0000", 2024-05-11T23:03:25.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4972999999999998e-05 2024-05-11T23:03:25.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:25.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999255+0000", 2024-05-11T23:03:25.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7425e-05 2024-05-11T23:03:25.996 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.996 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:25.996 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:25.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.996 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268028 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1253376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:25.996 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:03:25.996 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.986342523999999, 2024-05-11T23:03:25.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050280017000000003, 2024-05-11T23:03:25.996 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:25.996 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:25.996 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:25.996 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:25.996 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:25.997 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:25.997 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268028 2024-05-11T23:03:25.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.997 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:25.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:25.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:03:25.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:25.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:03:25.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:25.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:25.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949087+0000", 2024-05-11T23:03:25.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:03:25.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:25.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949089+0000", 2024-05-11T23:03:25.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8950000000000001e-06 2024-05-11T23:03:25.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.998 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:25.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949095+0000", 2024-05-11T23:03:25.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7540000000000003e-06 2024-05-11T23:03:25.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:25.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949114+0000", 2024-05-11T23:03:25.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9738e-05 2024-05-11T23:03:25.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:25.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999278+0000", 2024-05-11T23:03:25.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050163368999999999 2024-05-11T23:03:25.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:25.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:25.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:25.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999319+0000", 2024-05-11T23:03:25.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0979000000000002e-05 2024-05-11T23:03:26.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999366+0000", 2024-05-11T23:03:26.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7879e-05 2024-05-11T23:03:26.000 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.000 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.000 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.000 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268029 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1290240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:26.000 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:26.000 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.986302416000001, 2024-05-11T23:03:26.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050344904000000003, 2024-05-11T23:03:26.000 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.001 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.001 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.001 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.001 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268029 2024-05-11T23:03:26.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.001 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:26.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:26.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:26.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:03:26.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949130+0000", 2024-05-11T23:03:26.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-05-11T23:03:26.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949139+0000", 2024-05-11T23:03:26.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9800000000000004e-06 2024-05-11T23:03:26.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949145+0000", 2024-05-11T23:03:26.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0510000000000002e-06 2024-05-11T23:03:26.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999387+0000", 2024-05-11T23:03:26.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050242167999999997 2024-05-11T23:03:26.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999427+0000", 2024-05-11T23:03:26.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9566e-05 2024-05-11T23:03:26.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999471+0000", 2024-05-11T23:03:26.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4914999999999999e-05 2024-05-11T23:03:26.004 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.004 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.004 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.004 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268030 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1327104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:26.004 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:26.004 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.986277132, 2024-05-11T23:03:26.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050447394, 2024-05-11T23:03:26.005 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.005 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.005 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.005 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.005 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.005 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.005 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268030 2024-05-11T23:03:26.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.005 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:26.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:26.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:26.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0500000000000004e-07 2024-05-11T23:03:26.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949155+0000", 2024-05-11T23:03:26.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3819999999999998e-06 2024-05-11T23:03:26.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949162+0000", 2024-05-11T23:03:26.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9609999999999999e-06 2024-05-11T23:03:26.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949188+0000", 2024-05-11T23:03:26.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5877e-05 2024-05-11T23:03:26.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.007 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999494+0000", 2024-05-11T23:03:26.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050306265000000003 2024-05-11T23:03:26.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999534+0000", 2024-05-11T23:03:26.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0447000000000001e-05 2024-05-11T23:03:26.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999599+0000", 2024-05-11T23:03:26.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4956999999999995e-05 2024-05-11T23:03:26.008 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.008 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.008 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.008 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268031 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1363968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:26.009 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:03:26.009 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.986232032, 2024-05-11T23:03:26.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050511525000000002, 2024-05-11T23:03:26.009 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.009 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.009 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.009 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.009 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.009 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.009 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268031 2024-05-11T23:03:26.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.009 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:03:26.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:03:26.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949198+0000", 2024-05-11T23:03:26.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-05-11T23:03:26.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949200+0000", 2024-05-11T23:03:26.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-05-11T23:03:26.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949208+0000", 2024-05-11T23:03:26.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2840000000000001e-06 2024-05-11T23:03:26.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949214+0000", 2024-05-11T23:03:26.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9869999999999996e-06 2024-05-11T23:03:26.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999619+0000", 2024-05-11T23:03:26.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050405278999999997 2024-05-11T23:03:26.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999660+0000", 2024-05-11T23:03:26.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1427999999999997e-05 2024-05-11T23:03:26.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999708+0000", 2024-05-11T23:03:26.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8229999999999997e-05 2024-05-11T23:03:26.012 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.012 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.012 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.013 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268032 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1400832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:26.013 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:03:26.013 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.986202412000001, 2024-05-11T23:03:26.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050611863999999999, 2024-05-11T23:03:26.013 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.013 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.013 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.013 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.013 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268032 2024-05-11T23:03:26.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.013 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:03:26.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:03:26.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949230+0000", 2024-05-11T23:03:26.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.371e-06 2024-05-11T23:03:26.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949233+0000", 2024-05-11T23:03:26.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6599999999999999e-06 2024-05-11T23:03:26.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949241+0000", 2024-05-11T23:03:26.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8939999999999993e-06 2024-05-11T23:03:26.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949246+0000", 2024-05-11T23:03:26.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6590000000000004e-06 2024-05-11T23:03:26.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999733+0000", 2024-05-11T23:03:26.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050487268000000002 2024-05-11T23:03:26.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999784+0000", 2024-05-11T23:03:26.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0562000000000001e-05 2024-05-11T23:03:26.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.016 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.016 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999838+0000", 2024-05-11T23:03:26.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000002e-05 2024-05-11T23:03:26.016 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.017 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.017 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.017 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268033 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1437696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:26.017 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:03:26.017 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 12.986175285, 2024-05-11T23:03:26.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050697157999999999, 2024-05-11T23:03:26.017 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.017 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.017 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.017 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.017 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.017 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.017 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268033 2024-05-11T23:03:26.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.018 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:03:26.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:03:26.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950862+0000", 2024-05-11T23:03:26.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001607975 2024-05-11T23:03:26.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950866+0000", 2024-05-11T23:03:26.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7770000000000002e-06 2024-05-11T23:03:26.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950878+0000", 2024-05-11T23:03:26.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1421e-05 2024-05-11T23:03:26.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950885+0000", 2024-05-11T23:03:26.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-05-11T23:03:26.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999859+0000", 2024-05-11T23:03:26.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048973199000000002 2024-05-11T23:03:26.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999901+0000", 2024-05-11T23:03:26.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1943000000000001e-05 2024-05-11T23:03:26.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999951+0000", 2024-05-11T23:03:26.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0243e-05 2024-05-11T23:03:26.021 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.021 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.021 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.021 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.021 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.021 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:26.022 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:03:26.023 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:26.022+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:26.123 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:26.123+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:26.128 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:26.128 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:26.128 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:26.128 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:26.136 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:03:26.224 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:26.224+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:26.241 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:26.241 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:26.242 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:26.242 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:26.250 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:03:26.276 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-11T23:03:26.325 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:26.325+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:26.356 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:26.356 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:26.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 93.702016788999998, 2024-05-11T23:03:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:03:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.357 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:03:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:03:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:03:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:03:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:03:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:03:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:03:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:03:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:03:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:03:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:03:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:03:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:03:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:03:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 93.698635600000003, 2024-05-11T23:03:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:03:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.361 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:03:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:26.365 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 93.698616892999993, 2024-05-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 93.698598750000002, 2024-05-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.370 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:03:26.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:03:26.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:03:26.373 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 93.698571263000005, 2024-05-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.374 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 93.698550734999998, 2024-05-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.378 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 93.698530959999999, 2024-05-11T23:03:26.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:26.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:03:26.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:03:26.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 93.698512153999999, 2024-05-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:03:26.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 93.698493231, 2024-05-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.391 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 93.698474568999998, 2024-05-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:03:26.395 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:03:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:03:26.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:26.399 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 93.698129406999996, 2024-05-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 93.698103857000007, 2024-05-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.404 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:03:26.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:03:26.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 93.698021599, 2024-05-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.408 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:03:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:03:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:03:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:03:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:03:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:03:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:03:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:03:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 93.697999151999994, 2024-05-11T23:03:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:03:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.412 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:03:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:26.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:03:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:03:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:03:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:03:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:03:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:03:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:03:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:03:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:03:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:03:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:03:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:03:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:03:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:03:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:26.416 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 93.697979693999997, 2024-05-11T23:03:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:03:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:03:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:26.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:03:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:03:26.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:03:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:03:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:03:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:03:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:03:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:03:26.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:03:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:03:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:03:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:03:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:03:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 93.697959634, 2024-05-11T23:03:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:03:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:03:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:03:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:03:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:03:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:03:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:03:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:03:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:03:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:03:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:03:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:03:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:03:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:03:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.424 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 93.697940364000004, 2024-05-11T23:03:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:03:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:03:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:26.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:03:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:03:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:03:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:03:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:03:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:03:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:03:26.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:03:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:03:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:03:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:03:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:03:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:03:26.428 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 93.697394298000006, 2024-05-11T23:03:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:03:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.429 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:03:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:03:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:03:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:03:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:03:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:03:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:03:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:03:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:03:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:03:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:03:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:03:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:03:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:03:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:03:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 93.697362514000005, 2024-05-11T23:03:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:03:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.433 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:03:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:03:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:03:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:03:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:03:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:03:26.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:03:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:03:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:03:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:03:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:03:26.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:03:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:03:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:03:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:03:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.260117050000005, 2024-05-11T23:03:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:03:26.437 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:03:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:03:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:03:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:03:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:03:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.439 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:03:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:03:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:03:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:03:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:03:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:03:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:03:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:03:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:03:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:03:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:03:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:03:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:03:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:26.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:03:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:03:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:03:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:03:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:03:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:03:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.442 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.443 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.443 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:26.444 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:03:26.444 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:26.444+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:26.545 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:26.545+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:26.589 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:26.589 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:26.590 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:26.590 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:26.598 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:03:26.646 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:26.646+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:26.702 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:26.702 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:26.703 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:26.703 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:26.711 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:03:26.747 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:26.747+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-05-11T23:03:26.827 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:26.827 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:26.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:26.827 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:26.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.827 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:03:26.827 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:26.828 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 129.66315026699999, 2024-05-11T23:03:26.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:03:26.828 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.828 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.828 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:26.828 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.828 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:03:26.828 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:03:26.828 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:03:26.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.828 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:26.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:26.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:03:26.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:03:26.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:03:26.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:03:26.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:03:26.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:03:26.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:03:26.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:03:26.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:03:26.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:03:26.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:03:26.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:03:26.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:03:26.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:03:26.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:03:26.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:03:26.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:03:26.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:03:26.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:03:26.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:03:26.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:26.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:03:26.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:03:26.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:26.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:03:26.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:03:26.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:26.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:03:26.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:03:26.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:03:26.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:03:26.833 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.833 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.833 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.833 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78429 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:26.833 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:26.833 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.861880924999994, 2024-05-11T23:03:26.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044563497000000001, 2024-05-11T23:03:26.834 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.834 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.834 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.834 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.834 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.834 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.834 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78429 2024-05-11T23:03:26.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.834 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:26.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:26.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964872+0000", 2024-05-11T23:03:26.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035070300000000001 2024-05-11T23:03:26.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964876+0000", 2024-05-11T23:03:26.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5009999999999998e-06 2024-05-11T23:03:26.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964902+0000", 2024-05-11T23:03:26.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5715999999999999e-05 2024-05-11T23:03:26.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964910+0000", 2024-05-11T23:03:26.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3930000000000002e-06 2024-05-11T23:03:26.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008898+0000", 2024-05-11T23:03:26.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043988772000000002 2024-05-11T23:03:26.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008981+0000", 2024-05-11T23:03:26.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3140999999999996e-05 2024-05-11T23:03:26.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009085+0000", 2024-05-11T23:03:26.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010327099999999999 2024-05-11T23:03:26.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.837 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.837 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.837 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78431 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:26.837 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:26.837 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.861420616999993, 2024-05-11T23:03:26.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044542033000000002, 2024-05-11T23:03:26.838 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.838 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.838 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.838 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.838 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78431 2024-05-11T23:03:26.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.838 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:26.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:26.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965038+0000", 2024-05-11T23:03:26.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6993000000000001e-05 2024-05-11T23:03:26.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965043+0000", 2024-05-11T23:03:26.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7040000000000002e-06 2024-05-11T23:03:26.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965053+0000", 2024-05-11T23:03:26.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8539999999999992e-06 2024-05-11T23:03:26.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965061+0000", 2024-05-11T23:03:26.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9810000000000003e-06 2024-05-11T23:03:26.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009350+0000", 2024-05-11T23:03:26.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044288739000000001 2024-05-11T23:03:26.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009429+0000", 2024-05-11T23:03:26.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9354e-05 2024-05-11T23:03:26.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009524+0000", 2024-05-11T23:03:26.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4407999999999995e-05 2024-05-11T23:03:26.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.841 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.841 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.841 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78432 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:26.842 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:26.842 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.861329768999994, 2024-05-11T23:03:26.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044635421000000002, 2024-05-11T23:03:26.842 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.842 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.842 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.842 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.842 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.842 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.843 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78432 2024-05-11T23:03:26.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.843 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:26.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:26.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965343+0000", 2024-05-11T23:03:26.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027056500000000001 2024-05-11T23:03:26.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965347+0000", 2024-05-11T23:03:26.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3540000000000002e-06 2024-05-11T23:03:26.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965356+0000", 2024-05-11T23:03:26.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2140000000000002e-06 2024-05-11T23:03:26.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965363+0000", 2024-05-11T23:03:26.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4150000000000004e-06 2024-05-11T23:03:26.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009551+0000", 2024-05-11T23:03:26.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044187917 2024-05-11T23:03:26.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009626+0000", 2024-05-11T23:03:26.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5024000000000005e-05 2024-05-11T23:03:26.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009708+0000", 2024-05-11T23:03:26.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1932000000000001e-05 2024-05-11T23:03:26.846 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.846 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.846 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.846 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78433 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:26.846 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:26.846 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.861028881999999, 2024-05-11T23:03:26.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044564864000000003, 2024-05-11T23:03:26.846 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.846 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.846 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.846 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.847 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.847 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78433 2024-05-11T23:03:26.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.847 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:26.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:26.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966225+0000", 2024-05-11T23:03:26.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085220700000000001 2024-05-11T23:03:26.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966230+0000", 2024-05-11T23:03:26.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0690000000000003e-06 2024-05-11T23:03:26.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966239+0000", 2024-05-11T23:03:26.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-05-11T23:03:26.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966245+0000", 2024-05-11T23:03:26.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5520000000000001e-06 2024-05-11T23:03:26.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009743+0000", 2024-05-11T23:03:26.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043498311999999997 2024-05-11T23:03:26.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009848+0000", 2024-05-11T23:03:26.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104885 2024-05-11T23:03:26.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009938+0000", 2024-05-11T23:03:26.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9812e-05 2024-05-11T23:03:26.850 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.850 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.850 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.850 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78580 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 999424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:26.850 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:26.850 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.857916199000002, 2024-05-11T23:03:26.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044493076999999999, 2024-05-11T23:03:26.850 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.850 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.850 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.851 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.851 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78580 2024-05-11T23:03:26.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.851 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:26.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:26.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:26.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:03:26.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968489+0000", 2024-05-11T23:03:26.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.52e-06 2024-05-11T23:03:26.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968496+0000", 2024-05-11T23:03:26.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9419999999999998e-06 2024-05-11T23:03:26.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968501+0000", 2024-05-11T23:03:26.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8720000000000001e-06 2024-05-11T23:03:26.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012690+0000", 2024-05-11T23:03:26.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044189655000000001 2024-05-11T23:03:26.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012848+0000", 2024-05-11T23:03:26.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015758100000000001 2024-05-11T23:03:26.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012979+0000", 2024-05-11T23:03:26.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000131066 2024-05-11T23:03:26.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.854 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.854 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78581 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:26.854 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:26.854 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.857895194999998, 2024-05-11T23:03:26.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044616243, 2024-05-11T23:03:26.855 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.855 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.855 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.855 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78581 2024-05-11T23:03:26.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.855 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:26.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:26.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970227+0000", 2024-05-11T23:03:26.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017200049999999999 2024-05-11T23:03:26.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970232+0000", 2024-05-11T23:03:26.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000003e-06 2024-05-11T23:03:26.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970242+0000", 2024-05-11T23:03:26.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0192e-05 2024-05-11T23:03:26.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970249+0000", 2024-05-11T23:03:26.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.888e-06 2024-05-11T23:03:26.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013008+0000", 2024-05-11T23:03:26.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042759495000000002 2024-05-11T23:03:26.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013063+0000", 2024-05-11T23:03:26.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5071999999999997e-05 2024-05-11T23:03:26.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013123+0000", 2024-05-11T23:03:26.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9930999999999999e-05 2024-05-11T23:03:26.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.858 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.859 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78589 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:26.859 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:26.859 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.855975951000005, 2024-05-11T23:03:26.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044587697000000003, 2024-05-11T23:03:26.859 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.859 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.859 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.859 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78589 2024-05-11T23:03:26.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.859 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:26.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:26.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970427+0000", 2024-05-11T23:03:26.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:03:26.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970429+0000", 2024-05-11T23:03:26.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3800000000000001e-06 2024-05-11T23:03:26.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970435+0000", 2024-05-11T23:03:26.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2530000000000001e-06 2024-05-11T23:03:26.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970441+0000", 2024-05-11T23:03:26.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-05-11T23:03:26.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014822+0000", 2024-05-11T23:03:26.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04438168 2024-05-11T23:03:26.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014911+0000", 2024-05-11T23:03:26.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8882999999999996e-05 2024-05-11T23:03:26.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015014+0000", 2024-05-11T23:03:26.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010263399999999999 2024-05-11T23:03:26.862 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.862 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.862 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.863 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78590 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:26.863 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:26.863 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.855955152000007, 2024-05-11T23:03:26.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044880120000000003, 2024-05-11T23:03:26.863 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.863 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.863 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.863 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.863 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.863 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.863 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78590 2024-05-11T23:03:26.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.863 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:26.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:26.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970918+0000", 2024-05-11T23:03:26.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000470834 2024-05-11T23:03:26.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970925+0000", 2024-05-11T23:03:26.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1690000000000003e-06 2024-05-11T23:03:26.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970934+0000", 2024-05-11T23:03:26.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9099999999999994e-06 2024-05-11T23:03:26.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970941+0000", 2024-05-11T23:03:26.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7410000000000003e-06 2024-05-11T23:03:26.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015049+0000", 2024-05-11T23:03:26.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044108392000000003 2024-05-11T23:03:26.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015133+0000", 2024-05-11T23:03:26.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3568999999999999e-05 2024-05-11T23:03:26.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015327+0000", 2024-05-11T23:03:26.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019450500000000001 2024-05-11T23:03:26.867 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.867 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.867 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.867 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78591 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:26.867 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:26.867 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.855451736000006, 2024-05-11T23:03:26.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044618497, 2024-05-11T23:03:26.867 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.867 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.867 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.867 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.867 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.867 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.868 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78591 2024-05-11T23:03:26.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.868 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:26.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:26.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970951+0000", 2024-05-11T23:03:26.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:03:26.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970953+0000", 2024-05-11T23:03:26.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:03:26.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970956+0000", 2024-05-11T23:03:26.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-05-11T23:03:26.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970959+0000", 2024-05-11T23:03:26.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9409999999999999e-06 2024-05-11T23:03:26.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015367+0000", 2024-05-11T23:03:26.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044407621000000001 2024-05-11T23:03:26.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015453+0000", 2024-05-11T23:03:26.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6685000000000001e-05 2024-05-11T23:03:26.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015569+0000", 2024-05-11T23:03:26.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115396 2024-05-11T23:03:26.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.871 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.871 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.872 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78592 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:26.872 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:26.872 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.855438927999998, 2024-05-11T23:03:26.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044831546999999999, 2024-05-11T23:03:26.872 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.872 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.872 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.872 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.872 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78592 2024-05-11T23:03:26.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.872 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:26.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:26.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972226+0000", 2024-05-11T23:03:26.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00126246 2024-05-11T23:03:26.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972230+0000", 2024-05-11T23:03:26.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1860000000000002e-06 2024-05-11T23:03:26.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972237+0000", 2024-05-11T23:03:26.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:03:26.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972241+0000", 2024-05-11T23:03:26.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.284e-06 2024-05-11T23:03:26.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015611+0000", 2024-05-11T23:03:26.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043369461999999998 2024-05-11T23:03:26.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015695+0000", 2024-05-11T23:03:26.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4458999999999999e-05 2024-05-11T23:03:26.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015795+0000", 2024-05-11T23:03:26.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9475999999999994e-05 2024-05-11T23:03:26.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.875 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.875 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.876 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78593 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:26.876 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:26.876 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.854152009000003, 2024-05-11T23:03:26.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046045214000000001, 2024-05-11T23:03:26.876 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.876 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.876 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.876 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.876 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78593 2024-05-11T23:03:26.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.876 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:26.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:26.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:26.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7599999999999998e-07 2024-05-11T23:03:26.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972252+0000", 2024-05-11T23:03:26.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7349999999999999e-06 2024-05-11T23:03:26.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972256+0000", 2024-05-11T23:03:26.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5659999999999998e-06 2024-05-11T23:03:26.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972259+0000", 2024-05-11T23:03:26.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8040000000000002e-06 2024-05-11T23:03:26.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018088+0000", 2024-05-11T23:03:26.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045829316000000002 2024-05-11T23:03:26.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018179+0000", 2024-05-11T23:03:26.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1299999999999997e-05 2024-05-11T23:03:26.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018295+0000", 2024-05-11T23:03:26.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116117 2024-05-11T23:03:26.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.880 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.880 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.880 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78594 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:26.880 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:26.880 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.854139649000004, 2024-05-11T23:03:26.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046245204999999998, 2024-05-11T23:03:26.880 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.880 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.880 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.880 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78594 2024-05-11T23:03:26.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.881 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:26.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:26.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972263+0000", 2024-05-11T23:03:26.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:03:26.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972264+0000", 2024-05-11T23:03:26.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.206e-06 2024-05-11T23:03:26.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972267+0000", 2024-05-11T23:03:26.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0690000000000002e-06 2024-05-11T23:03:26.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972270+0000", 2024-05-11T23:03:26.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5610000000000001e-06 2024-05-11T23:03:26.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018330+0000", 2024-05-11T23:03:26.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04606085 2024-05-11T23:03:26.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018414+0000", 2024-05-11T23:03:26.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3125000000000001e-05 2024-05-11T23:03:26.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018508+0000", 2024-05-11T23:03:26.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4153000000000005e-05 2024-05-11T23:03:26.884 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.884 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.884 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.884 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78595 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:26.884 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:26.884 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.854126784000002, 2024-05-11T23:03:26.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046396698, 2024-05-11T23:03:26.884 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.884 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.884 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.885 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.885 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.885 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.885 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78595 2024-05-11T23:03:26.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.885 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:26.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:26.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972276+0000", 2024-05-11T23:03:26.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5600000000000002e-07 2024-05-11T23:03:26.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972277+0000", 2024-05-11T23:03:26.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.576e-06 2024-05-11T23:03:26.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972280+0000", 2024-05-11T23:03:26.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-05-11T23:03:26.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972283+0000", 2024-05-11T23:03:26.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:03:26.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018538+0000", 2024-05-11T23:03:26.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046255255000000002 2024-05-11T23:03:26.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018605+0000", 2024-05-11T23:03:26.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6847000000000006e-05 2024-05-11T23:03:26.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018672+0000", 2024-05-11T23:03:26.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7045000000000006e-05 2024-05-11T23:03:26.888 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.888 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.888 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.888 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78596 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:26.888 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:26.888 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.854115934999996, 2024-05-11T23:03:26.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046586806000000001, 2024-05-11T23:03:26.888 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.888 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.889 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.889 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.889 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.889 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.889 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78596 2024-05-11T23:03:26.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.889 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:26.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:26.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:26.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:03:26.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972288+0000", 2024-05-11T23:03:26.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2550000000000001e-06 2024-05-11T23:03:26.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972291+0000", 2024-05-11T23:03:26.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9490000000000001e-06 2024-05-11T23:03:26.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972293+0000", 2024-05-11T23:03:26.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5069999999999999e-06 2024-05-11T23:03:26.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018707+0000", 2024-05-11T23:03:26.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046413604999999997 2024-05-11T23:03:26.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018782+0000", 2024-05-11T23:03:26.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5556000000000006e-05 2024-05-11T23:03:26.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018873+0000", 2024-05-11T23:03:26.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0703000000000002e-05 2024-05-11T23:03:26.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.892 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.892 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78597 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:26.892 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:26.892 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.854106161999994, 2024-05-11T23:03:26.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046825787000000001, 2024-05-11T23:03:26.893 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.893 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.893 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.893 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78597 2024-05-11T23:03:26.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.893 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:26.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:26.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:26.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-05-11T23:03:26.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972297+0000", 2024-05-11T23:03:26.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3069999999999999e-06 2024-05-11T23:03:26.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972300+0000", 2024-05-11T23:03:26.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:03:26.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972303+0000", 2024-05-11T23:03:26.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5509999999999998e-06 2024-05-11T23:03:26.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018911+0000", 2024-05-11T23:03:26.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046608194999999998 2024-05-11T23:03:26.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018986+0000", 2024-05-11T23:03:26.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5360999999999999e-05 2024-05-11T23:03:26.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019122+0000", 2024-05-11T23:03:26.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013526 2024-05-11T23:03:26.897 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.897 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.897 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.897 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78598 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:26.897 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:26.897 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.854096261999999, 2024-05-11T23:03:26.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046998572000000002, 2024-05-11T23:03:26.897 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.897 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.897 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:26.897 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.897 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.898 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.898 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78598 2024-05-11T23:03:26.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.898 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:26.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:26.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:26.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:03:26.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972307+0000", 2024-05-11T23:03:26.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1799999999999999e-06 2024-05-11T23:03:26.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972310+0000", 2024-05-11T23:03:26.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:03:26.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972313+0000", 2024-05-11T23:03:26.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4760000000000001e-06 2024-05-11T23:03:26.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019149+0000", 2024-05-11T23:03:26.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046836056000000001 2024-05-11T23:03:26.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019203+0000", 2024-05-11T23:03:26.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-05 2024-05-11T23:03:26.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019304+0000", 2024-05-11T23:03:26.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101537 2024-05-11T23:03:26.901 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.901 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.901 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.902 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:105234 3.15 3:adf885e5:::rbd_object_map.111e913a615f.0000000000000025:head [writefull 0~39] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-11T23:03:26.902 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:26.902 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.818780552, 2024-05-11T23:03:26.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77797599100000003, 2024-05-11T23:03:26.902 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.902 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.902 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:26.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.902 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.902 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 105234 2024-05-11T23:03:26.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.902 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:26.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:26.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007623+0000", 2024-05-11T23:03:26.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2240000000000001e-06 2024-05-11T23:03:26.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007627+0000", 2024-05-11T23:03:26.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:03:26.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007636+0000", 2024-05-11T23:03:26.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2409999999999995e-06 2024-05-11T23:03:26.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007641+0000", 2024-05-11T23:03:26.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6810000000000003e-06 2024-05-11T23:03:26.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007676+0000", 2024-05-11T23:03:26.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5061999999999997e-05 2024-05-11T23:03:26.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:03:26.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007682+0000", 2024-05-11T23:03:26.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0399999999999998e-06 2024-05-11T23:03:26.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783880+0000", 2024-05-11T23:03:26.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77619752900000005 2024-05-11T23:03:26.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783995+0000", 2024-05-11T23:03:26.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011550800000000001 2024-05-11T23:03:26.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:03:26.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784082+0000", 2024-05-11T23:03:26.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6347999999999994e-05 2024-05-11T23:03:26.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:26.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784682+0000", 2024-05-11T23:03:26.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000600162 2024-05-11T23:03:26.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:26.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785539+0000", 2024-05-11T23:03:26.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085699099999999998 2024-05-11T23:03:26.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:26.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785572+0000", 2024-05-11T23:03:26.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3182999999999997e-05 2024-05-11T23:03:26.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785598+0000", 2024-05-11T23:03:26.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5522000000000001e-05 2024-05-11T23:03:26.907 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.907 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.907 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113761 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [call rbd.sparse_copyup in=999496b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:03:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.296093632999998, 2024-05-11T23:03:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048572964000000003, 2024-05-11T23:03:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113761 2024-05-11T23:03:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530309+0000", 2024-05-11T23:03:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:03:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531242+0000", 2024-05-11T23:03:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093316899999999997 2024-05-11T23:03:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531249+0000", 2024-05-11T23:03:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8140000000000003e-06 2024-05-11T23:03:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531253+0000", 2024-05-11T23:03:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4290000000000003e-06 2024-05-11T23:03:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570325+0000", 2024-05-11T23:03:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039071379000000003 2024-05-11T23:03:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570382+0000", 2024-05-11T23:03:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6988999999999999e-05 2024-05-11T23:03:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:03:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570555+0000", 2024-05-11T23:03:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017379799999999999 2024-05-11T23:03:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.575953+0000", 2024-05-11T23:03:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053975539999999997 2024-05-11T23:03:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578778+0000", 2024-05-11T23:03:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002825616 2024-05-11T23:03:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578837+0000", 2024-05-11T23:03:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9005000000000002e-05 2024-05-11T23:03:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578881+0000", 2024-05-11T23:03:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3970000000000001e-05 2024-05-11T23:03:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113762 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [delete] snapc 27=[27] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:03:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 52.295139681999999, 2024-05-11T23:03:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047916260000000002, 2024-05-11T23:03:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:26.913 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113762 2024-05-11T23:03:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:26.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531263+0000", 2024-05-11T23:03:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.85e-07 2024-05-11T23:03:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531265+0000", 2024-05-11T23:03:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.827e-06 2024-05-11T23:03:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531268+0000", 2024-05-11T23:03:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3780000000000001e-06 2024-05-11T23:03:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531271+0000", 2024-05-11T23:03:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8509999999999999e-06 2024-05-11T23:03:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572298+0000", 2024-05-11T23:03:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041026990999999999 2024-05-11T23:03:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572345+0000", 2024-05-11T23:03:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6878000000000001e-05 2024-05-11T23:03:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:03:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572443+0000", 2024-05-11T23:03:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8184999999999996e-05 2024-05-11T23:03:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578585+0000", 2024-05-11T23:03:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006141919 2024-05-11T23:03:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579126+0000", 2024-05-11T23:03:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00054063799999999997 2024-05-11T23:03:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579154+0000", 2024-05-11T23:03:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8042999999999999e-05 2024-05-11T23:03:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579179+0000", 2024-05-11T23:03:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4765000000000001e-05 2024-05-11T23:03:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:26.918 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:26.918 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:26.920 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:26.919+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:26.920 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:03:27.021 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:27.021+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:27.030 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:27.030 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:27.031 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:27.031 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:27.039 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:03:27.122 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:27.122+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:27.148 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:27.149 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:27.149 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:27.149 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:27.149 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:27.158 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:03:27.223 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:27.223+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:27.258 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 5, 7, 3, 6] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:03:27.258 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:03:27.258 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-05-11T23:03:27.258 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-11T23:03:27.258 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-11T23:03:27.258 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-05-11T23:03:27.312 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226051 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6049792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.994688514, 2024-05-11T23:03:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050622418000000002, 2024-05-11T23:03:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:27.313 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226051 2024-05-11T23:03:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.314 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:27.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317156+0000", 2024-05-11T23:03:27.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:27.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:27.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317157+0000", 2024-05-11T23:03:27.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.144e-06 2024-05-11T23:03:27.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.315 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:27.315 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317160+0000", 2024-05-11T23:03:27.315 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1350000000000001e-06 2024-05-11T23:03:27.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.315 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:27.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317162+0000", 2024-05-11T23:03:27.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5050000000000002e-06 2024-05-11T23:03:27.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:27.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367689+0000", 2024-05-11T23:03:27.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526797999999998 2024-05-11T23:03:27.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.316 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:27.316 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367735+0000", 2024-05-11T23:03:27.316 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5698999999999997e-05 2024-05-11T23:03:27.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.316 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.317 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:27.317 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367778+0000", 2024-05-11T23:03:27.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2928e-05 2024-05-11T23:03:27.317 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.317 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.317 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.317 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.317 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226052 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6060032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:27.317 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:27.317 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.994678457999999, 2024-05-11T23:03:27.317 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050830110999999997, 2024-05-11T23:03:27.317 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:27.317 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:27.317 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:27.318 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:27.318 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:27.318 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:27.318 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226052 2024-05-11T23:03:27.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.318 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:27.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:27.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:27.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.318 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:27.318 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:27.318 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:27.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317166+0000", 2024-05-11T23:03:27.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:03:27.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:27.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317167+0000", 2024-05-11T23:03:27.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.155e-06 2024-05-11T23:03:27.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.319 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:27.319 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317170+0000", 2024-05-11T23:03:27.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.061e-06 2024-05-11T23:03:27.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:27.320 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317172+0000", 2024-05-11T23:03:27.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4420000000000001e-06 2024-05-11T23:03:27.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:27.320 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367816+0000", 2024-05-11T23:03:27.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050644013000000002 2024-05-11T23:03:27.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:27.320 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367906+0000", 2024-05-11T23:03:27.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9739000000000003e-05 2024-05-11T23:03:27.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:27.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367995+0000", 2024-05-11T23:03:27.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9499000000000005e-05 2024-05-11T23:03:27.321 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.321 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.321 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.321 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226053 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6070272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:27.321 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:27.321 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.994668419, 2024-05-11T23:03:27.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051106912999999997, 2024-05-11T23:03:27.321 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:27.321 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:27.322 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:27.322 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:27.322 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:27.322 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:27.322 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226053 2024-05-11T23:03:27.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.322 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:27.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:27.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:27.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:27.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:27.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:27.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317176+0000", 2024-05-11T23:03:27.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0100000000000001e-07 2024-05-11T23:03:27.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:27.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317179+0000", 2024-05-11T23:03:27.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-05-11T23:03:27.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:27.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317182+0000", 2024-05-11T23:03:27.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3589999999999999e-06 2024-05-11T23:03:27.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:27.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317184+0000", 2024-05-11T23:03:27.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4339999999999999e-06 2024-05-11T23:03:27.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:27.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368061+0000", 2024-05-11T23:03:27.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050876927000000002 2024-05-11T23:03:27.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:27.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368179+0000", 2024-05-11T23:03:27.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118001 2024-05-11T23:03:27.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:27.325 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368282+0000", 2024-05-11T23:03:27.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102942 2024-05-11T23:03:27.325 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.325 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.325 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.325 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226054 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6080512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:27.325 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:27.325 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.994656097, 2024-05-11T23:03:27.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051322, 2024-05-11T23:03:27.325 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:27.326 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:27.326 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:27.326 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:27.326 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:27.326 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:27.326 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226054 2024-05-11T23:03:27.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.326 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:27.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:27.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:27.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:27.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:27.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:27.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:27.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:27.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:27.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317189+0000", 2024-05-11T23:03:27.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.217e-06 2024-05-11T23:03:27.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:27.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317192+0000", 2024-05-11T23:03:27.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0699999999999998e-06 2024-05-11T23:03:27.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:27.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317195+0000", 2024-05-11T23:03:27.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3829999999999999e-06 2024-05-11T23:03:27.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:27.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368328+0000", 2024-05-11T23:03:27.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051132999999999998 2024-05-11T23:03:27.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:27.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368429+0000", 2024-05-11T23:03:27.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010168800000000001 2024-05-11T23:03:27.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:27.329 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368510+0000", 2024-05-11T23:03:27.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0420999999999995e-05 2024-05-11T23:03:27.329 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.329 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.329 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.330 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226055 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6090752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:27.330 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:27.330 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.994646028999998, 2024-05-11T23:03:27.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051562835000000001, 2024-05-11T23:03:27.330 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:27.330 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:27.330 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:27.330 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:27.330 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:27.330 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:27.331 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226055 2024-05-11T23:03:27.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.331 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:27.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:27.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:27.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:27.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:27.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:27.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317256+0000", 2024-05-11T23:03:27.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8047000000000002e-05 2024-05-11T23:03:27.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:27.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317259+0000", 2024-05-11T23:03:27.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2590000000000001e-06 2024-05-11T23:03:27.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:27.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317267+0000", 2024-05-11T23:03:27.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8769999999999993e-06 2024-05-11T23:03:27.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:27.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317274+0000", 2024-05-11T23:03:27.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.187e-06 2024-05-11T23:03:27.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:27.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368544+0000", 2024-05-11T23:03:27.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051270299999999998 2024-05-11T23:03:27.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:27.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368658+0000", 2024-05-11T23:03:27.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000113347 2024-05-11T23:03:27.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:27.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368761+0000", 2024-05-11T23:03:27.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102818 2024-05-11T23:03:27.334 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.334 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.334 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.334 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226056 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6100992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:27.334 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:27.334 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.994560183000001, 2024-05-11T23:03:27.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051615576000000003, 2024-05-11T23:03:27.334 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:27.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:27.334 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:27.335 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:27.335 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:27.335 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:27.335 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226056 2024-05-11T23:03:27.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.335 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:27.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:27.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:27.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:27.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:27.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:27.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318915+0000", 2024-05-11T23:03:27.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001631291 2024-05-11T23:03:27.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:27.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318918+0000", 2024-05-11T23:03:27.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.512e-06 2024-05-11T23:03:27.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:27.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318927+0000", 2024-05-11T23:03:27.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1960000000000008e-06 2024-05-11T23:03:27.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:27.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318932+0000", 2024-05-11T23:03:27.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:03:27.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:27.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368805+0000", 2024-05-11T23:03:27.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049872998000000002 2024-05-11T23:03:27.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:27.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368853+0000", 2024-05-11T23:03:27.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8041999999999998e-05 2024-05-11T23:03:27.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:27.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368899+0000", 2024-05-11T23:03:27.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5766000000000002e-05 2024-05-11T23:03:27.338 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.338 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.338 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.338 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226061 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6152192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:27.338 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:27.338 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.992769096, 2024-05-11T23:03:27.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050675395999999998, 2024-05-11T23:03:27.339 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:27.339 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:27.339 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:27.339 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:27.339 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:27.339 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:27.339 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226061 2024-05-11T23:03:27.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.339 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:27.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:27.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:27.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:27.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:27.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.340 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:27.340 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:27.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:27.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:27.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319078+0000", 2024-05-11T23:03:27.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5670000000000002e-06 2024-05-11T23:03:27.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:27.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319085+0000", 2024-05-11T23:03:27.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2570000000000001e-06 2024-05-11T23:03:27.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:27.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319100+0000", 2024-05-11T23:03:27.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.489e-05 2024-05-11T23:03:27.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:27.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369584+0000", 2024-05-11T23:03:27.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050484345999999999 2024-05-11T23:03:27.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:27.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369671+0000", 2024-05-11T23:03:27.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6654000000000001e-05 2024-05-11T23:03:27.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:27.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369750+0000", 2024-05-11T23:03:27.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9257000000000004e-05 2024-05-11T23:03:27.343 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.343 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.343 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.343 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226062 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6162432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:27.343 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:27.343 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.992735551999999, 2024-05-11T23:03:27.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050761158000000001, 2024-05-11T23:03:27.343 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:27.343 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:27.343 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:27.343 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:27.343 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:27.343 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:27.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226062 2024-05-11T23:03:27.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.344 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:27.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:27.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:27.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:27.344 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:27.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:27.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319109+0000", 2024-05-11T23:03:27.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2800000000000002e-07 2024-05-11T23:03:27.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:27.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319111+0000", 2024-05-11T23:03:27.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4980000000000001e-06 2024-05-11T23:03:27.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:27.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319120+0000", 2024-05-11T23:03:27.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3999999999999992e-06 2024-05-11T23:03:27.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:27.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319129+0000", 2024-05-11T23:03:27.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7580000000000004e-06 2024-05-11T23:03:27.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:27.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369775+0000", 2024-05-11T23:03:27.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050645580000000003 2024-05-11T23:03:27.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:27.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369824+0000", 2024-05-11T23:03:27.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9506999999999999e-05 2024-05-11T23:03:27.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:27.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369869+0000", 2024-05-11T23:03:27.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4987000000000001e-05 2024-05-11T23:03:27.347 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.347 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.347 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.347 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226063 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6172672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:27.347 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:27.347 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.992706595000001, 2024-05-11T23:03:27.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050939784000000002, 2024-05-11T23:03:27.347 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:27.347 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:27.347 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:27.347 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:27.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:27.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:27.348 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226063 2024-05-11T23:03:27.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.348 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:27.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:27.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:27.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:27.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:27.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:27.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319138+0000", 2024-05-11T23:03:27.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.39e-07 2024-05-11T23:03:27.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:27.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319140+0000", 2024-05-11T23:03:27.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4820000000000001e-06 2024-05-11T23:03:27.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:27.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319148+0000", 2024-05-11T23:03:27.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6599999999999995e-06 2024-05-11T23:03:27.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:27.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319154+0000", 2024-05-11T23:03:27.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4169999999999997e-06 2024-05-11T23:03:27.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:27.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369905+0000", 2024-05-11T23:03:27.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050751009999999999 2024-05-11T23:03:27.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:27.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369991+0000", 2024-05-11T23:03:27.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5909999999999996e-05 2024-05-11T23:03:27.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:27.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370077+0000", 2024-05-11T23:03:27.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5865999999999994e-05 2024-05-11T23:03:27.351 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.351 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.351 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.351 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226064 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6182912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:27.351 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:27.351 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.992680399000001, 2024-05-11T23:03:27.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051038241999999998, 2024-05-11T23:03:27.351 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:27.351 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:27.352 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:27.352 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:27.352 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:27.352 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:27.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226064 2024-05-11T23:03:27.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.352 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:27.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:27.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:27.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:27.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:27.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:27.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319164+0000", 2024-05-11T23:03:27.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9299999999999999e-07 2024-05-11T23:03:27.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:27.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319166+0000", 2024-05-11T23:03:27.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.48e-06 2024-05-11T23:03:27.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:27.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319176+0000", 2024-05-11T23:03:27.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9669999999999996e-06 2024-05-11T23:03:27.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:27.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319180+0000", 2024-05-11T23:03:27.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8020000000000002e-06 2024-05-11T23:03:27.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:27.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370100+0000", 2024-05-11T23:03:27.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050919845999999998 2024-05-11T23:03:27.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:27.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370155+0000", 2024-05-11T23:03:27.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5040000000000002e-05 2024-05-11T23:03:27.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.355 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:27.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370202+0000", 2024-05-11T23:03:27.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6714e-05 2024-05-11T23:03:27.355 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.355 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.355 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.355 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226065 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6193152~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:27.355 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:27.355 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.992657828999999, 2024-05-11T23:03:27.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051225087000000002, 2024-05-11T23:03:27.356 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:27.356 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:27.356 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:27.356 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:27.356 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:27.356 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:27.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226065 2024-05-11T23:03:27.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.356 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:27.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:27.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:27.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:27.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:27.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:27.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320913+0000", 2024-05-11T23:03:27.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001727492 2024-05-11T23:03:27.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:27.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320919+0000", 2024-05-11T23:03:27.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.964e-06 2024-05-11T23:03:27.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:27.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320927+0000", 2024-05-11T23:03:27.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2239999999999998e-06 2024-05-11T23:03:27.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:27.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320934+0000", 2024-05-11T23:03:27.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8469999999999998e-06 2024-05-11T23:03:27.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:27.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370240+0000", 2024-05-11T23:03:27.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049306562999999998 2024-05-11T23:03:27.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:27.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370328+0000", 2024-05-11T23:03:27.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8122999999999996e-05 2024-05-11T23:03:27.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.359 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:27.359 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370411+0000", 2024-05-11T23:03:27.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2874000000000006e-05 2024-05-11T23:03:27.359 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.359 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.359 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.359 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226073 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6275072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:27.359 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:27.359 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.990819441999999, 2024-05-11T23:03:27.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050730475999999997, 2024-05-11T23:03:27.360 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:27.360 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:27.360 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:27.360 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:27.360 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:27.360 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:27.360 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226073 2024-05-11T23:03:27.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.360 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:27.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:27.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:27.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:27.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:27.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:27.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322914+0000", 2024-05-11T23:03:27.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001889153 2024-05-11T23:03:27.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:27.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322917+0000", 2024-05-11T23:03:27.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2100000000000002e-06 2024-05-11T23:03:27.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:27.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322924+0000", 2024-05-11T23:03:27.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9929999999999998e-06 2024-05-11T23:03:27.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:27.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322928+0000", 2024-05-11T23:03:27.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5549999999999996e-06 2024-05-11T23:03:27.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:27.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371578+0000", 2024-05-11T23:03:27.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048650001999999998 2024-05-11T23:03:27.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:27.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371671+0000", 2024-05-11T23:03:27.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2230000000000003e-05 2024-05-11T23:03:27.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:27.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371755+0000", 2024-05-11T23:03:27.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4332999999999994e-05 2024-05-11T23:03:27.363 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.363 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.363 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.364 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226085 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 5338112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:27.364 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:27.364 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.988728342999998, 2024-05-11T23:03:27.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050727622, 2024-05-11T23:03:27.364 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:27.364 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:27.364 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:27.364 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:27.364 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:27.364 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:27.364 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226085 2024-05-11T23:03:27.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.364 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:27.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:27.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:27.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:27.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:27.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:27.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324914+0000", 2024-05-11T23:03:27.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001798298 2024-05-11T23:03:27.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:27.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324918+0000", 2024-05-11T23:03:27.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3889999999999998e-06 2024-05-11T23:03:27.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:27.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324928+0000", 2024-05-11T23:03:27.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4339999999999998e-06 2024-05-11T23:03:27.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:27.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324936+0000", 2024-05-11T23:03:27.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2770000000000005e-06 2024-05-11T23:03:27.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:27.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373664+0000", 2024-05-11T23:03:27.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048728207000000003 2024-05-11T23:03:27.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:27.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373757+0000", 2024-05-11T23:03:27.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3332999999999996e-05 2024-05-11T23:03:27.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:27.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373843+0000", 2024-05-11T23:03:27.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5684000000000002e-05 2024-05-11T23:03:27.367 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.367 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.367 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.368 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232555 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7789568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:27.368 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:27.368 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.511718018, 2024-05-11T23:03:27.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050583086999999999, 2024-05-11T23:03:27.368 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:27.368 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:27.368 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:27.368 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:27.368 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:27.368 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:27.368 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232555 2024-05-11T23:03:27.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.368 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:27.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:27.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:27.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:27.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:27.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:27.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:27.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:03:27.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:27.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800127+0000", 2024-05-11T23:03:27.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.207e-06 2024-05-11T23:03:27.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:27.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800132+0000", 2024-05-11T23:03:27.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:03:27.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:27.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800134+0000", 2024-05-11T23:03:27.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5459999999999998e-06 2024-05-11T23:03:27.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:27.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850548+0000", 2024-05-11T23:03:27.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050414182000000002 2024-05-11T23:03:27.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:27.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850635+0000", 2024-05-11T23:03:27.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6853000000000003e-05 2024-05-11T23:03:27.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:27.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850709+0000", 2024-05-11T23:03:27.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3585999999999996e-05 2024-05-11T23:03:27.371 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.371 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.372 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.372 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232556 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7799808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:27.372 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:27.372 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.511706363999998, 2024-05-11T23:03:27.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050749599999999999, 2024-05-11T23:03:27.372 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:27.372 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:27.372 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:27.372 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:27.372 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:27.372 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:27.372 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232556 2024-05-11T23:03:27.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.372 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:27.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:27.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:27.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:27.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:27.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:27.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801914+0000", 2024-05-11T23:03:27.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017766360000000001 2024-05-11T23:03:27.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:27.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801919+0000", 2024-05-11T23:03:27.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.42e-06 2024-05-11T23:03:27.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:27.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801927+0000", 2024-05-11T23:03:27.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1999999999999994e-06 2024-05-11T23:03:27.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:27.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801931+0000", 2024-05-11T23:03:27.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4279999999999998e-06 2024-05-11T23:03:27.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:27.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850740+0000", 2024-05-11T23:03:27.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048808658999999997 2024-05-11T23:03:27.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:27.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850819+0000", 2024-05-11T23:03:27.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9102000000000003e-05 2024-05-11T23:03:27.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:27.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850887+0000", 2024-05-11T23:03:27.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8155000000000001e-05 2024-05-11T23:03:27.375 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.375 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.376 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.376 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232578 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8025088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:27.376 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:27.376 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.508413126000001, 2024-05-11T23:03:27.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050679473000000003, 2024-05-11T23:03:27.376 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:27.376 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:27.376 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:27.376 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:27.376 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:27.377 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:27.377 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232578 2024-05-11T23:03:27.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.377 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:27.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:27.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:27.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:27.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:27.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:27.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:27.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:27.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:27.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803432+0000", 2024-05-11T23:03:27.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.139e-06 2024-05-11T23:03:27.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:27.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803435+0000", 2024-05-11T23:03:27.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2440000000000002e-06 2024-05-11T23:03:27.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:27.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803439+0000", 2024-05-11T23:03:27.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3129999999999999e-06 2024-05-11T23:03:27.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:27.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.853953+0000", 2024-05-11T23:03:27.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050514846000000002 2024-05-11T23:03:27.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:27.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854035+0000", 2024-05-11T23:03:27.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1148000000000003e-05 2024-05-11T23:03:27.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:27.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854110+0000", 2024-05-11T23:03:27.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5562000000000006e-05 2024-05-11T23:03:27.380 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.380 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.380 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.380 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232579 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8035328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:27.380 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:27.380 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.508400521999999, 2024-05-11T23:03:27.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050886656000000002, 2024-05-11T23:03:27.381 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:27.381 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:27.381 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:27.381 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:27.381 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:27.381 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:27.381 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232579 2024-05-11T23:03:27.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.381 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:27.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:27.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:27.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:27.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:27.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:27.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803914+0000", 2024-05-11T23:03:27.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00047026700000000002 2024-05-11T23:03:27.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:27.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803918+0000", 2024-05-11T23:03:27.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3460000000000004e-06 2024-05-11T23:03:27.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:27.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803927+0000", 2024-05-11T23:03:27.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5030000000000003e-06 2024-05-11T23:03:27.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:27.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803933+0000", 2024-05-11T23:03:27.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0310000000000004e-06 2024-05-11T23:03:27.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:27.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854153+0000", 2024-05-11T23:03:27.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050219321999999997 2024-05-11T23:03:27.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:27.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854242+0000", 2024-05-11T23:03:27.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9457999999999996e-05 2024-05-11T23:03:27.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:27.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854330+0000", 2024-05-11T23:03:27.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7729e-05 2024-05-11T23:03:27.384 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.384 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.384 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.384 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232581 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8055808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:27.385 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:27.385 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.507877045000001, 2024-05-11T23:03:27.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050664439999999998, 2024-05-11T23:03:27.385 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:27.385 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:27.385 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:27.385 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:27.385 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:27.385 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:27.385 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232581 2024-05-11T23:03:27.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.385 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:27.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:27.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:27.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:27.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:27.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:27.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:27.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:03:27.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:27.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803969+0000", 2024-05-11T23:03:27.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0889999999999998e-06 2024-05-11T23:03:27.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:27.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803978+0000", 2024-05-11T23:03:27.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3070000000000003e-06 2024-05-11T23:03:27.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:27.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803983+0000", 2024-05-11T23:03:27.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1599999999999997e-06 2024-05-11T23:03:27.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:27.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854509+0000", 2024-05-11T23:03:27.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526339000000003 2024-05-11T23:03:27.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:27.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854576+0000", 2024-05-11T23:03:27.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7102999999999998e-05 2024-05-11T23:03:27.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:27.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854631+0000", 2024-05-11T23:03:27.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4997999999999998e-05 2024-05-11T23:03:27.388 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.388 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.388 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.389 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232582 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8066048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:27.389 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:27.389 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.507854299999998, 2024-05-11T23:03:27.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050753983000000003, 2024-05-11T23:03:27.389 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:27.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:27.389 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:27.389 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:27.389 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:27.389 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:27.389 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232582 2024-05-11T23:03:27.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.389 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:27.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:27.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:27.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:27.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:27.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:27.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805917+0000", 2024-05-11T23:03:27.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001927432 2024-05-11T23:03:27.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:27.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805920+0000", 2024-05-11T23:03:27.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1219999999999999e-06 2024-05-11T23:03:27.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:27.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805933+0000", 2024-05-11T23:03:27.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2727e-05 2024-05-11T23:03:27.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:27.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805941+0000", 2024-05-11T23:03:27.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7090000000000002e-06 2024-05-11T23:03:27.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:27.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854657+0000", 2024-05-11T23:03:27.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048716164999999999 2024-05-11T23:03:27.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:27.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854699+0000", 2024-05-11T23:03:27.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2531999999999999e-05 2024-05-11T23:03:27.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:27.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854744+0000", 2024-05-11T23:03:27.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4295999999999997e-05 2024-05-11T23:03:27.393 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.393 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.393 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.393 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232593 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8178688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:27.393 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:27.393 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 24.505654907, 2024-05-11T23:03:27.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050735178999999998, 2024-05-11T23:03:27.393 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:27.393 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:27.393 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:27.393 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:27.394 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:27.394 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:27.394 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232593 2024-05-11T23:03:27.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.394 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:27.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:27.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:27.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:27.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:27.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:27.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:27.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807913+0000", 2024-05-11T23:03:27.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017240319999999999 2024-05-11T23:03:27.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:27.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807916+0000", 2024-05-11T23:03:27.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4589999999999998e-06 2024-05-11T23:03:27.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:27.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807924+0000", 2024-05-11T23:03:27.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.272e-06 2024-05-11T23:03:27.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:27.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807928+0000", 2024-05-11T23:03:27.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5369999999999999e-06 2024-05-11T23:03:27.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:27.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856768+0000", 2024-05-11T23:03:27.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048839489999999999 2024-05-11T23:03:27.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:27.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856845+0000", 2024-05-11T23:03:27.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7746999999999999e-05 2024-05-11T23:03:27.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:27.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:27.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:27.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856924+0000", 2024-05-11T23:03:27.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8641999999999997e-05 2024-05-11T23:03:27.397 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.397 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.397 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.397 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:27.397 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:27.397 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:27.399 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:03:27.400 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:27.400 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-11T23:03:27.400 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:27.400 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:27.398+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:27.499 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:27.500+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:27.549 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:03:27.550 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:27.550 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:27.551 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:27.551 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:27.559 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:03:27.600 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:27.600+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:27.677 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:27.678 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:27.696 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:27.696 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:27.701 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:27.701+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:27.710 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:27.710 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:27.713 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:27.713 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:27.720 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:03:27.730 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:03:27.731 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:03:27.749 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:27.749 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:27.767 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:27.786 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:27.787 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:27.802 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:27.802+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:27.821 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:27.821 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:27.854 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:27.854 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:27.864 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:03:27.905 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:27.905+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:28.004 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:28.004+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:28.014 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:28.014 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:28.023 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:03:28.106 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:28.106+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:28.173 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:28.173 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:28.181 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:03:28.225 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:28.225+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:28.293 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:28.293 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:28.303 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:03:28.344 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:28.344+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:28.408 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:28.409+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:28.452 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:28.452 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:28.462 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:03:28.546 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:28.546+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:28.611 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:28.610+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:28.611 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:28.611 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:28.620 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:03:28.711 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:28.711+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:28.769 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:28.770 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:28.779 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:03:28.802 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-11T23:03:28.820 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:28.820+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:28.911 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:28.912 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:28.914 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:28.913+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:28.917 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:03:29.015 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:29.015+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:29.064 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:29.064 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:29.073 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:03:29.116 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:29.116+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:29.217 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:29.217+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:29.219 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:29.219 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:29.228 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:03:29.318 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:29.318+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:29.374 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:29.374 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:29.382 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:03:29.420 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:29.420+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:29.522 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:29.522+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:29.529 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:29.529 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:29.538 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:03:29.624 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:29.624+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:29.643 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:29.643 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:29.652 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:03:29.724 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:29.724+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:29.758 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:29.758 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:29.767 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:03:29.791 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-05-11T23:03:29.833 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:03:29.852 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:29.852+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:29.919 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:29.920 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:29.930 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:29.927+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:29.931 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:03:29.991 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:29.992 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:30.009 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:30.010 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:30.026 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:30.027 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:30.028 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:30.027+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:30.044 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:03:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:03:30.061 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:30.062 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:30.075 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:30.075 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:30.079 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:30.079 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:30.084 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:03:30.097 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:30.097 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:30.114 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:30.168 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:30.168+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:30.230 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:30.230 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:30.230 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:30.230 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:30.230 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:30.230+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:30.239 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:03:30.331 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:30.331+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:30.383 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:30.383 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:30.383 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:30.383 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:30.383 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:30.391 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:03:30.432 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:30.432+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:30.532 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:30.532+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:30.542 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:03:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 133.37506028799999, 2024-05-11T23:03:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:03:30.543 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:03:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:03:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:03:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.544 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:03:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:03:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:03:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:03:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:30.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:03:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:03:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:03:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:03:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:03:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:03:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:03:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:03:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:03:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:03:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:03:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:03:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:03:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:03:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:03:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:03:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:03:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:03:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:03:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:03:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:03:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:03:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:03:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:03:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.549 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78429 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.573790946000003, 2024-05-11T23:03:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044563497000000001, 2024-05-11T23:03:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78429 2024-05-11T23:03:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:30.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:30.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964872+0000", 2024-05-11T23:03:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035070300000000001 2024-05-11T23:03:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964876+0000", 2024-05-11T23:03:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5009999999999998e-06 2024-05-11T23:03:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964902+0000", 2024-05-11T23:03:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5715999999999999e-05 2024-05-11T23:03:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964910+0000", 2024-05-11T23:03:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3930000000000002e-06 2024-05-11T23:03:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008898+0000", 2024-05-11T23:03:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043988772000000002 2024-05-11T23:03:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008981+0000", 2024-05-11T23:03:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3140999999999996e-05 2024-05-11T23:03:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009085+0000", 2024-05-11T23:03:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010327099999999999 2024-05-11T23:03:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78431 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.573330638000002, 2024-05-11T23:03:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044542033000000002, 2024-05-11T23:03:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:30.554 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78431 2024-05-11T23:03:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:30.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965038+0000", 2024-05-11T23:03:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6993000000000001e-05 2024-05-11T23:03:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965043+0000", 2024-05-11T23:03:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7040000000000002e-06 2024-05-11T23:03:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965053+0000", 2024-05-11T23:03:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8539999999999992e-06 2024-05-11T23:03:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965061+0000", 2024-05-11T23:03:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9810000000000003e-06 2024-05-11T23:03:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009350+0000", 2024-05-11T23:03:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044288739000000001 2024-05-11T23:03:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009429+0000", 2024-05-11T23:03:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9354e-05 2024-05-11T23:03:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009524+0000", 2024-05-11T23:03:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4407999999999995e-05 2024-05-11T23:03:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.558 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78432 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.573239790000002, 2024-05-11T23:03:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044635421000000002, 2024-05-11T23:03:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78432 2024-05-11T23:03:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:30.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965343+0000", 2024-05-11T23:03:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027056500000000001 2024-05-11T23:03:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:30.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965347+0000", 2024-05-11T23:03:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3540000000000002e-06 2024-05-11T23:03:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965356+0000", 2024-05-11T23:03:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2140000000000002e-06 2024-05-11T23:03:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:30.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965363+0000", 2024-05-11T23:03:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4150000000000004e-06 2024-05-11T23:03:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009551+0000", 2024-05-11T23:03:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044187917 2024-05-11T23:03:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:30.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009626+0000", 2024-05-11T23:03:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5024000000000005e-05 2024-05-11T23:03:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009708+0000", 2024-05-11T23:03:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1932000000000001e-05 2024-05-11T23:03:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78433 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.572938902999994, 2024-05-11T23:03:30.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044564864000000003, 2024-05-11T23:03:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78433 2024-05-11T23:03:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966225+0000", 2024-05-11T23:03:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085220700000000001 2024-05-11T23:03:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966230+0000", 2024-05-11T23:03:30.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0690000000000003e-06 2024-05-11T23:03:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966239+0000", 2024-05-11T23:03:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-05-11T23:03:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966245+0000", 2024-05-11T23:03:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5520000000000001e-06 2024-05-11T23:03:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009743+0000", 2024-05-11T23:03:30.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043498311999999997 2024-05-11T23:03:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009848+0000", 2024-05-11T23:03:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104885 2024-05-11T23:03:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009938+0000", 2024-05-11T23:03:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9812e-05 2024-05-11T23:03:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78580 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 999424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.569826219999996, 2024-05-11T23:03:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044493076999999999, 2024-05-11T23:03:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78580 2024-05-11T23:03:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:03:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968489+0000", 2024-05-11T23:03:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.52e-06 2024-05-11T23:03:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968496+0000", 2024-05-11T23:03:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9419999999999998e-06 2024-05-11T23:03:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968501+0000", 2024-05-11T23:03:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8720000000000001e-06 2024-05-11T23:03:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012690+0000", 2024-05-11T23:03:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044189655000000001 2024-05-11T23:03:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012848+0000", 2024-05-11T23:03:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015758100000000001 2024-05-11T23:03:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012979+0000", 2024-05-11T23:03:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000131066 2024-05-11T23:03:30.571 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78581 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.569805216000006, 2024-05-11T23:03:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044616243, 2024-05-11T23:03:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:30.572 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78581 2024-05-11T23:03:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970227+0000", 2024-05-11T23:03:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017200049999999999 2024-05-11T23:03:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970232+0000", 2024-05-11T23:03:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000003e-06 2024-05-11T23:03:30.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970242+0000", 2024-05-11T23:03:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0192e-05 2024-05-11T23:03:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970249+0000", 2024-05-11T23:03:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.888e-06 2024-05-11T23:03:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013008+0000", 2024-05-11T23:03:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042759495000000002 2024-05-11T23:03:30.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013063+0000", 2024-05-11T23:03:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5071999999999997e-05 2024-05-11T23:03:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013123+0000", 2024-05-11T23:03:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9930999999999999e-05 2024-05-11T23:03:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.576 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78589 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.567885971999999, 2024-05-11T23:03:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044587697000000003, 2024-05-11T23:03:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78589 2024-05-11T23:03:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:30.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970427+0000", 2024-05-11T23:03:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:03:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970429+0000", 2024-05-11T23:03:30.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3800000000000001e-06 2024-05-11T23:03:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970435+0000", 2024-05-11T23:03:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2530000000000001e-06 2024-05-11T23:03:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970441+0000", 2024-05-11T23:03:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-05-11T23:03:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014822+0000", 2024-05-11T23:03:30.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04438168 2024-05-11T23:03:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014911+0000", 2024-05-11T23:03:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8882999999999996e-05 2024-05-11T23:03:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015014+0000", 2024-05-11T23:03:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010263399999999999 2024-05-11T23:03:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78590 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.567865173000001, 2024-05-11T23:03:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044880120000000003, 2024-05-11T23:03:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78590 2024-05-11T23:03:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970918+0000", 2024-05-11T23:03:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000470834 2024-05-11T23:03:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970925+0000", 2024-05-11T23:03:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1690000000000003e-06 2024-05-11T23:03:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970934+0000", 2024-05-11T23:03:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9099999999999994e-06 2024-05-11T23:03:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970941+0000", 2024-05-11T23:03:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7410000000000003e-06 2024-05-11T23:03:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015049+0000", 2024-05-11T23:03:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044108392000000003 2024-05-11T23:03:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015133+0000", 2024-05-11T23:03:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3568999999999999e-05 2024-05-11T23:03:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015327+0000", 2024-05-11T23:03:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019450500000000001 2024-05-11T23:03:30.585 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78591 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.567361757, 2024-05-11T23:03:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044618497, 2024-05-11T23:03:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:30.586 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78591 2024-05-11T23:03:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970951+0000", 2024-05-11T23:03:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:03:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970953+0000", 2024-05-11T23:03:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:03:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970956+0000", 2024-05-11T23:03:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-05-11T23:03:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970959+0000", 2024-05-11T23:03:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9409999999999999e-06 2024-05-11T23:03:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015367+0000", 2024-05-11T23:03:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044407621000000001 2024-05-11T23:03:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015453+0000", 2024-05-11T23:03:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6685000000000001e-05 2024-05-11T23:03:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015569+0000", 2024-05-11T23:03:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115396 2024-05-11T23:03:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.590 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78592 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.567348949000007, 2024-05-11T23:03:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044831546999999999, 2024-05-11T23:03:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78592 2024-05-11T23:03:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:30.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972226+0000", 2024-05-11T23:03:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00126246 2024-05-11T23:03:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:30.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972230+0000", 2024-05-11T23:03:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1860000000000002e-06 2024-05-11T23:03:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972237+0000", 2024-05-11T23:03:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:03:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972241+0000", 2024-05-11T23:03:30.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.284e-06 2024-05-11T23:03:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015611+0000", 2024-05-11T23:03:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043369461999999998 2024-05-11T23:03:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015695+0000", 2024-05-11T23:03:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4458999999999999e-05 2024-05-11T23:03:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015795+0000", 2024-05-11T23:03:30.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9475999999999994e-05 2024-05-11T23:03:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78593 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.566062029999998, 2024-05-11T23:03:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046045214000000001, 2024-05-11T23:03:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:30.595 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78593 2024-05-11T23:03:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:30.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7599999999999998e-07 2024-05-11T23:03:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972252+0000", 2024-05-11T23:03:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7349999999999999e-06 2024-05-11T23:03:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972256+0000", 2024-05-11T23:03:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5659999999999998e-06 2024-05-11T23:03:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:30.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972259+0000", 2024-05-11T23:03:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8040000000000002e-06 2024-05-11T23:03:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018088+0000", 2024-05-11T23:03:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045829316000000002 2024-05-11T23:03:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018179+0000", 2024-05-11T23:03:30.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1299999999999997e-05 2024-05-11T23:03:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018295+0000", 2024-05-11T23:03:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116117 2024-05-11T23:03:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.599 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78594 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.566049669999998, 2024-05-11T23:03:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046245204999999998, 2024-05-11T23:03:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78594 2024-05-11T23:03:30.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.601 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:30.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:30.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:30.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:30.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:30.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:30.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972263+0000", 2024-05-11T23:03:30.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:03:30.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:30.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972264+0000", 2024-05-11T23:03:30.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.206e-06 2024-05-11T23:03:30.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:30.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972267+0000", 2024-05-11T23:03:30.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0690000000000002e-06 2024-05-11T23:03:30.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:30.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972270+0000", 2024-05-11T23:03:30.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5610000000000001e-06 2024-05-11T23:03:30.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018330+0000", 2024-05-11T23:03:30.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04606085 2024-05-11T23:03:30.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:30.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018414+0000", 2024-05-11T23:03:30.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3125000000000001e-05 2024-05-11T23:03:30.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:30.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018508+0000", 2024-05-11T23:03:30.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4153000000000005e-05 2024-05-11T23:03:30.603 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.604 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.604 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.604 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78595 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:30.604 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:30.604 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.566036804999996, 2024-05-11T23:03:30.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046396698, 2024-05-11T23:03:30.604 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:30.604 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:30.604 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:30.604 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:30.604 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:30.604 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:30.604 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78595 2024-05-11T23:03:30.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.605 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:30.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:30.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:30.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:30.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:30.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:30.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972276+0000", 2024-05-11T23:03:30.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5600000000000002e-07 2024-05-11T23:03:30.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:30.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972277+0000", 2024-05-11T23:03:30.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.576e-06 2024-05-11T23:03:30.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:30.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972280+0000", 2024-05-11T23:03:30.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-05-11T23:03:30.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:30.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972283+0000", 2024-05-11T23:03:30.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:03:30.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018538+0000", 2024-05-11T23:03:30.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046255255000000002 2024-05-11T23:03:30.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:30.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018605+0000", 2024-05-11T23:03:30.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6847000000000006e-05 2024-05-11T23:03:30.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:30.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018672+0000", 2024-05-11T23:03:30.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7045000000000006e-05 2024-05-11T23:03:30.608 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.608 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.608 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.608 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78596 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:30.608 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:30.608 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.566025956000004, 2024-05-11T23:03:30.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046586806000000001, 2024-05-11T23:03:30.608 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:30.608 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:30.608 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:30.608 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:30.608 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:30.608 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:30.609 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78596 2024-05-11T23:03:30.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.609 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:30.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:30.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:30.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:30.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:30.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:30.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:30.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:03:30.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:30.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972288+0000", 2024-05-11T23:03:30.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2550000000000001e-06 2024-05-11T23:03:30.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:30.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972291+0000", 2024-05-11T23:03:30.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9490000000000001e-06 2024-05-11T23:03:30.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:30.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972293+0000", 2024-05-11T23:03:30.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5069999999999999e-06 2024-05-11T23:03:30.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018707+0000", 2024-05-11T23:03:30.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046413604999999997 2024-05-11T23:03:30.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:30.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018782+0000", 2024-05-11T23:03:30.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5556000000000006e-05 2024-05-11T23:03:30.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:30.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018873+0000", 2024-05-11T23:03:30.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0703000000000002e-05 2024-05-11T23:03:30.612 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.612 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.612 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.612 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78597 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:30.613 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:30.613 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.566016183000002, 2024-05-11T23:03:30.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046825787000000001, 2024-05-11T23:03:30.613 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:30.613 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:30.613 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:30.613 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:30.613 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:30.613 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:30.613 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78597 2024-05-11T23:03:30.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.613 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:30.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:30.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:30.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:30.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:30.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:30.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:30.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-05-11T23:03:30.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:30.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972297+0000", 2024-05-11T23:03:30.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3069999999999999e-06 2024-05-11T23:03:30.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:30.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972300+0000", 2024-05-11T23:03:30.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:03:30.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:30.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972303+0000", 2024-05-11T23:03:30.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5509999999999998e-06 2024-05-11T23:03:30.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018911+0000", 2024-05-11T23:03:30.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046608194999999998 2024-05-11T23:03:30.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:30.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018986+0000", 2024-05-11T23:03:30.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5360999999999999e-05 2024-05-11T23:03:30.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:30.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019122+0000", 2024-05-11T23:03:30.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013526 2024-05-11T23:03:30.616 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.616 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.616 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78598 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 74.566006282999993, 2024-05-11T23:03:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046998572000000002, 2024-05-11T23:03:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78598 2024-05-11T23:03:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:30.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:03:30.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:30.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972307+0000", 2024-05-11T23:03:30.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1799999999999999e-06 2024-05-11T23:03:30.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:30.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972310+0000", 2024-05-11T23:03:30.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:03:30.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:30.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972313+0000", 2024-05-11T23:03:30.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4760000000000001e-06 2024-05-11T23:03:30.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019149+0000", 2024-05-11T23:03:30.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046836056000000001 2024-05-11T23:03:30.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:30.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019203+0000", 2024-05-11T23:03:30.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-05 2024-05-11T23:03:30.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:30.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019304+0000", 2024-05-11T23:03:30.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101537 2024-05-11T23:03:30.621 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.621 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.621 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.621 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:105234 3.15 3:adf885e5:::rbd_object_map.111e913a615f.0000000000000025:head [writefull 0~39] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-11T23:03:30.621 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:30.621 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.530690573000001, 2024-05-11T23:03:30.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77797599100000003, 2024-05-11T23:03:30.621 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:30.621 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:30.622 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:30.622 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:30.622 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:30.622 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:30.622 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 105234 2024-05-11T23:03:30.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.622 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:30.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:30.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:30.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:30.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:30.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:30.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007623+0000", 2024-05-11T23:03:30.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2240000000000001e-06 2024-05-11T23:03:30.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:30.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007627+0000", 2024-05-11T23:03:30.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:03:30.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:30.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007636+0000", 2024-05-11T23:03:30.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2409999999999995e-06 2024-05-11T23:03:30.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:30.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007641+0000", 2024-05-11T23:03:30.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6810000000000003e-06 2024-05-11T23:03:30.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007676+0000", 2024-05-11T23:03:30.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5061999999999997e-05 2024-05-11T23:03:30.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:03:30.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007682+0000", 2024-05-11T23:03:30.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0399999999999998e-06 2024-05-11T23:03:30.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783880+0000", 2024-05-11T23:03:30.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77619752900000005 2024-05-11T23:03:30.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:30.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783995+0000", 2024-05-11T23:03:30.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011550800000000001 2024-05-11T23:03:30.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:03:30.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784082+0000", 2024-05-11T23:03:30.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6347999999999994e-05 2024-05-11T23:03:30.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:30.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784682+0000", 2024-05-11T23:03:30.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000600162 2024-05-11T23:03:30.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:30.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785539+0000", 2024-05-11T23:03:30.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085699099999999998 2024-05-11T23:03:30.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:30.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785572+0000", 2024-05-11T23:03:30.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3182999999999997e-05 2024-05-11T23:03:30.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:30.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785598+0000", 2024-05-11T23:03:30.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5522000000000001e-05 2024-05-11T23:03:30.627 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.627 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.627 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.627 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113761 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [call rbd.sparse_copyup in=999496b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:03:30.628 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:30.628 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.008003653999999, 2024-05-11T23:03:30.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048572964000000003, 2024-05-11T23:03:30.628 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:30.628 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:30.628 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:30.628 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:30.628 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:30.628 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:30.628 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113761 2024-05-11T23:03:30.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.628 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:30.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:30.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:30.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:30.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:30.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:30.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530309+0000", 2024-05-11T23:03:30.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:03:30.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:30.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531242+0000", 2024-05-11T23:03:30.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093316899999999997 2024-05-11T23:03:30.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:30.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531249+0000", 2024-05-11T23:03:30.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8140000000000003e-06 2024-05-11T23:03:30.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:30.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531253+0000", 2024-05-11T23:03:30.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4290000000000003e-06 2024-05-11T23:03:30.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570325+0000", 2024-05-11T23:03:30.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039071379000000003 2024-05-11T23:03:30.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:30.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570382+0000", 2024-05-11T23:03:30.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6988999999999999e-05 2024-05-11T23:03:30.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:03:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570555+0000", 2024-05-11T23:03:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017379799999999999 2024-05-11T23:03:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.575953+0000", 2024-05-11T23:03:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053975539999999997 2024-05-11T23:03:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578778+0000", 2024-05-11T23:03:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002825616 2024-05-11T23:03:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578837+0000", 2024-05-11T23:03:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9005000000000002e-05 2024-05-11T23:03:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578881+0000", 2024-05-11T23:03:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3970000000000001e-05 2024-05-11T23:03:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113762 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [delete] snapc 27=[27] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:03:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:30.633 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 56.007049703, 2024-05-11T23:03:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047916260000000002, 2024-05-11T23:03:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113762 2024-05-11T23:03:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531263+0000", 2024-05-11T23:03:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.85e-07 2024-05-11T23:03:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531265+0000", 2024-05-11T23:03:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.827e-06 2024-05-11T23:03:30.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531268+0000", 2024-05-11T23:03:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3780000000000001e-06 2024-05-11T23:03:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531271+0000", 2024-05-11T23:03:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8509999999999999e-06 2024-05-11T23:03:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572298+0000", 2024-05-11T23:03:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041026990999999999 2024-05-11T23:03:30.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572345+0000", 2024-05-11T23:03:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6878000000000001e-05 2024-05-11T23:03:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:03:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572443+0000", 2024-05-11T23:03:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8184999999999996e-05 2024-05-11T23:03:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578585+0000", 2024-05-11T23:03:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006141919 2024-05-11T23:03:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579126+0000", 2024-05-11T23:03:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00054063799999999997 2024-05-11T23:03:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579154+0000", 2024-05-11T23:03:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8042999999999999e-05 2024-05-11T23:03:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579179+0000", 2024-05-11T23:03:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4765000000000001e-05 2024-05-11T23:03:30.638 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.639 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.639 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.639 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:30.639 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:30.639 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:30.640 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:03:30.641 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:30.640+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:30.741 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:30.740+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:30.757 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:30.758 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:30.767 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:03:30.851 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:30.851+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:30.926 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:30.926 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:30.926 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:30.926 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:30.926 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:30.936 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:03:30.942 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:30.942+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:31.044 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:31.044+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:31.087 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:31.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226051 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6049792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.769233456999999, 2024-05-11T23:03:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050622418000000002, 2024-05-11T23:03:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226051 2024-05-11T23:03:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317156+0000", 2024-05-11T23:03:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317157+0000", 2024-05-11T23:03:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.144e-06 2024-05-11T23:03:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317160+0000", 2024-05-11T23:03:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1350000000000001e-06 2024-05-11T23:03:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317162+0000", 2024-05-11T23:03:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5050000000000002e-06 2024-05-11T23:03:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367689+0000", 2024-05-11T23:03:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526797999999998 2024-05-11T23:03:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367735+0000", 2024-05-11T23:03:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5698999999999997e-05 2024-05-11T23:03:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367778+0000", 2024-05-11T23:03:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2928e-05 2024-05-11T23:03:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.091 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226052 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6060032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.769223401000001, 2024-05-11T23:03:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050830110999999997, 2024-05-11T23:03:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226052 2024-05-11T23:03:31.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317166+0000", 2024-05-11T23:03:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:03:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317167+0000", 2024-05-11T23:03:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.155e-06 2024-05-11T23:03:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317170+0000", 2024-05-11T23:03:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.061e-06 2024-05-11T23:03:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317172+0000", 2024-05-11T23:03:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4420000000000001e-06 2024-05-11T23:03:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367816+0000", 2024-05-11T23:03:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050644013000000002 2024-05-11T23:03:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367906+0000", 2024-05-11T23:03:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9739000000000003e-05 2024-05-11T23:03:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367995+0000", 2024-05-11T23:03:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9499000000000005e-05 2024-05-11T23:03:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226053 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6070272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.769213361999999, 2024-05-11T23:03:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051106912999999997, 2024-05-11T23:03:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.096 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226053 2024-05-11T23:03:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317176+0000", 2024-05-11T23:03:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0100000000000001e-07 2024-05-11T23:03:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317179+0000", 2024-05-11T23:03:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-05-11T23:03:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317182+0000", 2024-05-11T23:03:31.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3589999999999999e-06 2024-05-11T23:03:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317184+0000", 2024-05-11T23:03:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4339999999999999e-06 2024-05-11T23:03:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368061+0000", 2024-05-11T23:03:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050876927000000002 2024-05-11T23:03:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368179+0000", 2024-05-11T23:03:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118001 2024-05-11T23:03:31.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368282+0000", 2024-05-11T23:03:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102942 2024-05-11T23:03:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226054 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6080512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.769201039999999, 2024-05-11T23:03:31.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051322, 2024-05-11T23:03:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226054 2024-05-11T23:03:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317189+0000", 2024-05-11T23:03:31.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.217e-06 2024-05-11T23:03:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317192+0000", 2024-05-11T23:03:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0699999999999998e-06 2024-05-11T23:03:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317195+0000", 2024-05-11T23:03:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3829999999999999e-06 2024-05-11T23:03:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368328+0000", 2024-05-11T23:03:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051132999999999998 2024-05-11T23:03:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368429+0000", 2024-05-11T23:03:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010168800000000001 2024-05-11T23:03:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368510+0000", 2024-05-11T23:03:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0420999999999995e-05 2024-05-11T23:03:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226055 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6090752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.769190972000001, 2024-05-11T23:03:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051562835000000001, 2024-05-11T23:03:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226055 2024-05-11T23:03:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.105 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317256+0000", 2024-05-11T23:03:31.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8047000000000002e-05 2024-05-11T23:03:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317259+0000", 2024-05-11T23:03:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2590000000000001e-06 2024-05-11T23:03:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317267+0000", 2024-05-11T23:03:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8769999999999993e-06 2024-05-11T23:03:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317274+0000", 2024-05-11T23:03:31.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.187e-06 2024-05-11T23:03:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368544+0000", 2024-05-11T23:03:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051270299999999998 2024-05-11T23:03:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368658+0000", 2024-05-11T23:03:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000113347 2024-05-11T23:03:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368761+0000", 2024-05-11T23:03:31.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102818 2024-05-11T23:03:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226056 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6100992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.769105125999999, 2024-05-11T23:03:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051615576000000003, 2024-05-11T23:03:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.109 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226056 2024-05-11T23:03:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318915+0000", 2024-05-11T23:03:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001631291 2024-05-11T23:03:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318918+0000", 2024-05-11T23:03:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.512e-06 2024-05-11T23:03:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318927+0000", 2024-05-11T23:03:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1960000000000008e-06 2024-05-11T23:03:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318932+0000", 2024-05-11T23:03:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:03:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368805+0000", 2024-05-11T23:03:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049872998000000002 2024-05-11T23:03:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368853+0000", 2024-05-11T23:03:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8041999999999998e-05 2024-05-11T23:03:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368899+0000", 2024-05-11T23:03:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5766000000000002e-05 2024-05-11T23:03:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226061 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6152192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.767314038999999, 2024-05-11T23:03:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050675395999999998, 2024-05-11T23:03:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226061 2024-05-11T23:03:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:31.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319078+0000", 2024-05-11T23:03:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5670000000000002e-06 2024-05-11T23:03:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319085+0000", 2024-05-11T23:03:31.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2570000000000001e-06 2024-05-11T23:03:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319100+0000", 2024-05-11T23:03:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.489e-05 2024-05-11T23:03:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369584+0000", 2024-05-11T23:03:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050484345999999999 2024-05-11T23:03:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369671+0000", 2024-05-11T23:03:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6654000000000001e-05 2024-05-11T23:03:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369750+0000", 2024-05-11T23:03:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9257000000000004e-05 2024-05-11T23:03:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226062 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6162432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.767280495000001, 2024-05-11T23:03:31.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050761158000000001, 2024-05-11T23:03:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226062 2024-05-11T23:03:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319109+0000", 2024-05-11T23:03:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2800000000000002e-07 2024-05-11T23:03:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319111+0000", 2024-05-11T23:03:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4980000000000001e-06 2024-05-11T23:03:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319120+0000", 2024-05-11T23:03:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3999999999999992e-06 2024-05-11T23:03:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319129+0000", 2024-05-11T23:03:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7580000000000004e-06 2024-05-11T23:03:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369775+0000", 2024-05-11T23:03:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050645580000000003 2024-05-11T23:03:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369824+0000", 2024-05-11T23:03:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9506999999999999e-05 2024-05-11T23:03:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369869+0000", 2024-05-11T23:03:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4987000000000001e-05 2024-05-11T23:03:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226063 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6172672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.767251538, 2024-05-11T23:03:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050939784000000002, 2024-05-11T23:03:31.121 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226063 2024-05-11T23:03:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319138+0000", 2024-05-11T23:03:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.39e-07 2024-05-11T23:03:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319140+0000", 2024-05-11T23:03:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4820000000000001e-06 2024-05-11T23:03:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319148+0000", 2024-05-11T23:03:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6599999999999995e-06 2024-05-11T23:03:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319154+0000", 2024-05-11T23:03:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4169999999999997e-06 2024-05-11T23:03:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369905+0000", 2024-05-11T23:03:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050751009999999999 2024-05-11T23:03:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369991+0000", 2024-05-11T23:03:31.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5909999999999996e-05 2024-05-11T23:03:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370077+0000", 2024-05-11T23:03:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5865999999999994e-05 2024-05-11T23:03:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226064 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6182912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.767225342, 2024-05-11T23:03:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051038241999999998, 2024-05-11T23:03:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.125 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226064 2024-05-11T23:03:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319164+0000", 2024-05-11T23:03:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9299999999999999e-07 2024-05-11T23:03:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319166+0000", 2024-05-11T23:03:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.48e-06 2024-05-11T23:03:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319176+0000", 2024-05-11T23:03:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9669999999999996e-06 2024-05-11T23:03:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319180+0000", 2024-05-11T23:03:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8020000000000002e-06 2024-05-11T23:03:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370100+0000", 2024-05-11T23:03:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050919845999999998 2024-05-11T23:03:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370155+0000", 2024-05-11T23:03:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5040000000000002e-05 2024-05-11T23:03:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370202+0000", 2024-05-11T23:03:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6714e-05 2024-05-11T23:03:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226065 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6193152~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:31.129 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.767202772000001, 2024-05-11T23:03:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051225087000000002, 2024-05-11T23:03:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226065 2024-05-11T23:03:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320913+0000", 2024-05-11T23:03:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001727492 2024-05-11T23:03:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320919+0000", 2024-05-11T23:03:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.964e-06 2024-05-11T23:03:31.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320927+0000", 2024-05-11T23:03:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2239999999999998e-06 2024-05-11T23:03:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320934+0000", 2024-05-11T23:03:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8469999999999998e-06 2024-05-11T23:03:31.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370240+0000", 2024-05-11T23:03:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049306562999999998 2024-05-11T23:03:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370328+0000", 2024-05-11T23:03:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8122999999999996e-05 2024-05-11T23:03:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370411+0000", 2024-05-11T23:03:31.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2874000000000006e-05 2024-05-11T23:03:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226073 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6275072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.765364385000002, 2024-05-11T23:03:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050730475999999997, 2024-05-11T23:03:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.134 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226073 2024-05-11T23:03:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322914+0000", 2024-05-11T23:03:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001889153 2024-05-11T23:03:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322917+0000", 2024-05-11T23:03:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2100000000000002e-06 2024-05-11T23:03:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322924+0000", 2024-05-11T23:03:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9929999999999998e-06 2024-05-11T23:03:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322928+0000", 2024-05-11T23:03:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5549999999999996e-06 2024-05-11T23:03:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371578+0000", 2024-05-11T23:03:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048650001999999998 2024-05-11T23:03:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371671+0000", 2024-05-11T23:03:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2230000000000003e-05 2024-05-11T23:03:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371755+0000", 2024-05-11T23:03:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4332999999999994e-05 2024-05-11T23:03:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226085 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 5338112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.763273286, 2024-05-11T23:03:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050727622, 2024-05-11T23:03:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.138 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226085 2024-05-11T23:03:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324914+0000", 2024-05-11T23:03:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001798298 2024-05-11T23:03:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324918+0000", 2024-05-11T23:03:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3889999999999998e-06 2024-05-11T23:03:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324928+0000", 2024-05-11T23:03:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4339999999999998e-06 2024-05-11T23:03:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324936+0000", 2024-05-11T23:03:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2770000000000005e-06 2024-05-11T23:03:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373664+0000", 2024-05-11T23:03:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048728207000000003 2024-05-11T23:03:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373757+0000", 2024-05-11T23:03:31.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3332999999999996e-05 2024-05-11T23:03:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373843+0000", 2024-05-11T23:03:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5684000000000002e-05 2024-05-11T23:03:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232555 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7789568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.286262960999998, 2024-05-11T23:03:31.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050583086999999999, 2024-05-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232555 2024-05-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800127+0000", 2024-05-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.207e-06 2024-05-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800132+0000", 2024-05-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800134+0000", 2024-05-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5459999999999998e-06 2024-05-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850548+0000", 2024-05-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050414182000000002 2024-05-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850635+0000", 2024-05-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6853000000000003e-05 2024-05-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850709+0000", 2024-05-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3585999999999996e-05 2024-05-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232556 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7799808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.286251307000001, 2024-05-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050749599999999999, 2024-05-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232556 2024-05-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801914+0000", 2024-05-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017766360000000001 2024-05-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801919+0000", 2024-05-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.42e-06 2024-05-11T23:03:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801927+0000", 2024-05-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1999999999999994e-06 2024-05-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801931+0000", 2024-05-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4279999999999998e-06 2024-05-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850740+0000", 2024-05-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048808658999999997 2024-05-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850819+0000", 2024-05-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9102000000000003e-05 2024-05-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850887+0000", 2024-05-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8155000000000001e-05 2024-05-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232578 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8025088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.282958068999999, 2024-05-11T23:03:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050679473000000003, 2024-05-11T23:03:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232578 2024-05-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803432+0000", 2024-05-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.139e-06 2024-05-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803435+0000", 2024-05-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2440000000000002e-06 2024-05-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803439+0000", 2024-05-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3129999999999999e-06 2024-05-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.853953+0000", 2024-05-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050514846000000002 2024-05-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854035+0000", 2024-05-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1148000000000003e-05 2024-05-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854110+0000", 2024-05-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5562000000000006e-05 2024-05-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232579 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8035328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.282945465000001, 2024-05-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050886656000000002, 2024-05-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232579 2024-05-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803914+0000", 2024-05-11T23:03:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00047026700000000002 2024-05-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803918+0000", 2024-05-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3460000000000004e-06 2024-05-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803927+0000", 2024-05-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5030000000000003e-06 2024-05-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803933+0000", 2024-05-11T23:03:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0310000000000004e-06 2024-05-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854153+0000", 2024-05-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050219321999999997 2024-05-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854242+0000", 2024-05-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9457999999999996e-05 2024-05-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854330+0000", 2024-05-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7729e-05 2024-05-11T23:03:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232581 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8055808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.282421987999999, 2024-05-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050664439999999998, 2024-05-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232581 2024-05-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803969+0000", 2024-05-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0889999999999998e-06 2024-05-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803978+0000", 2024-05-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3070000000000003e-06 2024-05-11T23:03:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803983+0000", 2024-05-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1599999999999997e-06 2024-05-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854509+0000", 2024-05-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526339000000003 2024-05-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854576+0000", 2024-05-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7102999999999998e-05 2024-05-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854631+0000", 2024-05-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4997999999999998e-05 2024-05-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232582 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8066048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.282399243, 2024-05-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050753983000000003, 2024-05-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232582 2024-05-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805917+0000", 2024-05-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001927432 2024-05-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805920+0000", 2024-05-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1219999999999999e-06 2024-05-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805933+0000", 2024-05-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2727e-05 2024-05-11T23:03:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805941+0000", 2024-05-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7090000000000002e-06 2024-05-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854657+0000", 2024-05-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048716164999999999 2024-05-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854699+0000", 2024-05-11T23:03:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2531999999999999e-05 2024-05-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854744+0000", 2024-05-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4295999999999997e-05 2024-05-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232593 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8178688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.280199849999999, 2024-05-11T23:03:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050735178999999998, 2024-05-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232593 2024-05-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807913+0000", 2024-05-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017240319999999999 2024-05-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807916+0000", 2024-05-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4589999999999998e-06 2024-05-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807924+0000", 2024-05-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.272e-06 2024-05-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807928+0000", 2024-05-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5369999999999999e-06 2024-05-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856768+0000", 2024-05-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048839489999999999 2024-05-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856845+0000", 2024-05-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7746999999999999e-05 2024-05-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856924+0000", 2024-05-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8641999999999997e-05 2024-05-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.173 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:31.175 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:03:31.175 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:03:31.175+0000 7fee2b4d6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32856) UID: 0 2024-05-11T23:03:31.275 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:31.275+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:31.325 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:31.325 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:31.334 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:03:31.377 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:31.377+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:31.443 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:31.444 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:31.452 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:03:31.477 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:31.478+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:31.563 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:64049 3.8 3:11d98727:::rbd_object_map.111e8974a681:head [read 0~38] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-05-11T23:03:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 90.783228184999999, 2024-05-11T23:03:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99884604399999999, 2024-05-11T23:03:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 64049 2024-05-11T23:03:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779883+0000", 2024-05-11T23:03:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.57e-06 2024-05-11T23:03:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779887+0000", 2024-05-11T23:03:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5709999999999999e-06 2024-05-11T23:03:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779898+0000", 2024-05-11T23:03:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.042e-05 2024-05-11T23:03:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779905+0000", 2024-05-11T23:03:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0110000000000003e-06 2024-05-11T23:03:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779962+0000", 2024-05-11T23:03:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7395999999999997e-05 2024-05-11T23:03:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779969+0000", 2024-05-11T23:03:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2459999999999997e-06 2024-05-11T23:03:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778379+0000", 2024-05-11T23:03:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99840986300000001 2024-05-11T23:03:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778564+0000", 2024-05-11T23:03:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018420900000000001 2024-05-11T23:03:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778727+0000", 2024-05-11T23:03:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016375800000000001 2024-05-11T23:03:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172773 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.959121766999999, 2024-05-11T23:03:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058721316000000003, 2024-05-11T23:03:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172773 2024-05-11T23:03:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3500000000000002e-07 2024-05-11T23:03:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603990+0000", 2024-05-11T23:03:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3549999999999999e-06 2024-05-11T23:03:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603997+0000", 2024-05-11T23:03:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3400000000000003e-06 2024-05-11T23:03:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604002+0000", 2024-05-11T23:03:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0509999999999996e-06 2024-05-11T23:03:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662697+0000", 2024-05-11T23:03:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058695487999999997 2024-05-11T23:03:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662709+0000", 2024-05-11T23:03:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1647e-05 2024-05-11T23:03:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172774 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.959100384999999, 2024-05-11T23:03:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058745040999999998, 2024-05-11T23:03:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172774 2024-05-11T23:03:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:03:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604012+0000", 2024-05-11T23:03:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2199999999999999e-06 2024-05-11T23:03:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604018+0000", 2024-05-11T23:03:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1770000000000003e-06 2024-05-11T23:03:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604023+0000", 2024-05-11T23:03:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0089999999999998e-06 2024-05-11T23:03:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662742+0000", 2024-05-11T23:03:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058719187999999999 2024-05-11T23:03:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662754+0000", 2024-05-11T23:03:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2095e-05 2024-05-11T23:03:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172775 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.959080172999997, 2024-05-11T23:03:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058764743000000001, 2024-05-11T23:03:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172775 2024-05-11T23:03:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604030+0000", 2024-05-11T23:03:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6699999999999999e-07 2024-05-11T23:03:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604032+0000", 2024-05-11T23:03:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9530000000000002e-06 2024-05-11T23:03:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604037+0000", 2024-05-11T23:03:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8590000000000001e-06 2024-05-11T23:03:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604042+0000", 2024-05-11T23:03:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8609999999999997e-06 2024-05-11T23:03:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662782+0000", 2024-05-11T23:03:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058739731000000003 2024-05-11T23:03:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662794+0000", 2024-05-11T23:03:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1972e-05 2024-05-11T23:03:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172776 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.959060882999999, 2024-05-11T23:03:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058778696999999998, 2024-05-11T23:03:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172776 2024-05-11T23:03:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2000000000000001e-07 2024-05-11T23:03:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604051+0000", 2024-05-11T23:03:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2879999999999999e-06 2024-05-11T23:03:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604057+0000", 2024-05-11T23:03:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:03:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604062+0000", 2024-05-11T23:03:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6800000000000001e-06 2024-05-11T23:03:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662820+0000", 2024-05-11T23:03:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058758444999999999 2024-05-11T23:03:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662827+0000", 2024-05-11T23:03:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1930000000000004e-06 2024-05-11T23:03:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172777 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.959041577000001, 2024-05-11T23:03:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058799918999999999, 2024-05-11T23:03:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172777 2024-05-11T23:03:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:03:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604075+0000", 2024-05-11T23:03:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.109e-06 2024-05-11T23:03:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604083+0000", 2024-05-11T23:03:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9170000000000006e-06 2024-05-11T23:03:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604089+0000", 2024-05-11T23:03:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3399999999999997e-06 2024-05-11T23:03:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662859+0000", 2024-05-11T23:03:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058770172000000002 2024-05-11T23:03:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662868+0000", 2024-05-11T23:03:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0289999999999994e-06 2024-05-11T23:03:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172778 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.959013487, 2024-05-11T23:03:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058845435000000001, 2024-05-11T23:03:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172778 2024-05-11T23:03:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5400000000000002e-07 2024-05-11T23:03:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604099+0000", 2024-05-11T23:03:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4650000000000001e-06 2024-05-11T23:03:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604105+0000", 2024-05-11T23:03:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.314e-06 2024-05-11T23:03:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604110+0000", 2024-05-11T23:03:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6310000000000002e-06 2024-05-11T23:03:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662931+0000", 2024-05-11T23:03:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058820960999999998 2024-05-11T23:03:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662941+0000", 2024-05-11T23:03:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.061e-05 2024-05-11T23:03:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172779 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.958993327000002, 2024-05-11T23:03:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058859163999999999, 2024-05-11T23:03:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172779 2024-05-11T23:03:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:03:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604119+0000", 2024-05-11T23:03:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2529999999999999e-06 2024-05-11T23:03:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604125+0000", 2024-05-11T23:03:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1670000000000004e-06 2024-05-11T23:03:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604130+0000", 2024-05-11T23:03:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.702e-06 2024-05-11T23:03:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662967+0000", 2024-05-11T23:03:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058837792999999999 2024-05-11T23:03:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662975+0000", 2024-05-11T23:03:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8490000000000006e-06 2024-05-11T23:03:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172780 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.958973176999997, 2024-05-11T23:03:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058872696000000002, 2024-05-11T23:03:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172780 2024-05-11T23:03:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604137+0000", 2024-05-11T23:03:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4400000000000001e-07 2024-05-11T23:03:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604139+0000", 2024-05-11T23:03:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3429999999999998e-06 2024-05-11T23:03:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604145+0000", 2024-05-11T23:03:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1940000000000003e-06 2024-05-11T23:03:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604150+0000", 2024-05-11T23:03:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:03:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663002+0000", 2024-05-11T23:03:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058851829000000001 2024-05-11T23:03:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663009+0000", 2024-05-11T23:03:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.199e-06 2024-05-11T23:03:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172781 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.958941336999999, 2024-05-11T23:03:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058875553999999997, 2024-05-11T23:03:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172781 2024-05-11T23:03:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604171+0000", 2024-05-11T23:03:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.21e-06 2024-05-11T23:03:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604176+0000", 2024-05-11T23:03:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2970000000000003e-06 2024-05-11T23:03:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604179+0000", 2024-05-11T23:03:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8459999999999999e-06 2024-05-11T23:03:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663033+0000", 2024-05-11T23:03:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058854565999999997 2024-05-11T23:03:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663044+0000", 2024-05-11T23:03:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0229e-05 2024-05-11T23:03:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172782 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.958926589000001, 2024-05-11T23:03:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058894375999999998, 2024-05-11T23:03:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172782 2024-05-11T23:03:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-05-11T23:03:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604184+0000", 2024-05-11T23:03:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:03:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604187+0000", 2024-05-11T23:03:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.044e-06 2024-05-11T23:03:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604190+0000", 2024-05-11T23:03:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6460000000000002e-06 2024-05-11T23:03:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663070+0000", 2024-05-11T23:03:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058879510000000003 2024-05-11T23:03:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663077+0000", 2024-05-11T23:03:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6669999999999996e-06 2024-05-11T23:03:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172783 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.958916270000003, 2024-05-11T23:03:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058913946000000002, 2024-05-11T23:03:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172783 2024-05-11T23:03:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-05-11T23:03:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604195+0000", 2024-05-11T23:03:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.19e-06 2024-05-11T23:03:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604198+0000", 2024-05-11T23:03:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:03:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604200+0000", 2024-05-11T23:03:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4719999999999998e-06 2024-05-11T23:03:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663100+0000", 2024-05-11T23:03:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058900300000000003 2024-05-11T23:03:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663107+0000", 2024-05-11T23:03:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7449999999999998e-06 2024-05-11T23:03:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172784 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.958906251000002, 2024-05-11T23:03:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933728999999997, 2024-05-11T23:03:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172784 2024-05-11T23:03:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.03e-07 2024-05-11T23:03:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604205+0000", 2024-05-11T23:03:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2300000000000001e-06 2024-05-11T23:03:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604207+0000", 2024-05-11T23:03:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8729999999999999e-06 2024-05-11T23:03:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604210+0000", 2024-05-11T23:03:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.858e-06 2024-05-11T23:03:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663130+0000", 2024-05-11T23:03:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058919314 2024-05-11T23:03:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663137+0000", 2024-05-11T23:03:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2509999999999997e-06 2024-05-11T23:03:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172785 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.958893981000003, 2024-05-11T23:03:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058951664000000001, 2024-05-11T23:03:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172785 2024-05-11T23:03:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604216+0000", 2024-05-11T23:03:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:03:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604218+0000", 2024-05-11T23:03:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7549999999999999e-06 2024-05-11T23:03:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604222+0000", 2024-05-11T23:03:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6700000000000002e-06 2024-05-11T23:03:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604227+0000", 2024-05-11T23:03:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4399999999999998e-06 2024-05-11T23:03:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663160+0000", 2024-05-11T23:03:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933119999999999 2024-05-11T23:03:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663167+0000", 2024-05-11T23:03:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1840000000000002e-06 2024-05-11T23:03:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172786 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.958876926999999, 2024-05-11T23:03:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058966991000000003, 2024-05-11T23:03:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172786 2024-05-11T23:03:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604233+0000", 2024-05-11T23:03:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6899999999999999e-07 2024-05-11T23:03:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604235+0000", 2024-05-11T23:03:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8369999999999999e-06 2024-05-11T23:03:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604240+0000", 2024-05-11T23:03:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0329999999999999e-06 2024-05-11T23:03:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604244+0000", 2024-05-11T23:03:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9859999999999996e-06 2024-05-11T23:03:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663192+0000", 2024-05-11T23:03:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058948852000000003 2024-05-11T23:03:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663199+0000", 2024-05-11T23:03:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0140000000000001e-06 2024-05-11T23:03:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172787 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.958860581000003, 2024-05-11T23:03:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058982018999999997, 2024-05-11T23:03:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172787 2024-05-11T23:03:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605916+0000", 2024-05-11T23:03:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00166752 2024-05-11T23:03:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605921+0000", 2024-05-11T23:03:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0669999999999999e-06 2024-05-11T23:03:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605934+0000", 2024-05-11T23:03:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2271e-05 2024-05-11T23:03:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605940+0000", 2024-05-11T23:03:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6880000000000002e-06 2024-05-11T23:03:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663224+0000", 2024-05-11T23:03:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057283251 2024-05-11T23:03:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663231+0000", 2024-05-11T23:03:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2219999999999996e-06 2024-05-11T23:03:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172793 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.957093733999997, 2024-05-11T23:03:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057402373999999999, 2024-05-11T23:03:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172793 2024-05-11T23:03:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3300000000000001e-07 2024-05-11T23:03:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606017+0000", 2024-05-11T23:03:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4920000000000001e-06 2024-05-11T23:03:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606020+0000", 2024-05-11T23:03:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:03:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606023+0000", 2024-05-11T23:03:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6680000000000001e-06 2024-05-11T23:03:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663411+0000", 2024-05-11T23:03:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057388206999999997 2024-05-11T23:03:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663418+0000", 2024-05-11T23:03:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.815e-06 2024-05-11T23:03:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172795 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.957052306999998, 2024-05-11T23:03:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057423015000000001, 2024-05-11T23:03:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172795 2024-05-11T23:03:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9499999999999998e-07 2024-05-11T23:03:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606059+0000", 2024-05-11T23:03:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-05-11T23:03:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606062+0000", 2024-05-11T23:03:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.977e-06 2024-05-11T23:03:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606064+0000", 2024-05-11T23:03:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7410000000000001e-06 2024-05-11T23:03:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663473+0000", 2024-05-11T23:03:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057408624999999998 2024-05-11T23:03:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663480+0000", 2024-05-11T23:03:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1049999999999997e-06 2024-05-11T23:03:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172796 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.957041744000001, 2024-05-11T23:03:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057449235000000001, 2024-05-11T23:03:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172796 2024-05-11T23:03:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606069+0000", 2024-05-11T23:03:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:03:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606072+0000", 2024-05-11T23:03:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8700000000000001e-06 2024-05-11T23:03:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606074+0000", 2024-05-11T23:03:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6230000000000001e-06 2024-05-11T23:03:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663510+0000", 2024-05-11T23:03:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05743529 2024-05-11T23:03:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663517+0000", 2024-05-11T23:03:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1230000000000002e-06 2024-05-11T23:03:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172797 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.957031712000003, 2024-05-11T23:03:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057470420000000001, 2024-05-11T23:03:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172797 2024-05-11T23:03:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607915+0000", 2024-05-11T23:03:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018375609999999999 2024-05-11T23:03:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607920+0000", 2024-05-11T23:03:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8899999999999998e-06 2024-05-11T23:03:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607930+0000", 2024-05-11T23:03:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4429999999999992e-06 2024-05-11T23:03:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607936+0000", 2024-05-11T23:03:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7089999999999996e-06 2024-05-11T23:03:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663541+0000", 2024-05-11T23:03:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055604537000000002 2024-05-11T23:03:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663548+0000", 2024-05-11T23:03:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2799999999999998e-06 2024-05-11T23:03:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:31.642 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:31.644 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:03:31.644 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:31.644+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:31.745 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:31.745+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:31.754 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:31.754 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:31.763 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:03:31.817 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-11T23:03:31.848 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:31.848+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:31.930 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:31.930 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:31.940 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:03:31.984 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:31.984+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:32.049 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:32.048+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:32.056 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:77128 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_get in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-11T23:03:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 77.040038000999999, 2024-05-11T23:03:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91737855899999998, 2024-05-11T23:03:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 77128 2024-05-11T23:03:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016003+0000", 2024-05-11T23:03:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5379999999999999e-06 2024-05-11T23:03:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016011+0000", 2024-05-11T23:03:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6410000000000002e-06 2024-05-11T23:03:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016026+0000", 2024-05-11T23:03:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4779e-05 2024-05-11T23:03:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016032+0000", 2024-05-11T23:03:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.066e-06 2024-05-11T23:03:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016093+0000", 2024-05-11T23:03:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0878000000000002e-05 2024-05-11T23:03:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016100+0000", 2024-05-11T23:03:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6799999999999993e-06 2024-05-11T23:03:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933302+0000", 2024-05-11T23:03:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91720135300000005 2024-05-11T23:03:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933380+0000", 2024-05-11T23:03:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8623999999999998e-05 2024-05-11T23:03:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98950 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.923957111999997, 2024-05-11T23:03:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021419438999999998, 2024-05-11T23:03:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98950 2024-05-11T23:03:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.34e-07 2024-05-11T23:03:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132085+0000", 2024-05-11T23:03:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0049999999999999e-06 2024-05-11T23:03:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132093+0000", 2024-05-11T23:03:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6660000000000008e-06 2024-05-11T23:03:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132097+0000", 2024-05-11T23:03:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2590000000000002e-06 2024-05-11T23:03:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135114+0000", 2024-05-11T23:03:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003017573 2024-05-11T23:03:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153416+0000", 2024-05-11T23:03:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018301884000000001 2024-05-11T23:03:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153502+0000", 2024-05-11T23:03:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5717999999999995e-05 2024-05-11T23:03:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98959 3.1e 3:7807da99:::rbd_data.111ed2934718.000000000000006d:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.923464374999995, 2024-05-11T23:03:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020525928999999998, 2024-05-11T23:03:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98959 2024-05-11T23:03:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132576+0000", 2024-05-11T23:03:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4199999999999995e-07 2024-05-11T23:03:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132580+0000", 2024-05-11T23:03:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2910000000000001e-06 2024-05-11T23:03:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132588+0000", 2024-05-11T23:03:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7970000000000001e-06 2024-05-11T23:03:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132602+0000", 2024-05-11T23:03:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3743999999999999e-05 2024-05-11T23:03:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132655+0000", 2024-05-11T23:03:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2714999999999999e-05 2024-05-11T23:03:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132787+0000", 2024-05-11T23:03:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013273899999999999 2024-05-11T23:03:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153101+0000", 2024-05-11T23:03:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020313900999999999 2024-05-11T23:03:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98965 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.920940739000002, 2024-05-11T23:03:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018608983999999999, 2024-05-11T23:03:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98965 2024-05-11T23:03:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135100+0000", 2024-05-11T23:03:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.17e-06 2024-05-11T23:03:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135105+0000", 2024-05-11T23:03:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.899e-06 2024-05-11T23:03:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135122+0000", 2024-05-11T23:03:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7346999999999998e-05 2024-05-11T23:03:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135130+0000", 2024-05-11T23:03:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0639999999999994e-06 2024-05-11T23:03:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153533+0000", 2024-05-11T23:03:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018402753000000001 2024-05-11T23:03:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153615+0000", 2024-05-11T23:03:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1720999999999999e-05 2024-05-11T23:03:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153708+0000", 2024-05-11T23:03:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3029999999999995e-05 2024-05-11T23:03:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98966 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3760128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.920868612999996, 2024-05-11T23:03:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018716535999999999, 2024-05-11T23:03:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98966 2024-05-11T23:03:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135172+0000", 2024-05-11T23:03:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-05-11T23:03:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135175+0000", 2024-05-11T23:03:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-05-11T23:03:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135184+0000", 2024-05-11T23:03:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1840000000000005e-06 2024-05-11T23:03:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152903+0000", 2024-05-11T23:03:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017718357000000001 2024-05-11T23:03:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153736+0000", 2024-05-11T23:03:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00083324799999999997 2024-05-11T23:03:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153809+0000", 2024-05-11T23:03:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2995000000000002e-05 2024-05-11T23:03:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153888+0000", 2024-05-11T23:03:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8695999999999993e-05 2024-05-11T23:03:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "pg_update_log_missing_reply(3.1e epoch 72/53 rep_tid 104 lcod 72'363)", 2024-05-11T23:03:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.920805716999993, 2024-05-11T23:03:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017871951000000001, 2024-05-11T23:03:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135235+0000", 2024-05-11T23:03:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8899999999999999e-07 2024-05-11T23:03:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135238+0000", 2024-05-11T23:03:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.27e-06 2024-05-11T23:03:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152878+0000", 2024-05-11T23:03:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01763986 2024-05-11T23:03:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152896+0000", 2024-05-11T23:03:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8573e-05 2024-05-11T23:03:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152968+0000", 2024-05-11T23:03:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1780999999999996e-05 2024-05-11T23:03:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153106+0000", 2024-05-11T23:03:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013777799999999999 2024-05-11T23:03:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:239544 3.1a 3:5fe59cad:::rbd_data.111ebabb4604.000000000000016e:head [stat,delete] snapc 2f=[2f] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 28.401448984999998, 2024-05-11T23:03:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.50514007500000002, 2024-05-11T23:03:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 239544 2024-05-11T23:03:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654592+0000", 2024-05-11T23:03:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.404e-06 2024-05-11T23:03:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654598+0000", 2024-05-11T23:03:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7660000000000004e-06 2024-05-11T23:03:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654611+0000", 2024-05-11T23:03:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2676e-05 2024-05-11T23:03:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654620+0000", 2024-05-11T23:03:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2990000000000002e-06 2024-05-11T23:03:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654698+0000", 2024-05-11T23:03:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7739999999999998e-05 2024-05-11T23:03:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654710+0000", 2024-05-11T23:03:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2648e-05 2024-05-11T23:03:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.157889+0000", 2024-05-11T23:03:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.50317850799999997 2024-05-11T23:03:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.158050+0000", 2024-05-11T23:03:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016084500000000001 2024-05-11T23:03:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.159731+0000", 2024-05-11T23:03:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001681189 2024-05-11T23:03:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262364 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2613248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.469576119999999, 2024-05-11T23:03:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016312508, 2024-05-11T23:03:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262364 2024-05-11T23:03:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.13e-07 2024-05-11T23:03:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586467+0000", 2024-05-11T23:03:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-05-11T23:03:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586474+0000", 2024-05-11T23:03:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7739999999999999e-06 2024-05-11T23:03:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586483+0000", 2024-05-11T23:03:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0340000000000002e-06 2024-05-11T23:03:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602629+0000", 2024-05-11T23:03:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016145845999999998 2024-05-11T23:03:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602703+0000", 2024-05-11T23:03:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4671000000000004e-05 2024-05-11T23:03:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602776+0000", 2024-05-11T23:03:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2793000000000006e-05 2024-05-11T23:03:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262417 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.467829820999999, 2024-05-11T23:03:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016403456, 2024-05-11T23:03:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262417 2024-05-11T23:03:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6199999999999999e-07 2024-05-11T23:03:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588212+0000", 2024-05-11T23:03:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.075e-06 2024-05-11T23:03:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588218+0000", 2024-05-11T23:03:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8150000000000002e-06 2024-05-11T23:03:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588223+0000", 2024-05-11T23:03:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7090000000000001e-06 2024-05-11T23:03:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604461+0000", 2024-05-11T23:03:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016237900999999999 2024-05-11T23:03:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604540+0000", 2024-05-11T23:03:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9376999999999996e-05 2024-05-11T23:03:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604613+0000", 2024-05-11T23:03:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3217000000000001e-05 2024-05-11T23:03:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262418 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2396160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.467809912, 2024-05-11T23:03:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016591180000000001, 2024-05-11T23:03:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262418 2024-05-11T23:03:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6100000000000001e-07 2024-05-11T23:03:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588232+0000", 2024-05-11T23:03:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0370000000000001e-06 2024-05-11T23:03:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588238+0000", 2024-05-11T23:03:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9340000000000003e-06 2024-05-11T23:03:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588243+0000", 2024-05-11T23:03:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6179999999999997e-06 2024-05-11T23:03:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604642+0000", 2024-05-11T23:03:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0163991 2024-05-11T23:03:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604712+0000", 2024-05-11T23:03:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9935e-05 2024-05-11T23:03:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604821+0000", 2024-05-11T23:03:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000109095 2024-05-11T23:03:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262419 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2433024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.467790594, 2024-05-11T23:03:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016799692000000001, 2024-05-11T23:03:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262419 2024-05-11T23:03:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588252+0000", 2024-05-11T23:03:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0990000000000001e-06 2024-05-11T23:03:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588257+0000", 2024-05-11T23:03:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8159999999999999e-06 2024-05-11T23:03:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588262+0000", 2024-05-11T23:03:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8069999999999999e-06 2024-05-11T23:03:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604868+0000", 2024-05-11T23:03:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016605880999999999 2024-05-11T23:03:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604966+0000", 2024-05-11T23:03:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7597999999999996e-05 2024-05-11T23:03:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605049+0000", 2024-05-11T23:03:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3141999999999998e-05 2024-05-11T23:03:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262420 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2469888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.467770482999999, 2024-05-11T23:03:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016972514000000001, 2024-05-11T23:03:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262420 2024-05-11T23:03:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588271+0000", 2024-05-11T23:03:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6199999999999999e-06 2024-05-11T23:03:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588273+0000", 2024-05-11T23:03:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3209999999999999e-06 2024-05-11T23:03:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588279+0000", 2024-05-11T23:03:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2519999999999996e-06 2024-05-11T23:03:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588285+0000", 2024-05-11T23:03:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1440000000000002e-06 2024-05-11T23:03:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605083+0000", 2024-05-11T23:03:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016798418999999998 2024-05-11T23:03:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605168+0000", 2024-05-11T23:03:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4851000000000005e-05 2024-05-11T23:03:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605242+0000", 2024-05-11T23:03:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3906999999999996e-05 2024-05-11T23:03:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262421 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2506752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.467748337, 2024-05-11T23:03:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017141427000000001, 2024-05-11T23:03:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262421 2024-05-11T23:03:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588292+0000", 2024-05-11T23:03:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588318+0000", 2024-05-11T23:03:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6376e-05 2024-05-11T23:03:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588326+0000", 2024-05-11T23:03:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6699999999999994e-06 2024-05-11T23:03:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588332+0000", 2024-05-11T23:03:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8540000000000002e-06 2024-05-11T23:03:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605274+0000", 2024-05-11T23:03:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016942024999999999 2024-05-11T23:03:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605360+0000", 2024-05-11T23:03:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6304999999999994e-05 2024-05-11T23:03:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605433+0000", 2024-05-11T23:03:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2772000000000001e-05 2024-05-11T23:03:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262422 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2543616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.467699112999998, 2024-05-11T23:03:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017283705, 2024-05-11T23:03:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262422 2024-05-11T23:03:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:03:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588343+0000", 2024-05-11T23:03:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.34e-06 2024-05-11T23:03:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588350+0000", 2024-05-11T23:03:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1569999999999997e-06 2024-05-11T23:03:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588354+0000", 2024-05-11T23:03:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.865e-06 2024-05-11T23:03:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605466+0000", 2024-05-11T23:03:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017111498999999999 2024-05-11T23:03:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605551+0000", 2024-05-11T23:03:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5210999999999994e-05 2024-05-11T23:03:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605624+0000", 2024-05-11T23:03:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3153999999999998e-05 2024-05-11T23:03:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262423 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2580480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.467678864, 2024-05-11T23:03:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017451194, 2024-05-11T23:03:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262423 2024-05-11T23:03:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588363+0000", 2024-05-11T23:03:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.855e-06 2024-05-11T23:03:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588369+0000", 2024-05-11T23:03:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1260000000000003e-06 2024-05-11T23:03:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588375+0000", 2024-05-11T23:03:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9630000000000003e-06 2024-05-11T23:03:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605656+0000", 2024-05-11T23:03:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017280865999999999 2024-05-11T23:03:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605737+0000", 2024-05-11T23:03:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0773999999999996e-05 2024-05-11T23:03:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605812+0000", 2024-05-11T23:03:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5203999999999994e-05 2024-05-11T23:03:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262424 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2617344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.467657345999999, 2024-05-11T23:03:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017617544999999998, 2024-05-11T23:03:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262424 2024-05-11T23:03:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588383+0000", 2024-05-11T23:03:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8299999999999997e-07 2024-05-11T23:03:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588385+0000", 2024-05-11T23:03:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2869999999999999e-06 2024-05-11T23:03:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588391+0000", 2024-05-11T23:03:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1689999999999997e-06 2024-05-11T23:03:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588396+0000", 2024-05-11T23:03:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5719999999999996e-06 2024-05-11T23:03:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605837+0000", 2024-05-11T23:03:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017440812999999999 2024-05-11T23:03:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605917+0000", 2024-05-11T23:03:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9924000000000003e-05 2024-05-11T23:03:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.606000+0000", 2024-05-11T23:03:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3296999999999999e-05 2024-05-11T23:03:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262896 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.396652709000001, 2024-05-11T23:03:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016295738000000001, 2024-05-11T23:03:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262896 2024-05-11T23:03:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659390+0000", 2024-05-11T23:03:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1440000000000001e-06 2024-05-11T23:03:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659397+0000", 2024-05-11T23:03:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0010000000000004e-06 2024-05-11T23:03:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659402+0000", 2024-05-11T23:03:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2179999999999998e-06 2024-05-11T23:03:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675487+0000", 2024-05-11T23:03:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016085706000000002 2024-05-11T23:03:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675580+0000", 2024-05-11T23:03:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2884e-05 2024-05-11T23:03:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675683+0000", 2024-05-11T23:03:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010236 2024-05-11T23:03:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262897 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.396617368000001, 2024-05-11T23:03:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016500932999999999, 2024-05-11T23:03:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262897 2024-05-11T23:03:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659423+0000", 2024-05-11T23:03:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9699999999999996e-07 2024-05-11T23:03:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659425+0000", 2024-05-11T23:03:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3199999999999998e-06 2024-05-11T23:03:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659433+0000", 2024-05-11T23:03:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3660000000000003e-06 2024-05-11T23:03:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659445+0000", 2024-05-11T23:03:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2102e-05 2024-05-11T23:03:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675727+0000", 2024-05-11T23:03:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016282331000000001 2024-05-11T23:03:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675815+0000", 2024-05-11T23:03:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7484999999999993e-05 2024-05-11T23:03:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675923+0000", 2024-05-11T23:03:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000108732 2024-05-11T23:03:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262898 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.396586249999999, 2024-05-11T23:03:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016685855999999999, 2024-05-11T23:03:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262898 2024-05-11T23:03:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659454+0000", 2024-05-11T23:03:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2200000000000004e-07 2024-05-11T23:03:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659457+0000", 2024-05-11T23:03:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.779e-06 2024-05-11T23:03:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659463+0000", 2024-05-11T23:03:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6660000000000002e-06 2024-05-11T23:03:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659472+0000", 2024-05-11T23:03:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2139999999999996e-06 2024-05-11T23:03:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675956+0000", 2024-05-11T23:03:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016484431000000001 2024-05-11T23:03:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676039+0000", 2024-05-11T23:03:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3358999999999999e-05 2024-05-11T23:03:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676139+0000", 2024-05-11T23:03:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9784999999999994e-05 2024-05-11T23:03:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262899 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 19.396560419, 2024-05-11T23:03:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017030087999999999, 2024-05-11T23:03:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262899 2024-05-11T23:03:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659480+0000", 2024-05-11T23:03:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7700000000000004e-07 2024-05-11T23:03:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659482+0000", 2024-05-11T23:03:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1390000000000002e-06 2024-05-11T23:03:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659489+0000", 2024-05-11T23:03:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8279999999999997e-06 2024-05-11T23:03:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659494+0000", 2024-05-11T23:03:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9019999999999998e-06 2024-05-11T23:03:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676167+0000", 2024-05-11T23:03:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016673621 2024-05-11T23:03:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676329+0000", 2024-05-11T23:03:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016206099999999999 2024-05-11T23:03:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676509+0000", 2024-05-11T23:03:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017996 2024-05-11T23:03:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:32.141 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:32.141 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:03:32.142 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:03:32.184 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:32.184+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:32.251 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:32.250+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:32.294 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:32.295 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:32.303 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-11T23:03:32.310 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:32.328 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:32.346 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:32.351 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:32.351+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:32.363 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3: {} 2024-05-11T23:03:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-11T23:03:32.381 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:32.399 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:32.401 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 5, 7, 3, 6] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:03:32.401 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:03:32.401 INFO:tasks.thrashosds.thrasher:Killing osd 3, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-11T23:03:32.402 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-11T23:03:32.402 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T23:03:32.418 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:32.418 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:32.428 INFO:teuthology.orchestra.run.smithi005.stderr:admin_socket: exception: exception: [Errno 104] Connection reset by peer 2024-05-11T23:03:32.433 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-11T23:03:32.434 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-11T23:03:32.438 INFO:tasks.ceph.osd.3:Stopped 2024-05-11T23:03:32.439 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.3 2024-05-11T23:03:32.439 INFO:tasks.thrashosds.thrasher:remote for osd 3 is ubuntu@smithi005.front.sepia.ceph.com 2024-05-11T23:03:32.439 DEBUG:teuthology.orchestra.run.smithi005:> type ceph-objectstore-tool 2024-05-11T23:03:32.441 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:32.454 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:32.452+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:32.462 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-05-11T23:03:32.462 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op list-pgs 2024-05-11T23:03:32.539 INFO:teuthology.orchestra.run.smithi005.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-05-11T23:03:32.542 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-11T23:03:32.542 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:03:32.583 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:32.583+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:32.655 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:32.655+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:32.661 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:32.661 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:32.670 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:03:32.755 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:32.755+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:32.798 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-05-11T23:03:32.822 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:32.822 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:32.832 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:03:32.876 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:32.876+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:32.940 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:32.940 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:32.948 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:03:32.989 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:32.989+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:33.057 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:33.057+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:33.093 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:33.093 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:33.102 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:03:33.158 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:33.158+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:33.206 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:33.206 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:33.214 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:03:33.259 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:33.259+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:33.319 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:33.319 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:33.328 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:03:33.371 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:33.371+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:33.433 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:33.433 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:33.441 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:03:33.483 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:33.482+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:33.561 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:33.560+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:33.586 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:33.586 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:33.595 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:03:33.661 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:33.661+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:33.704 INFO:teuthology.orchestra.run.smithi005.stdout:3.7 2024-05-11T23:03:33.705 INFO:teuthology.orchestra.run.smithi005.stdout:3.5 2024-05-11T23:03:33.705 INFO:teuthology.orchestra.run.smithi005.stdout:3.c 2024-05-11T23:03:33.705 INFO:teuthology.orchestra.run.smithi005.stdout:3.2 2024-05-11T23:03:33.705 INFO:teuthology.orchestra.run.smithi005.stdout:3.1b 2024-05-11T23:03:33.705 INFO:teuthology.orchestra.run.smithi005.stdout:3.8 2024-05-11T23:03:33.705 INFO:teuthology.orchestra.run.smithi005.stdout:3.1 2024-05-11T23:03:33.705 INFO:teuthology.orchestra.run.smithi005.stdout:3.1e 2024-05-11T23:03:33.705 INFO:teuthology.orchestra.run.smithi005.stdout:3.19 2024-05-11T23:03:33.705 INFO:teuthology.orchestra.run.smithi005.stdout:3.18 2024-05-11T23:03:33.705 INFO:teuthology.orchestra.run.smithi005.stdout:3.15 2024-05-11T23:03:33.705 INFO:teuthology.orchestra.run.smithi005.stdout:3.b 2024-05-11T23:03:33.705 INFO:teuthology.orchestra.run.smithi005.stdout:2.1 2024-05-11T23:03:33.741 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:33.741 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:33.750 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:03:33.762 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:33.762+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:33.863 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:33.863+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:33.871 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:33.871 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:33.881 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:03:33.964 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:33.964+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:34.031 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:34.032 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:34.041 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:03:34.084 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:34.084+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:34.150 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:34.150 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:34.159 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:03:34.166 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:34.166+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:34.232 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op export --pgid 3.15 --file /var/log/ceph/exp.3.15.3 2024-05-11T23:03:34.268 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:34.268+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:34.311 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:34.311 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:34.321 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:03:34.405 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:34.405+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:34.454 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:03:34.469 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:34.469+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:34.472 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:34.472 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:34.481 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:03:34.570 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:34.570+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:34.583 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:34.584 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:34.615 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:34.616 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:34.638 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:34.638 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:34.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:34.638 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:34.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.638 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:64049 3.8 3:11d98727:::rbd_object_map.111e8974a681:head [read 0~38] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-05-11T23:03:34.639 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:34.639 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 93.857580992999999, 2024-05-11T23:03:34.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99884604399999999, 2024-05-11T23:03:34.639 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.639 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.642 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:34.642 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.642 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.642 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.642 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 64049 2024-05-11T23:03:34.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.642 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:34.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:34.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779883+0000", 2024-05-11T23:03:34.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.57e-06 2024-05-11T23:03:34.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779887+0000", 2024-05-11T23:03:34.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5709999999999999e-06 2024-05-11T23:03:34.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779898+0000", 2024-05-11T23:03:34.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.042e-05 2024-05-11T23:03:34.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779905+0000", 2024-05-11T23:03:34.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0110000000000003e-06 2024-05-11T23:03:34.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779962+0000", 2024-05-11T23:03:34.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7395999999999997e-05 2024-05-11T23:03:34.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:34.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779969+0000", 2024-05-11T23:03:34.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2459999999999997e-06 2024-05-11T23:03:34.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778379+0000", 2024-05-11T23:03:34.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99840986300000001 2024-05-11T23:03:34.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:34.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778564+0000", 2024-05-11T23:03:34.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018420900000000001 2024-05-11T23:03:34.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:34.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778727+0000", 2024-05-11T23:03:34.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016375800000000001 2024-05-11T23:03:34.646 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.647 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.647 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172773 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:34.647 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:34.647 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.033474575, 2024-05-11T23:03:34.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058721316000000003, 2024-05-11T23:03:34.647 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.647 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.647 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:34.647 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.647 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.647 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.647 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172773 2024-05-11T23:03:34.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.648 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:34.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:34.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:34.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3500000000000002e-07 2024-05-11T23:03:34.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603990+0000", 2024-05-11T23:03:34.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3549999999999999e-06 2024-05-11T23:03:34.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603997+0000", 2024-05-11T23:03:34.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3400000000000003e-06 2024-05-11T23:03:34.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604002+0000", 2024-05-11T23:03:34.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0509999999999996e-06 2024-05-11T23:03:34.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662697+0000", 2024-05-11T23:03:34.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058695487999999997 2024-05-11T23:03:34.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:34.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662709+0000", 2024-05-11T23:03:34.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1647e-05 2024-05-11T23:03:34.650 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.650 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.651 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.651 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172774 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:34.651 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:34.651 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.033453193, 2024-05-11T23:03:34.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058745040999999998, 2024-05-11T23:03:34.651 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.651 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.651 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:34.651 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.651 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.651 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.651 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172774 2024-05-11T23:03:34.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.651 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:34.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:34.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:34.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:03:34.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604012+0000", 2024-05-11T23:03:34.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2199999999999999e-06 2024-05-11T23:03:34.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604018+0000", 2024-05-11T23:03:34.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1770000000000003e-06 2024-05-11T23:03:34.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604023+0000", 2024-05-11T23:03:34.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0089999999999998e-06 2024-05-11T23:03:34.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662742+0000", 2024-05-11T23:03:34.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058719187999999999 2024-05-11T23:03:34.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:34.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662754+0000", 2024-05-11T23:03:34.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2095e-05 2024-05-11T23:03:34.654 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.654 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.654 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.655 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172775 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:34.655 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:34.655 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.033432980999997, 2024-05-11T23:03:34.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058764743000000001, 2024-05-11T23:03:34.655 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.655 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.655 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:34.655 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.655 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.655 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.655 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172775 2024-05-11T23:03:34.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.655 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:34.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:34.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604030+0000", 2024-05-11T23:03:34.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6699999999999999e-07 2024-05-11T23:03:34.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604032+0000", 2024-05-11T23:03:34.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9530000000000002e-06 2024-05-11T23:03:34.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604037+0000", 2024-05-11T23:03:34.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8590000000000001e-06 2024-05-11T23:03:34.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604042+0000", 2024-05-11T23:03:34.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8609999999999997e-06 2024-05-11T23:03:34.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662782+0000", 2024-05-11T23:03:34.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058739731000000003 2024-05-11T23:03:34.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:34.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662794+0000", 2024-05-11T23:03:34.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1972e-05 2024-05-11T23:03:34.658 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.658 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.658 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.658 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172776 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:34.659 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:34.659 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.033413691, 2024-05-11T23:03:34.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058778696999999998, 2024-05-11T23:03:34.659 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.659 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.659 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:34.659 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.659 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.659 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.659 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172776 2024-05-11T23:03:34.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.659 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:34.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:34.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:34.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2000000000000001e-07 2024-05-11T23:03:34.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604051+0000", 2024-05-11T23:03:34.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2879999999999999e-06 2024-05-11T23:03:34.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604057+0000", 2024-05-11T23:03:34.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:03:34.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604062+0000", 2024-05-11T23:03:34.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6800000000000001e-06 2024-05-11T23:03:34.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662820+0000", 2024-05-11T23:03:34.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058758444999999999 2024-05-11T23:03:34.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:34.662 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662827+0000", 2024-05-11T23:03:34.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1930000000000004e-06 2024-05-11T23:03:34.662 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.662 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.662 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.662 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172777 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:34.662 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:34.662 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.033394385000001, 2024-05-11T23:03:34.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058799918999999999, 2024-05-11T23:03:34.663 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.663 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.663 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:34.663 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.663 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.663 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.663 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172777 2024-05-11T23:03:34.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.663 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.663 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:34.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:34.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:34.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:03:34.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604075+0000", 2024-05-11T23:03:34.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.109e-06 2024-05-11T23:03:34.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604083+0000", 2024-05-11T23:03:34.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9170000000000006e-06 2024-05-11T23:03:34.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604089+0000", 2024-05-11T23:03:34.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3399999999999997e-06 2024-05-11T23:03:34.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662859+0000", 2024-05-11T23:03:34.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058770172000000002 2024-05-11T23:03:34.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:34.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662868+0000", 2024-05-11T23:03:34.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0289999999999994e-06 2024-05-11T23:03:34.666 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.666 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.666 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.666 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172778 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:34.666 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:34.666 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.033366295, 2024-05-11T23:03:34.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058845435000000001, 2024-05-11T23:03:34.666 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.666 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:34.667 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.667 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.667 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172778 2024-05-11T23:03:34.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.667 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:34.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.667 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.667 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:34.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:34.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5400000000000002e-07 2024-05-11T23:03:34.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604099+0000", 2024-05-11T23:03:34.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4650000000000001e-06 2024-05-11T23:03:34.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604105+0000", 2024-05-11T23:03:34.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.314e-06 2024-05-11T23:03:34.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604110+0000", 2024-05-11T23:03:34.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6310000000000002e-06 2024-05-11T23:03:34.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662931+0000", 2024-05-11T23:03:34.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058820960999999998 2024-05-11T23:03:34.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:34.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662941+0000", 2024-05-11T23:03:34.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.061e-05 2024-05-11T23:03:34.670 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.670 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.670 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.670 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172779 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:34.670 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:34.670 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.033346135000002, 2024-05-11T23:03:34.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058859163999999999, 2024-05-11T23:03:34.670 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.671 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.671 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:34.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.671 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172779 2024-05-11T23:03:34.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.671 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:34.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.671 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.671 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:34.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:34.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:03:34.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604119+0000", 2024-05-11T23:03:34.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2529999999999999e-06 2024-05-11T23:03:34.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604125+0000", 2024-05-11T23:03:34.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1670000000000004e-06 2024-05-11T23:03:34.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604130+0000", 2024-05-11T23:03:34.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.702e-06 2024-05-11T23:03:34.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662967+0000", 2024-05-11T23:03:34.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058837792999999999 2024-05-11T23:03:34.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:34.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662975+0000", 2024-05-11T23:03:34.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8490000000000006e-06 2024-05-11T23:03:34.674 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.674 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.674 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.674 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172780 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:34.674 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:34.674 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.033325984999998, 2024-05-11T23:03:34.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058872696000000002, 2024-05-11T23:03:34.674 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.674 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.675 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:34.675 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.675 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.675 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.675 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172780 2024-05-11T23:03:34.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.675 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.675 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.675 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:34.675 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.675 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:34.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.676 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.676 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604137+0000", 2024-05-11T23:03:34.676 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4400000000000001e-07 2024-05-11T23:03:34.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604139+0000", 2024-05-11T23:03:34.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3429999999999998e-06 2024-05-11T23:03:34.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604145+0000", 2024-05-11T23:03:34.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1940000000000003e-06 2024-05-11T23:03:34.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.677 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.677 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604150+0000", 2024-05-11T23:03:34.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:03:34.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.678 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663002+0000", 2024-05-11T23:03:34.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058851829000000001 2024-05-11T23:03:34.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:34.678 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663009+0000", 2024-05-11T23:03:34.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.199e-06 2024-05-11T23:03:34.678 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.678 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.678 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.678 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172781 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:34.679 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:34.679 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.033294144999999, 2024-05-11T23:03:34.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058875553999999997, 2024-05-11T23:03:34.679 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.679 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.679 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:34.679 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.679 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.679 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.679 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172781 2024-05-11T23:03:34.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.679 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:34.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:34.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:34.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:34.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604171+0000", 2024-05-11T23:03:34.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.21e-06 2024-05-11T23:03:34.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604176+0000", 2024-05-11T23:03:34.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2970000000000003e-06 2024-05-11T23:03:34.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604179+0000", 2024-05-11T23:03:34.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8459999999999999e-06 2024-05-11T23:03:34.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663033+0000", 2024-05-11T23:03:34.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058854565999999997 2024-05-11T23:03:34.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:34.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663044+0000", 2024-05-11T23:03:34.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0229e-05 2024-05-11T23:03:34.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.682 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.682 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172782 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:34.682 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:34.682 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.033279397000001, 2024-05-11T23:03:34.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058894375999999998, 2024-05-11T23:03:34.683 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.683 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.683 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:34.683 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.683 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.683 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.683 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172782 2024-05-11T23:03:34.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.683 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:34.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:34.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:34.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-05-11T23:03:34.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604184+0000", 2024-05-11T23:03:34.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:03:34.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604187+0000", 2024-05-11T23:03:34.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.044e-06 2024-05-11T23:03:34.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604190+0000", 2024-05-11T23:03:34.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6460000000000002e-06 2024-05-11T23:03:34.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663070+0000", 2024-05-11T23:03:34.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058879510000000003 2024-05-11T23:03:34.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:34.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663077+0000", 2024-05-11T23:03:34.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6669999999999996e-06 2024-05-11T23:03:34.686 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.686 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.686 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.687 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172783 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:34.687 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:34.687 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.033269077999996, 2024-05-11T23:03:34.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058913946000000002, 2024-05-11T23:03:34.687 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.687 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.687 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:34.687 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.687 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.687 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.687 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172783 2024-05-11T23:03:34.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.687 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:34.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:34.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:34.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-05-11T23:03:34.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604195+0000", 2024-05-11T23:03:34.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.19e-06 2024-05-11T23:03:34.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604198+0000", 2024-05-11T23:03:34.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:03:34.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604200+0000", 2024-05-11T23:03:34.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4719999999999998e-06 2024-05-11T23:03:34.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663100+0000", 2024-05-11T23:03:34.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058900300000000003 2024-05-11T23:03:34.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.690 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:34.690 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663107+0000", 2024-05-11T23:03:34.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7449999999999998e-06 2024-05-11T23:03:34.690 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.690 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.690 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.690 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172784 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:34.690 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:34.690 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.033259059000002, 2024-05-11T23:03:34.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933728999999997, 2024-05-11T23:03:34.691 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.691 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.691 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:34.691 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.691 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.691 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.691 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172784 2024-05-11T23:03:34.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.691 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:34.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:34.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:34.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.03e-07 2024-05-11T23:03:34.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604205+0000", 2024-05-11T23:03:34.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2300000000000001e-06 2024-05-11T23:03:34.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604207+0000", 2024-05-11T23:03:34.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8729999999999999e-06 2024-05-11T23:03:34.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604210+0000", 2024-05-11T23:03:34.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.858e-06 2024-05-11T23:03:34.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.693 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663130+0000", 2024-05-11T23:03:34.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058919314 2024-05-11T23:03:34.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.693 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:34.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663137+0000", 2024-05-11T23:03:34.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2509999999999997e-06 2024-05-11T23:03:34.694 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.694 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.694 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.694 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172785 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:34.694 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:34.694 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.033246789000003, 2024-05-11T23:03:34.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058951664000000001, 2024-05-11T23:03:34.694 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.694 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.694 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:34.694 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.695 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.695 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172785 2024-05-11T23:03:34.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.695 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:34.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.695 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.695 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:34.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604216+0000", 2024-05-11T23:03:34.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:03:34.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604218+0000", 2024-05-11T23:03:34.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7549999999999999e-06 2024-05-11T23:03:34.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604222+0000", 2024-05-11T23:03:34.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6700000000000002e-06 2024-05-11T23:03:34.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604227+0000", 2024-05-11T23:03:34.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4399999999999998e-06 2024-05-11T23:03:34.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663160+0000", 2024-05-11T23:03:34.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933119999999999 2024-05-11T23:03:34.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:34.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663167+0000", 2024-05-11T23:03:34.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1840000000000002e-06 2024-05-11T23:03:34.697 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.698 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.698 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.698 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172786 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:34.698 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:34.698 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.033229734999999, 2024-05-11T23:03:34.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058966991000000003, 2024-05-11T23:03:34.698 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.698 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.698 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:34.698 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.698 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.698 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.698 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172786 2024-05-11T23:03:34.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.699 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:34.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:34.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.699 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.699 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604233+0000", 2024-05-11T23:03:34.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6899999999999999e-07 2024-05-11T23:03:34.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604235+0000", 2024-05-11T23:03:34.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8369999999999999e-06 2024-05-11T23:03:34.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604240+0000", 2024-05-11T23:03:34.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0329999999999999e-06 2024-05-11T23:03:34.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604244+0000", 2024-05-11T23:03:34.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9859999999999996e-06 2024-05-11T23:03:34.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663192+0000", 2024-05-11T23:03:34.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058948852000000003 2024-05-11T23:03:34.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:34.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663199+0000", 2024-05-11T23:03:34.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0140000000000001e-06 2024-05-11T23:03:34.701 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.701 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.701 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.701 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172787 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:34.702 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:34.702 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.033213388999997, 2024-05-11T23:03:34.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058982018999999997, 2024-05-11T23:03:34.702 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.702 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.702 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:34.702 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.702 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.702 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.702 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172787 2024-05-11T23:03:34.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.702 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:34.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:34.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605916+0000", 2024-05-11T23:03:34.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00166752 2024-05-11T23:03:34.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605921+0000", 2024-05-11T23:03:34.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0669999999999999e-06 2024-05-11T23:03:34.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.704 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605934+0000", 2024-05-11T23:03:34.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2271e-05 2024-05-11T23:03:34.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.704 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605940+0000", 2024-05-11T23:03:34.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6880000000000002e-06 2024-05-11T23:03:34.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.704 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663224+0000", 2024-05-11T23:03:34.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057283251 2024-05-11T23:03:34.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:34.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663231+0000", 2024-05-11T23:03:34.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2219999999999996e-06 2024-05-11T23:03:34.705 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.705 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.705 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.705 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172793 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:34.705 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:34.705 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.031446541999998, 2024-05-11T23:03:34.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057402373999999999, 2024-05-11T23:03:34.706 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.706 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.706 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:34.706 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.707 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.707 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.707 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172793 2024-05-11T23:03:34.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.707 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:34.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:34.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:34.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3300000000000001e-07 2024-05-11T23:03:34.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606017+0000", 2024-05-11T23:03:34.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4920000000000001e-06 2024-05-11T23:03:34.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606020+0000", 2024-05-11T23:03:34.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:03:34.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606023+0000", 2024-05-11T23:03:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6680000000000001e-06 2024-05-11T23:03:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663411+0000", 2024-05-11T23:03:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057388206999999997 2024-05-11T23:03:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663418+0000", 2024-05-11T23:03:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.815e-06 2024-05-11T23:03:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172795 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.031405114999998, 2024-05-11T23:03:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057423015000000001, 2024-05-11T23:03:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172795 2024-05-11T23:03:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:34.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9499999999999998e-07 2024-05-11T23:03:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606059+0000", 2024-05-11T23:03:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-05-11T23:03:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606062+0000", 2024-05-11T23:03:34.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.977e-06 2024-05-11T23:03:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606064+0000", 2024-05-11T23:03:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7410000000000001e-06 2024-05-11T23:03:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663473+0000", 2024-05-11T23:03:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057408624999999998 2024-05-11T23:03:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663480+0000", 2024-05-11T23:03:34.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1049999999999997e-06 2024-05-11T23:03:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172796 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.031394552000002, 2024-05-11T23:03:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057449235000000001, 2024-05-11T23:03:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.714 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172796 2024-05-11T23:03:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606069+0000", 2024-05-11T23:03:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:03:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606072+0000", 2024-05-11T23:03:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8700000000000001e-06 2024-05-11T23:03:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606074+0000", 2024-05-11T23:03:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6230000000000001e-06 2024-05-11T23:03:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663510+0000", 2024-05-11T23:03:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05743529 2024-05-11T23:03:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663517+0000", 2024-05-11T23:03:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1230000000000002e-06 2024-05-11T23:03:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172797 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 36.031384520000003, 2024-05-11T23:03:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057470420000000001, 2024-05-11T23:03:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172797 2024-05-11T23:03:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607915+0000", 2024-05-11T23:03:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018375609999999999 2024-05-11T23:03:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607920+0000", 2024-05-11T23:03:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8899999999999998e-06 2024-05-11T23:03:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607930+0000", 2024-05-11T23:03:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4429999999999992e-06 2024-05-11T23:03:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607936+0000", 2024-05-11T23:03:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7089999999999996e-06 2024-05-11T23:03:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663541+0000", 2024-05-11T23:03:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055604537000000002 2024-05-11T23:03:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663548+0000", 2024-05-11T23:03:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2799999999999998e-06 2024-05-11T23:03:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.721 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.721 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:34.723 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:03:34.723 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-05-11T23:03:34.724 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:34.725 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:34.725 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:34.723+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:34.816 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-11T23:03:34.824 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:34.823+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:34.885 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:34.885 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:34.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.885 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:281050 3.9 3:943c3e4a:::rbd_header.111e913a615f:head [watch watch cookie 139809575191376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-05-11T23:03:34.885 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:34.884740+0000", 2024-05-11T23:03:34.885 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.00068677500000000004, 2024-05-11T23:03:34.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00071328299999999999, 2024-05-11T23:03:34.885 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:34.885 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:34.885 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "waiting for sub ops", 2024-05-11T23:03:34.886 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:34.886 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:34.886 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:34.886 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 281050 2024-05-11T23:03:34.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.886 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:34.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:34.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:34.884740+0000", 2024-05-11T23:03:34.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:34.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:34.884740+0000", 2024-05-11T23:03:34.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:34.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:34.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:34.884740+0000", 2024-05-11T23:03:34.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3099999999999997e-07 2024-05-11T23:03:34.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:34.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:34.884744+0000", 2024-05-11T23:03:34.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1109999999999999e-06 2024-05-11T23:03:34.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:34.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:34.884752+0000", 2024-05-11T23:03:34.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9209999999999998e-06 2024-05-11T23:03:34.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:34.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:34.884757+0000", 2024-05-11T23:03:34.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8849999999999998e-06 2024-05-11T23:03:34.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.888 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:34.888 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:34.884791+0000", 2024-05-11T23:03:34.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3170999999999998e-05 2024-05-11T23:03:34.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:34.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:34.884832+0000", 2024-05-11T23:03:34.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.1217999999999997e-05 2024-05-11T23:03:34.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:34.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:34.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for subops from 7", 2024-05-11T23:03:34.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:34.884934+0000", 2024-05-11T23:03:34.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010254700000000001 2024-05-11T23:03:34.889 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.889 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:34.889 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.889 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:34.890 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-05-11T23:03:34.890 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 1 2024-05-11T23:03:34.890 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:34.896 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:03:34.912 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:34.912 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:34.924 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:34.924+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:34.934 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:34.935 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:34.938 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-11T23:03:34.939 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-11T23:03:35.014 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:35.015 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:35.015 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:35.015 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:35.015 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:35.024 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:03:35.025 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:35.025+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:35.126 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:35.126+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:35.136 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:35.137 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:35.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:35.137 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:35.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.137 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:77128 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_get in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-11T23:03:35.137 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:35.137 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 80.119988694, 2024-05-11T23:03:35.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91737855899999998, 2024-05-11T23:03:35.137 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:35.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:35.137 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:35.137 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:35.137 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:35.138 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.138 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 77128 2024-05-11T23:03:35.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.138 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:35.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:35.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:35.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:35.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:35.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:35.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016003+0000", 2024-05-11T23:03:35.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5379999999999999e-06 2024-05-11T23:03:35.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:35.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016011+0000", 2024-05-11T23:03:35.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6410000000000002e-06 2024-05-11T23:03:35.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:35.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016026+0000", 2024-05-11T23:03:35.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4779e-05 2024-05-11T23:03:35.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016032+0000", 2024-05-11T23:03:35.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.066e-06 2024-05-11T23:03:35.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016093+0000", 2024-05-11T23:03:35.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0878000000000002e-05 2024-05-11T23:03:35.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:35.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016100+0000", 2024-05-11T23:03:35.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6799999999999993e-06 2024-05-11T23:03:35.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933302+0000", 2024-05-11T23:03:35.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91720135300000005 2024-05-11T23:03:35.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:35.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933380+0000", 2024-05-11T23:03:35.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8623999999999998e-05 2024-05-11T23:03:35.141 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.141 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.141 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.142 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98950 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:35.142 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:35.142 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 69.003907804999997, 2024-05-11T23:03:35.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021419438999999998, 2024-05-11T23:03:35.142 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:35.142 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:35.142 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:35.142 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:35.142 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:35.142 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.142 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98950 2024-05-11T23:03:35.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.142 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:35.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:35.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:35.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:35.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:35.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:35.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:35.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.34e-07 2024-05-11T23:03:35.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:35.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132085+0000", 2024-05-11T23:03:35.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0049999999999999e-06 2024-05-11T23:03:35.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:35.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132093+0000", 2024-05-11T23:03:35.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6660000000000008e-06 2024-05-11T23:03:35.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132097+0000", 2024-05-11T23:03:35.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2590000000000002e-06 2024-05-11T23:03:35.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135114+0000", 2024-05-11T23:03:35.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003017573 2024-05-11T23:03:35.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:35.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153416+0000", 2024-05-11T23:03:35.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018301884000000001 2024-05-11T23:03:35.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:35.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153502+0000", 2024-05-11T23:03:35.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5717999999999995e-05 2024-05-11T23:03:35.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.145 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.146 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98959 3.1e 3:7807da99:::rbd_data.111ed2934718.000000000000006d:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:35.146 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:35.146 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 69.003415067999995, 2024-05-11T23:03:35.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020525928999999998, 2024-05-11T23:03:35.146 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:35.146 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:35.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:35.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:35.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:35.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.146 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98959 2024-05-11T23:03:35.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.146 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:35.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:35.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:35.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:35.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:35.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:35.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132576+0000", 2024-05-11T23:03:35.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4199999999999995e-07 2024-05-11T23:03:35.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:35.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132580+0000", 2024-05-11T23:03:35.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2910000000000001e-06 2024-05-11T23:03:35.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:35.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132588+0000", 2024-05-11T23:03:35.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7970000000000001e-06 2024-05-11T23:03:35.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132602+0000", 2024-05-11T23:03:35.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3743999999999999e-05 2024-05-11T23:03:35.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132655+0000", 2024-05-11T23:03:35.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2714999999999999e-05 2024-05-11T23:03:35.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:35.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132787+0000", 2024-05-11T23:03:35.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013273899999999999 2024-05-11T23:03:35.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:35.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153101+0000", 2024-05-11T23:03:35.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020313900999999999 2024-05-11T23:03:35.149 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.149 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.150 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.150 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98965 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:35.150 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:35.150 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 69.000891432000003, 2024-05-11T23:03:35.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018608983999999999, 2024-05-11T23:03:35.150 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:35.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:35.150 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:35.150 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:35.150 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:35.150 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.150 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98965 2024-05-11T23:03:35.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.151 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:35.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:35.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:35.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:35.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:35.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:35.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135100+0000", 2024-05-11T23:03:35.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.17e-06 2024-05-11T23:03:35.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:35.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135105+0000", 2024-05-11T23:03:35.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.899e-06 2024-05-11T23:03:35.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:35.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135122+0000", 2024-05-11T23:03:35.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7346999999999998e-05 2024-05-11T23:03:35.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135130+0000", 2024-05-11T23:03:35.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0639999999999994e-06 2024-05-11T23:03:35.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153533+0000", 2024-05-11T23:03:35.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018402753000000001 2024-05-11T23:03:35.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:35.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153615+0000", 2024-05-11T23:03:35.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1720999999999999e-05 2024-05-11T23:03:35.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:35.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153708+0000", 2024-05-11T23:03:35.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3029999999999995e-05 2024-05-11T23:03:35.154 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.154 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.154 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.154 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98966 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3760128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:35.154 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:35.154 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 69.000819305999997, 2024-05-11T23:03:35.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018716535999999999, 2024-05-11T23:03:35.154 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:35.154 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:35.154 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:35.154 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:35.154 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:35.155 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.155 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98966 2024-05-11T23:03:35.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.155 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:35.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:35.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:35.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:35.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:35.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:35.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135172+0000", 2024-05-11T23:03:35.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-05-11T23:03:35.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:35.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135175+0000", 2024-05-11T23:03:35.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-05-11T23:03:35.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:35.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135184+0000", 2024-05-11T23:03:35.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1840000000000005e-06 2024-05-11T23:03:35.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152903+0000", 2024-05-11T23:03:35.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017718357000000001 2024-05-11T23:03:35.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153736+0000", 2024-05-11T23:03:35.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00083324799999999997 2024-05-11T23:03:35.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:35.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153809+0000", 2024-05-11T23:03:35.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2995000000000002e-05 2024-05-11T23:03:35.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:35.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153888+0000", 2024-05-11T23:03:35.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8695999999999993e-05 2024-05-11T23:03:35.158 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.158 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.158 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.158 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "pg_update_log_missing_reply(3.1e epoch 72/53 rep_tid 104 lcod 72'363)", 2024-05-11T23:03:35.158 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:35.158 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 69.000756409999994, 2024-05-11T23:03:35.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017871951000000001, 2024-05-11T23:03:35.159 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:35.159 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:35.159 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:35.159 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:35.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:35.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:35.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:35.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:35.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135235+0000", 2024-05-11T23:03:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8899999999999999e-07 2024-05-11T23:03:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135238+0000", 2024-05-11T23:03:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.27e-06 2024-05-11T23:03:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152878+0000", 2024-05-11T23:03:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01763986 2024-05-11T23:03:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152896+0000", 2024-05-11T23:03:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8573e-05 2024-05-11T23:03:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152968+0000", 2024-05-11T23:03:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1780999999999996e-05 2024-05-11T23:03:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153106+0000", 2024-05-11T23:03:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013777799999999999 2024-05-11T23:03:35.161 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.162 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.162 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.162 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:239544 3.1a 3:5fe59cad:::rbd_data.111ebabb4604.000000000000016e:head [stat,delete] snapc 2f=[2f] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:35.162 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:35.162 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 31.481399677999999, 2024-05-11T23:03:35.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.50514007500000002, 2024-05-11T23:03:35.162 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:35.162 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:35.162 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:35.162 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:35.162 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:35.162 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.162 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 239544 2024-05-11T23:03:35.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.163 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:35.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:35.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:35.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:35.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:35.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:35.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654592+0000", 2024-05-11T23:03:35.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.404e-06 2024-05-11T23:03:35.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:35.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654598+0000", 2024-05-11T23:03:35.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7660000000000004e-06 2024-05-11T23:03:35.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:35.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654611+0000", 2024-05-11T23:03:35.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2676e-05 2024-05-11T23:03:35.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654620+0000", 2024-05-11T23:03:35.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2990000000000002e-06 2024-05-11T23:03:35.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654698+0000", 2024-05-11T23:03:35.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7739999999999998e-05 2024-05-11T23:03:35.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:35.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654710+0000", 2024-05-11T23:03:35.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2648e-05 2024-05-11T23:03:35.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.157889+0000", 2024-05-11T23:03:35.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.50317850799999997 2024-05-11T23:03:35.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:35.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.158050+0000", 2024-05-11T23:03:35.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016084500000000001 2024-05-11T23:03:35.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:35.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.159731+0000", 2024-05-11T23:03:35.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001681189 2024-05-11T23:03:35.166 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.166 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.166 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.167 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262364 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2613248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:35.167 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:35.167 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.549526813, 2024-05-11T23:03:35.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016312508, 2024-05-11T23:03:35.167 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:35.167 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:35.167 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:35.167 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:35.167 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:35.167 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.167 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262364 2024-05-11T23:03:35.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.167 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:35.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:35.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:35.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:35.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:35.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:35.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:35.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.13e-07 2024-05-11T23:03:35.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:35.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586467+0000", 2024-05-11T23:03:35.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-05-11T23:03:35.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:35.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586474+0000", 2024-05-11T23:03:35.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7739999999999999e-06 2024-05-11T23:03:35.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586483+0000", 2024-05-11T23:03:35.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0340000000000002e-06 2024-05-11T23:03:35.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602629+0000", 2024-05-11T23:03:35.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016145845999999998 2024-05-11T23:03:35.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:35.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602703+0000", 2024-05-11T23:03:35.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4671000000000004e-05 2024-05-11T23:03:35.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:35.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602776+0000", 2024-05-11T23:03:35.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2793000000000006e-05 2024-05-11T23:03:35.171 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.171 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.171 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.171 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262417 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:35.171 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:35.171 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.547780513999999, 2024-05-11T23:03:35.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016403456, 2024-05-11T23:03:35.171 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:35.171 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:35.171 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:35.171 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:35.171 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:35.171 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.171 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262417 2024-05-11T23:03:35.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.172 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:35.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:35.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:35.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:35.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:35.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:35.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:35.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6199999999999999e-07 2024-05-11T23:03:35.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:35.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588212+0000", 2024-05-11T23:03:35.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.075e-06 2024-05-11T23:03:35.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:35.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588218+0000", 2024-05-11T23:03:35.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8150000000000002e-06 2024-05-11T23:03:35.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588223+0000", 2024-05-11T23:03:35.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7090000000000001e-06 2024-05-11T23:03:35.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604461+0000", 2024-05-11T23:03:35.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016237900999999999 2024-05-11T23:03:35.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:35.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604540+0000", 2024-05-11T23:03:35.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9376999999999996e-05 2024-05-11T23:03:35.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:35.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604613+0000", 2024-05-11T23:03:35.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3217000000000001e-05 2024-05-11T23:03:35.175 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.175 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.175 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.175 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262418 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2396160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:35.175 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:35.175 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.547760605000001, 2024-05-11T23:03:35.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016591180000000001, 2024-05-11T23:03:35.175 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:35.175 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:35.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:35.175 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:35.175 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:35.176 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.176 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262418 2024-05-11T23:03:35.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.176 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:35.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:35.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:35.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:35.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:35.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:35.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:35.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6100000000000001e-07 2024-05-11T23:03:35.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:35.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588232+0000", 2024-05-11T23:03:35.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0370000000000001e-06 2024-05-11T23:03:35.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:35.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588238+0000", 2024-05-11T23:03:35.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9340000000000003e-06 2024-05-11T23:03:35.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588243+0000", 2024-05-11T23:03:35.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6179999999999997e-06 2024-05-11T23:03:35.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604642+0000", 2024-05-11T23:03:35.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0163991 2024-05-11T23:03:35.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:35.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604712+0000", 2024-05-11T23:03:35.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9935e-05 2024-05-11T23:03:35.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:35.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604821+0000", 2024-05-11T23:03:35.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000109095 2024-05-11T23:03:35.179 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.179 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.179 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.179 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262419 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2433024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:35.179 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:35.179 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.547741287000001, 2024-05-11T23:03:35.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016799692000000001, 2024-05-11T23:03:35.179 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:35.179 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:35.179 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:35.179 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:35.180 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:35.180 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.180 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262419 2024-05-11T23:03:35.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.180 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:35.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:35.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:35.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:35.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:35.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:35.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:35.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:35.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:35.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588252+0000", 2024-05-11T23:03:35.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0990000000000001e-06 2024-05-11T23:03:35.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:35.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588257+0000", 2024-05-11T23:03:35.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8159999999999999e-06 2024-05-11T23:03:35.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588262+0000", 2024-05-11T23:03:35.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8069999999999999e-06 2024-05-11T23:03:35.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604868+0000", 2024-05-11T23:03:35.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016605880999999999 2024-05-11T23:03:35.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:35.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604966+0000", 2024-05-11T23:03:35.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7597999999999996e-05 2024-05-11T23:03:35.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:35.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605049+0000", 2024-05-11T23:03:35.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3141999999999998e-05 2024-05-11T23:03:35.183 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.183 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.183 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.183 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262420 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2469888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:35.183 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:35.183 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.547721176, 2024-05-11T23:03:35.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016972514000000001, 2024-05-11T23:03:35.183 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:35.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:35.183 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:35.184 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:35.184 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:35.184 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.184 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262420 2024-05-11T23:03:35.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.184 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:35.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:35.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:35.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:35.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:35.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:35.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588271+0000", 2024-05-11T23:03:35.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6199999999999999e-06 2024-05-11T23:03:35.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:35.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588273+0000", 2024-05-11T23:03:35.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3209999999999999e-06 2024-05-11T23:03:35.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:35.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588279+0000", 2024-05-11T23:03:35.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2519999999999996e-06 2024-05-11T23:03:35.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588285+0000", 2024-05-11T23:03:35.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1440000000000002e-06 2024-05-11T23:03:35.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605083+0000", 2024-05-11T23:03:35.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016798418999999998 2024-05-11T23:03:35.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:35.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605168+0000", 2024-05-11T23:03:35.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4851000000000005e-05 2024-05-11T23:03:35.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:35.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605242+0000", 2024-05-11T23:03:35.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3906999999999996e-05 2024-05-11T23:03:35.187 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.187 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.187 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.188 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262421 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2506752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:35.188 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:35.188 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.54769903, 2024-05-11T23:03:35.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017141427000000001, 2024-05-11T23:03:35.188 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:35.188 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:35.188 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:35.188 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:35.188 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:35.188 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.188 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262421 2024-05-11T23:03:35.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.188 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:35.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:35.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:35.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:35.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:35.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:35.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588292+0000", 2024-05-11T23:03:35.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:35.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:35.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588318+0000", 2024-05-11T23:03:35.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6376e-05 2024-05-11T23:03:35.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:35.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588326+0000", 2024-05-11T23:03:35.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6699999999999994e-06 2024-05-11T23:03:35.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588332+0000", 2024-05-11T23:03:35.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8540000000000002e-06 2024-05-11T23:03:35.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605274+0000", 2024-05-11T23:03:35.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016942024999999999 2024-05-11T23:03:35.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:35.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605360+0000", 2024-05-11T23:03:35.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6304999999999994e-05 2024-05-11T23:03:35.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.191 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:35.191 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605433+0000", 2024-05-11T23:03:35.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2772000000000001e-05 2024-05-11T23:03:35.191 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.191 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.191 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.192 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262422 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2543616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:35.192 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:35.192 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.547649805999999, 2024-05-11T23:03:35.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017283705, 2024-05-11T23:03:35.192 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:35.192 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:35.192 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:35.192 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:35.192 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:35.192 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.192 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262422 2024-05-11T23:03:35.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.192 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:35.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:35.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:35.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:35.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:35.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:35.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:35.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:03:35.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:35.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588343+0000", 2024-05-11T23:03:35.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.34e-06 2024-05-11T23:03:35.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:35.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588350+0000", 2024-05-11T23:03:35.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1569999999999997e-06 2024-05-11T23:03:35.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588354+0000", 2024-05-11T23:03:35.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.865e-06 2024-05-11T23:03:35.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605466+0000", 2024-05-11T23:03:35.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017111498999999999 2024-05-11T23:03:35.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:35.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605551+0000", 2024-05-11T23:03:35.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5210999999999994e-05 2024-05-11T23:03:35.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:35.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605624+0000", 2024-05-11T23:03:35.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3153999999999998e-05 2024-05-11T23:03:35.195 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.195 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.196 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.196 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262423 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2580480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:35.196 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:35.196 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.547629557, 2024-05-11T23:03:35.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017451194, 2024-05-11T23:03:35.196 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:35.196 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:35.196 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:35.196 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:35.196 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:35.196 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.196 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262423 2024-05-11T23:03:35.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.197 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:35.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:35.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:35.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:35.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:35.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:35.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:35.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:35.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:35.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588363+0000", 2024-05-11T23:03:35.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.855e-06 2024-05-11T23:03:35.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:35.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588369+0000", 2024-05-11T23:03:35.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1260000000000003e-06 2024-05-11T23:03:35.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588375+0000", 2024-05-11T23:03:35.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9630000000000003e-06 2024-05-11T23:03:35.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605656+0000", 2024-05-11T23:03:35.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017280865999999999 2024-05-11T23:03:35.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:35.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605737+0000", 2024-05-11T23:03:35.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0773999999999996e-05 2024-05-11T23:03:35.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:35.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605812+0000", 2024-05-11T23:03:35.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5203999999999994e-05 2024-05-11T23:03:35.200 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.200 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.200 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.200 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262424 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2617344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:35.200 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:35.200 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.547608039, 2024-05-11T23:03:35.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017617544999999998, 2024-05-11T23:03:35.200 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:35.201 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:35.201 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:35.201 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:35.201 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:35.201 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.201 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262424 2024-05-11T23:03:35.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.201 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:35.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:35.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:35.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:35.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:35.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:35.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588383+0000", 2024-05-11T23:03:35.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8299999999999997e-07 2024-05-11T23:03:35.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:35.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588385+0000", 2024-05-11T23:03:35.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2869999999999999e-06 2024-05-11T23:03:35.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:35.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588391+0000", 2024-05-11T23:03:35.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1689999999999997e-06 2024-05-11T23:03:35.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588396+0000", 2024-05-11T23:03:35.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5719999999999996e-06 2024-05-11T23:03:35.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605837+0000", 2024-05-11T23:03:35.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017440812999999999 2024-05-11T23:03:35.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:35.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605917+0000", 2024-05-11T23:03:35.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9924000000000003e-05 2024-05-11T23:03:35.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:35.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.606000+0000", 2024-05-11T23:03:35.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3296999999999999e-05 2024-05-11T23:03:35.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.204 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.204 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262896 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:35.204 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:35.204 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.476603401999999, 2024-05-11T23:03:35.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016295738000000001, 2024-05-11T23:03:35.205 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:35.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:35.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:35.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:35.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:35.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262896 2024-05-11T23:03:35.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.205 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:35.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:35.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:35.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:35.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:35.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:35.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:35.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:35.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:35.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659390+0000", 2024-05-11T23:03:35.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1440000000000001e-06 2024-05-11T23:03:35.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:35.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659397+0000", 2024-05-11T23:03:35.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0010000000000004e-06 2024-05-11T23:03:35.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659402+0000", 2024-05-11T23:03:35.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2179999999999998e-06 2024-05-11T23:03:35.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675487+0000", 2024-05-11T23:03:35.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016085706000000002 2024-05-11T23:03:35.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:35.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675580+0000", 2024-05-11T23:03:35.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2884e-05 2024-05-11T23:03:35.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:35.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675683+0000", 2024-05-11T23:03:35.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010236 2024-05-11T23:03:35.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.208 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.208 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262897 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.476568060999998, 2024-05-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016500932999999999, 2024-05-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262897 2024-05-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:35.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:35.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:35.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:35.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:35.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659423+0000", 2024-05-11T23:03:35.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9699999999999996e-07 2024-05-11T23:03:35.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:35.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659425+0000", 2024-05-11T23:03:35.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3199999999999998e-06 2024-05-11T23:03:35.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:35.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659433+0000", 2024-05-11T23:03:35.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3660000000000003e-06 2024-05-11T23:03:35.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659445+0000", 2024-05-11T23:03:35.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2102e-05 2024-05-11T23:03:35.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675727+0000", 2024-05-11T23:03:35.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016282331000000001 2024-05-11T23:03:35.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:35.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675815+0000", 2024-05-11T23:03:35.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7484999999999993e-05 2024-05-11T23:03:35.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:35.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675923+0000", 2024-05-11T23:03:35.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000108732 2024-05-11T23:03:35.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.212 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.213 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262898 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:35.213 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:35.213 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.476536942999999, 2024-05-11T23:03:35.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016685855999999999, 2024-05-11T23:03:35.213 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:35.213 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:35.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:35.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:35.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:35.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.213 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262898 2024-05-11T23:03:35.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.213 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:35.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:35.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:35.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:35.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:35.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:35.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659454+0000", 2024-05-11T23:03:35.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2200000000000004e-07 2024-05-11T23:03:35.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:35.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659457+0000", 2024-05-11T23:03:35.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.779e-06 2024-05-11T23:03:35.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:35.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659463+0000", 2024-05-11T23:03:35.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6660000000000002e-06 2024-05-11T23:03:35.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659472+0000", 2024-05-11T23:03:35.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2139999999999996e-06 2024-05-11T23:03:35.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675956+0000", 2024-05-11T23:03:35.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016484431000000001 2024-05-11T23:03:35.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:35.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676039+0000", 2024-05-11T23:03:35.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3358999999999999e-05 2024-05-11T23:03:35.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:35.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676139+0000", 2024-05-11T23:03:35.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9784999999999994e-05 2024-05-11T23:03:35.216 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.216 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.217 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262899 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:35.217 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:35.217 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 22.476511112000001, 2024-05-11T23:03:35.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017030087999999999, 2024-05-11T23:03:35.217 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:35.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:35.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:35.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:35.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:35.217 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.217 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262899 2024-05-11T23:03:35.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.218 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:35.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:35.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:35.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:35.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:35.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:35.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:35.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659480+0000", 2024-05-11T23:03:35.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7700000000000004e-07 2024-05-11T23:03:35.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:35.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659482+0000", 2024-05-11T23:03:35.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1390000000000002e-06 2024-05-11T23:03:35.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:35.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659489+0000", 2024-05-11T23:03:35.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8279999999999997e-06 2024-05-11T23:03:35.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659494+0000", 2024-05-11T23:03:35.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9019999999999998e-06 2024-05-11T23:03:35.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:35.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676167+0000", 2024-05-11T23:03:35.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016673621 2024-05-11T23:03:35.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:35.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676329+0000", 2024-05-11T23:03:35.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016206099999999999 2024-05-11T23:03:35.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:35.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:35.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:35.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676509+0000", 2024-05-11T23:03:35.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017996 2024-05-11T23:03:35.220 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.221 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:35.221 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:35.221 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:35.222 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:03:35.266 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:35.266+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:35.329 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:35.329+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:35.337 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:35.338 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:35.338 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:35.338 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:35.347 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:03:35.380 INFO:teuthology.orchestra.run.smithi005.stderr:Exporting 3.15 info 3.15( v 128'553 (0'0,128'553] local-lis/les=76/77 n=100 ec=19/19 lis/c=76/76 les/c/f=77/77/0 sis=76) 2024-05-11T23:03:35.391 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:a80b56a9:::rbd_data.111e913a615f.0000000000000011:25# 2024-05-11T23:03:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:a80b56a9:::rbd_data.111e913a615f.0000000000000011:head# 2024-05-11T23:03:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:a84b8d2b:::rbd_data.111ef9e47166.000000000000008b:head# 2024-05-11T23:03:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:a8580d10:::rbd_data.111ebabb4604.0000000000000066:head# 2024-05-11T23:03:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:a879acfe:::rbd_data.111ebabb4604.0000000000000065:head# 2024-05-11T23:03:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:a8c1dbf6:::rbd_data.111e64023279.0000000000000043:20# 2024-05-11T23:03:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:a8c1dbf6:::rbd_data.111e64023279.0000000000000043:head# 2024-05-11T23:03:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:a8c550e1:::rbd_data.111edca93686.00000000000001bf:head# 2024-05-11T23:03:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:a8cd2134:::rbd_data.111ef9e47166.00000000000000cf:head# 2024-05-11T23:03:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:a8eae155:::rbd_data.111e7d9444a9.0000000000000009:head# 2024-05-11T23:03:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:a8f4e01b:::rbd_data.111edca93686.0000000000000162:head# 2024-05-11T23:03:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:a9023db9:::rbd_data.111ef9e47166.00000000000000b8:head# 2024-05-11T23:03:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:a92399d6:::rbd_data.111e479bff4d.0000000000000014:head# 2024-05-11T23:03:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:a9403309:::rbd_header.111ef9b37d2d:head# 2024-05-11T23:03:35.396 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:a94c607e:::rbd_data.111ee4903046.000000000000003f:head# 2024-05-11T23:03:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:a9e165c9:::rbd_data.111e5046b689.0000000000000015:1c# 2024-05-11T23:03:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:a9e165c9:::rbd_data.111e5046b689.0000000000000015:head# 2024-05-11T23:03:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:a9edbb24:::rbd_data.111ef9e47166.0000000000000114:head# 2024-05-11T23:03:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:a9f92ff2:::rbd_data.111ebabb4604.0000000000000106:head# 2024-05-11T23:03:35.397 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:aa0090a8:::rbd_data.111ed003fd7f.0000000000000007:head# 2024-05-11T23:03:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:aa325950:::rbd_data.111ee4903046.0000000000000034:head# 2024-05-11T23:03:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:aa4fca24:::rbd_data.111ef9e47166.0000000000000086:head# 2024-05-11T23:03:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:aa8fbce6:::rbd_data.111ef9e47166.00000000000000be:head# 2024-05-11T23:03:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:aa97d1e4:::rbd_data.111edca93686.0000000000000092:head# 2024-05-11T23:03:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:aad7891d:::rbd_data.111e50384849.000000000000005b:head# 2024-05-11T23:03:35.406 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:aaf589ab:::rbd_data.111e67f27da3.0000000000000034:head# 2024-05-11T23:03:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:aaf73702:::rbd_object_map.111e5046b689.000000000000001c:head# 2024-05-11T23:03:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ab251d05:::rbd_data.111ecc0f42c3.00000000000000af:19# 2024-05-11T23:03:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ab251d05:::rbd_data.111ecc0f42c3.00000000000000af:head# 2024-05-11T23:03:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ab2d1875:::rbd_data.111e7d9444a9.0000000000000002:22# 2024-05-11T23:03:35.417 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ab2d1875:::rbd_data.111e7d9444a9.0000000000000002:head# 2024-05-11T23:03:35.421 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ab37ba60:::rbd_data.111ebabb4604.00000000000000cc:head# 2024-05-11T23:03:35.421 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ab38d3f7:::rbd_data.111ed2934718.000000000000006c:24# 2024-05-11T23:03:35.423 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ab38d3f7:::rbd_data.111ed2934718.000000000000006c:head# 2024-05-11T23:03:35.423 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ab61b29b:::rbd_data.111ebabb4604.0000000000000122:head# 2024-05-11T23:03:35.423 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ab835b8b:::rbd_data.111ef9b37d2d.0000000000000022:1b# 2024-05-11T23:03:35.423 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ab835b8b:::rbd_data.111ef9b37d2d.0000000000000022:head# 2024-05-11T23:03:35.423 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:abd0fbe1:::rbd_data.111ed8ed3219.000000000000002f:head# 2024-05-11T23:03:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:abedf2fd:::rbd_data.111e1e89614a.0000000000000040:head# 2024-05-11T23:03:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ac0c7ea5:::rbd_data.111ebabb4604.00000000000000b3:head# 2024-05-11T23:03:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ac0f692a:::rbd_data.111ebabb4604.00000000000000d2:head# 2024-05-11T23:03:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ac16c1a0:::rbd_data.111edca93686.0000000000000072:head# 2024-05-11T23:03:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ac4f80fb:::rbd_data.111e64023279.000000000000002f:head# 2024-05-11T23:03:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ac5f45c8:::rbd_data.111e5e4d8275.0000000000000022:head# 2024-05-11T23:03:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ac7055dc:::rbd_data.111ebabb4604.0000000000000095:head# 2024-05-11T23:03:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ac751812:::rbd_data.111ebabb4604.00000000000000cd:head# 2024-05-11T23:03:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ac759760:::rbd_data.111e50384849.000000000000005a:head# 2024-05-11T23:03:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ac978c95:::rbd_data.111edca93686.00000000000001d3:head# 2024-05-11T23:03:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:aca8c3c6:::rbd_data.111e50384849.0000000000000046:head# 2024-05-11T23:03:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:acc2e1e9:::rbd_data.111e67f27da3.0000000000000014:head# 2024-05-11T23:03:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:accdadf5:::rbd_data.111ed8ed3219.00000000000000b6:head# 2024-05-11T23:03:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ace3afa0:::rbd_data.111e5e4d8275.0000000000000015:11# 2024-05-11T23:03:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ace3afa0:::rbd_data.111e5e4d8275.0000000000000015:head# 2024-05-11T23:03:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:acfc6fa6:::rbd_data.111e4e122501.000000000000004e:f# 2024-05-11T23:03:35.432 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:acfc6fa6:::rbd_data.111e4e122501.000000000000004e:head# 2024-05-11T23:03:35.432 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ad116bb8:::rbd_data.111ecc0f42c3.0000000000000181:19# 2024-05-11T23:03:35.432 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ad116bb8:::rbd_data.111ecc0f42c3.0000000000000181:head# 2024-05-11T23:03:35.432 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ad1e8f6b:::rbd_data.111ebabb4604.00000000000000ce:head# 2024-05-11T23:03:35.432 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ad29dc5d:::rbd_data.111edca93686.0000000000000113:head# 2024-05-11T23:03:35.432 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ad4c4d3c:::rbd_data.111edca93686.0000000000000040:head# 2024-05-11T23:03:35.432 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ad673748:::rbd_data.111ed8ed3219.0000000000000059:head# 2024-05-11T23:03:35.432 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ad6b93a6:::rbd_data.111ef7a115a1.0000000000000053:head# 2024-05-11T23:03:35.432 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ad6d1b8d:::rbd_data.111ed8ed3219.00000000000000a9:head# 2024-05-11T23:03:35.432 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ad715eb0:::rbd_data.111ebabb4604.000000000000000a:head# 2024-05-11T23:03:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ad96205b:::rbd_data.111ee4903046.0000000000000022:18# 2024-05-11T23:03:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ad96205b:::rbd_data.111ee4903046.0000000000000022:head# 2024-05-11T23:03:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:adec0c33:::rbd_data.111ee4903046.0000000000000045:head# 2024-05-11T23:03:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:adf885e5:::rbd_object_map.111e913a615f.0000000000000025:head# 2024-05-11T23:03:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:adf960fd:::rbd_data.111ebabb4604.00000000000000a2:head# 2024-05-11T23:03:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ae108ad7:::rbd_data.111ecc0f42c3.0000000000000028:19# 2024-05-11T23:03:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ae108ad7:::rbd_data.111ecc0f42c3.0000000000000028:head# 2024-05-11T23:03:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ae2ec3f2:::rbd_data.111e67f27da3.0000000000000000:head# 2024-05-11T23:03:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ae4aa414:::rbd_data.111e67f27da3.0000000000000026:head# 2024-05-11T23:03:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ae505ab8:::rbd_data.111ed2934718.0000000000000057:24# 2024-05-11T23:03:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ae505ab8:::rbd_data.111ed2934718.0000000000000057:head# 2024-05-11T23:03:35.433 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ae5932b3:::rbd_data.111e8974a681.0000000000000002:16# 2024-05-11T23:03:35.434 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:35.434+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:35.441 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ae5932b3:::rbd_data.111e8974a681.0000000000000002:head# 2024-05-11T23:03:35.441 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ae76d313:::rbd_data.111edca93686.00000000000000b5:head# 2024-05-11T23:03:35.442 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ae8454b4:::rbd_data.111ed8ed3219.00000000000000b3:head# 2024-05-11T23:03:35.442 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ae8bead3:::rbd_data.111ee4903046.000000000000004a:head# 2024-05-11T23:03:35.442 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:ae8c5dd8:::rbd_data.111edca93686.0000000000000158:head# 2024-05-11T23:03:35.442 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:aeb02784:::rbd_data.111e5e4d8275.0000000000000008:11# 2024-05-11T23:03:35.442 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:aeb02784:::rbd_data.111e5e4d8275.0000000000000008:head# 2024-05-11T23:03:35.443 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:aed4e7fe:::rbd_data.111e479bff4d.0000000000000037:head# 2024-05-11T23:03:35.448 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:aee04b84:::rbd_data.111ee4903046.000000000000002b:head# 2024-05-11T23:03:35.448 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:aee09837:::rbd_data.111ef9e47166.0000000000000089:head# 2024-05-11T23:03:35.448 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:af13a123:::rbd_data.111edca93686.000000000000014a:head# 2024-05-11T23:03:35.448 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:af229500:::rbd_data.111ee4903046.000000000000001a:head# 2024-05-11T23:03:35.448 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:af299a46:::rbd_data.111ecc0f42c3.0000000000000117:19# 2024-05-11T23:03:35.448 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:af299a46:::rbd_data.111ecc0f42c3.0000000000000117:head# 2024-05-11T23:03:35.448 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:af3d21b4:::rbd_data.111edca93686.0000000000000127:head# 2024-05-11T23:03:35.449 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:af65319d:::rbd_data.111ebabb4604.0000000000000073:head# 2024-05-11T23:03:35.449 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:af6f5356:::rbd_data.111e64023279.0000000000000063:20# 2024-05-11T23:03:35.449 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:af6f5356:::rbd_data.111e64023279.0000000000000063:head# 2024-05-11T23:03:35.449 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:afb8e764:::rbd_data.111e148bd16.000000000000000e:10# 2024-05-11T23:03:35.449 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:afb8e764:::rbd_data.111e148bd16.000000000000000e:head# 2024-05-11T23:03:35.451 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:afc839c8:::rbd_data.111ef9e47166.00000000000000a3:head# 2024-05-11T23:03:35.451 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:afd4c577:::rbd_data.111edca93686.0000000000000067:head# 2024-05-11T23:03:35.451 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:aff88f3b:::rbd_data.111ef7a115a1.0000000000000084:head# 2024-05-11T23:03:35.451 INFO:teuthology.orchestra.run.smithi005.stderr:Read #3:affd1311:::rbd_data.111ef9e47166.0000000000000111:head# 2024-05-11T23:03:35.451 INFO:teuthology.orchestra.run.smithi005.stderr:Export successful 2024-05-11T23:03:35.493 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:35.493 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:35.493 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:35.493 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:35.493 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:35.502 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:03:35.543 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:35.543+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:35.636 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:35.636+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:35.649 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:35.649 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:35.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:35.649 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:35.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.649 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116446 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8273920~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:35.649 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:35.649 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.272778422999998, 2024-05-11T23:03:35.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045535299000000001, 2024-05-11T23:03:35.649 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:35.650 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:35.650 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:35.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:35.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:35.650 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.650 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116446 2024-05-11T23:03:35.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.650 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:35.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:35.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:35.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:35.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:35.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:35.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375547+0000", 2024-05-11T23:03:35.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4000000000000001e-07 2024-05-11T23:03:35.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:35.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375549+0000", 2024-05-11T23:03:35.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.368e-06 2024-05-11T23:03:35.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:35.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375556+0000", 2024-05-11T23:03:35.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1840000000000004e-06 2024-05-11T23:03:35.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375560+0000", 2024-05-11T23:03:35.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.741e-06 2024-05-11T23:03:35.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:35.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420896+0000", 2024-05-11T23:03:35.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045335435 2024-05-11T23:03:35.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.652 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:35.652 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420987+0000", 2024-05-11T23:03:35.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1033999999999996e-05 2024-05-11T23:03:35.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:35.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421082+0000", 2024-05-11T23:03:35.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4896999999999996e-05 2024-05-11T23:03:35.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.653 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.653 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.653 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116447 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6316032~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:35.653 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:35.653 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.272756913000002, 2024-05-11T23:03:35.653 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045745830000000001, 2024-05-11T23:03:35.654 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:35.654 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:35.654 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:35.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:35.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:35.654 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.654 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116447 2024-05-11T23:03:35.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.654 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:35.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:35.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:35.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:35.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:35.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:35.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:35.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.58e-07 2024-05-11T23:03:35.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:35.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375570+0000", 2024-05-11T23:03:35.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:03:35.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:35.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375576+0000", 2024-05-11T23:03:35.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7939999999999999e-06 2024-05-11T23:03:35.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375581+0000", 2024-05-11T23:03:35.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9280000000000001e-06 2024-05-11T23:03:35.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:35.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421127+0000", 2024-05-11T23:03:35.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045545785999999998 2024-05-11T23:03:35.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:35.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421214+0000", 2024-05-11T23:03:35.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7039000000000005e-05 2024-05-11T23:03:35.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.657 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:35.657 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421314+0000", 2024-05-11T23:03:35.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9772999999999995e-05 2024-05-11T23:03:35.657 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.657 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.657 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.658 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116448 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8314880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:35.658 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:35.658 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.272737839000001, 2024-05-11T23:03:35.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045941753000000002, 2024-05-11T23:03:35.658 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:35.658 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:35.658 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:35.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:35.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:35.658 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.658 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116448 2024-05-11T23:03:35.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.658 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:35.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:35.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:35.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:35.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:35.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:35.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:35.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.77e-07 2024-05-11T23:03:35.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:35.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375589+0000", 2024-05-11T23:03:35.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.091e-06 2024-05-11T23:03:35.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:35.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375596+0000", 2024-05-11T23:03:35.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9179999999999997e-06 2024-05-11T23:03:35.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375601+0000", 2024-05-11T23:03:35.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0030000000000002e-06 2024-05-11T23:03:35.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:35.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421363+0000", 2024-05-11T23:03:35.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045761352999999998 2024-05-11T23:03:35.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:35.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421440+0000", 2024-05-11T23:03:35.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7585999999999999e-05 2024-05-11T23:03:35.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.661 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:35.661 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421529+0000", 2024-05-11T23:03:35.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8424999999999994e-05 2024-05-11T23:03:35.661 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.662 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.662 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.662 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116449 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6438912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:35.662 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:35.662 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.272717403999998, 2024-05-11T23:03:35.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046082208999999999, 2024-05-11T23:03:35.665 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:35.665 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:35.665 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:35.665 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:35.665 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:35.665 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.666 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116449 2024-05-11T23:03:35.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.666 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:35.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:35.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:35.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:35.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:35.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:35.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377225+0000", 2024-05-11T23:03:35.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016177030000000001 2024-05-11T23:03:35.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:35.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377229+0000", 2024-05-11T23:03:35.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8e-06 2024-05-11T23:03:35.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:35.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377238+0000", 2024-05-11T23:03:35.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2229999999999996e-06 2024-05-11T23:03:35.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377244+0000", 2024-05-11T23:03:35.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4060000000000002e-06 2024-05-11T23:03:35.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:35.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421520+0000", 2024-05-11T23:03:35.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044275222000000003 2024-05-11T23:03:35.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:35.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421595+0000", 2024-05-11T23:03:35.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4838000000000004e-05 2024-05-11T23:03:35.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:35.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421690+0000", 2024-05-11T23:03:35.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5017000000000002e-05 2024-05-11T23:03:35.669 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.669 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.669 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.669 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116458 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7094272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:35.669 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:35.669 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.270903187000002, 2024-05-11T23:03:35.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045706877999999999, 2024-05-11T23:03:35.670 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:35.670 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:35.670 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:35.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:35.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:35.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.670 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116458 2024-05-11T23:03:35.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.670 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:35.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.670 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:35.670 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:35.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:35.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:35.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:35.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:35.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9400000000000005e-07 2024-05-11T23:03:35.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:35.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377424+0000", 2024-05-11T23:03:35.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1859999999999999e-06 2024-05-11T23:03:35.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:35.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377431+0000", 2024-05-11T23:03:35.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4049999999999997e-06 2024-05-11T23:03:35.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377436+0000", 2024-05-11T23:03:35.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1399999999999999e-06 2024-05-11T23:03:35.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:35.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.422945+0000", 2024-05-11T23:03:35.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045509326000000003 2024-05-11T23:03:35.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:35.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423036+0000", 2024-05-11T23:03:35.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0649999999999994e-05 2024-05-11T23:03:35.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:35.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423128+0000", 2024-05-11T23:03:35.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2577000000000005e-05 2024-05-11T23:03:35.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.673 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.674 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116459 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7258112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:35.674 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:35.674 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.270882454999999, 2024-05-11T23:03:35.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045831945999999998, 2024-05-11T23:03:35.674 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:35.674 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:35.674 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:35.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:35.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:35.675 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.675 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116459 2024-05-11T23:03:35.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.675 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:35.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:35.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:35.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:35.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:35.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:35.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377443+0000", 2024-05-11T23:03:35.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999999e-07 2024-05-11T23:03:35.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:35.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377445+0000", 2024-05-11T23:03:35.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0209999999999998e-06 2024-05-11T23:03:35.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:35.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377451+0000", 2024-05-11T23:03:35.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.071e-06 2024-05-11T23:03:35.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377456+0000", 2024-05-11T23:03:35.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8289999999999998e-06 2024-05-11T23:03:35.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:35.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423155+0000", 2024-05-11T23:03:35.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045699447999999997 2024-05-11T23:03:35.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:35.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423200+0000", 2024-05-11T23:03:35.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5083000000000002e-05 2024-05-11T23:03:35.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:35.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423274+0000", 2024-05-11T23:03:35.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4046000000000002e-05 2024-05-11T23:03:35.678 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.678 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.678 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.679 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116460 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7299072~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:35.679 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:35.679 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.270863392999999, 2024-05-11T23:03:35.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046010987000000003, 2024-05-11T23:03:35.679 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:35.679 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:35.679 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:35.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:35.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:35.679 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.679 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116460 2024-05-11T23:03:35.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.679 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:35.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:35.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:35.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:35.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:35.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:35.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377462+0000", 2024-05-11T23:03:35.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:03:35.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:35.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377464+0000", 2024-05-11T23:03:35.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1600000000000001e-06 2024-05-11T23:03:35.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:35.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377469+0000", 2024-05-11T23:03:35.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5430000000000001e-06 2024-05-11T23:03:35.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377474+0000", 2024-05-11T23:03:35.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8509999999999998e-06 2024-05-11T23:03:35.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:35.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423312+0000", 2024-05-11T23:03:35.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045838080000000003 2024-05-11T23:03:35.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:35.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423401+0000", 2024-05-11T23:03:35.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8170000000000005e-05 2024-05-11T23:03:35.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:35.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423472+0000", 2024-05-11T23:03:35.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1847000000000005e-05 2024-05-11T23:03:35.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.683 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.683 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116461 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7421952~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:35.683 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:35.683 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.270844623000002, 2024-05-11T23:03:35.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046130749999999998, 2024-05-11T23:03:35.683 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:35.683 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:35.683 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:35.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:35.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:35.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.684 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116461 2024-05-11T23:03:35.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.684 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:35.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:35.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:35.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:35.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:35.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:35.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:35.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4700000000000002e-07 2024-05-11T23:03:35.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:35.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377483+0000", 2024-05-11T23:03:35.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0219999999999999e-06 2024-05-11T23:03:35.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:35.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377489+0000", 2024-05-11T23:03:35.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4420000000000003e-06 2024-05-11T23:03:35.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377494+0000", 2024-05-11T23:03:35.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9239999999999998e-06 2024-05-11T23:03:35.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:35.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423500+0000", 2024-05-11T23:03:35.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046006054999999997 2024-05-11T23:03:35.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:35.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423557+0000", 2024-05-11T23:03:35.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7169000000000001e-05 2024-05-11T23:03:35.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:35.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423611+0000", 2024-05-11T23:03:35.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3791e-05 2024-05-11T23:03:35.687 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.687 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.687 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.687 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116462 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7462912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:35.687 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:35.687 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.270825004999999, 2024-05-11T23:03:35.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046254666, 2024-05-11T23:03:35.687 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:35.688 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:35.688 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:35.688 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:35.688 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:35.688 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.688 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116462 2024-05-11T23:03:35.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.688 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:35.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:35.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:35.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:35.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:35.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:35.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:35.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:35.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:35.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377502+0000", 2024-05-11T23:03:35.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-05-11T23:03:35.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:35.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377508+0000", 2024-05-11T23:03:35.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4500000000000003e-06 2024-05-11T23:03:35.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377512+0000", 2024-05-11T23:03:35.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7700000000000001e-06 2024-05-11T23:03:35.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:35.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423649+0000", 2024-05-11T23:03:35.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046136859000000002 2024-05-11T23:03:35.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:35.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423704+0000", 2024-05-11T23:03:35.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4938000000000002e-05 2024-05-11T23:03:35.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:35.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423754+0000", 2024-05-11T23:03:35.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0145999999999997e-05 2024-05-11T23:03:35.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.691 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.691 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116463 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7585792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:35.692 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:35.692 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.270806579000002, 2024-05-11T23:03:35.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361763, 2024-05-11T23:03:35.692 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:35.692 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:35.692 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:35.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:35.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:35.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.692 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116463 2024-05-11T23:03:35.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.692 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:35.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:35.692 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:35.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:35.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:35.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:35.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377519+0000", 2024-05-11T23:03:35.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2600000000000002e-07 2024-05-11T23:03:35.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:35.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377521+0000", 2024-05-11T23:03:35.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1710000000000001e-06 2024-05-11T23:03:35.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:35.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377527+0000", 2024-05-11T23:03:35.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9649999999999996e-06 2024-05-11T23:03:35.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377532+0000", 2024-05-11T23:03:35.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.78e-06 2024-05-11T23:03:35.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:35.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423779+0000", 2024-05-11T23:03:35.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046247706 2024-05-11T23:03:35.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:35.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423829+0000", 2024-05-11T23:03:35.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9188999999999999e-05 2024-05-11T23:03:35.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:35.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423880+0000", 2024-05-11T23:03:35.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1425999999999999e-05 2024-05-11T23:03:35.695 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.696 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.696 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116464 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7749632~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:35.696 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:35.696 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.270787343000002, 2024-05-11T23:03:35.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046491719000000001, 2024-05-11T23:03:35.696 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:35.696 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:35.696 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:35.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:35.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:35.696 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.697 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116464 2024-05-11T23:03:35.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.697 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:35.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:35.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:35.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:35.697 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:35.697 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.697 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.697 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:35.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377538+0000", 2024-05-11T23:03:35.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4499999999999998e-07 2024-05-11T23:03:35.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:35.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377540+0000", 2024-05-11T23:03:35.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2340000000000001e-06 2024-05-11T23:03:35.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:35.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377546+0000", 2024-05-11T23:03:35.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8289999999999996e-06 2024-05-11T23:03:35.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377551+0000", 2024-05-11T23:03:35.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8559999999999998e-06 2024-05-11T23:03:35.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:35.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423913+0000", 2024-05-11T23:03:35.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361885999999998 2024-05-11T23:03:35.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:35.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423958+0000", 2024-05-11T23:03:35.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5513999999999998e-05 2024-05-11T23:03:35.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:35.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424029+0000", 2024-05-11T23:03:35.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1055000000000003e-05 2024-05-11T23:03:35.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.700 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.700 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.700 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116465 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7626752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:35.700 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:35.700 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.270768195000002, 2024-05-11T23:03:35.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046600057, 2024-05-11T23:03:35.700 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:35.700 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:35.700 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:35.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:35.700 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:35.701 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.701 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116465 2024-05-11T23:03:35.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.701 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:35.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:35.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:35.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:35.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:35.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:35.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:35.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3599999999999999e-07 2024-05-11T23:03:35.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:35.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377559+0000", 2024-05-11T23:03:35.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9640000000000002e-06 2024-05-11T23:03:35.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.702 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:35.702 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377572+0000", 2024-05-11T23:03:35.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3188e-05 2024-05-11T23:03:35.702 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377578+0000", 2024-05-11T23:03:35.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.942e-06 2024-05-11T23:03:35.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:35.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424058+0000", 2024-05-11T23:03:35.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046480021000000003 2024-05-11T23:03:35.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:35.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424103+0000", 2024-05-11T23:03:35.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5021999999999997e-05 2024-05-11T23:03:35.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:35.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424157+0000", 2024-05-11T23:03:35.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3483999999999998e-05 2024-05-11T23:03:35.704 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.704 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.704 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.704 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116466 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7790592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:35.704 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:35.704 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.270739613000003, 2024-05-11T23:03:35.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046743741999999998, 2024-05-11T23:03:35.704 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:35.704 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:35.704 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:35.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:35.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:35.705 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.705 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116466 2024-05-11T23:03:35.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.705 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:35.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:35.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:35.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:35.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:35.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:35.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377586+0000", 2024-05-11T23:03:35.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:03:35.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:35.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377588+0000", 2024-05-11T23:03:35.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3039999999999999e-06 2024-05-11T23:03:35.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:35.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377594+0000", 2024-05-11T23:03:35.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-05-11T23:03:35.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377600+0000", 2024-05-11T23:03:35.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7699999999999998e-06 2024-05-11T23:03:35.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:35.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424185+0000", 2024-05-11T23:03:35.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046585891999999997 2024-05-11T23:03:35.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.707 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.707 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:35.707 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424235+0000", 2024-05-11T23:03:35.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9356e-05 2024-05-11T23:03:35.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:35.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424329+0000", 2024-05-11T23:03:35.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4036999999999994e-05 2024-05-11T23:03:35.708 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.708 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.708 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.708 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116467 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7954432~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:35.708 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:35.708 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.270719325999998, 2024-05-11T23:03:35.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046930134999999998, 2024-05-11T23:03:35.709 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:35.709 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:35.709 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:35.709 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:35.709 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:35.709 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.709 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116467 2024-05-11T23:03:35.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.709 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:35.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:35.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:35.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:35.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:35.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:35.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379161+0000", 2024-05-11T23:03:35.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015555009999999999 2024-05-11T23:03:35.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:35.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379166+0000", 2024-05-11T23:03:35.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6689999999999997e-06 2024-05-11T23:03:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379176+0000", 2024-05-11T23:03:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0549e-05 2024-05-11T23:03:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379183+0000", 2024-05-11T23:03:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1199999999999996e-06 2024-05-11T23:03:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424359+0000", 2024-05-11T23:03:35.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045175949999999999 2024-05-11T23:03:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424433+0000", 2024-05-11T23:03:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3824000000000003e-05 2024-05-11T23:03:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424536+0000", 2024-05-11T23:03:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010252200000000001 2024-05-11T23:03:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.712 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116468 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7913472~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.269129106999998, 2024-05-11T23:03:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045525364999999998, 2024-05-11T23:03:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116468 2024-05-11T23:03:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3300000000000002e-07 2024-05-11T23:03:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379198+0000", 2024-05-11T23:03:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2670000000000001e-06 2024-05-11T23:03:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379206+0000", 2024-05-11T23:03:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7149999999999998e-06 2024-05-11T23:03:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379228+0000", 2024-05-11T23:03:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1429000000000001e-05 2024-05-11T23:03:35.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424574+0000", 2024-05-11T23:03:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045346209999999998 2024-05-11T23:03:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424627+0000", 2024-05-11T23:03:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3352e-05 2024-05-11T23:03:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424721+0000", 2024-05-11T23:03:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3858999999999997e-05 2024-05-11T23:03:35.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116469 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8077312~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.269079286999997, 2024-05-11T23:03:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045636687000000002, 2024-05-11T23:03:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.717 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116469 2024-05-11T23:03:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:35.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379246+0000", 2024-05-11T23:03:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.32e-07 2024-05-11T23:03:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379254+0000", 2024-05-11T23:03:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.807e-06 2024-05-11T23:03:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379261+0000", 2024-05-11T23:03:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.985e-06 2024-05-11T23:03:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379264+0000", 2024-05-11T23:03:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6480000000000001e-06 2024-05-11T23:03:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424744+0000", 2024-05-11T23:03:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045479638000000003 2024-05-11T23:03:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424797+0000", 2024-05-11T23:03:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2725000000000001e-05 2024-05-11T23:03:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:35.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424882+0000", 2024-05-11T23:03:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5451999999999994e-05 2024-05-11T23:03:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116470 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8118272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.269054075, 2024-05-11T23:03:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045755745, 2024-05-11T23:03:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:35.721 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116470 2024-05-11T23:03:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:35.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3999999999999997e-07 2024-05-11T23:03:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379273+0000", 2024-05-11T23:03:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-05-11T23:03:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379277+0000", 2024-05-11T23:03:35.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4020000000000004e-06 2024-05-11T23:03:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379280+0000", 2024-05-11T23:03:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-05-11T23:03:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424922+0000", 2024-05-11T23:03:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045642598999999999 2024-05-11T23:03:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424973+0000", 2024-05-11T23:03:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0674999999999998e-05 2024-05-11T23:03:35.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425026+0000", 2024-05-11T23:03:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3492000000000002e-05 2024-05-11T23:03:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116471 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8241152~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.269041291999997, 2024-05-11T23:03:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045867374000000002, 2024-05-11T23:03:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:35.725 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116471 2024-05-11T23:03:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:35.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379284+0000", 2024-05-11T23:03:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.53e-07 2024-05-11T23:03:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379285+0000", 2024-05-11T23:03:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.223e-06 2024-05-11T23:03:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:35.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379288+0000", 2024-05-11T23:03:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5379999999999998e-06 2024-05-11T23:03:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379291+0000", 2024-05-11T23:03:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6699999999999998e-06 2024-05-11T23:03:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425055+0000", 2024-05-11T23:03:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045763497 2024-05-11T23:03:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:35.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425099+0000", 2024-05-11T23:03:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3899000000000001e-05 2024-05-11T23:03:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425151+0000", 2024-05-11T23:03:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2293999999999998e-05 2024-05-11T23:03:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116472 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8282112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.269030293, 2024-05-11T23:03:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045973778999999999, 2024-05-11T23:03:35.729 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116472 2024-05-11T23:03:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381226+0000", 2024-05-11T23:03:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001931627 2024-05-11T23:03:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381231+0000", 2024-05-11T23:03:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8679999999999998e-06 2024-05-11T23:03:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381242+0000", 2024-05-11T23:03:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0702e-05 2024-05-11T23:03:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381249+0000", 2024-05-11T23:03:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9800000000000001e-06 2024-05-11T23:03:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425175+0000", 2024-05-11T23:03:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043926805999999999 2024-05-11T23:03:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425215+0000", 2024-05-11T23:03:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9737999999999998e-05 2024-05-11T23:03:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425268+0000", 2024-05-11T23:03:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3057999999999999e-05 2024-05-11T23:03:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116482 3.1b 3:db6e8843:::rbd_object_map.111e67f27da3.0000000000000028:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:35.733 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 60.267061750000003, 2024-05-11T23:03:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046526076, 2024-05-11T23:03:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116482 2024-05-11T23:03:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381264+0000", 2024-05-11T23:03:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5300000000000004e-07 2024-05-11T23:03:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381267+0000", 2024-05-11T23:03:35.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1999999999999999e-06 2024-05-11T23:03:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381274+0000", 2024-05-11T23:03:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7109999999999995e-06 2024-05-11T23:03:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381285+0000", 2024-05-11T23:03:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0856000000000001e-05 2024-05-11T23:03:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425292+0000", 2024-05-11T23:03:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044006376 2024-05-11T23:03:35.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425337+0000", 2024-05-11T23:03:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4907999999999998e-05 2024-05-11T23:03:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:03:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425507+0000", 2024-05-11T23:03:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017029999999999999 2024-05-11T23:03:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427668+0000", 2024-05-11T23:03:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021612430000000002 2024-05-11T23:03:35.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427683+0000", 2024-05-11T23:03:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5016e-05 2024-05-11T23:03:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427742+0000", 2024-05-11T23:03:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8811999999999999e-05 2024-05-11T23:03:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427789+0000", 2024-05-11T23:03:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7101000000000002e-05 2024-05-11T23:03:35.738 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:35.739 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:35.739 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:35.740 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:03:35.741 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:35.740+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:35.787 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-05-11T23:03:35.848 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:35.848+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:35.902 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:35.902 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:35.902 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:35.902 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:35.911 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:03:35.942 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:35.942+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:35.973 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --force --op remove --pgid 3.15 2024-05-11T23:03:36.015 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:36.015 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:36.015 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:36.024 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:03:36.068 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:36.068+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:36.130 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:36.131 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:36.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:36.131 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:36.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.131 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267820 3.16 3:6b963329:::rbd_data.111ec06c817a.000000000000000a:33 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:36.131 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:03:36.131 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.187151958000001, 2024-05-11T23:03:36.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050141218000000001, 2024-05-11T23:03:36.131 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.131 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.131 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.131 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267820 2024-05-11T23:03:36.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.132 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:03:36.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:03:36.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942754+0000", 2024-05-11T23:03:36.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8559000000000003e-05 2024-05-11T23:03:36.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942756+0000", 2024-05-11T23:03:36.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.604e-06 2024-05-11T23:03:36.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942763+0000", 2024-05-11T23:03:36.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7939999999999997e-06 2024-05-11T23:03:36.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942768+0000", 2024-05-11T23:03:36.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4410000000000001e-06 2024-05-11T23:03:36.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992601+0000", 2024-05-11T23:03:36.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049832754999999999 2024-05-11T23:03:36.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992687+0000", 2024-05-11T23:03:36.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5605000000000005e-05 2024-05-11T23:03:36.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992796+0000", 2024-05-11T23:03:36.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010946 2024-05-11T23:03:36.135 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.135 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.135 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.135 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267852 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:36.135 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:03:36.135 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.187030753999998, 2024-05-11T23:03:36.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050212600000000003, 2024-05-11T23:03:36.135 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.135 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.135 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.135 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.135 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.135 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.136 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267852 2024-05-11T23:03:36.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.136 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:03:36.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:03:36.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943228+0000", 2024-05-11T23:03:36.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00045180600000000002 2024-05-11T23:03:36.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943232+0000", 2024-05-11T23:03:36.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8399999999999997e-06 2024-05-11T23:03:36.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943240+0000", 2024-05-11T23:03:36.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8900000000000007e-06 2024-05-11T23:03:36.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943246+0000", 2024-05-11T23:03:36.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6239999999999996e-06 2024-05-11T23:03:36.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992828+0000", 2024-05-11T23:03:36.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049581247000000002 2024-05-11T23:03:36.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992905+0000", 2024-05-11T23:03:36.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7656000000000003e-05 2024-05-11T23:03:36.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992989+0000", 2024-05-11T23:03:36.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3536999999999997e-05 2024-05-11T23:03:36.139 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.139 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.139 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.139 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267854 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1134592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:36.139 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:03:36.139 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.186517612999999, 2024-05-11T23:03:36.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050077425000000002, 2024-05-11T23:03:36.139 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.139 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.139 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.139 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.139 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.139 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.140 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267854 2024-05-11T23:03:36.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.140 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:03:36.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:03:36.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943290+0000", 2024-05-11T23:03:36.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2300000000000002e-07 2024-05-11T23:03:36.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943291+0000", 2024-05-11T23:03:36.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6190000000000001e-06 2024-05-11T23:03:36.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943296+0000", 2024-05-11T23:03:36.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2889999999999999e-06 2024-05-11T23:03:36.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943299+0000", 2024-05-11T23:03:36.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1030000000000002e-06 2024-05-11T23:03:36.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993205+0000", 2024-05-11T23:03:36.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049906405000000001 2024-05-11T23:03:36.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993285+0000", 2024-05-11T23:03:36.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9752999999999993e-05 2024-05-11T23:03:36.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993367+0000", 2024-05-11T23:03:36.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1933000000000003e-05 2024-05-11T23:03:36.143 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.143 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.143 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.143 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267855 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1171456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:36.143 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:03:36.143 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.186500671000001, 2024-05-11T23:03:36.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050250346000000001, 2024-05-11T23:03:36.143 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.143 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.143 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.143 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.143 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.143 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.144 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267855 2024-05-11T23:03:36.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.144 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:03:36.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:03:36.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945226+0000", 2024-05-11T23:03:36.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001919235 2024-05-11T23:03:36.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945230+0000", 2024-05-11T23:03:36.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9580000000000001e-06 2024-05-11T23:03:36.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945238+0000", 2024-05-11T23:03:36.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1040000000000007e-06 2024-05-11T23:03:36.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945244+0000", 2024-05-11T23:03:36.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6919999999999996e-06 2024-05-11T23:03:36.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993399+0000", 2024-05-11T23:03:36.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048154651 2024-05-11T23:03:36.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993472+0000", 2024-05-11T23:03:36.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2780000000000005e-05 2024-05-11T23:03:36.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993557+0000", 2024-05-11T23:03:36.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4926000000000006e-05 2024-05-11T23:03:36.146 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.146 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.147 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.147 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267865 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1540096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:36.147 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:36.147 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.184349045000001, 2024-05-11T23:03:36.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050054409000000001, 2024-05-11T23:03:36.147 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.147 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.147 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.147 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.147 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.147 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.147 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267865 2024-05-11T23:03:36.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.147 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:36.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:36.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:36.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6600000000000002e-07 2024-05-11T23:03:36.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945461+0000", 2024-05-11T23:03:36.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2670000000000001e-06 2024-05-11T23:03:36.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945466+0000", 2024-05-11T23:03:36.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7200000000000003e-06 2024-05-11T23:03:36.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945471+0000", 2024-05-11T23:03:36.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9439999999999996e-06 2024-05-11T23:03:36.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995337+0000", 2024-05-11T23:03:36.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049865415000000003 2024-05-11T23:03:36.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995413+0000", 2024-05-11T23:03:36.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5989999999999996e-05 2024-05-11T23:03:36.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995512+0000", 2024-05-11T23:03:36.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9706999999999999e-05 2024-05-11T23:03:36.150 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.150 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.151 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.151 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268017 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:36.151 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947359+0000", 2024-05-11T23:03:36.151 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.18244816, 2024-05-11T23:03:36.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050017655000000001, 2024-05-11T23:03:36.151 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.151 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.151 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.151 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.151 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.151 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.151 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268017 2024-05-11T23:03:36.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.152 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947359+0000", 2024-05-11T23:03:36.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947359+0000", 2024-05-11T23:03:36.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947359+0000", 2024-05-11T23:03:36.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5000000000000003e-07 2024-05-11T23:03:36.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947411+0000", 2024-05-11T23:03:36.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1158e-05 2024-05-11T23:03:36.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947418+0000", 2024-05-11T23:03:36.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8269999999999998e-06 2024-05-11T23:03:36.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947424+0000", 2024-05-11T23:03:36.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6529999999999999e-06 2024-05-11T23:03:36.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997179+0000", 2024-05-11T23:03:36.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049754675999999998 2024-05-11T23:03:36.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997279+0000", 2024-05-11T23:03:36.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9789000000000003e-05 2024-05-11T23:03:36.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997377+0000", 2024-05-11T23:03:36.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8002000000000001e-05 2024-05-11T23:03:36.154 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.155 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.155 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.155 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268018 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 872448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:36.155 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:03:36.155 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.18237298, 2024-05-11T23:03:36.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050170927999999997, 2024-05-11T23:03:36.155 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.155 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.155 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.155 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.155 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.155 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.155 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268018 2024-05-11T23:03:36.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.156 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:03:36.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:03:36.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947435+0000", 2024-05-11T23:03:36.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3099999999999997e-07 2024-05-11T23:03:36.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947437+0000", 2024-05-11T23:03:36.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6340000000000001e-06 2024-05-11T23:03:36.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947443+0000", 2024-05-11T23:03:36.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7810000000000002e-06 2024-05-11T23:03:36.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947448+0000", 2024-05-11T23:03:36.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9679999999999997e-06 2024-05-11T23:03:36.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997418+0000", 2024-05-11T23:03:36.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049969587000000003 2024-05-11T23:03:36.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997504+0000", 2024-05-11T23:03:36.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6426000000000002e-05 2024-05-11T23:03:36.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997605+0000", 2024-05-11T23:03:36.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100901 2024-05-11T23:03:36.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.159 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.159 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.159 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268019 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:36.159 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:36.159 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.182352422000001, 2024-05-11T23:03:36.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050285074999999999, 2024-05-11T23:03:36.159 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.159 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.159 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.160 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.160 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.160 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.160 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268019 2024-05-11T23:03:36.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.160 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:36.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:36.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:36.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0999999999999999e-07 2024-05-11T23:03:36.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947457+0000", 2024-05-11T23:03:36.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-05-11T23:03:36.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947462+0000", 2024-05-11T23:03:36.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2410000000000003e-06 2024-05-11T23:03:36.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947467+0000", 2024-05-11T23:03:36.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8339999999999998e-06 2024-05-11T23:03:36.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997639+0000", 2024-05-11T23:03:36.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050171697000000001 2024-05-11T23:03:36.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997685+0000", 2024-05-11T23:03:36.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6146000000000002e-05 2024-05-11T23:03:36.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997740+0000", 2024-05-11T23:03:36.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4737000000000001e-05 2024-05-11T23:03:36.163 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.163 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.163 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.163 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268020 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:36.163 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:03:36.163 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.182334177000001, 2024-05-11T23:03:36.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050504089000000002, 2024-05-11T23:03:36.163 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.163 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.164 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.164 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.164 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.164 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.164 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268020 2024-05-11T23:03:36.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.164 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:03:36.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:03:36.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947591+0000", 2024-05-11T23:03:36.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011805 2024-05-11T23:03:36.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947594+0000", 2024-05-11T23:03:36.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1549999999999999e-06 2024-05-11T23:03:36.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947602+0000", 2024-05-11T23:03:36.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.174e-06 2024-05-11T23:03:36.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947608+0000", 2024-05-11T23:03:36.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.263e-06 2024-05-11T23:03:36.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997775+0000", 2024-05-11T23:03:36.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050166980999999999 2024-05-11T23:03:36.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997875+0000", 2024-05-11T23:03:36.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9844000000000002e-05 2024-05-11T23:03:36.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997977+0000", 2024-05-11T23:03:36.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101622 2024-05-11T23:03:36.167 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.167 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.167 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.167 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268021 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:36.167 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:03:36.167 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.182188716999999, 2024-05-11T23:03:36.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050498874999999999, 2024-05-11T23:03:36.168 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.168 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.168 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.168 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.168 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.168 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.168 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268021 2024-05-11T23:03:36.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.168 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:03:36.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:03:36.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947695+0000", 2024-05-11T23:03:36.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6247000000000004e-05 2024-05-11T23:03:36.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947698+0000", 2024-05-11T23:03:36.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9199999999999997e-06 2024-05-11T23:03:36.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947706+0000", 2024-05-11T23:03:36.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:03:36.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947711+0000", 2024-05-11T23:03:36.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6720000000000001e-06 2024-05-11T23:03:36.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998017+0000", 2024-05-11T23:03:36.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050305470999999997 2024-05-11T23:03:36.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998060+0000", 2024-05-11T23:03:36.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3374000000000002e-05 2024-05-11T23:03:36.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998117+0000", 2024-05-11T23:03:36.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6971000000000001e-05 2024-05-11T23:03:36.171 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.171 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.171 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.171 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268022 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:36.171 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:03:36.172 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.182085681, 2024-05-11T23:03:36.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050731361000000003, 2024-05-11T23:03:36.172 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.172 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.172 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.172 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.172 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.172 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.172 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268022 2024-05-11T23:03:36.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.172 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:03:36.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:03:36.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947722+0000", 2024-05-11T23:03:36.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8999999999999996e-07 2024-05-11T23:03:36.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947724+0000", 2024-05-11T23:03:36.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1639999999999999e-06 2024-05-11T23:03:36.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947730+0000", 2024-05-11T23:03:36.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6099999999999997e-06 2024-05-11T23:03:36.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947735+0000", 2024-05-11T23:03:36.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7189999999999998e-06 2024-05-11T23:03:36.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998140+0000", 2024-05-11T23:03:36.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050404254000000003 2024-05-11T23:03:36.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998239+0000", 2024-05-11T23:03:36.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9192999999999997e-05 2024-05-11T23:03:36.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998453+0000", 2024-05-11T23:03:36.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021383099999999999 2024-05-11T23:03:36.175 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.175 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.175 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.175 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268023 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1069056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:36.176 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:03:36.176 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.182063415999998, 2024-05-11T23:03:36.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050874509999999998, 2024-05-11T23:03:36.176 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.176 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.176 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.176 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.176 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.176 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.176 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268023 2024-05-11T23:03:36.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.176 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:03:36.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:03:36.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948934+0000", 2024-05-11T23:03:36.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011903339999999999 2024-05-11T23:03:36.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948938+0000", 2024-05-11T23:03:36.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1330000000000001e-06 2024-05-11T23:03:36.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948949+0000", 2024-05-11T23:03:36.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0872e-05 2024-05-11T23:03:36.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948956+0000", 2024-05-11T23:03:36.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.131e-06 2024-05-11T23:03:36.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998494+0000", 2024-05-11T23:03:36.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049537678000000002 2024-05-11T23:03:36.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998551+0000", 2024-05-11T23:03:36.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7256000000000002e-05 2024-05-11T23:03:36.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998618+0000", 2024-05-11T23:03:36.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7106000000000004e-05 2024-05-11T23:03:36.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.179 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.179 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268026 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1179648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:36.179 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:03:36.180 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.180786362999999, 2024-05-11T23:03:36.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050107977999999997, 2024-05-11T23:03:36.180 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.180 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.180 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.180 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268026 2024-05-11T23:03:36.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.180 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:03:36.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:03:36.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949029+0000", 2024-05-11T23:03:36.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2220000000000002e-06 2024-05-11T23:03:36.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949041+0000", 2024-05-11T23:03:36.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.241e-05 2024-05-11T23:03:36.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949049+0000", 2024-05-11T23:03:36.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.072e-06 2024-05-11T23:03:36.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949056+0000", 2024-05-11T23:03:36.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1380000000000001e-06 2024-05-11T23:03:36.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999016+0000", 2024-05-11T23:03:36.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049959212000000003 2024-05-11T23:03:36.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999067+0000", 2024-05-11T23:03:36.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1598000000000003e-05 2024-05-11T23:03:36.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999129+0000", 2024-05-11T23:03:36.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1326000000000002e-05 2024-05-11T23:03:36.183 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.183 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.183 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.184 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268027 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1216512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:36.184 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:36.184 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.180742212999998, 2024-05-11T23:03:36.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050189865, 2024-05-11T23:03:36.184 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.184 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.184 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.184 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268027 2024-05-11T23:03:36.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.184 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:36.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:36.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:36.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-05-11T23:03:36.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949068+0000", 2024-05-11T23:03:36.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.26e-06 2024-05-11T23:03:36.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949074+0000", 2024-05-11T23:03:36.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7050000000000002e-06 2024-05-11T23:03:36.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949080+0000", 2024-05-11T23:03:36.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3689999999999998e-06 2024-05-11T23:03:36.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999152+0000", 2024-05-11T23:03:36.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050072653000000002 2024-05-11T23:03:36.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999197+0000", 2024-05-11T23:03:36.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4972999999999998e-05 2024-05-11T23:03:36.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999255+0000", 2024-05-11T23:03:36.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7425e-05 2024-05-11T23:03:36.188 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.188 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.188 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.188 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268028 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1253376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:36.188 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:03:36.188 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.180720553, 2024-05-11T23:03:36.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050280017000000003, 2024-05-11T23:03:36.188 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.188 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.188 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.188 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.188 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.189 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.189 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268028 2024-05-11T23:03:36.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.189 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:03:36.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:03:36.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949087+0000", 2024-05-11T23:03:36.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:03:36.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949089+0000", 2024-05-11T23:03:36.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8950000000000001e-06 2024-05-11T23:03:36.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949095+0000", 2024-05-11T23:03:36.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7540000000000003e-06 2024-05-11T23:03:36.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949114+0000", 2024-05-11T23:03:36.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9738e-05 2024-05-11T23:03:36.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999278+0000", 2024-05-11T23:03:36.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050163368999999999 2024-05-11T23:03:36.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999319+0000", 2024-05-11T23:03:36.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0979000000000002e-05 2024-05-11T23:03:36.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999366+0000", 2024-05-11T23:03:36.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7879e-05 2024-05-11T23:03:36.192 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.192 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.192 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.192 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268029 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1290240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:36.192 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:36.192 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.180680445, 2024-05-11T23:03:36.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050344904000000003, 2024-05-11T23:03:36.192 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.192 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.192 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.193 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.193 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.193 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.193 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268029 2024-05-11T23:03:36.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.193 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:36.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:36.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:36.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:03:36.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949130+0000", 2024-05-11T23:03:36.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-05-11T23:03:36.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949139+0000", 2024-05-11T23:03:36.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9800000000000004e-06 2024-05-11T23:03:36.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949145+0000", 2024-05-11T23:03:36.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0510000000000002e-06 2024-05-11T23:03:36.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999387+0000", 2024-05-11T23:03:36.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050242167999999997 2024-05-11T23:03:36.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999427+0000", 2024-05-11T23:03:36.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9566e-05 2024-05-11T23:03:36.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999471+0000", 2024-05-11T23:03:36.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4914999999999999e-05 2024-05-11T23:03:36.196 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.196 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.196 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.196 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268030 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1327104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:36.196 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:36.196 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.180655161000001, 2024-05-11T23:03:36.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050447394, 2024-05-11T23:03:36.196 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.197 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.197 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.197 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.197 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.197 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.197 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268030 2024-05-11T23:03:36.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.197 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:36.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:36.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:36.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0500000000000004e-07 2024-05-11T23:03:36.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949155+0000", 2024-05-11T23:03:36.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3819999999999998e-06 2024-05-11T23:03:36.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949162+0000", 2024-05-11T23:03:36.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9609999999999999e-06 2024-05-11T23:03:36.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949188+0000", 2024-05-11T23:03:36.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5877e-05 2024-05-11T23:03:36.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999494+0000", 2024-05-11T23:03:36.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050306265000000003 2024-05-11T23:03:36.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999534+0000", 2024-05-11T23:03:36.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0447000000000001e-05 2024-05-11T23:03:36.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999599+0000", 2024-05-11T23:03:36.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4956999999999995e-05 2024-05-11T23:03:36.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.200 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.200 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268031 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1363968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:36.200 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:03:36.200 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.180610060999999, 2024-05-11T23:03:36.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050511525000000002, 2024-05-11T23:03:36.201 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.201 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.201 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.201 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268031 2024-05-11T23:03:36.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.201 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:03:36.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:03:36.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949198+0000", 2024-05-11T23:03:36.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-05-11T23:03:36.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949200+0000", 2024-05-11T23:03:36.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-05-11T23:03:36.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949208+0000", 2024-05-11T23:03:36.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2840000000000001e-06 2024-05-11T23:03:36.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949214+0000", 2024-05-11T23:03:36.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9869999999999996e-06 2024-05-11T23:03:36.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999619+0000", 2024-05-11T23:03:36.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050405278999999997 2024-05-11T23:03:36.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999660+0000", 2024-05-11T23:03:36.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1427999999999997e-05 2024-05-11T23:03:36.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999708+0000", 2024-05-11T23:03:36.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8229999999999997e-05 2024-05-11T23:03:36.204 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.204 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.205 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.205 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268032 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1400832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:36.205 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:03:36.205 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.180580441, 2024-05-11T23:03:36.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050611863999999999, 2024-05-11T23:03:36.205 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.205 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.205 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.205 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268032 2024-05-11T23:03:36.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.206 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:03:36.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:03:36.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949230+0000", 2024-05-11T23:03:36.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.371e-06 2024-05-11T23:03:36.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949233+0000", 2024-05-11T23:03:36.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6599999999999999e-06 2024-05-11T23:03:36.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949241+0000", 2024-05-11T23:03:36.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8939999999999993e-06 2024-05-11T23:03:36.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949246+0000", 2024-05-11T23:03:36.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6590000000000004e-06 2024-05-11T23:03:36.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999733+0000", 2024-05-11T23:03:36.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050487268000000002 2024-05-11T23:03:36.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999784+0000", 2024-05-11T23:03:36.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0562000000000001e-05 2024-05-11T23:03:36.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999838+0000", 2024-05-11T23:03:36.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000002e-05 2024-05-11T23:03:36.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.209 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.209 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268033 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1437696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:36.209 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:03:36.209 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 23.180553314000001, 2024-05-11T23:03:36.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050697157999999999, 2024-05-11T23:03:36.209 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.209 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.209 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.209 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.210 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268033 2024-05-11T23:03:36.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.210 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:03:36.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:03:36.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950862+0000", 2024-05-11T23:03:36.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001607975 2024-05-11T23:03:36.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950866+0000", 2024-05-11T23:03:36.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7770000000000002e-06 2024-05-11T23:03:36.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950878+0000", 2024-05-11T23:03:36.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1421e-05 2024-05-11T23:03:36.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950885+0000", 2024-05-11T23:03:36.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-05-11T23:03:36.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999859+0000", 2024-05-11T23:03:36.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048973199000000002 2024-05-11T23:03:36.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999901+0000", 2024-05-11T23:03:36.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1943000000000001e-05 2024-05-11T23:03:36.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999951+0000", 2024-05-11T23:03:36.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0243e-05 2024-05-11T23:03:36.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.213 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.213 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.213 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:36.214 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:03:36.215 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:36.214+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:36.315 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:36.315+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:36.360 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:36.360 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:36.360 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:36.360 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:36.369 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:03:36.453 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:36.453+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:36.513 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:36.513 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:36.514 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:36.514 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:36.517 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:36.517+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:36.522 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:03:36.618 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:36.618+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:36.669 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:36.670 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:36.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:36.670 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:36.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.670 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:36.670 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:36.670 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.014953413, 2024-05-11T23:03:36.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:03:36.670 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.670 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.670 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.671 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.671 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:03:36.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.671 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:36.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:36.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:36.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:03:36.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:03:36.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:03:36.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:03:36.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:03:36.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:03:36.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:03:36.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:03:36.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:03:36.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:03:36.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:03:36.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.674 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.674 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:03:36.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:03:36.675 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.675 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.675 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.675 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:36.675 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:36.675 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.01157222400001, 2024-05-11T23:03:36.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:03:36.675 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.675 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.675 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.676 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.676 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.676 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.676 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:03:36.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.676 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:36.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:36.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:36.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:36.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:03:36.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:03:36.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:03:36.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:03:36.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.678 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.678 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:03:36.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:03:36.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:03:36.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:03:36.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:03:36.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:03:36.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:03:36.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:03:36.680 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.680 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.680 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.680 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:36.680 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:36.680 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.011553517, 2024-05-11T23:03:36.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:03:36.680 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.680 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.680 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.680 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:03:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:36.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:03:36.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:03:36.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:03:36.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:03:36.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:03:36.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:03:36.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:03:36.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:03:36.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:03:36.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:03:36.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:03:36.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:03:36.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.684 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:03:36.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:03:36.684 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.684 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.685 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.685 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:36.685 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:36.685 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.011535374, 2024-05-11T23:03:36.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:03:36.685 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.685 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.685 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.685 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:03:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:03:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:03:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:03:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:03:36.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:03:36.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:03:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:03:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:03:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:03:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:03:36.690 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:03:36.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.691 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.691 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:03:36.691 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:03:36.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.691 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.691 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.691 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.691 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.691 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:36.691 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:36.691 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.01150788699999, 2024-05-11T23:03:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:03:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:03:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.692 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:03:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:03:36.693 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:03:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:03:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:03:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:03:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.694 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:03:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:03:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:03:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:03:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:03:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:03:36.695 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:03:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:03:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.011487359, 2024-05-11T23:03:36.696 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:03:36.697 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.697 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.697 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.697 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.697 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:03:36.697 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.697 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:36.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.698 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.698 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.698 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.698 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:36.698 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:36.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:03:36.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.699 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:03:36.699 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:03:36.699 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.699 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.699 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:03:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:03:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:03:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:03:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:03:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:03:36.700 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:03:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:03:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:03:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:03:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.701 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.702 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.702 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:36.702 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:36.702 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.011467584, 2024-05-11T23:03:36.702 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:03:36.702 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.702 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.702 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.702 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.702 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:03:36.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.703 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:36.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.703 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.703 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:36.703 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.703 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.703 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:03:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:03:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:03:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:03:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:03:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:03:36.704 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:03:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:03:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:03:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:03:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.705 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:03:36.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:03:36.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.706 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.706 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:03:36.706 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:03:36.706 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.706 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.706 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.706 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.706 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.011448778, 2024-05-11T23:03:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:03:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:03:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.707 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.708 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:03:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:03:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:03:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:03:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:03:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:03:36.709 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:03:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:03:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:03:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:03:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.710 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:03:36.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:03:36.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.711 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.711 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:03:36.711 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:03:36.711 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.711 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.711 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.711 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.711 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.712 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:36.712 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:36.712 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.011429855, 2024-05-11T23:03:36.712 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:03:36.712 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.712 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.712 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.712 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.712 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:03:36.712 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.713 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:36.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.713 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.713 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:36.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.713 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:36.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:03:36.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:03:36.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:03:36.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:03:36.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:03:36.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:03:36.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:03:36.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:03:36.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:03:36.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:03:36.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:03:36.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:03:36.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:03:36.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.716 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.716 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.717 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:36.717 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:36.717 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.011411193, 2024-05-11T23:03:36.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:03:36.717 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.717 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.717 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.717 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.718 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:03:36.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.718 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:36.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:36.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:36.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:03:36.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.719 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.719 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:03:36.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:03:36.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:03:36.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:03:36.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:03:36.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:03:36.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:03:36.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:03:36.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:03:36.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:03:36.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:03:36.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:03:36.721 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.721 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.722 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.722 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:36.722 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:36.722 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.011066031, 2024-05-11T23:03:36.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:03:36.722 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.722 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.722 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.722 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.722 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.723 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.723 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:03:36.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.723 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:36.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.723 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:36.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:03:36.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:03:36.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:03:36.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:03:36.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:03:36.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:03:36.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:03:36.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:03:36.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:03:36.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:03:36.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:03:36.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:03:36.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:03:36.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:03:36.726 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.727 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.727 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.727 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:36.727 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:36.727 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.01104048099999, 2024-05-11T23:03:36.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:03:36.727 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.727 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.727 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.727 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.728 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.728 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:03:36.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.728 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:36.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:36.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:36.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:03:36.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:03:36.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:03:36.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:03:36.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:03:36.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:03:36.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:03:36.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:03:36.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:03:36.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:03:36.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:03:36.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:03:36.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:03:36.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.731 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.732 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.732 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:36.732 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:36.732 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.010958223, 2024-05-11T23:03:36.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:03:36.732 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.732 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.732 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.733 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:03:36.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.733 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:36.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:36.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:36.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:03:36.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:03:36.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:03:36.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:03:36.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:03:36.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:03:36.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:03:36.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:03:36.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:03:36.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:03:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:03:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:03:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:03:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.736 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.010935776, 2024-05-11T23:03:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:03:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.737 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:03:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:36.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:03:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:03:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:03:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:03:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:03:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:03:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:03:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:03:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:03:36.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:03:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:03:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:03:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:03:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:03:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.741 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.010916318, 2024-05-11T23:03:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:03:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.742 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:03:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:03:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:03:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:03:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:03:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:03:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:03:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:03:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:03:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:03:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:03:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:03:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:03:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:03:36.746 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.010896258, 2024-05-11T23:03:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:03:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.747 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:03:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:03:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:03:36.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:03:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:03:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:03:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:03:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:03:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:03:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:03:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:03:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:03:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:03:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.751 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.01087698800001, 2024-05-11T23:03:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:03:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.752 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:03:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:03:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:03:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:03:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:03:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:03:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:03:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:03:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:03:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:03:36.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:03:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:03:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:03:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:03:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.01033092199999, 2024-05-11T23:03:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:03:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:03:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:36.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:03:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:03:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:03:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:03:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:03:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:03:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:03:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:03:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:03:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:03:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:03:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:03:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:03:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:03:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.01029913799999, 2024-05-11T23:03:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:03:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:03:36.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:03:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:03:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:03:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:03:36.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:03:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:03:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:03:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:03:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:03:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:03:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:03:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:03:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:03:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:03:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.573053673999993, 2024-05-11T23:03:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:03:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:03:36.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:03:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:03:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:03:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:03:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:03:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:03:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:03:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:03:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:03:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:03:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:36.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:03:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:03:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:03:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:03:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:03:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:03:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:03:36.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:03:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:03:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:03:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:03:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:36.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:03:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:03:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:36.773 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:36.773 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:36.775 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:03:36.775 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:36.775+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:36.876 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:36.876+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:36.886 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:36.886 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:36.886 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:36.886 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:36.896 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:03:36.940 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:03:36.977 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:36.977+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:37.002 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:37.002 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:37.002 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:37.002 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:37.002 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:37.010 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:03:37.066 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:37.066 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:37.084 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:37.097 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:37.095+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:37.101 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:37.101 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:37.101 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-05-11T23:03:37.122 INFO:teuthology.orchestra.run.smithi005.stdout: marking collection for removal 2024-05-11T23:03:37.122 INFO:teuthology.orchestra.run.smithi005.stdout:setting '_remove' omap key 2024-05-11T23:03:37.122 INFO:teuthology.orchestra.run.smithi005.stdout:finish_remove_pgs 3.15_head removing 3.15 2024-05-11T23:03:37.125 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:37.126 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:37.134 INFO:teuthology.orchestra.run.smithi005.stdout:Remove successful 2024-05-11T23:03:37.144 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:37.144 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:37.160 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:37.160 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:37.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:37.161 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:37.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.161 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:03:37.161 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:37.161 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 139.99652424499999, 2024-05-11T23:03:37.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:03:37.161 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:37.161 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:37.161 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:37.161 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:37.161 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:03:37.161 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:03:37.161 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:03:37.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.162 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:37.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:37.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:37.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:37.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:37.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:37.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:03:37.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:03:37.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:37.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:03:37.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:03:37.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:37.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:03:37.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:03:37.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:37.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:03:37.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:03:37.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:03:37.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:03:37.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:03:37.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:03:37.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:03:37.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:03:37.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:03:37.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:37.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:03:37.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:03:37.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:03:37.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:03:37.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:03:37.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:37.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:03:37.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:03:37.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:37.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:03:37.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:03:37.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:37.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:03:37.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:03:37.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:37.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:03:37.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:03:37.167 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.167 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.167 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.167 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78429 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:37.167 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:37.167 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.195254903000006, 2024-05-11T23:03:37.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044563497000000001, 2024-05-11T23:03:37.167 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:37.167 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:37.167 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:37.167 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:37.168 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:37.168 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:37.168 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78429 2024-05-11T23:03:37.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.168 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:37.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:37.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:37.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:37.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:37.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:37.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964872+0000", 2024-05-11T23:03:37.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035070300000000001 2024-05-11T23:03:37.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:37.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964876+0000", 2024-05-11T23:03:37.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5009999999999998e-06 2024-05-11T23:03:37.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:37.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964902+0000", 2024-05-11T23:03:37.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5715999999999999e-05 2024-05-11T23:03:37.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:37.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964910+0000", 2024-05-11T23:03:37.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3930000000000002e-06 2024-05-11T23:03:37.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008898+0000", 2024-05-11T23:03:37.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043988772000000002 2024-05-11T23:03:37.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:37.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008981+0000", 2024-05-11T23:03:37.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3140999999999996e-05 2024-05-11T23:03:37.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:37.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009085+0000", 2024-05-11T23:03:37.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010327099999999999 2024-05-11T23:03:37.171 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.171 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.171 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.171 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78431 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:37.171 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:37.171 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.194794595000005, 2024-05-11T23:03:37.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044542033000000002, 2024-05-11T23:03:37.172 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:37.172 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:37.172 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:37.172 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:37.172 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:37.172 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:37.172 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78431 2024-05-11T23:03:37.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.172 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:37.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:37.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:37.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:37.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:37.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:37.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965038+0000", 2024-05-11T23:03:37.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6993000000000001e-05 2024-05-11T23:03:37.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:37.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965043+0000", 2024-05-11T23:03:37.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7040000000000002e-06 2024-05-11T23:03:37.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:37.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965053+0000", 2024-05-11T23:03:37.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8539999999999992e-06 2024-05-11T23:03:37.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:37.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965061+0000", 2024-05-11T23:03:37.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9810000000000003e-06 2024-05-11T23:03:37.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009350+0000", 2024-05-11T23:03:37.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044288739000000001 2024-05-11T23:03:37.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:37.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009429+0000", 2024-05-11T23:03:37.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9354e-05 2024-05-11T23:03:37.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:37.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009524+0000", 2024-05-11T23:03:37.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4407999999999995e-05 2024-05-11T23:03:37.175 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.175 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.175 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.176 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78432 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:37.176 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:37.176 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.194703747000005, 2024-05-11T23:03:37.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044635421000000002, 2024-05-11T23:03:37.176 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:37.176 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:37.176 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:37.176 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:37.176 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:37.176 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:37.176 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78432 2024-05-11T23:03:37.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.176 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:37.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:37.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:37.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:37.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:37.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:37.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965343+0000", 2024-05-11T23:03:37.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027056500000000001 2024-05-11T23:03:37.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:37.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965347+0000", 2024-05-11T23:03:37.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3540000000000002e-06 2024-05-11T23:03:37.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:37.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965356+0000", 2024-05-11T23:03:37.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2140000000000002e-06 2024-05-11T23:03:37.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:37.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965363+0000", 2024-05-11T23:03:37.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4150000000000004e-06 2024-05-11T23:03:37.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009551+0000", 2024-05-11T23:03:37.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044187917 2024-05-11T23:03:37.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:37.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009626+0000", 2024-05-11T23:03:37.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5024000000000005e-05 2024-05-11T23:03:37.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:37.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009708+0000", 2024-05-11T23:03:37.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1932000000000001e-05 2024-05-11T23:03:37.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.180 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.180 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.180 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78433 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:37.180 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:37.180 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.194402859999997, 2024-05-11T23:03:37.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044564864000000003, 2024-05-11T23:03:37.180 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:37.180 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:37.180 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:37.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:37.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:37.180 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:37.180 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78433 2024-05-11T23:03:37.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.181 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:37.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:37.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:37.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:37.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:37.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:37.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966225+0000", 2024-05-11T23:03:37.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085220700000000001 2024-05-11T23:03:37.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:37.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966230+0000", 2024-05-11T23:03:37.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0690000000000003e-06 2024-05-11T23:03:37.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:37.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966239+0000", 2024-05-11T23:03:37.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-05-11T23:03:37.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:37.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966245+0000", 2024-05-11T23:03:37.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5520000000000001e-06 2024-05-11T23:03:37.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009743+0000", 2024-05-11T23:03:37.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043498311999999997 2024-05-11T23:03:37.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:37.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009848+0000", 2024-05-11T23:03:37.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104885 2024-05-11T23:03:37.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:37.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009938+0000", 2024-05-11T23:03:37.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9812e-05 2024-05-11T23:03:37.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.184 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.184 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.184 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78580 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 999424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:37.184 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:37.184 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.191290176999999, 2024-05-11T23:03:37.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044493076999999999, 2024-05-11T23:03:37.184 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:37.184 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:37.184 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:37.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:37.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:37.184 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:37.184 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78580 2024-05-11T23:03:37.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.185 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:37.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:37.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:37.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:37.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:37.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:37.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:37.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:03:37.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:37.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968489+0000", 2024-05-11T23:03:37.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.52e-06 2024-05-11T23:03:37.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:37.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968496+0000", 2024-05-11T23:03:37.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9419999999999998e-06 2024-05-11T23:03:37.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:37.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968501+0000", 2024-05-11T23:03:37.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8720000000000001e-06 2024-05-11T23:03:37.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012690+0000", 2024-05-11T23:03:37.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044189655000000001 2024-05-11T23:03:37.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:37.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012848+0000", 2024-05-11T23:03:37.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015758100000000001 2024-05-11T23:03:37.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:37.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012979+0000", 2024-05-11T23:03:37.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000131066 2024-05-11T23:03:37.188 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.188 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.188 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.188 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78581 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:37.188 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:37.188 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.191269172999995, 2024-05-11T23:03:37.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044616243, 2024-05-11T23:03:37.188 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:37.188 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:37.188 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:37.188 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:37.188 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:37.188 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:37.189 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78581 2024-05-11T23:03:37.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.189 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:37.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:37.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:37.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:37.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:37.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:37.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970227+0000", 2024-05-11T23:03:37.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017200049999999999 2024-05-11T23:03:37.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:37.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970232+0000", 2024-05-11T23:03:37.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000003e-06 2024-05-11T23:03:37.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:37.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970242+0000", 2024-05-11T23:03:37.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0192e-05 2024-05-11T23:03:37.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:37.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970249+0000", 2024-05-11T23:03:37.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.888e-06 2024-05-11T23:03:37.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013008+0000", 2024-05-11T23:03:37.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042759495000000002 2024-05-11T23:03:37.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:37.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013063+0000", 2024-05-11T23:03:37.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5071999999999997e-05 2024-05-11T23:03:37.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:37.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013123+0000", 2024-05-11T23:03:37.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9930999999999999e-05 2024-05-11T23:03:37.191 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.192 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.192 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.192 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78589 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:37.192 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:37.192 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.189349929000002, 2024-05-11T23:03:37.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044587697000000003, 2024-05-11T23:03:37.192 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:37.192 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:37.192 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:37.192 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:37.192 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:37.192 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:37.193 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78589 2024-05-11T23:03:37.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.193 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:37.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:37.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:37.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:37.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:37.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:37.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970427+0000", 2024-05-11T23:03:37.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:03:37.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:37.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970429+0000", 2024-05-11T23:03:37.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3800000000000001e-06 2024-05-11T23:03:37.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:37.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970435+0000", 2024-05-11T23:03:37.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2530000000000001e-06 2024-05-11T23:03:37.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:37.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970441+0000", 2024-05-11T23:03:37.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-05-11T23:03:37.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014822+0000", 2024-05-11T23:03:37.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04438168 2024-05-11T23:03:37.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:37.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014911+0000", 2024-05-11T23:03:37.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8882999999999996e-05 2024-05-11T23:03:37.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:37.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015014+0000", 2024-05-11T23:03:37.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010263399999999999 2024-05-11T23:03:37.196 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.196 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.196 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.196 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78590 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:37.196 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:37.196 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.189329130000004, 2024-05-11T23:03:37.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044880120000000003, 2024-05-11T23:03:37.196 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:37.196 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:37.196 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:37.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:37.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:37.196 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:37.197 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78590 2024-05-11T23:03:37.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.197 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:37.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:37.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:37.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:37.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:37.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:37.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970918+0000", 2024-05-11T23:03:37.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000470834 2024-05-11T23:03:37.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:37.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970925+0000", 2024-05-11T23:03:37.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1690000000000003e-06 2024-05-11T23:03:37.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:37.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970934+0000", 2024-05-11T23:03:37.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9099999999999994e-06 2024-05-11T23:03:37.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:37.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970941+0000", 2024-05-11T23:03:37.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7410000000000003e-06 2024-05-11T23:03:37.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015049+0000", 2024-05-11T23:03:37.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044108392000000003 2024-05-11T23:03:37.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:37.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015133+0000", 2024-05-11T23:03:37.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3568999999999999e-05 2024-05-11T23:03:37.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:37.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015327+0000", 2024-05-11T23:03:37.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019450500000000001 2024-05-11T23:03:37.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.200 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.200 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78591 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:37.201 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:37.201 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.188825714000004, 2024-05-11T23:03:37.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044618497, 2024-05-11T23:03:37.201 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:37.201 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:37.201 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:37.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:37.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:37.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:37.201 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78591 2024-05-11T23:03:37.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.201 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:37.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:37.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:37.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:37.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:37.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:37.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970951+0000", 2024-05-11T23:03:37.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:03:37.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:37.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970953+0000", 2024-05-11T23:03:37.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:03:37.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:37.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970956+0000", 2024-05-11T23:03:37.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-05-11T23:03:37.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:37.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970959+0000", 2024-05-11T23:03:37.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9409999999999999e-06 2024-05-11T23:03:37.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015367+0000", 2024-05-11T23:03:37.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044407621000000001 2024-05-11T23:03:37.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:37.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015453+0000", 2024-05-11T23:03:37.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6685000000000001e-05 2024-05-11T23:03:37.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:37.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015569+0000", 2024-05-11T23:03:37.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115396 2024-05-11T23:03:37.204 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.204 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.205 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.205 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78592 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:37.205 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:37.205 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.188812905999995, 2024-05-11T23:03:37.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044831546999999999, 2024-05-11T23:03:37.205 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:37.205 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:37.205 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:37.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:37.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:37.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:37.205 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78592 2024-05-11T23:03:37.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.206 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:37.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:37.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:37.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:37.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:37.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:37.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972226+0000", 2024-05-11T23:03:37.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00126246 2024-05-11T23:03:37.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:37.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972230+0000", 2024-05-11T23:03:37.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1860000000000002e-06 2024-05-11T23:03:37.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:37.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972237+0000", 2024-05-11T23:03:37.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:03:37.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:37.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972241+0000", 2024-05-11T23:03:37.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.284e-06 2024-05-11T23:03:37.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015611+0000", 2024-05-11T23:03:37.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043369461999999998 2024-05-11T23:03:37.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:37.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015695+0000", 2024-05-11T23:03:37.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4458999999999999e-05 2024-05-11T23:03:37.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:37.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015795+0000", 2024-05-11T23:03:37.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9475999999999994e-05 2024-05-11T23:03:37.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.209 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.209 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78593 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:37.209 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:37.209 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.187525987000001, 2024-05-11T23:03:37.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046045214000000001, 2024-05-11T23:03:37.209 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:37.209 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:37.209 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:37.209 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:37.209 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:37.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:37.210 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78593 2024-05-11T23:03:37.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.210 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:37.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:37.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:37.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:37.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:37.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:37.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:37.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7599999999999998e-07 2024-05-11T23:03:37.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:37.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972252+0000", 2024-05-11T23:03:37.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7349999999999999e-06 2024-05-11T23:03:37.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:37.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972256+0000", 2024-05-11T23:03:37.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5659999999999998e-06 2024-05-11T23:03:37.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:37.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972259+0000", 2024-05-11T23:03:37.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8040000000000002e-06 2024-05-11T23:03:37.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018088+0000", 2024-05-11T23:03:37.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045829316000000002 2024-05-11T23:03:37.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:37.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018179+0000", 2024-05-11T23:03:37.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1299999999999997e-05 2024-05-11T23:03:37.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:37.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018295+0000", 2024-05-11T23:03:37.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116117 2024-05-11T23:03:37.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.213 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.213 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78594 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:37.213 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:37.213 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.187513627000001, 2024-05-11T23:03:37.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046245204999999998, 2024-05-11T23:03:37.213 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:37.213 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:37.213 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:37.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:37.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:37.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:37.214 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78594 2024-05-11T23:03:37.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.214 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:37.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:37.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:37.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:37.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:37.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:37.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972263+0000", 2024-05-11T23:03:37.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:03:37.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:37.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972264+0000", 2024-05-11T23:03:37.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.206e-06 2024-05-11T23:03:37.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:37.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972267+0000", 2024-05-11T23:03:37.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0690000000000002e-06 2024-05-11T23:03:37.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:37.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972270+0000", 2024-05-11T23:03:37.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5610000000000001e-06 2024-05-11T23:03:37.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018330+0000", 2024-05-11T23:03:37.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04606085 2024-05-11T23:03:37.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:37.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018414+0000", 2024-05-11T23:03:37.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3125000000000001e-05 2024-05-11T23:03:37.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:37.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018508+0000", 2024-05-11T23:03:37.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4153000000000005e-05 2024-05-11T23:03:37.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.217 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.217 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78595 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:37.217 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:37.217 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.187500761999999, 2024-05-11T23:03:37.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046396698, 2024-05-11T23:03:37.217 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:37.217 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:37.217 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:37.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:37.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:37.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:37.218 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78595 2024-05-11T23:03:37.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.218 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:37.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:37.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:37.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:37.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:37.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:37.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972276+0000", 2024-05-11T23:03:37.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5600000000000002e-07 2024-05-11T23:03:37.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:37.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972277+0000", 2024-05-11T23:03:37.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.576e-06 2024-05-11T23:03:37.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:37.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972280+0000", 2024-05-11T23:03:37.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-05-11T23:03:37.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:37.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972283+0000", 2024-05-11T23:03:37.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:03:37.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018538+0000", 2024-05-11T23:03:37.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046255255000000002 2024-05-11T23:03:37.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:37.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018605+0000", 2024-05-11T23:03:37.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6847000000000006e-05 2024-05-11T23:03:37.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:37.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018672+0000", 2024-05-11T23:03:37.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7045000000000006e-05 2024-05-11T23:03:37.221 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.221 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.221 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.221 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78596 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:37.221 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:37.221 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.187489912999993, 2024-05-11T23:03:37.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046586806000000001, 2024-05-11T23:03:37.221 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:37.221 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:37.221 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:37.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:37.222 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:37.222 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:37.222 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78596 2024-05-11T23:03:37.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.222 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:37.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:37.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:37.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:37.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:37.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:37.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:37.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:03:37.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:37.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972288+0000", 2024-05-11T23:03:37.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2550000000000001e-06 2024-05-11T23:03:37.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:37.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972291+0000", 2024-05-11T23:03:37.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9490000000000001e-06 2024-05-11T23:03:37.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:37.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972293+0000", 2024-05-11T23:03:37.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5069999999999999e-06 2024-05-11T23:03:37.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018707+0000", 2024-05-11T23:03:37.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046413604999999997 2024-05-11T23:03:37.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:37.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018782+0000", 2024-05-11T23:03:37.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5556000000000006e-05 2024-05-11T23:03:37.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:37.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018873+0000", 2024-05-11T23:03:37.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0703000000000002e-05 2024-05-11T23:03:37.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.225 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.225 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.225 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78597 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:37.225 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:37.225 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.187480140000005, 2024-05-11T23:03:37.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046825787000000001, 2024-05-11T23:03:37.225 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:37.225 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:37.225 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:37.225 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:37.225 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:37.226 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:37.226 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78597 2024-05-11T23:03:37.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.226 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:37.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:37.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:37.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:37.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:37.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:37.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:37.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-05-11T23:03:37.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:37.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972297+0000", 2024-05-11T23:03:37.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3069999999999999e-06 2024-05-11T23:03:37.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:37.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972300+0000", 2024-05-11T23:03:37.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:03:37.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972303+0000", 2024-05-11T23:03:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5509999999999998e-06 2024-05-11T23:03:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018911+0000", 2024-05-11T23:03:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046608194999999998 2024-05-11T23:03:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018986+0000", 2024-05-11T23:03:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5360999999999999e-05 2024-05-11T23:03:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019122+0000", 2024-05-11T23:03:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013526 2024-05-11T23:03:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78598 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 81.187470239999996, 2024-05-11T23:03:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046998572000000002, 2024-05-11T23:03:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:37.229 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78598 2024-05-11T23:03:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:03:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972307+0000", 2024-05-11T23:03:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1799999999999999e-06 2024-05-11T23:03:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972310+0000", 2024-05-11T23:03:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:03:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972313+0000", 2024-05-11T23:03:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4760000000000001e-06 2024-05-11T23:03:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019149+0000", 2024-05-11T23:03:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046836056000000001 2024-05-11T23:03:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019203+0000", 2024-05-11T23:03:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-05 2024-05-11T23:03:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019304+0000", 2024-05-11T23:03:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101537 2024-05-11T23:03:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:105234 3.15 3:adf885e5:::rbd_object_map.111e913a615f.0000000000000025:head [writefull 0~39] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-11T23:03:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 67.152154530000004, 2024-05-11T23:03:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77797599100000003, 2024-05-11T23:03:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:37.233 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 105234 2024-05-11T23:03:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007623+0000", 2024-05-11T23:03:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2240000000000001e-06 2024-05-11T23:03:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007627+0000", 2024-05-11T23:03:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:03:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007636+0000", 2024-05-11T23:03:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2409999999999995e-06 2024-05-11T23:03:37.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007641+0000", 2024-05-11T23:03:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6810000000000003e-06 2024-05-11T23:03:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007676+0000", 2024-05-11T23:03:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5061999999999997e-05 2024-05-11T23:03:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:03:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007682+0000", 2024-05-11T23:03:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0399999999999998e-06 2024-05-11T23:03:37.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783880+0000", 2024-05-11T23:03:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77619752900000005 2024-05-11T23:03:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783995+0000", 2024-05-11T23:03:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011550800000000001 2024-05-11T23:03:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:03:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784082+0000", 2024-05-11T23:03:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6347999999999994e-05 2024-05-11T23:03:37.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784682+0000", 2024-05-11T23:03:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000600162 2024-05-11T23:03:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785539+0000", 2024-05-11T23:03:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085699099999999998 2024-05-11T23:03:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785572+0000", 2024-05-11T23:03:37.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3182999999999997e-05 2024-05-11T23:03:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785598+0000", 2024-05-11T23:03:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5522000000000001e-05 2024-05-11T23:03:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113761 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [call rbd.sparse_copyup in=999496b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:03:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 62.629467611000003, 2024-05-11T23:03:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048572964000000003, 2024-05-11T23:03:37.239 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113761 2024-05-11T23:03:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530309+0000", 2024-05-11T23:03:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:03:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531242+0000", 2024-05-11T23:03:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093316899999999997 2024-05-11T23:03:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531249+0000", 2024-05-11T23:03:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8140000000000003e-06 2024-05-11T23:03:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531253+0000", 2024-05-11T23:03:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4290000000000003e-06 2024-05-11T23:03:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570325+0000", 2024-05-11T23:03:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039071379000000003 2024-05-11T23:03:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570382+0000", 2024-05-11T23:03:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6988999999999999e-05 2024-05-11T23:03:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:03:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570555+0000", 2024-05-11T23:03:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017379799999999999 2024-05-11T23:03:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.575953+0000", 2024-05-11T23:03:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053975539999999997 2024-05-11T23:03:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578778+0000", 2024-05-11T23:03:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002825616 2024-05-11T23:03:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578837+0000", 2024-05-11T23:03:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9005000000000002e-05 2024-05-11T23:03:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578881+0000", 2024-05-11T23:03:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3970000000000001e-05 2024-05-11T23:03:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.244 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113762 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [delete] snapc 27=[27] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:03:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 62.628513660000003, 2024-05-11T23:03:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047916260000000002, 2024-05-11T23:03:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113762 2024-05-11T23:03:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.245 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:37.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:37.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:37.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:37.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:37.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:37.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:37.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531263+0000", 2024-05-11T23:03:37.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.85e-07 2024-05-11T23:03:37.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:37.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531265+0000", 2024-05-11T23:03:37.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.827e-06 2024-05-11T23:03:37.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:37.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531268+0000", 2024-05-11T23:03:37.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3780000000000001e-06 2024-05-11T23:03:37.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:37.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531271+0000", 2024-05-11T23:03:37.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8509999999999999e-06 2024-05-11T23:03:37.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:37.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572298+0000", 2024-05-11T23:03:37.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041026990999999999 2024-05-11T23:03:37.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:37.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572345+0000", 2024-05-11T23:03:37.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6878000000000001e-05 2024-05-11T23:03:37.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:03:37.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572443+0000", 2024-05-11T23:03:37.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8184999999999996e-05 2024-05-11T23:03:37.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:37.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578585+0000", 2024-05-11T23:03:37.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006141919 2024-05-11T23:03:37.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:37.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579126+0000", 2024-05-11T23:03:37.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00054063799999999997 2024-05-11T23:03:37.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:37.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579154+0000", 2024-05-11T23:03:37.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8042999999999999e-05 2024-05-11T23:03:37.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:37.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:37.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:37.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579179+0000", 2024-05-11T23:03:37.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4765000000000001e-05 2024-05-11T23:03:37.250 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.250 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.250 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.250 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:37.250 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:37.250 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:37.251 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:37.252 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:37.252 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:37.253 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:03:37.253 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-11T23:03:37.254 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:37.254 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-11T23:03:37.254 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:37.252+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:37.352 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:37.352+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:37.404 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:37.404 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:37.413 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:03:37.457 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:37.457+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:37.522 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:37.523 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:37.534 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:03:37.562 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:37.559+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:37.636 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.15.3 2024-05-11T23:03:37.683 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:37.683+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:37.691 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:37.691 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:37.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:03:37.761 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:37.760+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:37.807 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-11T23:03:37.861 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:37.860+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:37.875 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:37.876 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:37.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:03:37.962 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:37.962+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:38.007 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:38.007 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:38.017 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:03:38.062 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:38.061+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:38.163 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:38.163+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:38.171 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:38.171 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:38.180 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:03:38.264 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:38.264+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:38.290 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:38.290 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:38.298 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:03:38.364 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:38.365+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:38.408 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:38.408 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:38.416 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:03:38.465 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:38.465+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:38.535 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:38.535 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:38.544 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:03:38.587 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:38.587+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:38.649 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:38.650 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:38.658 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:03:38.667 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:38.667+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:38.747 INFO:teuthology.orchestra.run.smithi005.stdout:get_pg_num_history pg_num_history pg_num_history(e129 pg_nums {1={14=1},2={16=8},3={19=32}} deleted_pools ) 2024-05-11T23:03:38.747 INFO:teuthology.orchestra.run.smithi005.stdout:Importing pgid 3.15 2024-05-11T23:03:38.747 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:a80b56a9:::rbd_data.111e913a615f.0000000000000011:25# 2024-05-11T23:03:38.749 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:a80b56a9:::rbd_data.111e913a615f.0000000000000011:head# 2024-05-11T23:03:38.750 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 25=[]:{25=[25]} 2024-05-11T23:03:38.750 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:a84b8d2b:::rbd_data.111ef9e47166.000000000000008b:head# 2024-05-11T23:03:38.750 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 29=[]:{} 2024-05-11T23:03:38.750 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:a8580d10:::rbd_data.111ebabb4604.0000000000000066:head# 2024-05-11T23:03:38.750 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 2f=[]:{} 2024-05-11T23:03:38.750 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:a879acfe:::rbd_data.111ebabb4604.0000000000000065:head# 2024-05-11T23:03:38.751 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 2f=[]:{} 2024-05-11T23:03:38.751 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:a8c1dbf6:::rbd_data.111e64023279.0000000000000043:20# 2024-05-11T23:03:38.751 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:a8c1dbf6:::rbd_data.111e64023279.0000000000000043:head# 2024-05-11T23:03:38.751 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 20=[]:{20=[20]} 2024-05-11T23:03:38.752 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:a8c550e1:::rbd_data.111edca93686.00000000000001bf:head# 2024-05-11T23:03:38.752 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 26=[]:{} 2024-05-11T23:03:38.752 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:a8cd2134:::rbd_data.111ef9e47166.00000000000000cf:head# 2024-05-11T23:03:38.752 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 29=[]:{} 2024-05-11T23:03:38.752 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:a8eae155:::rbd_data.111e7d9444a9.0000000000000009:head# 2024-05-11T23:03:38.752 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 22=[]:{} 2024-05-11T23:03:38.753 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:a8f4e01b:::rbd_data.111edca93686.0000000000000162:head# 2024-05-11T23:03:38.753 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 26=[]:{} 2024-05-11T23:03:38.754 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:a9023db9:::rbd_data.111ef9e47166.00000000000000b8:head# 2024-05-11T23:03:38.754 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 29=[]:{} 2024-05-11T23:03:38.754 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:a92399d6:::rbd_data.111e479bff4d.0000000000000014:head# 2024-05-11T23:03:38.754 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 0=[]:{} 2024-05-11T23:03:38.754 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:a9403309:::rbd_header.111ef9b37d2d:head# 2024-05-11T23:03:38.754 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 0=[]:{} 2024-05-11T23:03:38.754 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:a94c607e:::rbd_data.111ee4903046.000000000000003f:head# 2024-05-11T23:03:38.755 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 18=[]:{} 2024-05-11T23:03:38.755 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:a9e165c9:::rbd_data.111e5046b689.0000000000000015:1c# 2024-05-11T23:03:38.756 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:a9e165c9:::rbd_data.111e5046b689.0000000000000015:head# 2024-05-11T23:03:38.756 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 1c=[]:{1c=[1c]} 2024-05-11T23:03:38.756 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:a9edbb24:::rbd_data.111ef9e47166.0000000000000114:head# 2024-05-11T23:03:38.756 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 29=[]:{} 2024-05-11T23:03:38.757 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:a9f92ff2:::rbd_data.111ebabb4604.0000000000000106:head# 2024-05-11T23:03:38.757 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 2f=[]:{} 2024-05-11T23:03:38.757 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:aa0090a8:::rbd_data.111ed003fd7f.0000000000000007:head# 2024-05-11T23:03:38.760 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 0=[]:{} 2024-05-11T23:03:38.764 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:aa325950:::rbd_data.111ee4903046.0000000000000034:head# 2024-05-11T23:03:38.765 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 18=[]:{} 2024-05-11T23:03:38.765 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:aa4fca24:::rbd_data.111ef9e47166.0000000000000086:head# 2024-05-11T23:03:38.766 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 29=[]:{} 2024-05-11T23:03:38.766 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:aa8fbce6:::rbd_data.111ef9e47166.00000000000000be:head# 2024-05-11T23:03:38.766 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 29=[]:{} 2024-05-11T23:03:38.766 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:aa97d1e4:::rbd_data.111edca93686.0000000000000092:head# 2024-05-11T23:03:38.766 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 26=[]:{} 2024-05-11T23:03:38.767 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:aad7891d:::rbd_data.111e50384849.000000000000005b:head# 2024-05-11T23:03:38.767 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 5=[]:{} 2024-05-11T23:03:38.767 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:aaf589ab:::rbd_data.111e67f27da3.0000000000000034:head# 2024-05-11T23:03:38.767 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 28=[]:{} 2024-05-11T23:03:38.767 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:aaf73702:::rbd_object_map.111e5046b689.000000000000001c:head# 2024-05-11T23:03:38.767 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 0=[]:{} 2024-05-11T23:03:38.768 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:38.767+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:38.768 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ab251d05:::rbd_data.111ecc0f42c3.00000000000000af:19# 2024-05-11T23:03:38.768 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ab251d05:::rbd_data.111ecc0f42c3.00000000000000af:head# 2024-05-11T23:03:38.768 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 19=[]:{19=[19]} 2024-05-11T23:03:38.768 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ab2d1875:::rbd_data.111e7d9444a9.0000000000000002:22# 2024-05-11T23:03:38.778 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ab2d1875:::rbd_data.111e7d9444a9.0000000000000002:head# 2024-05-11T23:03:38.778 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 22=[]:{22=[22]} 2024-05-11T23:03:38.780 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ab37ba60:::rbd_data.111ebabb4604.00000000000000cc:head# 2024-05-11T23:03:38.780 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 2f=[]:{} 2024-05-11T23:03:38.781 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ab38d3f7:::rbd_data.111ed2934718.000000000000006c:24# 2024-05-11T23:03:38.781 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ab38d3f7:::rbd_data.111ed2934718.000000000000006c:head# 2024-05-11T23:03:38.781 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 24=[]:{24=[24]} 2024-05-11T23:03:38.781 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ab61b29b:::rbd_data.111ebabb4604.0000000000000122:head# 2024-05-11T23:03:38.782 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 2f=[]:{} 2024-05-11T23:03:38.782 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ab835b8b:::rbd_data.111ef9b37d2d.0000000000000022:1b# 2024-05-11T23:03:38.782 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ab835b8b:::rbd_data.111ef9b37d2d.0000000000000022:head# 2024-05-11T23:03:38.783 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 1b=[]:{1b=[1b]} 2024-05-11T23:03:38.784 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:abd0fbe1:::rbd_data.111ed8ed3219.000000000000002f:head# 2024-05-11T23:03:38.784 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 27=[]:{} 2024-05-11T23:03:38.784 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:abedf2fd:::rbd_data.111e1e89614a.0000000000000040:head# 2024-05-11T23:03:38.784 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 0=[]:{} 2024-05-11T23:03:38.784 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ac0c7ea5:::rbd_data.111ebabb4604.00000000000000b3:head# 2024-05-11T23:03:38.784 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 2f=[]:{} 2024-05-11T23:03:38.784 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ac0f692a:::rbd_data.111ebabb4604.00000000000000d2:head# 2024-05-11T23:03:38.784 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 2f=[]:{} 2024-05-11T23:03:38.785 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ac16c1a0:::rbd_data.111edca93686.0000000000000072:head# 2024-05-11T23:03:38.785 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 26=[]:{} 2024-05-11T23:03:38.785 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ac4f80fb:::rbd_data.111e64023279.000000000000002f:head# 2024-05-11T23:03:38.785 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 20=[]:{} 2024-05-11T23:03:38.785 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ac5f45c8:::rbd_data.111e5e4d8275.0000000000000022:head# 2024-05-11T23:03:38.785 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 0=[]:{} 2024-05-11T23:03:38.786 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ac7055dc:::rbd_data.111ebabb4604.0000000000000095:head# 2024-05-11T23:03:38.786 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 2f=[]:{} 2024-05-11T23:03:38.786 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ac751812:::rbd_data.111ebabb4604.00000000000000cd:head# 2024-05-11T23:03:38.786 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 2f=[]:{} 2024-05-11T23:03:38.786 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ac759760:::rbd_data.111e50384849.000000000000005a:head# 2024-05-11T23:03:38.787 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 5=[]:{} 2024-05-11T23:03:38.788 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ac978c95:::rbd_data.111edca93686.00000000000001d3:head# 2024-05-11T23:03:38.788 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 26=[]:{} 2024-05-11T23:03:38.788 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:aca8c3c6:::rbd_data.111e50384849.0000000000000046:head# 2024-05-11T23:03:38.788 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 5=[]:{} 2024-05-11T23:03:38.788 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:acc2e1e9:::rbd_data.111e67f27da3.0000000000000014:head# 2024-05-11T23:03:38.788 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 28=[]:{} 2024-05-11T23:03:38.788 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:accdadf5:::rbd_data.111ed8ed3219.00000000000000b6:head# 2024-05-11T23:03:38.788 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 27=[]:{} 2024-05-11T23:03:38.788 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ace3afa0:::rbd_data.111e5e4d8275.0000000000000015:11# 2024-05-11T23:03:38.789 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ace3afa0:::rbd_data.111e5e4d8275.0000000000000015:head# 2024-05-11T23:03:38.789 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 11=[]:{11=[11]} 2024-05-11T23:03:38.790 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:acfc6fa6:::rbd_data.111e4e122501.000000000000004e:f# 2024-05-11T23:03:38.791 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:acfc6fa6:::rbd_data.111e4e122501.000000000000004e:head# 2024-05-11T23:03:38.791 INFO:teuthology.orchestra.run.smithi005.stdout:snapset f=[]:{f=[f]} 2024-05-11T23:03:38.791 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ad116bb8:::rbd_data.111ecc0f42c3.0000000000000181:19# 2024-05-11T23:03:38.791 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ad116bb8:::rbd_data.111ecc0f42c3.0000000000000181:head# 2024-05-11T23:03:38.791 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 19=[]:{19=[19]} 2024-05-11T23:03:38.794 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ad1e8f6b:::rbd_data.111ebabb4604.00000000000000ce:head# 2024-05-11T23:03:38.794 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 2f=[]:{} 2024-05-11T23:03:38.794 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ad29dc5d:::rbd_data.111edca93686.0000000000000113:head# 2024-05-11T23:03:38.794 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 26=[]:{} 2024-05-11T23:03:38.794 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ad4c4d3c:::rbd_data.111edca93686.0000000000000040:head# 2024-05-11T23:03:38.794 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 26=[]:{} 2024-05-11T23:03:38.794 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ad673748:::rbd_data.111ed8ed3219.0000000000000059:head# 2024-05-11T23:03:38.794 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 27=[]:{} 2024-05-11T23:03:38.794 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ad6b93a6:::rbd_data.111ef7a115a1.0000000000000053:head# 2024-05-11T23:03:38.794 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 13=[]:{} 2024-05-11T23:03:38.794 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ad6d1b8d:::rbd_data.111ed8ed3219.00000000000000a9:head# 2024-05-11T23:03:38.795 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 27=[]:{} 2024-05-11T23:03:38.795 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ad715eb0:::rbd_data.111ebabb4604.000000000000000a:head# 2024-05-11T23:03:38.795 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 2f=[]:{} 2024-05-11T23:03:38.795 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ad96205b:::rbd_data.111ee4903046.0000000000000022:18# 2024-05-11T23:03:38.796 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ad96205b:::rbd_data.111ee4903046.0000000000000022:head# 2024-05-11T23:03:38.796 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 18=[]:{18=[18]} 2024-05-11T23:03:38.796 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:adec0c33:::rbd_data.111ee4903046.0000000000000045:head# 2024-05-11T23:03:38.796 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 18=[]:{} 2024-05-11T23:03:38.796 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:adf885e5:::rbd_object_map.111e913a615f.0000000000000025:head# 2024-05-11T23:03:38.796 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 0=[]:{} 2024-05-11T23:03:38.796 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:adf960fd:::rbd_data.111ebabb4604.00000000000000a2:head# 2024-05-11T23:03:38.796 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 2f=[]:{} 2024-05-11T23:03:38.796 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ae108ad7:::rbd_data.111ecc0f42c3.0000000000000028:19# 2024-05-11T23:03:38.797 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ae108ad7:::rbd_data.111ecc0f42c3.0000000000000028:head# 2024-05-11T23:03:38.797 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 19=[]:{19=[19]} 2024-05-11T23:03:38.797 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ae2ec3f2:::rbd_data.111e67f27da3.0000000000000000:head# 2024-05-11T23:03:38.797 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 28=[]:{} 2024-05-11T23:03:38.797 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ae4aa414:::rbd_data.111e67f27da3.0000000000000026:head# 2024-05-11T23:03:38.797 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 28=[]:{} 2024-05-11T23:03:38.797 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ae505ab8:::rbd_data.111ed2934718.0000000000000057:24# 2024-05-11T23:03:38.798 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ae505ab8:::rbd_data.111ed2934718.0000000000000057:head# 2024-05-11T23:03:38.798 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 24=[]:{24=[24]} 2024-05-11T23:03:38.798 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ae5932b3:::rbd_data.111e8974a681.0000000000000002:16# 2024-05-11T23:03:38.803 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:38.803 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:38.808 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ae5932b3:::rbd_data.111e8974a681.0000000000000002:head# 2024-05-11T23:03:38.808 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 16=[]:{16=[16]} 2024-05-11T23:03:38.809 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ae76d313:::rbd_data.111edca93686.00000000000000b5:head# 2024-05-11T23:03:38.809 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 26=[]:{} 2024-05-11T23:03:38.809 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ae8454b4:::rbd_data.111ed8ed3219.00000000000000b3:head# 2024-05-11T23:03:38.809 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 27=[]:{} 2024-05-11T23:03:38.809 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ae8bead3:::rbd_data.111ee4903046.000000000000004a:head# 2024-05-11T23:03:38.810 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 18=[]:{} 2024-05-11T23:03:38.810 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:ae8c5dd8:::rbd_data.111edca93686.0000000000000158:head# 2024-05-11T23:03:38.810 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 26=[]:{} 2024-05-11T23:03:38.810 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:aeb02784:::rbd_data.111e5e4d8275.0000000000000008:11# 2024-05-11T23:03:38.811 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:aeb02784:::rbd_data.111e5e4d8275.0000000000000008:head# 2024-05-11T23:03:38.812 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 11=[]:{11=[11]} 2024-05-11T23:03:38.812 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:aed4e7fe:::rbd_data.111e479bff4d.0000000000000037:head# 2024-05-11T23:03:38.813 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:03:38.814 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 0=[]:{} 2024-05-11T23:03:38.814 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:aee04b84:::rbd_data.111ee4903046.000000000000002b:head# 2024-05-11T23:03:38.815 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 18=[]:{} 2024-05-11T23:03:38.816 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:aee09837:::rbd_data.111ef9e47166.0000000000000089:head# 2024-05-11T23:03:38.816 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 29=[]:{} 2024-05-11T23:03:38.816 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:af13a123:::rbd_data.111edca93686.000000000000014a:head# 2024-05-11T23:03:38.816 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 26=[]:{} 2024-05-11T23:03:38.816 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:af229500:::rbd_data.111ee4903046.000000000000001a:head# 2024-05-11T23:03:38.816 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 18=[]:{} 2024-05-11T23:03:38.816 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:af299a46:::rbd_data.111ecc0f42c3.0000000000000117:19# 2024-05-11T23:03:38.818 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:af299a46:::rbd_data.111ecc0f42c3.0000000000000117:head# 2024-05-11T23:03:38.818 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 19=[]:{19=[19]} 2024-05-11T23:03:38.818 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:af3d21b4:::rbd_data.111edca93686.0000000000000127:head# 2024-05-11T23:03:38.818 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 26=[]:{} 2024-05-11T23:03:38.818 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:af65319d:::rbd_data.111ebabb4604.0000000000000073:head# 2024-05-11T23:03:38.818 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 2f=[]:{} 2024-05-11T23:03:38.818 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:af6f5356:::rbd_data.111e64023279.0000000000000063:20# 2024-05-11T23:03:38.819 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:af6f5356:::rbd_data.111e64023279.0000000000000063:head# 2024-05-11T23:03:38.819 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 20=[]:{20=[20]} 2024-05-11T23:03:38.819 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:afb8e764:::rbd_data.111e148bd16.000000000000000e:10# 2024-05-11T23:03:38.819 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:afb8e764:::rbd_data.111e148bd16.000000000000000e:head# 2024-05-11T23:03:38.820 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 10=[]:{10=[10]} 2024-05-11T23:03:38.822 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:afc839c8:::rbd_data.111ef9e47166.00000000000000a3:head# 2024-05-11T23:03:38.823 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 29=[]:{} 2024-05-11T23:03:38.823 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:afd4c577:::rbd_data.111edca93686.0000000000000067:head# 2024-05-11T23:03:38.823 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 26=[]:{} 2024-05-11T23:03:38.823 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:aff88f3b:::rbd_data.111ef7a115a1.0000000000000084:head# 2024-05-11T23:03:38.823 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 13=[]:{} 2024-05-11T23:03:38.824 INFO:teuthology.orchestra.run.smithi005.stdout:Write #3:affd1311:::rbd_data.111ef9e47166.0000000000000111:head# 2024-05-11T23:03:38.824 INFO:teuthology.orchestra.run.smithi005.stdout:snapset 29=[]:{} 2024-05-11T23:03:38.824 INFO:teuthology.orchestra.run.smithi005.stdout:write_pg epoch 129 info 3.15( v 128'553 (0'0,128'553] local-lis/les=76/77 n=100 ec=19/19 lis/c=76/76 les/c/f=77/77/0 sis=76) 2024-05-11T23:03:38.826 INFO:teuthology.orchestra.run.smithi005.stdout:Import successful 2024-05-11T23:03:38.835 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-05-11T23:03:38.891 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:38.891+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:38.930 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:38.931 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:38.941 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:03:38.985 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:38.985+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:39.050 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:39.050 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:39.056 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:03:39.069 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:39.069+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:39.170 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:39.170+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:39.203 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:39.203 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:39.212 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:03:39.254 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:03:39.295 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:39.295+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:39.356 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:39.356 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:39.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /var/log/ceph/exp.3.15.3 2024-05-11T23:03:39.365 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:03:39.386 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:39.386 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:39.386 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:39.386+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:39.404 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:39.404 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:39.421 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-05-11T23:03:39.439 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:39.439 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:39.457 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:39.457 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:39.473 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:39.473+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:39.475 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:39.475 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:39.493 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:39.493 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:39.496 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-11T23:03:39.496 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-11T23:03:39.510 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:39.510 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:39.511 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:39.511 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:39.519 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:03:39.574 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:39.574+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:39.624 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:39.625 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:39.625 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:39.633 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:03:39.675 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:39.675+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:39.775 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:39.775+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:39.780 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:39.780 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:39.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:39.780 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:39.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.780 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.125843238, 2024-05-11T23:03:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:03:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:03:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:39.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:03:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:39.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:03:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:03:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:03:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:03:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:03:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:03:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:39.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:03:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:03:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:03:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:03:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:03:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:03:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.784 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.12246204900001, 2024-05-11T23:03:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:03:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:03:39.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:39.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:03:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:03:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:03:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:03:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:03:39.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:03:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:03:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:03:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:03:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:03:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:03:39.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:03:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.122443342, 2024-05-11T23:03:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:03:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:39.789 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:03:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:03:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:03:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:03:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:03:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:03:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:03:39.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:03:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:03:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:03:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:03:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:03:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:03:39.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:03:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:03:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.12242519900001, 2024-05-11T23:03:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:03:39.793 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:03:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:03:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:03:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:03:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:39.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:03:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:03:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:03:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:03:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:03:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:03:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:39.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:03:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:03:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:03:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:03:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.797 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.12239771199999, 2024-05-11T23:03:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:03:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:03:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.798 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:03:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:03:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:03:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:03:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:03:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:03:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:03:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:03:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:39.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:03:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:03:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:03:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:03:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:03:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:03:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.122377184, 2024-05-11T23:03:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:03:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:03:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:39.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:39.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:03:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:03:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:03:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:03:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:03:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:39.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:03:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:03:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:03:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:03:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:03:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:03:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:39.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:03:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:03:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.122357409, 2024-05-11T23:03:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:03:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:39.806 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:03:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:39.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:03:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:03:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:03:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:03:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:03:39.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:03:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:03:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:03:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:03:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:03:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:03:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:03:39.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:03:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:03:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.122338603, 2024-05-11T23:03:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:03:39.810 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:03:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:39.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:03:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:03:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:03:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:03:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:03:39.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:03:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:03:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:03:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:03:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:03:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:03:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:03:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:03:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:03:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.12231968, 2024-05-11T23:03:39.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:03:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:03:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:03:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:03:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:03:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:03:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:03:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:03:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:03:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:03:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:03:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:03:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:03:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:03:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:03:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:39.818 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.122301018, 2024-05-11T23:03:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:03:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:03:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:03:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:03:39.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:03:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:03:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:03:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:03:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:03:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:03:39.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:03:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:03:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:03:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:03:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:03:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.121955856, 2024-05-11T23:03:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:03:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:03:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:39.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:03:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:03:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:03:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:03:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:03:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:03:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:03:39.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:03:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:03:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:03:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:03:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:03:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:03:39.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:03:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.121930306, 2024-05-11T23:03:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:03:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:39.827 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:03:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.828 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:03:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:03:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:03:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:03:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:03:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:03:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:03:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:03:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:03:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:03:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:03:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:03:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:03:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.121848048, 2024-05-11T23:03:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:03:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:39.831 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:03:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:39.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:03:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:03:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:03:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:03:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:03:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:03:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:03:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:03:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:03:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:03:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:03:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:03:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:03:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.121825601, 2024-05-11T23:03:39.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:03:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:03:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:03:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:03:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:03:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:03:39.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:03:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:03:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:03:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:03:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:03:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:03:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:03:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:03:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:03:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:03:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.121806143, 2024-05-11T23:03:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:03:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:03:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:39.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:03:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:03:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:03:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:03:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:03:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:03:39.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:03:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:03:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:03:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:03:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:03:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:03:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:03:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.121786083, 2024-05-11T23:03:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:03:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:39.844 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:03:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:03:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:03:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:03:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:03:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:03:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:03:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:03:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:03:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:03:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:03:39.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:03:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:03:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.12176681299999, 2024-05-11T23:03:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:03:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:39.848 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:03:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:39.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:03:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:03:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:03:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:39.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:03:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:03:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:03:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:03:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:03:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:03:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:03:39.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:03:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:03:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:03:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.121220747, 2024-05-11T23:03:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:03:39.852 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:03:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:39.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:03:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:03:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:03:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:03:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:03:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:03:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:03:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:03:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:03:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:03:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:03:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:03:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:03:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:03:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:39.856 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.12118896299999, 2024-05-11T23:03:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:03:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:03:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:03:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:03:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:03:39.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:03:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:03:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:03:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:03:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:03:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:03:39.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:03:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:03:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:03:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:03:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:03:39.860 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 97.683943498999994, 2024-05-11T23:03:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:03:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:03:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:03:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:03:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:03:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:03:39.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:03:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:03:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:03:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:03:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:03:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:03:39.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:03:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:03:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:03:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:03:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:03:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:03:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:03:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:03:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:03:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:03:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:03:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:03:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:03:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.865 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.866 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.866 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:39.866 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:39.866 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:39.867 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:03:39.876 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:39.876+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:39.977 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:39.977+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:39.983 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:39.983 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:39.983 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:39.983 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:39.992 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:03:40.077 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:40.077+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:40.138 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:40.138 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:40.138 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:40.138 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:40.138 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:40.147 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:03:40.188 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:40.188+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:40.279 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:40.279+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:40.294 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:03:40.294 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 143.13036380599999, 2024-05-11T23:03:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:03:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:03:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:03:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:03:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:40.295 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:03:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:03:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:03:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:03:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.296 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:03:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:03:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:03:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:03:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:03:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:03:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.297 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:03:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:03:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:03:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:03:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:03:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:03:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:03:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.298 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:03:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:03:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:03:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:03:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:03:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:03:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:03:40.299 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:03:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:03:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:03:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:03:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.300 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78429 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.329094463999994, 2024-05-11T23:03:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044563497000000001, 2024-05-11T23:03:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78429 2024-05-11T23:03:40.301 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:40.302 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964872+0000", 2024-05-11T23:03:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035070300000000001 2024-05-11T23:03:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964876+0000", 2024-05-11T23:03:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5009999999999998e-06 2024-05-11T23:03:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.303 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964902+0000", 2024-05-11T23:03:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5715999999999999e-05 2024-05-11T23:03:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964910+0000", 2024-05-11T23:03:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3930000000000002e-06 2024-05-11T23:03:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008898+0000", 2024-05-11T23:03:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043988772000000002 2024-05-11T23:03:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.304 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008981+0000", 2024-05-11T23:03:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3140999999999996e-05 2024-05-11T23:03:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009085+0000", 2024-05-11T23:03:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010327099999999999 2024-05-11T23:03:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78431 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:40.305 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.328634156000007, 2024-05-11T23:03:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044542033000000002, 2024-05-11T23:03:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78431 2024-05-11T23:03:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:40.306 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965038+0000", 2024-05-11T23:03:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6993000000000001e-05 2024-05-11T23:03:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:40.307 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965043+0000", 2024-05-11T23:03:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7040000000000002e-06 2024-05-11T23:03:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965053+0000", 2024-05-11T23:03:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8539999999999992e-06 2024-05-11T23:03:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965061+0000", 2024-05-11T23:03:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9810000000000003e-06 2024-05-11T23:03:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.308 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009350+0000", 2024-05-11T23:03:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044288739000000001 2024-05-11T23:03:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009429+0000", 2024-05-11T23:03:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9354e-05 2024-05-11T23:03:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009524+0000", 2024-05-11T23:03:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4407999999999995e-05 2024-05-11T23:03:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.309 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78432 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.328543307999993, 2024-05-11T23:03:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044635421000000002, 2024-05-11T23:03:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78432 2024-05-11T23:03:40.310 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965343+0000", 2024-05-11T23:03:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027056500000000001 2024-05-11T23:03:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.311 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965347+0000", 2024-05-11T23:03:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3540000000000002e-06 2024-05-11T23:03:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965356+0000", 2024-05-11T23:03:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2140000000000002e-06 2024-05-11T23:03:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965363+0000", 2024-05-11T23:03:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4150000000000004e-06 2024-05-11T23:03:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.312 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009551+0000", 2024-05-11T23:03:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044187917 2024-05-11T23:03:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009626+0000", 2024-05-11T23:03:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5024000000000005e-05 2024-05-11T23:03:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009708+0000", 2024-05-11T23:03:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1932000000000001e-05 2024-05-11T23:03:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78433 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.328242420999999, 2024-05-11T23:03:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044564864000000003, 2024-05-11T23:03:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78433 2024-05-11T23:03:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:40.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966225+0000", 2024-05-11T23:03:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085220700000000001 2024-05-11T23:03:40.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966230+0000", 2024-05-11T23:03:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0690000000000003e-06 2024-05-11T23:03:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966239+0000", 2024-05-11T23:03:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-05-11T23:03:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966245+0000", 2024-05-11T23:03:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5520000000000001e-06 2024-05-11T23:03:40.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009743+0000", 2024-05-11T23:03:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043498311999999997 2024-05-11T23:03:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009848+0000", 2024-05-11T23:03:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104885 2024-05-11T23:03:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009938+0000", 2024-05-11T23:03:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9812e-05 2024-05-11T23:03:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78580 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 999424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.325129738000001, 2024-05-11T23:03:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044493076999999999, 2024-05-11T23:03:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:40.318 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78580 2024-05-11T23:03:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:40.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:03:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968489+0000", 2024-05-11T23:03:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.52e-06 2024-05-11T23:03:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:40.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968496+0000", 2024-05-11T23:03:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9419999999999998e-06 2024-05-11T23:03:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968501+0000", 2024-05-11T23:03:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8720000000000001e-06 2024-05-11T23:03:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012690+0000", 2024-05-11T23:03:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044189655000000001 2024-05-11T23:03:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012848+0000", 2024-05-11T23:03:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015758100000000001 2024-05-11T23:03:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012979+0000", 2024-05-11T23:03:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000131066 2024-05-11T23:03:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78581 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:40.322 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.325108733999997, 2024-05-11T23:03:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044616243, 2024-05-11T23:03:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78581 2024-05-11T23:03:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970227+0000", 2024-05-11T23:03:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017200049999999999 2024-05-11T23:03:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970232+0000", 2024-05-11T23:03:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000003e-06 2024-05-11T23:03:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970242+0000", 2024-05-11T23:03:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0192e-05 2024-05-11T23:03:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970249+0000", 2024-05-11T23:03:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.888e-06 2024-05-11T23:03:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013008+0000", 2024-05-11T23:03:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042759495000000002 2024-05-11T23:03:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013063+0000", 2024-05-11T23:03:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5071999999999997e-05 2024-05-11T23:03:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013123+0000", 2024-05-11T23:03:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9930999999999999e-05 2024-05-11T23:03:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.326 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78589 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.323189490000004, 2024-05-11T23:03:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044587697000000003, 2024-05-11T23:03:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:40.327 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78589 2024-05-11T23:03:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970427+0000", 2024-05-11T23:03:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:03:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970429+0000", 2024-05-11T23:03:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3800000000000001e-06 2024-05-11T23:03:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970435+0000", 2024-05-11T23:03:40.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2530000000000001e-06 2024-05-11T23:03:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970441+0000", 2024-05-11T23:03:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-05-11T23:03:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014822+0000", 2024-05-11T23:03:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04438168 2024-05-11T23:03:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014911+0000", 2024-05-11T23:03:40.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8882999999999996e-05 2024-05-11T23:03:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015014+0000", 2024-05-11T23:03:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010263399999999999 2024-05-11T23:03:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78590 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.323168691000006, 2024-05-11T23:03:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044880120000000003, 2024-05-11T23:03:40.331 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78590 2024-05-11T23:03:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:40.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970918+0000", 2024-05-11T23:03:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000470834 2024-05-11T23:03:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970925+0000", 2024-05-11T23:03:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1690000000000003e-06 2024-05-11T23:03:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:40.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970934+0000", 2024-05-11T23:03:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9099999999999994e-06 2024-05-11T23:03:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970941+0000", 2024-05-11T23:03:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7410000000000003e-06 2024-05-11T23:03:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015049+0000", 2024-05-11T23:03:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044108392000000003 2024-05-11T23:03:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:40.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015133+0000", 2024-05-11T23:03:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3568999999999999e-05 2024-05-11T23:03:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015327+0000", 2024-05-11T23:03:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019450500000000001 2024-05-11T23:03:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78591 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.322665275000006, 2024-05-11T23:03:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044618497, 2024-05-11T23:03:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:40.336 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78591 2024-05-11T23:03:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:40.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970951+0000", 2024-05-11T23:03:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:03:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970953+0000", 2024-05-11T23:03:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:03:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970956+0000", 2024-05-11T23:03:40.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-05-11T23:03:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970959+0000", 2024-05-11T23:03:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9409999999999999e-06 2024-05-11T23:03:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015367+0000", 2024-05-11T23:03:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044407621000000001 2024-05-11T23:03:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015453+0000", 2024-05-11T23:03:40.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6685000000000001e-05 2024-05-11T23:03:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015569+0000", 2024-05-11T23:03:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115396 2024-05-11T23:03:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78592 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.322652466999998, 2024-05-11T23:03:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044831546999999999, 2024-05-11T23:03:40.340 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78592 2024-05-11T23:03:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:40.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972226+0000", 2024-05-11T23:03:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00126246 2024-05-11T23:03:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972230+0000", 2024-05-11T23:03:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1860000000000002e-06 2024-05-11T23:03:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972237+0000", 2024-05-11T23:03:40.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:03:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972241+0000", 2024-05-11T23:03:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.284e-06 2024-05-11T23:03:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015611+0000", 2024-05-11T23:03:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043369461999999998 2024-05-11T23:03:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015695+0000", 2024-05-11T23:03:40.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4458999999999999e-05 2024-05-11T23:03:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015795+0000", 2024-05-11T23:03:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9475999999999994e-05 2024-05-11T23:03:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78593 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.321365548000003, 2024-05-11T23:03:40.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046045214000000001, 2024-05-11T23:03:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78593 2024-05-11T23:03:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7599999999999998e-07 2024-05-11T23:03:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972252+0000", 2024-05-11T23:03:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7349999999999999e-06 2024-05-11T23:03:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:40.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972256+0000", 2024-05-11T23:03:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5659999999999998e-06 2024-05-11T23:03:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972259+0000", 2024-05-11T23:03:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8040000000000002e-06 2024-05-11T23:03:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018088+0000", 2024-05-11T23:03:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045829316000000002 2024-05-11T23:03:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018179+0000", 2024-05-11T23:03:40.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1299999999999997e-05 2024-05-11T23:03:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018295+0000", 2024-05-11T23:03:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116117 2024-05-11T23:03:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78594 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.321353188000003, 2024-05-11T23:03:40.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046245204999999998, 2024-05-11T23:03:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78594 2024-05-11T23:03:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972263+0000", 2024-05-11T23:03:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:03:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972264+0000", 2024-05-11T23:03:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.206e-06 2024-05-11T23:03:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972267+0000", 2024-05-11T23:03:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0690000000000002e-06 2024-05-11T23:03:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972270+0000", 2024-05-11T23:03:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5610000000000001e-06 2024-05-11T23:03:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018330+0000", 2024-05-11T23:03:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04606085 2024-05-11T23:03:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.351 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018414+0000", 2024-05-11T23:03:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3125000000000001e-05 2024-05-11T23:03:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018508+0000", 2024-05-11T23:03:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4153000000000005e-05 2024-05-11T23:03:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78595 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.321340323000001, 2024-05-11T23:03:40.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046396698, 2024-05-11T23:03:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78595 2024-05-11T23:03:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972276+0000", 2024-05-11T23:03:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5600000000000002e-07 2024-05-11T23:03:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972277+0000", 2024-05-11T23:03:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.576e-06 2024-05-11T23:03:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972280+0000", 2024-05-11T23:03:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-05-11T23:03:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972283+0000", 2024-05-11T23:03:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:03:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018538+0000", 2024-05-11T23:03:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046255255000000002 2024-05-11T23:03:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:40.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018605+0000", 2024-05-11T23:03:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6847000000000006e-05 2024-05-11T23:03:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018672+0000", 2024-05-11T23:03:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7045000000000006e-05 2024-05-11T23:03:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78596 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.321329473999995, 2024-05-11T23:03:40.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046586806000000001, 2024-05-11T23:03:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78596 2024-05-11T23:03:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:03:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972288+0000", 2024-05-11T23:03:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2550000000000001e-06 2024-05-11T23:03:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:40.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972291+0000", 2024-05-11T23:03:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9490000000000001e-06 2024-05-11T23:03:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972293+0000", 2024-05-11T23:03:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5069999999999999e-06 2024-05-11T23:03:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018707+0000", 2024-05-11T23:03:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046413604999999997 2024-05-11T23:03:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018782+0000", 2024-05-11T23:03:40.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5556000000000006e-05 2024-05-11T23:03:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018873+0000", 2024-05-11T23:03:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0703000000000002e-05 2024-05-11T23:03:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78597 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.321319700999993, 2024-05-11T23:03:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046825787000000001, 2024-05-11T23:03:40.360 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78597 2024-05-11T23:03:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-05-11T23:03:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972297+0000", 2024-05-11T23:03:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3069999999999999e-06 2024-05-11T23:03:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972300+0000", 2024-05-11T23:03:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:03:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972303+0000", 2024-05-11T23:03:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5509999999999998e-06 2024-05-11T23:03:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018911+0000", 2024-05-11T23:03:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046608194999999998 2024-05-11T23:03:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:40.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018986+0000", 2024-05-11T23:03:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5360999999999999e-05 2024-05-11T23:03:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019122+0000", 2024-05-11T23:03:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013526 2024-05-11T23:03:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78598 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 84.321309800999998, 2024-05-11T23:03:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046998572000000002, 2024-05-11T23:03:40.364 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78598 2024-05-11T23:03:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:03:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972307+0000", 2024-05-11T23:03:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1799999999999999e-06 2024-05-11T23:03:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972310+0000", 2024-05-11T23:03:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:03:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972313+0000", 2024-05-11T23:03:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4760000000000001e-06 2024-05-11T23:03:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019149+0000", 2024-05-11T23:03:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046836056000000001 2024-05-11T23:03:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019203+0000", 2024-05-11T23:03:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-05 2024-05-11T23:03:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019304+0000", 2024-05-11T23:03:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101537 2024-05-11T23:03:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:105234 3.15 3:adf885e5:::rbd_object_map.111e913a615f.0000000000000025:head [writefull 0~39] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-11T23:03:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.285994091000006, 2024-05-11T23:03:40.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77797599100000003, 2024-05-11T23:03:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 105234 2024-05-11T23:03:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007623+0000", 2024-05-11T23:03:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2240000000000001e-06 2024-05-11T23:03:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007627+0000", 2024-05-11T23:03:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:03:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007636+0000", 2024-05-11T23:03:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2409999999999995e-06 2024-05-11T23:03:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007641+0000", 2024-05-11T23:03:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6810000000000003e-06 2024-05-11T23:03:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007676+0000", 2024-05-11T23:03:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5061999999999997e-05 2024-05-11T23:03:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:03:40.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007682+0000", 2024-05-11T23:03:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0399999999999998e-06 2024-05-11T23:03:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783880+0000", 2024-05-11T23:03:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77619752900000005 2024-05-11T23:03:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783995+0000", 2024-05-11T23:03:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011550800000000001 2024-05-11T23:03:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:03:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784082+0000", 2024-05-11T23:03:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6347999999999994e-05 2024-05-11T23:03:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784682+0000", 2024-05-11T23:03:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000600162 2024-05-11T23:03:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785539+0000", 2024-05-11T23:03:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085699099999999998 2024-05-11T23:03:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785572+0000", 2024-05-11T23:03:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3182999999999997e-05 2024-05-11T23:03:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785598+0000", 2024-05-11T23:03:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5522000000000001e-05 2024-05-11T23:03:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.376 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113761 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [call rbd.sparse_copyup in=999496b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:03:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.763307171999998, 2024-05-11T23:03:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048572964000000003, 2024-05-11T23:03:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113761 2024-05-11T23:03:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.377 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530309+0000", 2024-05-11T23:03:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:03:40.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531242+0000", 2024-05-11T23:03:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093316899999999997 2024-05-11T23:03:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531249+0000", 2024-05-11T23:03:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8140000000000003e-06 2024-05-11T23:03:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531253+0000", 2024-05-11T23:03:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4290000000000003e-06 2024-05-11T23:03:40.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570325+0000", 2024-05-11T23:03:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039071379000000003 2024-05-11T23:03:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570382+0000", 2024-05-11T23:03:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6988999999999999e-05 2024-05-11T23:03:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:03:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570555+0000", 2024-05-11T23:03:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017379799999999999 2024-05-11T23:03:40.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.575953+0000", 2024-05-11T23:03:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053975539999999997 2024-05-11T23:03:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578778+0000", 2024-05-11T23:03:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002825616 2024-05-11T23:03:40.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578837+0000", 2024-05-11T23:03:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9005000000000002e-05 2024-05-11T23:03:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578881+0000", 2024-05-11T23:03:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3970000000000001e-05 2024-05-11T23:03:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113762 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [delete] snapc 27=[27] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:03:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.762353220999998, 2024-05-11T23:03:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047916260000000002, 2024-05-11T23:03:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113762 2024-05-11T23:03:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:40.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531263+0000", 2024-05-11T23:03:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.85e-07 2024-05-11T23:03:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531265+0000", 2024-05-11T23:03:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.827e-06 2024-05-11T23:03:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531268+0000", 2024-05-11T23:03:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3780000000000001e-06 2024-05-11T23:03:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531271+0000", 2024-05-11T23:03:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8509999999999999e-06 2024-05-11T23:03:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572298+0000", 2024-05-11T23:03:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041026990999999999 2024-05-11T23:03:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572345+0000", 2024-05-11T23:03:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6878000000000001e-05 2024-05-11T23:03:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:03:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572443+0000", 2024-05-11T23:03:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8184999999999996e-05 2024-05-11T23:03:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:40.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578585+0000", 2024-05-11T23:03:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006141919 2024-05-11T23:03:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579126+0000", 2024-05-11T23:03:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00054063799999999997 2024-05-11T23:03:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579154+0000", 2024-05-11T23:03:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8042999999999999e-05 2024-05-11T23:03:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579179+0000", 2024-05-11T23:03:40.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4765000000000001e-05 2024-05-11T23:03:40.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.388 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:40.388 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:40.388 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:40.389 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:40.390+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:40.390 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:03:40.490 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:40.490+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:40.504 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:40.504 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:40.504 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:40.504 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:40.512 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:03:40.590 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:40.590+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:40.622 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:40.622 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:40.622 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:40.622 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:40.622 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:40.631 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:03:40.691 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:40.691+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:40.782 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:40.782 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:40.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:40.783 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:40.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.783 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226051 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6049792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:40.783 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:40.783 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 38.464031708, 2024-05-11T23:03:40.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050622418000000002, 2024-05-11T23:03:40.783 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:40.783 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:40.783 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:40.783 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:40.783 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:40.784 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.784 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226051 2024-05-11T23:03:40.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.784 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:40.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:40.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:40.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:40.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:40.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:40.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317156+0000", 2024-05-11T23:03:40.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:40.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:40.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317157+0000", 2024-05-11T23:03:40.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.144e-06 2024-05-11T23:03:40.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:40.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317160+0000", 2024-05-11T23:03:40.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1350000000000001e-06 2024-05-11T23:03:40.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317162+0000", 2024-05-11T23:03:40.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5050000000000002e-06 2024-05-11T23:03:40.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367689+0000", 2024-05-11T23:03:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526797999999998 2024-05-11T23:03:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367735+0000", 2024-05-11T23:03:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5698999999999997e-05 2024-05-11T23:03:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367778+0000", 2024-05-11T23:03:40.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2928e-05 2024-05-11T23:03:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226052 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6060032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 38.464021652, 2024-05-11T23:03:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050830110999999997, 2024-05-11T23:03:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:40.788 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226052 2024-05-11T23:03:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:40.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:40.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:40.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317166+0000", 2024-05-11T23:03:40.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:03:40.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:40.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317167+0000", 2024-05-11T23:03:40.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.155e-06 2024-05-11T23:03:40.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:40.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317170+0000", 2024-05-11T23:03:40.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.061e-06 2024-05-11T23:03:40.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317172+0000", 2024-05-11T23:03:40.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4420000000000001e-06 2024-05-11T23:03:40.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:40.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367816+0000", 2024-05-11T23:03:40.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050644013000000002 2024-05-11T23:03:40.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:40.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367906+0000", 2024-05-11T23:03:40.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9739000000000003e-05 2024-05-11T23:03:40.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:40.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367995+0000", 2024-05-11T23:03:40.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9499000000000005e-05 2024-05-11T23:03:40.792 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.792 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.793 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.793 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226053 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6070272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:40.793 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:40.793 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 38.464011612999997, 2024-05-11T23:03:40.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051106912999999997, 2024-05-11T23:03:40.793 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:40.793 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:40.794 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:40.794 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:40.794 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:40.794 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226053 2024-05-11T23:03:40.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.794 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:40.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.794 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:40.794 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:40.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:40.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:40.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:40.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317176+0000", 2024-05-11T23:03:40.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0100000000000001e-07 2024-05-11T23:03:40.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.795 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:40.795 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317179+0000", 2024-05-11T23:03:40.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-05-11T23:03:40.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:40.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317182+0000", 2024-05-11T23:03:40.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3589999999999999e-06 2024-05-11T23:03:40.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317184+0000", 2024-05-11T23:03:40.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4339999999999999e-06 2024-05-11T23:03:40.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:40.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368061+0000", 2024-05-11T23:03:40.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050876927000000002 2024-05-11T23:03:40.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:40.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368179+0000", 2024-05-11T23:03:40.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118001 2024-05-11T23:03:40.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:40.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368282+0000", 2024-05-11T23:03:40.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102942 2024-05-11T23:03:40.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.798 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.798 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226054 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6080512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:40.799 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:40.799 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 38.463999291, 2024-05-11T23:03:40.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051322, 2024-05-11T23:03:40.799 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:40.799 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:40.799 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:40.799 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:40.799 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:40.799 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.799 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226054 2024-05-11T23:03:40.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.799 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:40.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:40.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:40.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:40.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:40.800 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:40.800 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:40.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:40.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:40.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317189+0000", 2024-05-11T23:03:40.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.217e-06 2024-05-11T23:03:40.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:40.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317192+0000", 2024-05-11T23:03:40.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0699999999999998e-06 2024-05-11T23:03:40.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317195+0000", 2024-05-11T23:03:40.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3829999999999999e-06 2024-05-11T23:03:40.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:40.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368328+0000", 2024-05-11T23:03:40.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051132999999999998 2024-05-11T23:03:40.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:40.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368429+0000", 2024-05-11T23:03:40.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010168800000000001 2024-05-11T23:03:40.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:40.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368510+0000", 2024-05-11T23:03:40.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0420999999999995e-05 2024-05-11T23:03:40.803 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.803 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.803 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.803 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226055 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6090752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:40.803 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:40.803 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 38.463989222999999, 2024-05-11T23:03:40.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051562835000000001, 2024-05-11T23:03:40.804 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:40.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:40.804 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:40.804 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:40.804 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:40.804 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.804 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226055 2024-05-11T23:03:40.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.804 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:40.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.804 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:40.804 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:40.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:40.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:40.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:40.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317256+0000", 2024-05-11T23:03:40.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8047000000000002e-05 2024-05-11T23:03:40.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:40.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317259+0000", 2024-05-11T23:03:40.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2590000000000001e-06 2024-05-11T23:03:40.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:40.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317267+0000", 2024-05-11T23:03:40.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8769999999999993e-06 2024-05-11T23:03:40.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317274+0000", 2024-05-11T23:03:40.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.187e-06 2024-05-11T23:03:40.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:40.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368544+0000", 2024-05-11T23:03:40.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051270299999999998 2024-05-11T23:03:40.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:40.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368658+0000", 2024-05-11T23:03:40.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000113347 2024-05-11T23:03:40.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:40.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368761+0000", 2024-05-11T23:03:40.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102818 2024-05-11T23:03:40.808 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.808 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.808 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.808 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226056 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6100992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:40.808 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:40.808 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 38.463903377000001, 2024-05-11T23:03:40.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051615576000000003, 2024-05-11T23:03:40.808 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:40.808 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:40.809 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:40.809 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:40.809 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:40.809 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.809 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226056 2024-05-11T23:03:40.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.809 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:40.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:40.809 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:40.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:40.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:40.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:40.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318915+0000", 2024-05-11T23:03:40.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001631291 2024-05-11T23:03:40.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:40.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318918+0000", 2024-05-11T23:03:40.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.512e-06 2024-05-11T23:03:40.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:40.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318927+0000", 2024-05-11T23:03:40.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1960000000000008e-06 2024-05-11T23:03:40.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318932+0000", 2024-05-11T23:03:40.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:03:40.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:40.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368805+0000", 2024-05-11T23:03:40.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049872998000000002 2024-05-11T23:03:40.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:40.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368853+0000", 2024-05-11T23:03:40.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8041999999999998e-05 2024-05-11T23:03:40.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:40.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368899+0000", 2024-05-11T23:03:40.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5766000000000002e-05 2024-05-11T23:03:40.813 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.813 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.813 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.813 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226061 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6152192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:40.813 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:40.813 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 38.46211229, 2024-05-11T23:03:40.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050675395999999998, 2024-05-11T23:03:40.813 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:40.813 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:40.813 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:40.814 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:40.814 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:40.814 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.814 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226061 2024-05-11T23:03:40.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.814 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:40.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:40.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:40.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:40.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:40.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:40.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:40.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:40.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:40.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319078+0000", 2024-05-11T23:03:40.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5670000000000002e-06 2024-05-11T23:03:40.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:40.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319085+0000", 2024-05-11T23:03:40.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2570000000000001e-06 2024-05-11T23:03:40.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319100+0000", 2024-05-11T23:03:40.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.489e-05 2024-05-11T23:03:40.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:40.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369584+0000", 2024-05-11T23:03:40.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050484345999999999 2024-05-11T23:03:40.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:40.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369671+0000", 2024-05-11T23:03:40.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6654000000000001e-05 2024-05-11T23:03:40.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:40.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369750+0000", 2024-05-11T23:03:40.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9257000000000004e-05 2024-05-11T23:03:40.818 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.818 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.818 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.818 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226062 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6162432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:40.818 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:40.818 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 38.462078746000003, 2024-05-11T23:03:40.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050761158000000001, 2024-05-11T23:03:40.818 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:40.819 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:40.819 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:40.819 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:40.819 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:40.819 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.819 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226062 2024-05-11T23:03:40.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.819 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:40.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:40.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:40.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:40.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:40.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:40.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319109+0000", 2024-05-11T23:03:40.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2800000000000002e-07 2024-05-11T23:03:40.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:40.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319111+0000", 2024-05-11T23:03:40.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4980000000000001e-06 2024-05-11T23:03:40.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:40.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319120+0000", 2024-05-11T23:03:40.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3999999999999992e-06 2024-05-11T23:03:40.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319129+0000", 2024-05-11T23:03:40.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7580000000000004e-06 2024-05-11T23:03:40.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:40.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369775+0000", 2024-05-11T23:03:40.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050645580000000003 2024-05-11T23:03:40.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.822 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:40.822 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369824+0000", 2024-05-11T23:03:40.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9506999999999999e-05 2024-05-11T23:03:40.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:40.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369869+0000", 2024-05-11T23:03:40.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4987000000000001e-05 2024-05-11T23:03:40.823 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.823 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.823 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.823 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226063 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6172672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:40.823 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:40.823 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 38.462049788999998, 2024-05-11T23:03:40.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050939784000000002, 2024-05-11T23:03:40.824 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:40.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:40.824 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:40.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:40.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:40.824 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226063 2024-05-11T23:03:40.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.824 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:40.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:40.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:40.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:40.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:40.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:40.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319138+0000", 2024-05-11T23:03:40.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.39e-07 2024-05-11T23:03:40.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:40.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319140+0000", 2024-05-11T23:03:40.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4820000000000001e-06 2024-05-11T23:03:40.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:40.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319148+0000", 2024-05-11T23:03:40.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6599999999999995e-06 2024-05-11T23:03:40.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319154+0000", 2024-05-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4169999999999997e-06 2024-05-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369905+0000", 2024-05-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050751009999999999 2024-05-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369991+0000", 2024-05-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5909999999999996e-05 2024-05-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370077+0000", 2024-05-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5865999999999994e-05 2024-05-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226064 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6182912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 38.462023592999998, 2024-05-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051038241999999998, 2024-05-11T23:03:40.828 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:40.829 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:40.829 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:40.829 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:40.829 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:40.829 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.829 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226064 2024-05-11T23:03:40.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.829 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:40.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:40.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:40.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:40.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:40.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:40.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319164+0000", 2024-05-11T23:03:40.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9299999999999999e-07 2024-05-11T23:03:40.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:40.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319166+0000", 2024-05-11T23:03:40.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.48e-06 2024-05-11T23:03:40.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:40.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319176+0000", 2024-05-11T23:03:40.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9669999999999996e-06 2024-05-11T23:03:40.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319180+0000", 2024-05-11T23:03:40.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8020000000000002e-06 2024-05-11T23:03:40.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:40.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370100+0000", 2024-05-11T23:03:40.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050919845999999998 2024-05-11T23:03:40.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:40.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370155+0000", 2024-05-11T23:03:40.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5040000000000002e-05 2024-05-11T23:03:40.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:40.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370202+0000", 2024-05-11T23:03:40.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6714e-05 2024-05-11T23:03:40.833 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.833 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.833 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.833 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226065 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6193152~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:40.833 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:40.833 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 38.462001022999999, 2024-05-11T23:03:40.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051225087000000002, 2024-05-11T23:03:40.833 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:40.834 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:40.834 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:40.834 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:40.834 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:40.834 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.834 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226065 2024-05-11T23:03:40.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.834 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:40.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:40.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:40.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:40.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:40.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:40.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320913+0000", 2024-05-11T23:03:40.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001727492 2024-05-11T23:03:40.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:40.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320919+0000", 2024-05-11T23:03:40.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.964e-06 2024-05-11T23:03:40.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:40.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320927+0000", 2024-05-11T23:03:40.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2239999999999998e-06 2024-05-11T23:03:40.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.836 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.836 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320934+0000", 2024-05-11T23:03:40.836 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8469999999999998e-06 2024-05-11T23:03:40.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:40.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370240+0000", 2024-05-11T23:03:40.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049306562999999998 2024-05-11T23:03:40.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.837 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:40.837 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370328+0000", 2024-05-11T23:03:40.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8122999999999996e-05 2024-05-11T23:03:40.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:40.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370411+0000", 2024-05-11T23:03:40.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2874000000000006e-05 2024-05-11T23:03:40.838 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.838 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.838 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.838 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226073 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6275072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:40.838 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:40.838 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 38.460162636, 2024-05-11T23:03:40.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050730475999999997, 2024-05-11T23:03:40.839 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:40.839 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:40.839 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:40.839 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:40.839 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:40.839 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.839 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226073 2024-05-11T23:03:40.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.839 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:40.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:40.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:40.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:40.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:40.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:40.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322914+0000", 2024-05-11T23:03:40.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001889153 2024-05-11T23:03:40.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:40.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322917+0000", 2024-05-11T23:03:40.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2100000000000002e-06 2024-05-11T23:03:40.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:40.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322924+0000", 2024-05-11T23:03:40.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9929999999999998e-06 2024-05-11T23:03:40.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322928+0000", 2024-05-11T23:03:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5549999999999996e-06 2024-05-11T23:03:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371578+0000", 2024-05-11T23:03:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048650001999999998 2024-05-11T23:03:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371671+0000", 2024-05-11T23:03:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2230000000000003e-05 2024-05-11T23:03:40.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:40.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371755+0000", 2024-05-11T23:03:40.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4332999999999994e-05 2024-05-11T23:03:40.843 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.843 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.843 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.843 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226085 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 5338112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:40.843 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:40.843 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 38.458071537000002, 2024-05-11T23:03:40.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050727622, 2024-05-11T23:03:40.844 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:40.844 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:40.844 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:40.844 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:40.844 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:40.844 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226085 2024-05-11T23:03:40.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.844 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:40.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:40.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:40.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:40.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:40.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:40.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324914+0000", 2024-05-11T23:03:40.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001798298 2024-05-11T23:03:40.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:40.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324918+0000", 2024-05-11T23:03:40.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3889999999999998e-06 2024-05-11T23:03:40.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:40.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324928+0000", 2024-05-11T23:03:40.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4339999999999998e-06 2024-05-11T23:03:40.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324936+0000", 2024-05-11T23:03:40.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2770000000000005e-06 2024-05-11T23:03:40.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:40.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373664+0000", 2024-05-11T23:03:40.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048728207000000003 2024-05-11T23:03:40.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:40.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373757+0000", 2024-05-11T23:03:40.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3332999999999996e-05 2024-05-11T23:03:40.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:40.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373843+0000", 2024-05-11T23:03:40.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5684000000000002e-05 2024-05-11T23:03:40.848 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.848 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.848 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.848 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232555 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7789568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:40.848 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:40.848 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.981061212, 2024-05-11T23:03:40.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050583086999999999, 2024-05-11T23:03:40.849 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:40.849 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:40.849 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:40.849 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:40.849 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:40.849 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.849 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232555 2024-05-11T23:03:40.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.849 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:40.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:40.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:40.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:40.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:40.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:40.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:40.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:03:40.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:40.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800127+0000", 2024-05-11T23:03:40.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.207e-06 2024-05-11T23:03:40.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:40.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800132+0000", 2024-05-11T23:03:40.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:03:40.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800134+0000", 2024-05-11T23:03:40.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5459999999999998e-06 2024-05-11T23:03:40.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:40.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850548+0000", 2024-05-11T23:03:40.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050414182000000002 2024-05-11T23:03:40.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:40.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850635+0000", 2024-05-11T23:03:40.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6853000000000003e-05 2024-05-11T23:03:40.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:40.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850709+0000", 2024-05-11T23:03:40.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3585999999999996e-05 2024-05-11T23:03:40.853 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.853 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.853 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.853 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232556 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7799808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:40.853 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:40.854 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.981049558000002, 2024-05-11T23:03:40.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050749599999999999, 2024-05-11T23:03:40.854 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:40.854 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:40.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:40.854 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:40.854 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:40.854 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.854 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232556 2024-05-11T23:03:40.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.854 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:40.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:40.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801914+0000", 2024-05-11T23:03:40.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017766360000000001 2024-05-11T23:03:40.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:40.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801919+0000", 2024-05-11T23:03:40.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.42e-06 2024-05-11T23:03:40.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:40.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801927+0000", 2024-05-11T23:03:40.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1999999999999994e-06 2024-05-11T23:03:40.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801931+0000", 2024-05-11T23:03:40.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4279999999999998e-06 2024-05-11T23:03:40.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:40.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850740+0000", 2024-05-11T23:03:40.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048808658999999997 2024-05-11T23:03:40.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:40.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850819+0000", 2024-05-11T23:03:40.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9102000000000003e-05 2024-05-11T23:03:40.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:40.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850887+0000", 2024-05-11T23:03:40.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8155000000000001e-05 2024-05-11T23:03:40.858 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.858 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.858 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.858 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232578 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8025088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:40.859 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:40.859 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.977756319999997, 2024-05-11T23:03:40.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050679473000000003, 2024-05-11T23:03:40.859 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:40.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:40.859 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:40.859 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:40.859 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:40.859 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.859 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232578 2024-05-11T23:03:40.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.859 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:40.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:40.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:40.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:40.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:40.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:40.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:40.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:40.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:40.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803432+0000", 2024-05-11T23:03:40.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.139e-06 2024-05-11T23:03:40.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:40.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803435+0000", 2024-05-11T23:03:40.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2440000000000002e-06 2024-05-11T23:03:40.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803439+0000", 2024-05-11T23:03:40.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3129999999999999e-06 2024-05-11T23:03:40.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:40.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.853953+0000", 2024-05-11T23:03:40.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050514846000000002 2024-05-11T23:03:40.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:40.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854035+0000", 2024-05-11T23:03:40.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1148000000000003e-05 2024-05-11T23:03:40.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:40.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854110+0000", 2024-05-11T23:03:40.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5562000000000006e-05 2024-05-11T23:03:40.863 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.863 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.864 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.864 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232579 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8035328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:40.864 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:40.864 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.977743715999999, 2024-05-11T23:03:40.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050886656000000002, 2024-05-11T23:03:40.864 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:40.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:40.864 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:40.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:40.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:40.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.865 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232579 2024-05-11T23:03:40.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.865 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:40.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:40.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:40.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:40.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:40.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:40.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803914+0000", 2024-05-11T23:03:40.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00047026700000000002 2024-05-11T23:03:40.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:40.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803918+0000", 2024-05-11T23:03:40.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3460000000000004e-06 2024-05-11T23:03:40.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:40.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803927+0000", 2024-05-11T23:03:40.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5030000000000003e-06 2024-05-11T23:03:40.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803933+0000", 2024-05-11T23:03:40.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0310000000000004e-06 2024-05-11T23:03:40.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:40.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854153+0000", 2024-05-11T23:03:40.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050219321999999997 2024-05-11T23:03:40.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:40.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854242+0000", 2024-05-11T23:03:40.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9457999999999996e-05 2024-05-11T23:03:40.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:40.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854330+0000", 2024-05-11T23:03:40.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7729e-05 2024-05-11T23:03:40.868 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.868 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.868 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.869 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232581 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8055808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:40.869 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:40.869 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.977220238999998, 2024-05-11T23:03:40.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050664439999999998, 2024-05-11T23:03:40.869 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:40.869 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:40.869 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:40.869 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:40.869 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:40.869 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.870 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232581 2024-05-11T23:03:40.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.870 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:40.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:40.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:40.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:40.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:40.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:40.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:40.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:03:40.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:40.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803969+0000", 2024-05-11T23:03:40.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0889999999999998e-06 2024-05-11T23:03:40.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:40.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803978+0000", 2024-05-11T23:03:40.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3070000000000003e-06 2024-05-11T23:03:40.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803983+0000", 2024-05-11T23:03:40.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1599999999999997e-06 2024-05-11T23:03:40.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:40.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854509+0000", 2024-05-11T23:03:40.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526339000000003 2024-05-11T23:03:40.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:40.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854576+0000", 2024-05-11T23:03:40.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7102999999999998e-05 2024-05-11T23:03:40.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:40.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854631+0000", 2024-05-11T23:03:40.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4997999999999998e-05 2024-05-11T23:03:40.873 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.873 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.873 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.874 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232582 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8066048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:40.874 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:40.874 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.977197494000002, 2024-05-11T23:03:40.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050753983000000003, 2024-05-11T23:03:40.874 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:40.874 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:40.874 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:40.874 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:40.874 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:40.874 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.875 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232582 2024-05-11T23:03:40.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.875 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:40.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:40.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:40.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:40.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:40.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:40.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805917+0000", 2024-05-11T23:03:40.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001927432 2024-05-11T23:03:40.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:40.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805920+0000", 2024-05-11T23:03:40.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1219999999999999e-06 2024-05-11T23:03:40.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:40.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805933+0000", 2024-05-11T23:03:40.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2727e-05 2024-05-11T23:03:40.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805941+0000", 2024-05-11T23:03:40.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7090000000000002e-06 2024-05-11T23:03:40.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:40.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854657+0000", 2024-05-11T23:03:40.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048716164999999999 2024-05-11T23:03:40.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:40.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854699+0000", 2024-05-11T23:03:40.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2531999999999999e-05 2024-05-11T23:03:40.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:40.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854744+0000", 2024-05-11T23:03:40.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4295999999999997e-05 2024-05-11T23:03:40.878 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.878 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.878 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.879 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232593 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8178688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:40.879 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:40.879 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 37.974998100999997, 2024-05-11T23:03:40.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050735178999999998, 2024-05-11T23:03:40.879 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:40.879 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:40.879 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:40.879 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:40.879 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:40.879 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:40.879 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232593 2024-05-11T23:03:40.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.879 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:40.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:40.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:40.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:40.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:40.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:40.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:40.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807913+0000", 2024-05-11T23:03:40.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017240319999999999 2024-05-11T23:03:40.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:40.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807916+0000", 2024-05-11T23:03:40.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4589999999999998e-06 2024-05-11T23:03:40.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:40.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807924+0000", 2024-05-11T23:03:40.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.272e-06 2024-05-11T23:03:40.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:40.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807928+0000", 2024-05-11T23:03:40.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5369999999999999e-06 2024-05-11T23:03:40.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:40.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856768+0000", 2024-05-11T23:03:40.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048839489999999999 2024-05-11T23:03:40.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:40.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856845+0000", 2024-05-11T23:03:40.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7746999999999999e-05 2024-05-11T23:03:40.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:40.883 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:40.883 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:40.883 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856924+0000", 2024-05-11T23:03:40.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8641999999999997e-05 2024-05-11T23:03:40.883 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.883 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.883 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.883 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:40.883 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:40.884 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:40.884 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-11T23:03:40.885 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:40.884+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:40.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:03:40.986 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:40.986+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:41.038 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:41.038 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:41.038 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:41.038 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:41.048 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:03:41.092 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:41.092+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:41.158 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:41.158 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:41.158 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:41.158 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:41.158 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:41.168 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:03:41.209 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:41.209+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:41.289 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:41.289+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:41.319 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:41.319 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:41.319 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:41.319 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:41.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.319 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:64049 3.8 3:11d98727:::rbd_object_map.111e8974a681:head [read 0~38] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-05-11T23:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 100.538738662, 2024-05-11T23:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99884604399999999, 2024-05-11T23:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 64049 2024-05-11T23:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.320 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779883+0000", 2024-05-11T23:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.57e-06 2024-05-11T23:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.321 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779887+0000", 2024-05-11T23:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5709999999999999e-06 2024-05-11T23:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779898+0000", 2024-05-11T23:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.042e-05 2024-05-11T23:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779905+0000", 2024-05-11T23:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0110000000000003e-06 2024-05-11T23:03:41.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779962+0000", 2024-05-11T23:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7395999999999997e-05 2024-05-11T23:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779969+0000", 2024-05-11T23:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2459999999999997e-06 2024-05-11T23:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778379+0000", 2024-05-11T23:03:41.323 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99840986300000001 2024-05-11T23:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778564+0000", 2024-05-11T23:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018420900000000001 2024-05-11T23:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778727+0000", 2024-05-11T23:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016375800000000001 2024-05-11T23:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.324 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172773 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.714632244000001, 2024-05-11T23:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058721316000000003, 2024-05-11T23:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172773 2024-05-11T23:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.325 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:41.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:41.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:41.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:41.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.326 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:41.326 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:41.326 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3500000000000002e-07 2024-05-11T23:03:41.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603990+0000", 2024-05-11T23:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3549999999999999e-06 2024-05-11T23:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603997+0000", 2024-05-11T23:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3400000000000003e-06 2024-05-11T23:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604002+0000", 2024-05-11T23:03:41.327 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0509999999999996e-06 2024-05-11T23:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662697+0000", 2024-05-11T23:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058695487999999997 2024-05-11T23:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662709+0000", 2024-05-11T23:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1647e-05 2024-05-11T23:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.328 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172774 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.714610862000001, 2024-05-11T23:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058745040999999998, 2024-05-11T23:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172774 2024-05-11T23:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.329 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:41.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:41.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:41.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:41.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:41.330 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:41.330 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:03:41.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.330 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.330 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:41.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604012+0000", 2024-05-11T23:03:41.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2199999999999999e-06 2024-05-11T23:03:41.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:41.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604018+0000", 2024-05-11T23:03:41.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1770000000000003e-06 2024-05-11T23:03:41.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:41.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604023+0000", 2024-05-11T23:03:41.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0089999999999998e-06 2024-05-11T23:03:41.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.331 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.331 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.331 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662742+0000", 2024-05-11T23:03:41.331 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058719187999999999 2024-05-11T23:03:41.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.332 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:41.332 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662754+0000", 2024-05-11T23:03:41.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2095e-05 2024-05-11T23:03:41.332 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.332 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.332 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.332 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172775 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:41.332 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:41.332 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.714590649999998, 2024-05-11T23:03:41.332 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058764743000000001, 2024-05-11T23:03:41.332 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:41.333 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:41.333 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:41.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:41.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:41.333 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:41.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172775 2024-05-11T23:03:41.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.333 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:41.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:41.333 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:41.333 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.333 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.333 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:41.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:41.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:41.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604030+0000", 2024-05-11T23:03:41.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6699999999999999e-07 2024-05-11T23:03:41.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.334 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:41.334 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604032+0000", 2024-05-11T23:03:41.334 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9530000000000002e-06 2024-05-11T23:03:41.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.334 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:41.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604037+0000", 2024-05-11T23:03:41.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8590000000000001e-06 2024-05-11T23:03:41.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:41.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604042+0000", 2024-05-11T23:03:41.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8609999999999997e-06 2024-05-11T23:03:41.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.335 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.335 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662782+0000", 2024-05-11T23:03:41.335 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058739731000000003 2024-05-11T23:03:41.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.335 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.336 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:41.336 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662794+0000", 2024-05-11T23:03:41.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1972e-05 2024-05-11T23:03:41.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.336 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.336 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172776 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:41.336 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:41.336 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.714571360000001, 2024-05-11T23:03:41.336 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058778696999999998, 2024-05-11T23:03:41.336 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:41.336 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:41.336 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:41.337 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:41.337 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:41.337 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:41.337 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172776 2024-05-11T23:03:41.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.337 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:41.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:41.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:41.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.337 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.337 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:41.337 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:41.337 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:41.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:41.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2000000000000001e-07 2024-05-11T23:03:41.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:41.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604051+0000", 2024-05-11T23:03:41.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2879999999999999e-06 2024-05-11T23:03:41.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.338 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.338 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:41.338 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604057+0000", 2024-05-11T23:03:41.338 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:03:41.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:41.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604062+0000", 2024-05-11T23:03:41.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6800000000000001e-06 2024-05-11T23:03:41.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662820+0000", 2024-05-11T23:03:41.339 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058758444999999999 2024-05-11T23:03:41.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.339 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.339 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:41.339 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662827+0000", 2024-05-11T23:03:41.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1930000000000004e-06 2024-05-11T23:03:41.340 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.340 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.340 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.340 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172777 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:41.340 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:41.340 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.714552054000002, 2024-05-11T23:03:41.340 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058799918999999999, 2024-05-11T23:03:41.340 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:41.340 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:41.340 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:41.340 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:41.340 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:41.340 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:41.341 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172777 2024-05-11T23:03:41.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.341 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:41.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:41.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:41.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:41.341 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:41.341 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.341 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.341 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:41.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:41.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:03:41.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:41.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604075+0000", 2024-05-11T23:03:41.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.109e-06 2024-05-11T23:03:41.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:41.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604083+0000", 2024-05-11T23:03:41.342 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9170000000000006e-06 2024-05-11T23:03:41.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.342 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.342 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:41.342 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604089+0000", 2024-05-11T23:03:41.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3399999999999997e-06 2024-05-11T23:03:41.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662859+0000", 2024-05-11T23:03:41.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058770172000000002 2024-05-11T23:03:41.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.343 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:41.343 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662868+0000", 2024-05-11T23:03:41.343 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0289999999999994e-06 2024-05-11T23:03:41.343 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.343 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.343 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.344 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172778 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:41.344 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:41.344 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.714523964000001, 2024-05-11T23:03:41.344 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058845435000000001, 2024-05-11T23:03:41.344 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:41.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:41.344 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:41.344 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:41.344 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:41.344 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:41.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172778 2024-05-11T23:03:41.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.344 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:41.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.344 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:41.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:41.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:41.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:41.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:41.345 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:41.345 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5400000000000002e-07 2024-05-11T23:03:41.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.345 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.345 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:41.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604099+0000", 2024-05-11T23:03:41.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4650000000000001e-06 2024-05-11T23:03:41.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:41.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604105+0000", 2024-05-11T23:03:41.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.314e-06 2024-05-11T23:03:41.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:41.346 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604110+0000", 2024-05-11T23:03:41.346 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6310000000000002e-06 2024-05-11T23:03:41.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.346 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662931+0000", 2024-05-11T23:03:41.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058820960999999998 2024-05-11T23:03:41.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.347 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:41.347 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662941+0000", 2024-05-11T23:03:41.347 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.061e-05 2024-05-11T23:03:41.347 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.347 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.347 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.347 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.347 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172779 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:41.347 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:41.348 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.714503804000003, 2024-05-11T23:03:41.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058859163999999999, 2024-05-11T23:03:41.348 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:41.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:41.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:41.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:41.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:41.348 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:41.348 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172779 2024-05-11T23:03:41.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.348 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:41.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.348 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:41.348 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:41.348 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:41.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:41.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:41.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:41.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:03:41.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.349 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:41.349 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604119+0000", 2024-05-11T23:03:41.349 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2529999999999999e-06 2024-05-11T23:03:41.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:41.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604125+0000", 2024-05-11T23:03:41.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1670000000000004e-06 2024-05-11T23:03:41.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:41.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604130+0000", 2024-05-11T23:03:41.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.702e-06 2024-05-11T23:03:41.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.350 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.350 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.350 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662967+0000", 2024-05-11T23:03:41.350 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058837792999999999 2024-05-11T23:03:41.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.351 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:41.351 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662975+0000", 2024-05-11T23:03:41.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8490000000000006e-06 2024-05-11T23:03:41.351 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.351 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.351 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.351 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.351 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172780 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:41.351 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:41.351 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.714483653999999, 2024-05-11T23:03:41.351 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058872696000000002, 2024-05-11T23:03:41.351 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:41.351 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:41.352 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:41.352 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:41.352 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:41.352 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:41.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172780 2024-05-11T23:03:41.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.352 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:41.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:41.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:41.352 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.352 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:41.352 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:41.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:41.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604137+0000", 2024-05-11T23:03:41.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4400000000000001e-07 2024-05-11T23:03:41.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:41.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604139+0000", 2024-05-11T23:03:41.353 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3429999999999998e-06 2024-05-11T23:03:41.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.353 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.353 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:41.353 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604145+0000", 2024-05-11T23:03:41.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1940000000000003e-06 2024-05-11T23:03:41.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:41.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604150+0000", 2024-05-11T23:03:41.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:03:41.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.354 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663002+0000", 2024-05-11T23:03:41.354 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058851829000000001 2024-05-11T23:03:41.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.354 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.354 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:41.355 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663009+0000", 2024-05-11T23:03:41.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.199e-06 2024-05-11T23:03:41.355 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.355 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.355 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.355 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.355 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172781 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:41.355 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:41.355 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.714451814, 2024-05-11T23:03:41.355 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058875553999999997, 2024-05-11T23:03:41.355 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:41.355 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:41.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:41.355 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:41.356 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:41.356 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:41.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172781 2024-05-11T23:03:41.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.356 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:41.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:41.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:41.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.356 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.356 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:41.356 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:41.356 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:41.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:41.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:41.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:41.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604171+0000", 2024-05-11T23:03:41.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.21e-06 2024-05-11T23:03:41.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.357 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:41.357 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604176+0000", 2024-05-11T23:03:41.357 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2970000000000003e-06 2024-05-11T23:03:41.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:41.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604179+0000", 2024-05-11T23:03:41.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8459999999999999e-06 2024-05-11T23:03:41.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663033+0000", 2024-05-11T23:03:41.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058854565999999997 2024-05-11T23:03:41.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.358 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.358 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:41.358 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663044+0000", 2024-05-11T23:03:41.358 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0229e-05 2024-05-11T23:03:41.358 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.359 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.359 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.359 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172782 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:41.359 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:41.359 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.714437066000002, 2024-05-11T23:03:41.359 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058894375999999998, 2024-05-11T23:03:41.359 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:41.359 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:41.359 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:41.359 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:41.359 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:41.359 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:41.359 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172782 2024-05-11T23:03:41.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.360 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:41.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:41.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:41.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:41.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:41.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.360 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:41.360 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:41.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-05-11T23:03:41.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:41.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604184+0000", 2024-05-11T23:03:41.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:03:41.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:41.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604187+0000", 2024-05-11T23:03:41.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.044e-06 2024-05-11T23:03:41.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:41.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604190+0000", 2024-05-11T23:03:41.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6460000000000002e-06 2024-05-11T23:03:41.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663070+0000", 2024-05-11T23:03:41.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058879510000000003 2024-05-11T23:03:41.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:41.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663077+0000", 2024-05-11T23:03:41.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6669999999999996e-06 2024-05-11T23:03:41.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.362 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.363 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172783 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:41.363 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:41.363 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.714426746999997, 2024-05-11T23:03:41.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058913946000000002, 2024-05-11T23:03:41.363 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:41.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:41.363 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:41.363 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:41.363 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:41.363 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:41.363 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172783 2024-05-11T23:03:41.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.363 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:41.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:41.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:41.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:41.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:41.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:41.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:41.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-05-11T23:03:41.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:41.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604195+0000", 2024-05-11T23:03:41.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.19e-06 2024-05-11T23:03:41.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:41.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604198+0000", 2024-05-11T23:03:41.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:03:41.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:41.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604200+0000", 2024-05-11T23:03:41.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4719999999999998e-06 2024-05-11T23:03:41.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663100+0000", 2024-05-11T23:03:41.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058900300000000003 2024-05-11T23:03:41.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.366 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:41.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663107+0000", 2024-05-11T23:03:41.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7449999999999998e-06 2024-05-11T23:03:41.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.366 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.366 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172784 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:41.367 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:41.367 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.714416728000003, 2024-05-11T23:03:41.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933728999999997, 2024-05-11T23:03:41.367 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:41.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:41.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:41.367 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:41.367 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:41.367 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:41.367 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172784 2024-05-11T23:03:41.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.367 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:41.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:41.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:41.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:41.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:41.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:41.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:41.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.03e-07 2024-05-11T23:03:41.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:41.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604205+0000", 2024-05-11T23:03:41.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2300000000000001e-06 2024-05-11T23:03:41.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:41.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604207+0000", 2024-05-11T23:03:41.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8729999999999999e-06 2024-05-11T23:03:41.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:41.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604210+0000", 2024-05-11T23:03:41.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.858e-06 2024-05-11T23:03:41.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663130+0000", 2024-05-11T23:03:41.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058919314 2024-05-11T23:03:41.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:41.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663137+0000", 2024-05-11T23:03:41.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2509999999999997e-06 2024-05-11T23:03:41.370 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.370 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.370 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.370 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172785 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:41.370 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:41.370 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.714404457999997, 2024-05-11T23:03:41.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058951664000000001, 2024-05-11T23:03:41.371 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:41.371 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:41.371 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:41.371 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:41.371 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:41.371 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:41.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172785 2024-05-11T23:03:41.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.371 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:41.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.371 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:41.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:41.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:41.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:41.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:41.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604216+0000", 2024-05-11T23:03:41.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:03:41.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.372 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:41.372 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604218+0000", 2024-05-11T23:03:41.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7549999999999999e-06 2024-05-11T23:03:41.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.372 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:41.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604222+0000", 2024-05-11T23:03:41.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6700000000000002e-06 2024-05-11T23:03:41.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:41.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604227+0000", 2024-05-11T23:03:41.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4399999999999998e-06 2024-05-11T23:03:41.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663160+0000", 2024-05-11T23:03:41.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933119999999999 2024-05-11T23:03:41.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:41.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663167+0000", 2024-05-11T23:03:41.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1840000000000002e-06 2024-05-11T23:03:41.374 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.374 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.374 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.374 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172786 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:41.374 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:41.374 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.714387404, 2024-05-11T23:03:41.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058966991000000003, 2024-05-11T23:03:41.374 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:41.375 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:41.375 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:41.375 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:41.375 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:41.375 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:41.375 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172786 2024-05-11T23:03:41.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.375 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:41.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:41.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:41.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:41.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:41.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:41.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604233+0000", 2024-05-11T23:03:41.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6899999999999999e-07 2024-05-11T23:03:41.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:41.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604235+0000", 2024-05-11T23:03:41.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8369999999999999e-06 2024-05-11T23:03:41.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:41.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604240+0000", 2024-05-11T23:03:41.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0329999999999999e-06 2024-05-11T23:03:41.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:41.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604244+0000", 2024-05-11T23:03:41.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9859999999999996e-06 2024-05-11T23:03:41.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663192+0000", 2024-05-11T23:03:41.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058948852000000003 2024-05-11T23:03:41.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:41.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663199+0000", 2024-05-11T23:03:41.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0140000000000001e-06 2024-05-11T23:03:41.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.378 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.378 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172787 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:41.378 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:41.378 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.714371057999998, 2024-05-11T23:03:41.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058982018999999997, 2024-05-11T23:03:41.378 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:41.378 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:41.378 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:41.378 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:41.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:41.379 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:41.379 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172787 2024-05-11T23:03:41.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.379 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:41.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:41.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:41.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.379 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:41.379 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:41.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:41.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605916+0000", 2024-05-11T23:03:41.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00166752 2024-05-11T23:03:41.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:41.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605921+0000", 2024-05-11T23:03:41.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0669999999999999e-06 2024-05-11T23:03:41.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.380 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:41.380 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605934+0000", 2024-05-11T23:03:41.380 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2271e-05 2024-05-11T23:03:41.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.380 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:41.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605940+0000", 2024-05-11T23:03:41.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6880000000000002e-06 2024-05-11T23:03:41.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663224+0000", 2024-05-11T23:03:41.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057283251 2024-05-11T23:03:41.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:41.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663231+0000", 2024-05-11T23:03:41.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2219999999999996e-06 2024-05-11T23:03:41.381 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.381 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.381 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.382 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172793 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:41.382 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:41.382 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.712604210999999, 2024-05-11T23:03:41.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057402373999999999, 2024-05-11T23:03:41.382 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:41.382 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:41.382 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:41.382 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:41.382 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:41.382 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:41.382 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172793 2024-05-11T23:03:41.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.383 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:41.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:41.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:41.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:41.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:41.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:41.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:41.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3300000000000001e-07 2024-05-11T23:03:41.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:41.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606017+0000", 2024-05-11T23:03:41.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4920000000000001e-06 2024-05-11T23:03:41.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:41.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606020+0000", 2024-05-11T23:03:41.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:03:41.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:41.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606023+0000", 2024-05-11T23:03:41.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6680000000000001e-06 2024-05-11T23:03:41.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663411+0000", 2024-05-11T23:03:41.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057388206999999997 2024-05-11T23:03:41.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:41.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663418+0000", 2024-05-11T23:03:41.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.815e-06 2024-05-11T23:03:41.385 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.385 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.385 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.386 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172795 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:41.386 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:41.386 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.712562783999999, 2024-05-11T23:03:41.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057423015000000001, 2024-05-11T23:03:41.386 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:41.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:41.386 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:41.386 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:41.386 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:41.386 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:41.386 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172795 2024-05-11T23:03:41.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.386 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:41.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:41.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:41.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:41.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:41.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:41.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:41.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9499999999999998e-07 2024-05-11T23:03:41.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:41.390 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606059+0000", 2024-05-11T23:03:41.390 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-05-11T23:03:41.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.390 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:41.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606062+0000", 2024-05-11T23:03:41.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.977e-06 2024-05-11T23:03:41.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:41.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606064+0000", 2024-05-11T23:03:41.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7410000000000001e-06 2024-05-11T23:03:41.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663473+0000", 2024-05-11T23:03:41.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057408624999999998 2024-05-11T23:03:41.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:41.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663480+0000", 2024-05-11T23:03:41.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1049999999999997e-06 2024-05-11T23:03:41.392 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.392 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.392 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.392 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172796 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:41.392 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:41.392 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.712552221000003, 2024-05-11T23:03:41.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057449235000000001, 2024-05-11T23:03:41.392 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:41.392 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:41.392 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:41.392 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:41.393 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:41.393 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:41.393 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172796 2024-05-11T23:03:41.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.393 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:41.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:41.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:41.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:41.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:41.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:41.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:41.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:41.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:41.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606069+0000", 2024-05-11T23:03:41.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:03:41.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.394 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:41.394 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606072+0000", 2024-05-11T23:03:41.394 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8700000000000001e-06 2024-05-11T23:03:41.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:41.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606074+0000", 2024-05-11T23:03:41.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6230000000000001e-06 2024-05-11T23:03:41.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663510+0000", 2024-05-11T23:03:41.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05743529 2024-05-11T23:03:41.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.395 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.395 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:41.395 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663517+0000", 2024-05-11T23:03:41.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1230000000000002e-06 2024-05-11T23:03:41.396 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.396 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.396 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.396 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172797 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:41.396 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:41.396 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.712542188999997, 2024-05-11T23:03:41.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057470420000000001, 2024-05-11T23:03:41.396 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:41.396 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:41.396 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:41.396 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:41.396 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:41.396 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:41.397 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172797 2024-05-11T23:03:41.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.397 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:41.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:41.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:41.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:41.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:41.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:41.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:41.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607915+0000", 2024-05-11T23:03:41.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018375609999999999 2024-05-11T23:03:41.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:41.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607920+0000", 2024-05-11T23:03:41.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8899999999999998e-06 2024-05-11T23:03:41.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:41.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607930+0000", 2024-05-11T23:03:41.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4429999999999992e-06 2024-05-11T23:03:41.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:41.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607936+0000", 2024-05-11T23:03:41.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7089999999999996e-06 2024-05-11T23:03:41.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:41.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663541+0000", 2024-05-11T23:03:41.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055604537000000002 2024-05-11T23:03:41.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:41.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:41.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:41.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663548+0000", 2024-05-11T23:03:41.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2799999999999998e-06 2024-05-11T23:03:41.399 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.399 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.400 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.400 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:41.400 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:41.400 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:41.401 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:03:41.402 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:41.401+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:41.497 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:03:41.541 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:41.541+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:41.551 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:41.551 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:41.551 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:41.551 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:41.561 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:03:41.602 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:41.601+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:41.626 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:41.626 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:41.644 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:41.644 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:41.661 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:41.662 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:41.662 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-05-11T23:03:41.680 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:41.680 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:41.698 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:41.698 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:41.702 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:41.702+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:41.713 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:41.713 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:41.716 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:41.716 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:41.722 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:03:41.733 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:41.734 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:41.737 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-11T23:03:41.737 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-11T23:03:41.803 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:41.804+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:41.844 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:41.844 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:41.846 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-05-11T23:03:41.856 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:03:41.905 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:41.904+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:41.976 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:41.976 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:41.985 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:03:42.026 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:42.027+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:42.107 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:42.107+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:42.132 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:42.132 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:42.141 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:03:42.225 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:42.226+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:42.286 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:42.286 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:42.295 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:03:42.311 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:42.312+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:42.415 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:42.415+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:42.439 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:42.439 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:42.448 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:03:42.516 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:42.516+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:42.552 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:42.552 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:42.561 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:03:42.619 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:42.620+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:42.665 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:42.666 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:42.674 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:03:42.720 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:42.720+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:42.822 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:42.822 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:42.824 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:42.824+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:42.828 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:03:42.926 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:42.926+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:43.136 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:43.136 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:43.136 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:03:43.137 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:43.136+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:43.237 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:43.238+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:43.243 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:43.243 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:43.252 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:03:43.338 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:43.338+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:43.358 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:43.358 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:43.367 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:03:43.439 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:43.439+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:43.472 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:43.472 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:43.481 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:03:43.540 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:43.541+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:43.588 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:43.588 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:43.597 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:03:43.640 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:43.641+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:43.719 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:43.720 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:43.729 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:03:43.738 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:03:43.741 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:43.742+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:43.843 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:43.843 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:43.843 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:43.844+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:43.850 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:03:43.862 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-11T23:03:43.918 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:43.941 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:43.941 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:43.943 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:43.944+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:43.964 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:43.964 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-05-11T23:03:43.987 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:43.987 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:43.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:43.987 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:43.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.987 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226051 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6049792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:43.987 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:43.987 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.669373872000001, 2024-05-11T23:03:43.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050622418000000002, 2024-05-11T23:03:43.988 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:43.988 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:43.988 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:43.988 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:43.988 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:43.988 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:43.988 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226051 2024-05-11T23:03:43.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.988 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:43.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:43.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:43.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:43.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:43.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:43.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:43.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:43.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317156+0000", 2024-05-11T23:03:43.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:43.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:43.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317157+0000", 2024-05-11T23:03:43.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.144e-06 2024-05-11T23:03:43.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:43.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317160+0000", 2024-05-11T23:03:43.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1350000000000001e-06 2024-05-11T23:03:43.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:43.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317162+0000", 2024-05-11T23:03:43.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5050000000000002e-06 2024-05-11T23:03:43.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:43.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367689+0000", 2024-05-11T23:03:43.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526797999999998 2024-05-11T23:03:43.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:43.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367735+0000", 2024-05-11T23:03:43.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5698999999999997e-05 2024-05-11T23:03:43.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:43.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367778+0000", 2024-05-11T23:03:43.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2928e-05 2024-05-11T23:03:43.991 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:43.991 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:43.991 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:43.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.992 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226052 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6060032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:43.992 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:43.992 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.669363816000001, 2024-05-11T23:03:43.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050830110999999997, 2024-05-11T23:03:43.992 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:43.992 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:43.992 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:43.992 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:43.992 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:43.992 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:43.992 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226052 2024-05-11T23:03:43.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.992 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:43.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:43.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:43.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:43.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:43.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:43.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:43.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:43.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317166+0000", 2024-05-11T23:03:43.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:03:43.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:43.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317167+0000", 2024-05-11T23:03:43.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.155e-06 2024-05-11T23:03:43.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:43.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317170+0000", 2024-05-11T23:03:43.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.061e-06 2024-05-11T23:03:43.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:43.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317172+0000", 2024-05-11T23:03:43.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4420000000000001e-06 2024-05-11T23:03:43.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:43.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367816+0000", 2024-05-11T23:03:43.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050644013000000002 2024-05-11T23:03:43.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367906+0000", 2024-05-11T23:03:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9739000000000003e-05 2024-05-11T23:03:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367995+0000", 2024-05-11T23:03:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9499000000000005e-05 2024-05-11T23:03:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226053 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6070272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.669353776999998, 2024-05-11T23:03:43.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051106912999999997, 2024-05-11T23:03:43.997 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:43.997 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:43.997 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:43.997 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:43.997 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:43.997 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:43.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226053 2024-05-11T23:03:43.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.997 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:43.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:43.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:43.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:43.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:43.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:43.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:43.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:43.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317176+0000", 2024-05-11T23:03:43.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0100000000000001e-07 2024-05-11T23:03:43.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:43.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317179+0000", 2024-05-11T23:03:43.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-05-11T23:03:43.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317182+0000", 2024-05-11T23:03:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3589999999999999e-06 2024-05-11T23:03:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317184+0000", 2024-05-11T23:03:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4339999999999999e-06 2024-05-11T23:03:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368061+0000", 2024-05-11T23:03:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050876927000000002 2024-05-11T23:03:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368179+0000", 2024-05-11T23:03:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118001 2024-05-11T23:03:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368282+0000", 2024-05-11T23:03:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102942 2024-05-11T23:03:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226054 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6080512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:44.000 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.669341455000001, 2024-05-11T23:03:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051322, 2024-05-11T23:03:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226054 2024-05-11T23:03:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317189+0000", 2024-05-11T23:03:44.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.217e-06 2024-05-11T23:03:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317192+0000", 2024-05-11T23:03:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0699999999999998e-06 2024-05-11T23:03:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317195+0000", 2024-05-11T23:03:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3829999999999999e-06 2024-05-11T23:03:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368328+0000", 2024-05-11T23:03:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051132999999999998 2024-05-11T23:03:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368429+0000", 2024-05-11T23:03:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010168800000000001 2024-05-11T23:03:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368510+0000", 2024-05-11T23:03:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0420999999999995e-05 2024-05-11T23:03:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226055 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6090752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.669331387, 2024-05-11T23:03:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051562835000000001, 2024-05-11T23:03:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226055 2024-05-11T23:03:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.005 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317256+0000", 2024-05-11T23:03:44.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8047000000000002e-05 2024-05-11T23:03:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317259+0000", 2024-05-11T23:03:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2590000000000001e-06 2024-05-11T23:03:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317267+0000", 2024-05-11T23:03:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8769999999999993e-06 2024-05-11T23:03:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.007 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317274+0000", 2024-05-11T23:03:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.187e-06 2024-05-11T23:03:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368544+0000", 2024-05-11T23:03:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051270299999999998 2024-05-11T23:03:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368658+0000", 2024-05-11T23:03:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000113347 2024-05-11T23:03:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368761+0000", 2024-05-11T23:03:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102818 2024-05-11T23:03:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226056 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6100992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.669245541000002, 2024-05-11T23:03:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051615576000000003, 2024-05-11T23:03:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:44.009 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226056 2024-05-11T23:03:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318915+0000", 2024-05-11T23:03:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001631291 2024-05-11T23:03:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318918+0000", 2024-05-11T23:03:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.512e-06 2024-05-11T23:03:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318927+0000", 2024-05-11T23:03:44.011 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1960000000000008e-06 2024-05-11T23:03:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318932+0000", 2024-05-11T23:03:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:03:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368805+0000", 2024-05-11T23:03:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049872998000000002 2024-05-11T23:03:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368853+0000", 2024-05-11T23:03:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8041999999999998e-05 2024-05-11T23:03:44.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368899+0000", 2024-05-11T23:03:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5766000000000002e-05 2024-05-11T23:03:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226061 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6152192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.667454454000001, 2024-05-11T23:03:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050675395999999998, 2024-05-11T23:03:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.013 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.014 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226061 2024-05-11T23:03:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319078+0000", 2024-05-11T23:03:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5670000000000002e-06 2024-05-11T23:03:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319085+0000", 2024-05-11T23:03:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2570000000000001e-06 2024-05-11T23:03:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319100+0000", 2024-05-11T23:03:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.489e-05 2024-05-11T23:03:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369584+0000", 2024-05-11T23:03:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050484345999999999 2024-05-11T23:03:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:44.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369671+0000", 2024-05-11T23:03:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6654000000000001e-05 2024-05-11T23:03:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369750+0000", 2024-05-11T23:03:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9257000000000004e-05 2024-05-11T23:03:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226062 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6162432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:44.018 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.667420909999997, 2024-05-11T23:03:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050761158000000001, 2024-05-11T23:03:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226062 2024-05-11T23:03:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319109+0000", 2024-05-11T23:03:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2800000000000002e-07 2024-05-11T23:03:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319111+0000", 2024-05-11T23:03:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4980000000000001e-06 2024-05-11T23:03:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319120+0000", 2024-05-11T23:03:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3999999999999992e-06 2024-05-11T23:03:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319129+0000", 2024-05-11T23:03:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7580000000000004e-06 2024-05-11T23:03:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369775+0000", 2024-05-11T23:03:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050645580000000003 2024-05-11T23:03:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369824+0000", 2024-05-11T23:03:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9506999999999999e-05 2024-05-11T23:03:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369869+0000", 2024-05-11T23:03:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4987000000000001e-05 2024-05-11T23:03:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226063 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6172672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:44.022 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.667391952999999, 2024-05-11T23:03:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050939784000000002, 2024-05-11T23:03:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226063 2024-05-11T23:03:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319138+0000", 2024-05-11T23:03:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.39e-07 2024-05-11T23:03:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319140+0000", 2024-05-11T23:03:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4820000000000001e-06 2024-05-11T23:03:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319148+0000", 2024-05-11T23:03:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6599999999999995e-06 2024-05-11T23:03:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319154+0000", 2024-05-11T23:03:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4169999999999997e-06 2024-05-11T23:03:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369905+0000", 2024-05-11T23:03:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050751009999999999 2024-05-11T23:03:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369991+0000", 2024-05-11T23:03:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5909999999999996e-05 2024-05-11T23:03:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370077+0000", 2024-05-11T23:03:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5865999999999994e-05 2024-05-11T23:03:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226064 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6182912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.667365756999999, 2024-05-11T23:03:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051038241999999998, 2024-05-11T23:03:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226064 2024-05-11T23:03:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319164+0000", 2024-05-11T23:03:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9299999999999999e-07 2024-05-11T23:03:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319166+0000", 2024-05-11T23:03:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.48e-06 2024-05-11T23:03:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319176+0000", 2024-05-11T23:03:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9669999999999996e-06 2024-05-11T23:03:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319180+0000", 2024-05-11T23:03:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8020000000000002e-06 2024-05-11T23:03:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370100+0000", 2024-05-11T23:03:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050919845999999998 2024-05-11T23:03:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370155+0000", 2024-05-11T23:03:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5040000000000002e-05 2024-05-11T23:03:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370202+0000", 2024-05-11T23:03:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6714e-05 2024-05-11T23:03:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.030 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226065 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6193152~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.667343187, 2024-05-11T23:03:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051225087000000002, 2024-05-11T23:03:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226065 2024-05-11T23:03:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320913+0000", 2024-05-11T23:03:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001727492 2024-05-11T23:03:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320919+0000", 2024-05-11T23:03:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.964e-06 2024-05-11T23:03:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320927+0000", 2024-05-11T23:03:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2239999999999998e-06 2024-05-11T23:03:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320934+0000", 2024-05-11T23:03:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8469999999999998e-06 2024-05-11T23:03:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370240+0000", 2024-05-11T23:03:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049306562999999998 2024-05-11T23:03:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370328+0000", 2024-05-11T23:03:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8122999999999996e-05 2024-05-11T23:03:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370411+0000", 2024-05-11T23:03:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2874000000000006e-05 2024-05-11T23:03:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.034 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226073 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6275072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.665504800000001, 2024-05-11T23:03:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050730475999999997, 2024-05-11T23:03:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226073 2024-05-11T23:03:44.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322914+0000", 2024-05-11T23:03:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001889153 2024-05-11T23:03:44.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322917+0000", 2024-05-11T23:03:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2100000000000002e-06 2024-05-11T23:03:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322924+0000", 2024-05-11T23:03:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9929999999999998e-06 2024-05-11T23:03:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322928+0000", 2024-05-11T23:03:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5549999999999996e-06 2024-05-11T23:03:44.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371578+0000", 2024-05-11T23:03:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048650001999999998 2024-05-11T23:03:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371671+0000", 2024-05-11T23:03:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2230000000000003e-05 2024-05-11T23:03:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371755+0000", 2024-05-11T23:03:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4332999999999994e-05 2024-05-11T23:03:44.038 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226085 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 5338112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.663413701000003, 2024-05-11T23:03:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050727622, 2024-05-11T23:03:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.039 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226085 2024-05-11T23:03:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324914+0000", 2024-05-11T23:03:44.041 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001798298 2024-05-11T23:03:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324918+0000", 2024-05-11T23:03:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3889999999999998e-06 2024-05-11T23:03:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324928+0000", 2024-05-11T23:03:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4339999999999998e-06 2024-05-11T23:03:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324936+0000", 2024-05-11T23:03:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2770000000000005e-06 2024-05-11T23:03:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373664+0000", 2024-05-11T23:03:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048728207000000003 2024-05-11T23:03:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373757+0000", 2024-05-11T23:03:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3332999999999996e-05 2024-05-11T23:03:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373843+0000", 2024-05-11T23:03:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5684000000000002e-05 2024-05-11T23:03:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232555 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7789568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:44.044 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.186403376000001, 2024-05-11T23:03:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050583086999999999, 2024-05-11T23:03:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232555 2024-05-11T23:03:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:03:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800127+0000", 2024-05-11T23:03:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.207e-06 2024-05-11T23:03:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800132+0000", 2024-05-11T23:03:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:03:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800134+0000", 2024-05-11T23:03:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5459999999999998e-06 2024-05-11T23:03:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850548+0000", 2024-05-11T23:03:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050414182000000002 2024-05-11T23:03:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850635+0000", 2024-05-11T23:03:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6853000000000003e-05 2024-05-11T23:03:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850709+0000", 2024-05-11T23:03:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3585999999999996e-05 2024-05-11T23:03:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.048 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232556 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7799808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.186391722000003, 2024-05-11T23:03:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050749599999999999, 2024-05-11T23:03:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232556 2024-05-11T23:03:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.049 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801914+0000", 2024-05-11T23:03:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017766360000000001 2024-05-11T23:03:44.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801919+0000", 2024-05-11T23:03:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.42e-06 2024-05-11T23:03:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801927+0000", 2024-05-11T23:03:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1999999999999994e-06 2024-05-11T23:03:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801931+0000", 2024-05-11T23:03:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4279999999999998e-06 2024-05-11T23:03:44.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850740+0000", 2024-05-11T23:03:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048808658999999997 2024-05-11T23:03:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850819+0000", 2024-05-11T23:03:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9102000000000003e-05 2024-05-11T23:03:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850887+0000", 2024-05-11T23:03:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8155000000000001e-05 2024-05-11T23:03:44.052 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232578 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8025088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.183098483999999, 2024-05-11T23:03:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050679473000000003, 2024-05-11T23:03:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.053 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232578 2024-05-11T23:03:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.054 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803432+0000", 2024-05-11T23:03:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.139e-06 2024-05-11T23:03:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803435+0000", 2024-05-11T23:03:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2440000000000002e-06 2024-05-11T23:03:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803439+0000", 2024-05-11T23:03:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3129999999999999e-06 2024-05-11T23:03:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.853953+0000", 2024-05-11T23:03:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050514846000000002 2024-05-11T23:03:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854035+0000", 2024-05-11T23:03:44.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1148000000000003e-05 2024-05-11T23:03:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854110+0000", 2024-05-11T23:03:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5562000000000006e-05 2024-05-11T23:03:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232579 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8035328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.18308588, 2024-05-11T23:03:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050886656000000002, 2024-05-11T23:03:44.057 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232579 2024-05-11T23:03:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803914+0000", 2024-05-11T23:03:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00047026700000000002 2024-05-11T23:03:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803918+0000", 2024-05-11T23:03:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3460000000000004e-06 2024-05-11T23:03:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803927+0000", 2024-05-11T23:03:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5030000000000003e-06 2024-05-11T23:03:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803933+0000", 2024-05-11T23:03:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0310000000000004e-06 2024-05-11T23:03:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854153+0000", 2024-05-11T23:03:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050219321999999997 2024-05-11T23:03:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:44.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854242+0000", 2024-05-11T23:03:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9457999999999996e-05 2024-05-11T23:03:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854330+0000", 2024-05-11T23:03:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7729e-05 2024-05-11T23:03:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232581 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8055808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.182562402999999, 2024-05-11T23:03:44.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050664439999999998, 2024-05-11T23:03:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232581 2024-05-11T23:03:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:03:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803969+0000", 2024-05-11T23:03:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0889999999999998e-06 2024-05-11T23:03:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803978+0000", 2024-05-11T23:03:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3070000000000003e-06 2024-05-11T23:03:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803983+0000", 2024-05-11T23:03:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1599999999999997e-06 2024-05-11T23:03:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854509+0000", 2024-05-11T23:03:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526339000000003 2024-05-11T23:03:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854576+0000", 2024-05-11T23:03:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7102999999999998e-05 2024-05-11T23:03:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854631+0000", 2024-05-11T23:03:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4997999999999998e-05 2024-05-11T23:03:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232582 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8066048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.182539658000003, 2024-05-11T23:03:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050753983000000003, 2024-05-11T23:03:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232582 2024-05-11T23:03:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805917+0000", 2024-05-11T23:03:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001927432 2024-05-11T23:03:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805920+0000", 2024-05-11T23:03:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1219999999999999e-06 2024-05-11T23:03:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805933+0000", 2024-05-11T23:03:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2727e-05 2024-05-11T23:03:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805941+0000", 2024-05-11T23:03:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7090000000000002e-06 2024-05-11T23:03:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854657+0000", 2024-05-11T23:03:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048716164999999999 2024-05-11T23:03:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854699+0000", 2024-05-11T23:03:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2531999999999999e-05 2024-05-11T23:03:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854744+0000", 2024-05-11T23:03:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4295999999999997e-05 2024-05-11T23:03:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232593 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8178688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:44.069 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.180340264999998, 2024-05-11T23:03:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050735178999999998, 2024-05-11T23:03:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232593 2024-05-11T23:03:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807913+0000", 2024-05-11T23:03:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017240319999999999 2024-05-11T23:03:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807916+0000", 2024-05-11T23:03:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4589999999999998e-06 2024-05-11T23:03:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807924+0000", 2024-05-11T23:03:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.272e-06 2024-05-11T23:03:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807928+0000", 2024-05-11T23:03:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5369999999999999e-06 2024-05-11T23:03:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856768+0000", 2024-05-11T23:03:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048839489999999999 2024-05-11T23:03:44.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856845+0000", 2024-05-11T23:03:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7746999999999999e-05 2024-05-11T23:03:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856924+0000", 2024-05-11T23:03:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8641999999999997e-05 2024-05-11T23:03:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.073 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.074 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.074 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:44.074 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:03:44.075 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:44.075 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:44.075 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:44.076 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:44.076 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:44.077 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-11T23:03:44.077 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:44.077 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-11T23:03:44.078 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:44.077+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:44.178 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:44.178+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:44.226 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:44.226 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:44.226 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:44.227 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:44.235 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:03:44.280 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:44.280+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:44.380 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:44.381+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:44.384 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:44.384 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:44.384 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:44.384 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:44.384 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:44.393 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:03:44.395 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 5, 7, 3, 6] out_osds: [4] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-05-11T23:03:44.395 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:03:44.395 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-05-11T23:03:44.395 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 4 2024-05-11T23:03:44.481 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:44.482+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:44.506 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:44.507 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:44.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:44.507 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:44.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.507 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:64049 3.8 3:11d98727:::rbd_object_map.111e8974a681:head [read 0~38] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-05-11T23:03:44.507 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:44.507 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 103.726173265, 2024-05-11T23:03:44.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99884604399999999, 2024-05-11T23:03:44.507 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.507 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.507 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:44.507 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.507 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.508 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.508 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 64049 2024-05-11T23:03:44.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.508 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:44.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:44.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779883+0000", 2024-05-11T23:03:44.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.57e-06 2024-05-11T23:03:44.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779887+0000", 2024-05-11T23:03:44.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5709999999999999e-06 2024-05-11T23:03:44.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779898+0000", 2024-05-11T23:03:44.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.042e-05 2024-05-11T23:03:44.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779905+0000", 2024-05-11T23:03:44.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0110000000000003e-06 2024-05-11T23:03:44.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779962+0000", 2024-05-11T23:03:44.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7395999999999997e-05 2024-05-11T23:03:44.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:44.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779969+0000", 2024-05-11T23:03:44.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2459999999999997e-06 2024-05-11T23:03:44.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778379+0000", 2024-05-11T23:03:44.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99840986300000001 2024-05-11T23:03:44.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:44.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778564+0000", 2024-05-11T23:03:44.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018420900000000001 2024-05-11T23:03:44.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778727+0000", 2024-05-11T23:03:44.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016375800000000001 2024-05-11T23:03:44.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.512 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.512 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172773 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:44.512 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:44.512 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.902066847, 2024-05-11T23:03:44.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058721316000000003, 2024-05-11T23:03:44.512 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.512 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.512 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:44.512 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.512 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.512 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.513 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172773 2024-05-11T23:03:44.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.513 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:44.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:44.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:44.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3500000000000002e-07 2024-05-11T23:03:44.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603990+0000", 2024-05-11T23:03:44.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3549999999999999e-06 2024-05-11T23:03:44.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603997+0000", 2024-05-11T23:03:44.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3400000000000003e-06 2024-05-11T23:03:44.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604002+0000", 2024-05-11T23:03:44.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0509999999999996e-06 2024-05-11T23:03:44.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662697+0000", 2024-05-11T23:03:44.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058695487999999997 2024-05-11T23:03:44.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662709+0000", 2024-05-11T23:03:44.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1647e-05 2024-05-11T23:03:44.516 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.516 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.516 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.516 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172774 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:44.516 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:44.516 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.902045465, 2024-05-11T23:03:44.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058745040999999998, 2024-05-11T23:03:44.516 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.516 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.516 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:44.517 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.517 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.517 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.517 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172774 2024-05-11T23:03:44.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.517 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:44.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:44.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:44.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:03:44.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604012+0000", 2024-05-11T23:03:44.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2199999999999999e-06 2024-05-11T23:03:44.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604018+0000", 2024-05-11T23:03:44.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1770000000000003e-06 2024-05-11T23:03:44.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604023+0000", 2024-05-11T23:03:44.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0089999999999998e-06 2024-05-11T23:03:44.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662742+0000", 2024-05-11T23:03:44.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058719187999999999 2024-05-11T23:03:44.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662754+0000", 2024-05-11T23:03:44.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2095e-05 2024-05-11T23:03:44.520 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.520 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.520 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.520 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172775 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:44.520 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:44.520 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.902025252999998, 2024-05-11T23:03:44.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058764743000000001, 2024-05-11T23:03:44.520 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.520 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.520 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:44.520 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.521 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.521 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.521 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172775 2024-05-11T23:03:44.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.521 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:44.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:44.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604030+0000", 2024-05-11T23:03:44.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6699999999999999e-07 2024-05-11T23:03:44.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604032+0000", 2024-05-11T23:03:44.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9530000000000002e-06 2024-05-11T23:03:44.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604037+0000", 2024-05-11T23:03:44.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8590000000000001e-06 2024-05-11T23:03:44.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604042+0000", 2024-05-11T23:03:44.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8609999999999997e-06 2024-05-11T23:03:44.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662782+0000", 2024-05-11T23:03:44.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058739731000000003 2024-05-11T23:03:44.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662794+0000", 2024-05-11T23:03:44.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1972e-05 2024-05-11T23:03:44.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.524 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.524 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172776 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:44.524 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:44.525 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.902005963000001, 2024-05-11T23:03:44.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058778696999999998, 2024-05-11T23:03:44.525 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.525 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.525 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:44.525 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.525 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.525 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.525 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172776 2024-05-11T23:03:44.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.525 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:44.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:44.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:44.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2000000000000001e-07 2024-05-11T23:03:44.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604051+0000", 2024-05-11T23:03:44.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2879999999999999e-06 2024-05-11T23:03:44.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604057+0000", 2024-05-11T23:03:44.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:03:44.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604062+0000", 2024-05-11T23:03:44.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6800000000000001e-06 2024-05-11T23:03:44.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662820+0000", 2024-05-11T23:03:44.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058758444999999999 2024-05-11T23:03:44.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662827+0000", 2024-05-11T23:03:44.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1930000000000004e-06 2024-05-11T23:03:44.530 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.530 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.530 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.530 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172777 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:44.530 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:44.530 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.901986657000002, 2024-05-11T23:03:44.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058799918999999999, 2024-05-11T23:03:44.530 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.531 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:44.531 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.531 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.531 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172777 2024-05-11T23:03:44.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.531 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:44.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:44.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:44.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:03:44.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604075+0000", 2024-05-11T23:03:44.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.109e-06 2024-05-11T23:03:44.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604083+0000", 2024-05-11T23:03:44.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9170000000000006e-06 2024-05-11T23:03:44.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604089+0000", 2024-05-11T23:03:44.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3399999999999997e-06 2024-05-11T23:03:44.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662859+0000", 2024-05-11T23:03:44.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058770172000000002 2024-05-11T23:03:44.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662868+0000", 2024-05-11T23:03:44.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0289999999999994e-06 2024-05-11T23:03:44.534 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.534 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.534 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.534 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172778 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:44.534 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:44.534 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.901958567000001, 2024-05-11T23:03:44.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058845435000000001, 2024-05-11T23:03:44.534 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.534 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.534 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:44.535 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.535 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.535 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.535 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172778 2024-05-11T23:03:44.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.535 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:44.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:44.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:44.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5400000000000002e-07 2024-05-11T23:03:44.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604099+0000", 2024-05-11T23:03:44.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4650000000000001e-06 2024-05-11T23:03:44.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604105+0000", 2024-05-11T23:03:44.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.314e-06 2024-05-11T23:03:44.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604110+0000", 2024-05-11T23:03:44.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6310000000000002e-06 2024-05-11T23:03:44.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662931+0000", 2024-05-11T23:03:44.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058820960999999998 2024-05-11T23:03:44.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662941+0000", 2024-05-11T23:03:44.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.061e-05 2024-05-11T23:03:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172779 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.901938407000003, 2024-05-11T23:03:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058859163999999999, 2024-05-11T23:03:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.538 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172779 2024-05-11T23:03:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:03:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604119+0000", 2024-05-11T23:03:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2529999999999999e-06 2024-05-11T23:03:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604125+0000", 2024-05-11T23:03:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1670000000000004e-06 2024-05-11T23:03:44.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604130+0000", 2024-05-11T23:03:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.702e-06 2024-05-11T23:03:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662967+0000", 2024-05-11T23:03:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058837792999999999 2024-05-11T23:03:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662975+0000", 2024-05-11T23:03:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8490000000000006e-06 2024-05-11T23:03:44.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172780 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.901918256999998, 2024-05-11T23:03:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058872696000000002, 2024-05-11T23:03:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.542 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172780 2024-05-11T23:03:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604137+0000", 2024-05-11T23:03:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4400000000000001e-07 2024-05-11T23:03:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604139+0000", 2024-05-11T23:03:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3429999999999998e-06 2024-05-11T23:03:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604145+0000", 2024-05-11T23:03:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1940000000000003e-06 2024-05-11T23:03:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604150+0000", 2024-05-11T23:03:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:03:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663002+0000", 2024-05-11T23:03:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058851829000000001 2024-05-11T23:03:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663009+0000", 2024-05-11T23:03:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.199e-06 2024-05-11T23:03:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172781 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.901886417, 2024-05-11T23:03:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058875553999999997, 2024-05-11T23:03:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172781 2024-05-11T23:03:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.547 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604171+0000", 2024-05-11T23:03:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.21e-06 2024-05-11T23:03:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604176+0000", 2024-05-11T23:03:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2970000000000003e-06 2024-05-11T23:03:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.548 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604179+0000", 2024-05-11T23:03:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8459999999999999e-06 2024-05-11T23:03:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663033+0000", 2024-05-11T23:03:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058854565999999997 2024-05-11T23:03:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663044+0000", 2024-05-11T23:03:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0229e-05 2024-05-11T23:03:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.549 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172782 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.901871669000002, 2024-05-11T23:03:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058894375999999998, 2024-05-11T23:03:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172782 2024-05-11T23:03:44.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.551 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-05-11T23:03:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604184+0000", 2024-05-11T23:03:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:03:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604187+0000", 2024-05-11T23:03:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.044e-06 2024-05-11T23:03:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.552 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604190+0000", 2024-05-11T23:03:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6460000000000002e-06 2024-05-11T23:03:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663070+0000", 2024-05-11T23:03:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058879510000000003 2024-05-11T23:03:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663077+0000", 2024-05-11T23:03:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6669999999999996e-06 2024-05-11T23:03:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.553 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172783 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.901861349999997, 2024-05-11T23:03:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058913946000000002, 2024-05-11T23:03:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.554 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172783 2024-05-11T23:03:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-05-11T23:03:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604195+0000", 2024-05-11T23:03:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.19e-06 2024-05-11T23:03:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604198+0000", 2024-05-11T23:03:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:03:44.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604200+0000", 2024-05-11T23:03:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4719999999999998e-06 2024-05-11T23:03:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663100+0000", 2024-05-11T23:03:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058900300000000003 2024-05-11T23:03:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.557 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663107+0000", 2024-05-11T23:03:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7449999999999998e-06 2024-05-11T23:03:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172784 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.901851331000003, 2024-05-11T23:03:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933728999999997, 2024-05-11T23:03:44.558 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172784 2024-05-11T23:03:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.559 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.03e-07 2024-05-11T23:03:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604205+0000", 2024-05-11T23:03:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2300000000000001e-06 2024-05-11T23:03:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.560 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604207+0000", 2024-05-11T23:03:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8729999999999999e-06 2024-05-11T23:03:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604210+0000", 2024-05-11T23:03:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.858e-06 2024-05-11T23:03:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663130+0000", 2024-05-11T23:03:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058919314 2024-05-11T23:03:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.561 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663137+0000", 2024-05-11T23:03:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2509999999999997e-06 2024-05-11T23:03:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172785 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.901839060999997, 2024-05-11T23:03:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058951664000000001, 2024-05-11T23:03:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.562 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172785 2024-05-11T23:03:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604216+0000", 2024-05-11T23:03:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:03:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604218+0000", 2024-05-11T23:03:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7549999999999999e-06 2024-05-11T23:03:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604222+0000", 2024-05-11T23:03:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6700000000000002e-06 2024-05-11T23:03:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604227+0000", 2024-05-11T23:03:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4399999999999998e-06 2024-05-11T23:03:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663160+0000", 2024-05-11T23:03:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933119999999999 2024-05-11T23:03:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.565 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663167+0000", 2024-05-11T23:03:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1840000000000002e-06 2024-05-11T23:03:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172786 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.901822007, 2024-05-11T23:03:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058966991000000003, 2024-05-11T23:03:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:44.566 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172786 2024-05-11T23:03:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604233+0000", 2024-05-11T23:03:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6899999999999999e-07 2024-05-11T23:03:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604235+0000", 2024-05-11T23:03:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8369999999999999e-06 2024-05-11T23:03:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604240+0000", 2024-05-11T23:03:44.568 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0329999999999999e-06 2024-05-11T23:03:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604244+0000", 2024-05-11T23:03:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9859999999999996e-06 2024-05-11T23:03:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663192+0000", 2024-05-11T23:03:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058948852000000003 2024-05-11T23:03:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.569 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663199+0000", 2024-05-11T23:03:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0140000000000001e-06 2024-05-11T23:03:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172787 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.901805660999997, 2024-05-11T23:03:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058982018999999997, 2024-05-11T23:03:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.570 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172787 2024-05-11T23:03:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605916+0000", 2024-05-11T23:03:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00166752 2024-05-11T23:03:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605921+0000", 2024-05-11T23:03:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0669999999999999e-06 2024-05-11T23:03:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605934+0000", 2024-05-11T23:03:44.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2271e-05 2024-05-11T23:03:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605940+0000", 2024-05-11T23:03:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6880000000000002e-06 2024-05-11T23:03:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663224+0000", 2024-05-11T23:03:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057283251 2024-05-11T23:03:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663231+0000", 2024-05-11T23:03:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2219999999999996e-06 2024-05-11T23:03:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172793 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.900038813999998, 2024-05-11T23:03:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057402373999999999, 2024-05-11T23:03:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.574 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172793 2024-05-11T23:03:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3300000000000001e-07 2024-05-11T23:03:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606017+0000", 2024-05-11T23:03:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4920000000000001e-06 2024-05-11T23:03:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606020+0000", 2024-05-11T23:03:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:03:44.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606023+0000", 2024-05-11T23:03:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6680000000000001e-06 2024-05-11T23:03:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663411+0000", 2024-05-11T23:03:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057388206999999997 2024-05-11T23:03:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663418+0000", 2024-05-11T23:03:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.815e-06 2024-05-11T23:03:44.577 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172795 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.899997386999999, 2024-05-11T23:03:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057423015000000001, 2024-05-11T23:03:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.578 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172795 2024-05-11T23:03:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9499999999999998e-07 2024-05-11T23:03:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606059+0000", 2024-05-11T23:03:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-05-11T23:03:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606062+0000", 2024-05-11T23:03:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.977e-06 2024-05-11T23:03:44.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606064+0000", 2024-05-11T23:03:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7410000000000001e-06 2024-05-11T23:03:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663473+0000", 2024-05-11T23:03:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057408624999999998 2024-05-11T23:03:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663480+0000", 2024-05-11T23:03:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1049999999999997e-06 2024-05-11T23:03:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172796 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.899986824000003, 2024-05-11T23:03:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057449235000000001, 2024-05-11T23:03:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.582 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172796 2024-05-11T23:03:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:44.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606069+0000", 2024-05-11T23:03:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:03:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606072+0000", 2024-05-11T23:03:44.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8700000000000001e-06 2024-05-11T23:03:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606074+0000", 2024-05-11T23:03:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6230000000000001e-06 2024-05-11T23:03:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663510+0000", 2024-05-11T23:03:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05743529 2024-05-11T23:03:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663517+0000", 2024-05-11T23:03:44.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1230000000000002e-06 2024-05-11T23:03:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172797 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.899976791999997, 2024-05-11T23:03:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057470420000000001, 2024-05-11T23:03:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:44.586 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172797 2024-05-11T23:03:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:44.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607915+0000", 2024-05-11T23:03:44.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018375609999999999 2024-05-11T23:03:44.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607920+0000", 2024-05-11T23:03:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8899999999999998e-06 2024-05-11T23:03:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607930+0000", 2024-05-11T23:03:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4429999999999992e-06 2024-05-11T23:03:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:44.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607936+0000", 2024-05-11T23:03:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7089999999999996e-06 2024-05-11T23:03:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663541+0000", 2024-05-11T23:03:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055604537000000002 2024-05-11T23:03:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663548+0000", 2024-05-11T23:03:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2799999999999998e-06 2024-05-11T23:03:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:44.591 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:44.591 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:44.592 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:03:44.593 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:44.592+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:44.693 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:44.693+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:44.705 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-11T23:03:44.705+0000 7fd16247e640 -1 mon.a@0(leader).osd e141 definitely_dead 0 2024-05-11T23:03:44.742 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:44.742 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:44.742 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:44.742 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:44.751 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:03:44.794 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:44.794+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:44.855 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-05-11T23:03:44.855 INFO:teuthology.orchestra.run.smithi005.stderr:marked in osd.4. 2024-05-11T23:03:44.878 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-05-11T23:03:44.894 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:44.894+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:44.907 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:44.907 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:44.908 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:44.908 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:44.908 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:44.916 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:03:45.001 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:45.001+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:45.068 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:45.068 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:45.068 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:45.068 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:45.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.069 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:77128 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_get in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-11T23:03:45.069 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:45.069 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 90.051590748999999, 2024-05-11T23:03:45.069 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91737855899999998, 2024-05-11T23:03:45.069 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:45.069 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:45.069 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:45.069 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:45.069 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:45.069 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.069 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 77128 2024-05-11T23:03:45.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.069 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:45.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:45.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:45.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:45.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:45.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.070 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:45.070 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016003+0000", 2024-05-11T23:03:45.070 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5379999999999999e-06 2024-05-11T23:03:45.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.070 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:45.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016011+0000", 2024-05-11T23:03:45.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6410000000000002e-06 2024-05-11T23:03:45.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:45.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016026+0000", 2024-05-11T23:03:45.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4779e-05 2024-05-11T23:03:45.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.071 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.071 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016032+0000", 2024-05-11T23:03:45.071 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.066e-06 2024-05-11T23:03:45.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016093+0000", 2024-05-11T23:03:45.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0878000000000002e-05 2024-05-11T23:03:45.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:45.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016100+0000", 2024-05-11T23:03:45.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6799999999999993e-06 2024-05-11T23:03:45.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.072 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.072 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.072 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933302+0000", 2024-05-11T23:03:45.072 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91720135300000005 2024-05-11T23:03:45.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933380+0000", 2024-05-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8623999999999998e-05 2024-05-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98950 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.935509859999996, 2024-05-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021419438999999998, 2024-05-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:45.073 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:45.074 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:45.074 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:45.074 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:45.074 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.074 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98950 2024-05-11T23:03:45.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.074 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:45.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.074 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:45.074 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:45.074 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.074 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:45.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:45.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:45.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:45.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.34e-07 2024-05-11T23:03:45.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.075 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:45.075 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132085+0000", 2024-05-11T23:03:45.075 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0049999999999999e-06 2024-05-11T23:03:45.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:45.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132093+0000", 2024-05-11T23:03:45.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6660000000000008e-06 2024-05-11T23:03:45.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132097+0000", 2024-05-11T23:03:45.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2590000000000002e-06 2024-05-11T23:03:45.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.076 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.076 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.076 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135114+0000", 2024-05-11T23:03:45.076 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003017573 2024-05-11T23:03:45.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:45.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153416+0000", 2024-05-11T23:03:45.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018301884000000001 2024-05-11T23:03:45.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.077 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:45.077 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153502+0000", 2024-05-11T23:03:45.077 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5717999999999995e-05 2024-05-11T23:03:45.077 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.077 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.077 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.078 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98959 3.1e 3:7807da99:::rbd_data.111ed2934718.000000000000006d:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:45.078 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:45.078 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.935017122999994, 2024-05-11T23:03:45.078 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020525928999999998, 2024-05-11T23:03:45.078 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:45.078 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:45.078 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:45.078 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:45.078 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:45.078 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.078 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98959 2024-05-11T23:03:45.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.078 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:45.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.078 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:45.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:45.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:45.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:45.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:45.079 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132576+0000", 2024-05-11T23:03:45.079 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4199999999999995e-07 2024-05-11T23:03:45.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.079 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:45.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132580+0000", 2024-05-11T23:03:45.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2910000000000001e-06 2024-05-11T23:03:45.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:45.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132588+0000", 2024-05-11T23:03:45.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7970000000000001e-06 2024-05-11T23:03:45.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.080 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.080 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132602+0000", 2024-05-11T23:03:45.080 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3743999999999999e-05 2024-05-11T23:03:45.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132655+0000", 2024-05-11T23:03:45.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2714999999999999e-05 2024-05-11T23:03:45.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:45.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132787+0000", 2024-05-11T23:03:45.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013273899999999999 2024-05-11T23:03:45.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.081 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:45.081 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153101+0000", 2024-05-11T23:03:45.081 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020313900999999999 2024-05-11T23:03:45.081 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.081 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.081 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.082 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98965 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:45.082 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:45.082 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.932493487000002, 2024-05-11T23:03:45.082 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018608983999999999, 2024-05-11T23:03:45.082 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:45.082 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:45.082 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:45.082 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:45.082 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:45.082 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.082 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98965 2024-05-11T23:03:45.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.083 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:45.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:45.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:45.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:45.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:45.083 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.083 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:45.083 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135100+0000", 2024-05-11T23:03:45.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.17e-06 2024-05-11T23:03:45.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:45.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135105+0000", 2024-05-11T23:03:45.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.899e-06 2024-05-11T23:03:45.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:45.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135122+0000", 2024-05-11T23:03:45.084 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7346999999999998e-05 2024-05-11T23:03:45.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.084 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.084 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135130+0000", 2024-05-11T23:03:45.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0639999999999994e-06 2024-05-11T23:03:45.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153533+0000", 2024-05-11T23:03:45.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018402753000000001 2024-05-11T23:03:45.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:45.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153615+0000", 2024-05-11T23:03:45.085 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1720999999999999e-05 2024-05-11T23:03:45.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.085 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:45.085 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153708+0000", 2024-05-11T23:03:45.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3029999999999995e-05 2024-05-11T23:03:45.086 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.086 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.086 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.086 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98966 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3760128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:45.086 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:45.086 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.932421360999996, 2024-05-11T23:03:45.086 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018716535999999999, 2024-05-11T23:03:45.086 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:45.086 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:45.086 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:45.086 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:45.086 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:45.087 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.087 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98966 2024-05-11T23:03:45.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.087 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:45.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:45.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:45.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.087 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:45.087 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:45.087 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:45.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135172+0000", 2024-05-11T23:03:45.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-05-11T23:03:45.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:45.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135175+0000", 2024-05-11T23:03:45.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-05-11T23:03:45.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:45.088 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135184+0000", 2024-05-11T23:03:45.088 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1840000000000005e-06 2024-05-11T23:03:45.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.088 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152903+0000", 2024-05-11T23:03:45.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017718357000000001 2024-05-11T23:03:45.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153736+0000", 2024-05-11T23:03:45.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00083324799999999997 2024-05-11T23:03:45.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:45.089 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153809+0000", 2024-05-11T23:03:45.089 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2995000000000002e-05 2024-05-11T23:03:45.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.089 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:45.090 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153888+0000", 2024-05-11T23:03:45.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8695999999999993e-05 2024-05-11T23:03:45.090 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.090 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.090 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.090 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "pg_update_log_missing_reply(3.1e epoch 72/53 rep_tid 104 lcod 72'363)", 2024-05-11T23:03:45.090 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:45.090 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.932358464999993, 2024-05-11T23:03:45.090 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017871951000000001, 2024-05-11T23:03:45.090 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:45.090 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:45.090 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:45.091 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:45.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:45.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:45.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:45.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:45.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.091 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:45.091 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135235+0000", 2024-05-11T23:03:45.091 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8899999999999999e-07 2024-05-11T23:03:45.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:45.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135238+0000", 2024-05-11T23:03:45.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.27e-06 2024-05-11T23:03:45.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:45.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152878+0000", 2024-05-11T23:03:45.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01763986 2024-05-11T23:03:45.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.092 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.092 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152896+0000", 2024-05-11T23:03:45.092 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8573e-05 2024-05-11T23:03:45.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152968+0000", 2024-05-11T23:03:45.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1780999999999996e-05 2024-05-11T23:03:45.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.093 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:45.093 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153106+0000", 2024-05-11T23:03:45.093 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013777799999999999 2024-05-11T23:03:45.093 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.093 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.093 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.093 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:239544 3.1a 3:5fe59cad:::rbd_data.111ebabb4604.000000000000016e:head [stat,delete] snapc 2f=[2f] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:45.093 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:45.094 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 41.413001733000002, 2024-05-11T23:03:45.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.50514007500000002, 2024-05-11T23:03:45.094 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:45.094 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:45.094 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:45.094 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:45.094 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:45.094 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.094 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 239544 2024-05-11T23:03:45.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.094 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:45.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.094 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:45.094 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:45.094 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654592+0000", 2024-05-11T23:03:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.404e-06 2024-05-11T23:03:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:45.095 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654598+0000", 2024-05-11T23:03:45.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7660000000000004e-06 2024-05-11T23:03:45.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:45.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654611+0000", 2024-05-11T23:03:45.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2676e-05 2024-05-11T23:03:45.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654620+0000", 2024-05-11T23:03:45.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2990000000000002e-06 2024-05-11T23:03:45.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.096 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.096 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654698+0000", 2024-05-11T23:03:45.096 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7739999999999998e-05 2024-05-11T23:03:45.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:45.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654710+0000", 2024-05-11T23:03:45.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2648e-05 2024-05-11T23:03:45.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.157889+0000", 2024-05-11T23:03:45.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.50317850799999997 2024-05-11T23:03:45.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.097 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:45.097 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.158050+0000", 2024-05-11T23:03:45.097 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016084500000000001 2024-05-11T23:03:45.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.098 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:45.098 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.159731+0000", 2024-05-11T23:03:45.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001681189 2024-05-11T23:03:45.098 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.098 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.098 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.098 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262364 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2613248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:45.098 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:45.098 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.481128867999999, 2024-05-11T23:03:45.098 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016312508, 2024-05-11T23:03:45.098 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:45.099 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:45.099 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:45.099 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:45.099 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:45.099 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.099 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262364 2024-05-11T23:03:45.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.099 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:45.099 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.099 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:45.099 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:45.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:45.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:45.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.100 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:45.100 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:45.100 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.13e-07 2024-05-11T23:03:45.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:45.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586467+0000", 2024-05-11T23:03:45.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-05-11T23:03:45.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:45.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586474+0000", 2024-05-11T23:03:45.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7739999999999999e-06 2024-05-11T23:03:45.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.101 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.101 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586483+0000", 2024-05-11T23:03:45.101 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0340000000000002e-06 2024-05-11T23:03:45.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602629+0000", 2024-05-11T23:03:45.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016145845999999998 2024-05-11T23:03:45.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:45.102 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602703+0000", 2024-05-11T23:03:45.102 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4671000000000004e-05 2024-05-11T23:03:45.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.102 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602776+0000", 2024-05-11T23:03:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2793000000000006e-05 2024-05-11T23:03:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262417 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.479382569000002, 2024-05-11T23:03:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016403456, 2024-05-11T23:03:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:45.103 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262417 2024-05-11T23:03:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.104 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6199999999999999e-07 2024-05-11T23:03:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588212+0000", 2024-05-11T23:03:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.075e-06 2024-05-11T23:03:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588218+0000", 2024-05-11T23:03:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8150000000000002e-06 2024-05-11T23:03:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588223+0000", 2024-05-11T23:03:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7090000000000001e-06 2024-05-11T23:03:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604461+0000", 2024-05-11T23:03:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016237900999999999 2024-05-11T23:03:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604540+0000", 2024-05-11T23:03:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9376999999999996e-05 2024-05-11T23:03:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604613+0000", 2024-05-11T23:03:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3217000000000001e-05 2024-05-11T23:03:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262418 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2396160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.47936266, 2024-05-11T23:03:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016591180000000001, 2024-05-11T23:03:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262418 2024-05-11T23:03:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6100000000000001e-07 2024-05-11T23:03:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588232+0000", 2024-05-11T23:03:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0370000000000001e-06 2024-05-11T23:03:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588238+0000", 2024-05-11T23:03:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9340000000000003e-06 2024-05-11T23:03:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588243+0000", 2024-05-11T23:03:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6179999999999997e-06 2024-05-11T23:03:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604642+0000", 2024-05-11T23:03:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0163991 2024-05-11T23:03:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604712+0000", 2024-05-11T23:03:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9935e-05 2024-05-11T23:03:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604821+0000", 2024-05-11T23:03:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000109095 2024-05-11T23:03:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262419 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2433024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.479343342, 2024-05-11T23:03:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016799692000000001, 2024-05-11T23:03:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262419 2024-05-11T23:03:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588252+0000", 2024-05-11T23:03:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0990000000000001e-06 2024-05-11T23:03:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588257+0000", 2024-05-11T23:03:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8159999999999999e-06 2024-05-11T23:03:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588262+0000", 2024-05-11T23:03:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8069999999999999e-06 2024-05-11T23:03:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604868+0000", 2024-05-11T23:03:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016605880999999999 2024-05-11T23:03:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604966+0000", 2024-05-11T23:03:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7597999999999996e-05 2024-05-11T23:03:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605049+0000", 2024-05-11T23:03:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3141999999999998e-05 2024-05-11T23:03:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262420 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2469888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.479323231000002, 2024-05-11T23:03:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016972514000000001, 2024-05-11T23:03:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262420 2024-05-11T23:03:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588271+0000", 2024-05-11T23:03:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6199999999999999e-06 2024-05-11T23:03:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588273+0000", 2024-05-11T23:03:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3209999999999999e-06 2024-05-11T23:03:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588279+0000", 2024-05-11T23:03:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2519999999999996e-06 2024-05-11T23:03:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588285+0000", 2024-05-11T23:03:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1440000000000002e-06 2024-05-11T23:03:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605083+0000", 2024-05-11T23:03:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016798418999999998 2024-05-11T23:03:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605168+0000", 2024-05-11T23:03:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4851000000000005e-05 2024-05-11T23:03:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605242+0000", 2024-05-11T23:03:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3906999999999996e-05 2024-05-11T23:03:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262421 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2506752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.479301085000003, 2024-05-11T23:03:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017141427000000001, 2024-05-11T23:03:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262421 2024-05-11T23:03:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:45.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588292+0000", 2024-05-11T23:03:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588318+0000", 2024-05-11T23:03:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6376e-05 2024-05-11T23:03:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:45.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588326+0000", 2024-05-11T23:03:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6699999999999994e-06 2024-05-11T23:03:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588332+0000", 2024-05-11T23:03:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8540000000000002e-06 2024-05-11T23:03:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605274+0000", 2024-05-11T23:03:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016942024999999999 2024-05-11T23:03:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.122 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605360+0000", 2024-05-11T23:03:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6304999999999994e-05 2024-05-11T23:03:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605433+0000", 2024-05-11T23:03:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2772000000000001e-05 2024-05-11T23:03:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262422 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2543616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:45.123 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.479251861000002, 2024-05-11T23:03:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017283705, 2024-05-11T23:03:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262422 2024-05-11T23:03:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:03:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588343+0000", 2024-05-11T23:03:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.34e-06 2024-05-11T23:03:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588350+0000", 2024-05-11T23:03:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1569999999999997e-06 2024-05-11T23:03:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588354+0000", 2024-05-11T23:03:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.865e-06 2024-05-11T23:03:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605466+0000", 2024-05-11T23:03:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017111498999999999 2024-05-11T23:03:45.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605551+0000", 2024-05-11T23:03:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5210999999999994e-05 2024-05-11T23:03:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605624+0000", 2024-05-11T23:03:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3153999999999998e-05 2024-05-11T23:03:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.127 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262423 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2580480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.479231612, 2024-05-11T23:03:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017451194, 2024-05-11T23:03:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262423 2024-05-11T23:03:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:45.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588363+0000", 2024-05-11T23:03:45.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.855e-06 2024-05-11T23:03:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588369+0000", 2024-05-11T23:03:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1260000000000003e-06 2024-05-11T23:03:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588375+0000", 2024-05-11T23:03:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9630000000000003e-06 2024-05-11T23:03:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605656+0000", 2024-05-11T23:03:45.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017280865999999999 2024-05-11T23:03:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605737+0000", 2024-05-11T23:03:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0773999999999996e-05 2024-05-11T23:03:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605812+0000", 2024-05-11T23:03:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5203999999999994e-05 2024-05-11T23:03:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262424 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2617344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.479210094000003, 2024-05-11T23:03:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017617544999999998, 2024-05-11T23:03:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262424 2024-05-11T23:03:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:45.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588383+0000", 2024-05-11T23:03:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8299999999999997e-07 2024-05-11T23:03:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:45.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588385+0000", 2024-05-11T23:03:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2869999999999999e-06 2024-05-11T23:03:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588391+0000", 2024-05-11T23:03:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1689999999999997e-06 2024-05-11T23:03:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588396+0000", 2024-05-11T23:03:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5719999999999996e-06 2024-05-11T23:03:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605837+0000", 2024-05-11T23:03:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017440812999999999 2024-05-11T23:03:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605917+0000", 2024-05-11T23:03:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9924000000000003e-05 2024-05-11T23:03:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.606000+0000", 2024-05-11T23:03:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3296999999999999e-05 2024-05-11T23:03:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262896 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.408205457000001, 2024-05-11T23:03:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016295738000000001, 2024-05-11T23:03:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262896 2024-05-11T23:03:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:45.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659390+0000", 2024-05-11T23:03:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1440000000000001e-06 2024-05-11T23:03:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659397+0000", 2024-05-11T23:03:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0010000000000004e-06 2024-05-11T23:03:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659402+0000", 2024-05-11T23:03:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2179999999999998e-06 2024-05-11T23:03:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675487+0000", 2024-05-11T23:03:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016085706000000002 2024-05-11T23:03:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675580+0000", 2024-05-11T23:03:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2884e-05 2024-05-11T23:03:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675683+0000", 2024-05-11T23:03:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010236 2024-05-11T23:03:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262897 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.408170116000001, 2024-05-11T23:03:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016500932999999999, 2024-05-11T23:03:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262897 2024-05-11T23:03:45.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659423+0000", 2024-05-11T23:03:45.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9699999999999996e-07 2024-05-11T23:03:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659425+0000", 2024-05-11T23:03:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3199999999999998e-06 2024-05-11T23:03:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659433+0000", 2024-05-11T23:03:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3660000000000003e-06 2024-05-11T23:03:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659445+0000", 2024-05-11T23:03:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2102e-05 2024-05-11T23:03:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675727+0000", 2024-05-11T23:03:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016282331000000001 2024-05-11T23:03:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675815+0000", 2024-05-11T23:03:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7484999999999993e-05 2024-05-11T23:03:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:45.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675923+0000", 2024-05-11T23:03:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000108732 2024-05-11T23:03:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262898 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.408138997999998, 2024-05-11T23:03:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016685855999999999, 2024-05-11T23:03:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:45.144 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262898 2024-05-11T23:03:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659454+0000", 2024-05-11T23:03:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2200000000000004e-07 2024-05-11T23:03:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659457+0000", 2024-05-11T23:03:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.779e-06 2024-05-11T23:03:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659463+0000", 2024-05-11T23:03:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6660000000000002e-06 2024-05-11T23:03:45.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659472+0000", 2024-05-11T23:03:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2139999999999996e-06 2024-05-11T23:03:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675956+0000", 2024-05-11T23:03:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016484431000000001 2024-05-11T23:03:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676039+0000", 2024-05-11T23:03:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3358999999999999e-05 2024-05-11T23:03:45.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676139+0000", 2024-05-11T23:03:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9784999999999994e-05 2024-05-11T23:03:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262899 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 32.408113167000003, 2024-05-11T23:03:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017030087999999999, 2024-05-11T23:03:45.148 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262899 2024-05-11T23:03:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659480+0000", 2024-05-11T23:03:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7700000000000004e-07 2024-05-11T23:03:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659482+0000", 2024-05-11T23:03:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1390000000000002e-06 2024-05-11T23:03:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659489+0000", 2024-05-11T23:03:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8279999999999997e-06 2024-05-11T23:03:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659494+0000", 2024-05-11T23:03:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9019999999999998e-06 2024-05-11T23:03:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676167+0000", 2024-05-11T23:03:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016673621 2024-05-11T23:03:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:45.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676329+0000", 2024-05-11T23:03:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016206099999999999 2024-05-11T23:03:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676509+0000", 2024-05-11T23:03:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017996 2024-05-11T23:03:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:45.152 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:45.152 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:45.154 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:03:45.154 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:45.154+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:45.254 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:45.255+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:45.271 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:45.271 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:45.271 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:45.281 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:03:45.355 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:45.356+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:45.428 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:45.428 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:45.428 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:45.429 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:45.429 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:45.437 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:03:45.483 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:45.483+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:45.544 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:45.544 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:45.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:45.544 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116446 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8273920~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.167972141000007, 2024-05-11T23:03:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045535299000000001, 2024-05-11T23:03:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116446 2024-05-11T23:03:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375547+0000", 2024-05-11T23:03:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4000000000000001e-07 2024-05-11T23:03:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375549+0000", 2024-05-11T23:03:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.368e-06 2024-05-11T23:03:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375556+0000", 2024-05-11T23:03:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1840000000000004e-06 2024-05-11T23:03:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375560+0000", 2024-05-11T23:03:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.741e-06 2024-05-11T23:03:45.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420896+0000", 2024-05-11T23:03:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045335435 2024-05-11T23:03:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420987+0000", 2024-05-11T23:03:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1033999999999996e-05 2024-05-11T23:03:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421082+0000", 2024-05-11T23:03:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4896999999999996e-05 2024-05-11T23:03:45.548 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116447 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6316032~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.167950630999997, 2024-05-11T23:03:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045745830000000001, 2024-05-11T23:03:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.549 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116447 2024-05-11T23:03:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.58e-07 2024-05-11T23:03:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375570+0000", 2024-05-11T23:03:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:03:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375576+0000", 2024-05-11T23:03:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7939999999999999e-06 2024-05-11T23:03:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375581+0000", 2024-05-11T23:03:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9280000000000001e-06 2024-05-11T23:03:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421127+0000", 2024-05-11T23:03:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045545785999999998 2024-05-11T23:03:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421214+0000", 2024-05-11T23:03:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7039000000000005e-05 2024-05-11T23:03:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:45.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421314+0000", 2024-05-11T23:03:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9772999999999995e-05 2024-05-11T23:03:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116448 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8314880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.167931557000003, 2024-05-11T23:03:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045941753000000002, 2024-05-11T23:03:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.553 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116448 2024-05-11T23:03:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.77e-07 2024-05-11T23:03:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375589+0000", 2024-05-11T23:03:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.091e-06 2024-05-11T23:03:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375596+0000", 2024-05-11T23:03:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9179999999999997e-06 2024-05-11T23:03:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375601+0000", 2024-05-11T23:03:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0030000000000002e-06 2024-05-11T23:03:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421363+0000", 2024-05-11T23:03:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045761352999999998 2024-05-11T23:03:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421440+0000", 2024-05-11T23:03:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7585999999999999e-05 2024-05-11T23:03:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421529+0000", 2024-05-11T23:03:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8424999999999994e-05 2024-05-11T23:03:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.557 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116449 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6438912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.167911122000007, 2024-05-11T23:03:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046082208999999999, 2024-05-11T23:03:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:45.558 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116449 2024-05-11T23:03:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:45.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377225+0000", 2024-05-11T23:03:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016177030000000001 2024-05-11T23:03:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377229+0000", 2024-05-11T23:03:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8e-06 2024-05-11T23:03:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377238+0000", 2024-05-11T23:03:45.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2229999999999996e-06 2024-05-11T23:03:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377244+0000", 2024-05-11T23:03:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4060000000000002e-06 2024-05-11T23:03:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421520+0000", 2024-05-11T23:03:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044275222000000003 2024-05-11T23:03:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421595+0000", 2024-05-11T23:03:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4838000000000004e-05 2024-05-11T23:03:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421690+0000", 2024-05-11T23:03:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5017000000000002e-05 2024-05-11T23:03:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116458 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7094272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.166096905000003, 2024-05-11T23:03:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045706877999999999, 2024-05-11T23:03:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116458 2024-05-11T23:03:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:45.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9400000000000005e-07 2024-05-11T23:03:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377424+0000", 2024-05-11T23:03:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1859999999999999e-06 2024-05-11T23:03:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377431+0000", 2024-05-11T23:03:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4049999999999997e-06 2024-05-11T23:03:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377436+0000", 2024-05-11T23:03:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1399999999999999e-06 2024-05-11T23:03:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.422945+0000", 2024-05-11T23:03:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045509326000000003 2024-05-11T23:03:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423036+0000", 2024-05-11T23:03:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0649999999999994e-05 2024-05-11T23:03:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423128+0000", 2024-05-11T23:03:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2577000000000005e-05 2024-05-11T23:03:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116459 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7258112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.166076172999993, 2024-05-11T23:03:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045831945999999998, 2024-05-11T23:03:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:45.567 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116459 2024-05-11T23:03:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377443+0000", 2024-05-11T23:03:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999999e-07 2024-05-11T23:03:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377445+0000", 2024-05-11T23:03:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0209999999999998e-06 2024-05-11T23:03:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377451+0000", 2024-05-11T23:03:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.071e-06 2024-05-11T23:03:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377456+0000", 2024-05-11T23:03:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8289999999999998e-06 2024-05-11T23:03:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423155+0000", 2024-05-11T23:03:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045699447999999997 2024-05-11T23:03:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423200+0000", 2024-05-11T23:03:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5083000000000002e-05 2024-05-11T23:03:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423274+0000", 2024-05-11T23:03:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4046000000000002e-05 2024-05-11T23:03:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116460 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7299072~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.166057111000001, 2024-05-11T23:03:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046010987000000003, 2024-05-11T23:03:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.571 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116460 2024-05-11T23:03:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377462+0000", 2024-05-11T23:03:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:03:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377464+0000", 2024-05-11T23:03:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1600000000000001e-06 2024-05-11T23:03:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377469+0000", 2024-05-11T23:03:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5430000000000001e-06 2024-05-11T23:03:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377474+0000", 2024-05-11T23:03:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8509999999999998e-06 2024-05-11T23:03:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423312+0000", 2024-05-11T23:03:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045838080000000003 2024-05-11T23:03:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423401+0000", 2024-05-11T23:03:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8170000000000005e-05 2024-05-11T23:03:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423472+0000", 2024-05-11T23:03:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1847000000000005e-05 2024-05-11T23:03:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116461 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7421952~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:45.575 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.166038341000004, 2024-05-11T23:03:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046130749999999998, 2024-05-11T23:03:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116461 2024-05-11T23:03:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4700000000000002e-07 2024-05-11T23:03:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377483+0000", 2024-05-11T23:03:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0219999999999999e-06 2024-05-11T23:03:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377489+0000", 2024-05-11T23:03:45.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4420000000000003e-06 2024-05-11T23:03:45.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377494+0000", 2024-05-11T23:03:45.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9239999999999998e-06 2024-05-11T23:03:45.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423500+0000", 2024-05-11T23:03:45.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046006054999999997 2024-05-11T23:03:45.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:45.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423557+0000", 2024-05-11T23:03:45.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7169000000000001e-05 2024-05-11T23:03:45.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:45.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423611+0000", 2024-05-11T23:03:45.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3791e-05 2024-05-11T23:03:45.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.579 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.579 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.579 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116462 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7462912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:45.579 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:45.579 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.166018722999993, 2024-05-11T23:03:45.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046254666, 2024-05-11T23:03:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116462 2024-05-11T23:03:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377502+0000", 2024-05-11T23:03:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-05-11T23:03:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377508+0000", 2024-05-11T23:03:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4500000000000003e-06 2024-05-11T23:03:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377512+0000", 2024-05-11T23:03:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7700000000000001e-06 2024-05-11T23:03:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423649+0000", 2024-05-11T23:03:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046136859000000002 2024-05-11T23:03:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:45.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423704+0000", 2024-05-11T23:03:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4938000000000002e-05 2024-05-11T23:03:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423754+0000", 2024-05-11T23:03:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0145999999999997e-05 2024-05-11T23:03:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116463 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7585792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:45.583 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.166000296999997, 2024-05-11T23:03:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361763, 2024-05-11T23:03:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116463 2024-05-11T23:03:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:45.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377519+0000", 2024-05-11T23:03:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2600000000000002e-07 2024-05-11T23:03:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377521+0000", 2024-05-11T23:03:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1710000000000001e-06 2024-05-11T23:03:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377527+0000", 2024-05-11T23:03:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9649999999999996e-06 2024-05-11T23:03:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377532+0000", 2024-05-11T23:03:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.78e-06 2024-05-11T23:03:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423779+0000", 2024-05-11T23:03:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046247706 2024-05-11T23:03:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423829+0000", 2024-05-11T23:03:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9188999999999999e-05 2024-05-11T23:03:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423880+0000", 2024-05-11T23:03:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1425999999999999e-05 2024-05-11T23:03:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.587 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116464 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7749632~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.165981060999997, 2024-05-11T23:03:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046491719000000001, 2024-05-11T23:03:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116464 2024-05-11T23:03:45.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377538+0000", 2024-05-11T23:03:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4499999999999998e-07 2024-05-11T23:03:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377540+0000", 2024-05-11T23:03:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2340000000000001e-06 2024-05-11T23:03:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377546+0000", 2024-05-11T23:03:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8289999999999996e-06 2024-05-11T23:03:45.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377551+0000", 2024-05-11T23:03:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8559999999999998e-06 2024-05-11T23:03:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423913+0000", 2024-05-11T23:03:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361885999999998 2024-05-11T23:03:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423958+0000", 2024-05-11T23:03:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5513999999999998e-05 2024-05-11T23:03:45.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424029+0000", 2024-05-11T23:03:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1055000000000003e-05 2024-05-11T23:03:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116465 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7626752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.165961913000004, 2024-05-11T23:03:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046600057, 2024-05-11T23:03:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.592 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116465 2024-05-11T23:03:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3599999999999999e-07 2024-05-11T23:03:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377559+0000", 2024-05-11T23:03:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9640000000000002e-06 2024-05-11T23:03:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377572+0000", 2024-05-11T23:03:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3188e-05 2024-05-11T23:03:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377578+0000", 2024-05-11T23:03:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.942e-06 2024-05-11T23:03:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424058+0000", 2024-05-11T23:03:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046480021000000003 2024-05-11T23:03:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:45.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424103+0000", 2024-05-11T23:03:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5021999999999997e-05 2024-05-11T23:03:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424157+0000", 2024-05-11T23:03:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3483999999999998e-05 2024-05-11T23:03:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116466 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7790592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:45.596 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.165933331000005, 2024-05-11T23:03:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046743741999999998, 2024-05-11T23:03:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116466 2024-05-11T23:03:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377586+0000", 2024-05-11T23:03:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:03:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377588+0000", 2024-05-11T23:03:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3039999999999999e-06 2024-05-11T23:03:45.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377594+0000", 2024-05-11T23:03:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-05-11T23:03:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377600+0000", 2024-05-11T23:03:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7699999999999998e-06 2024-05-11T23:03:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424185+0000", 2024-05-11T23:03:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046585891999999997 2024-05-11T23:03:45.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424235+0000", 2024-05-11T23:03:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9356e-05 2024-05-11T23:03:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424329+0000", 2024-05-11T23:03:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4036999999999994e-05 2024-05-11T23:03:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116467 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7954432~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.165913044000007, 2024-05-11T23:03:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046930134999999998, 2024-05-11T23:03:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116467 2024-05-11T23:03:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379161+0000", 2024-05-11T23:03:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015555009999999999 2024-05-11T23:03:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379166+0000", 2024-05-11T23:03:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6689999999999997e-06 2024-05-11T23:03:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379176+0000", 2024-05-11T23:03:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0549e-05 2024-05-11T23:03:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379183+0000", 2024-05-11T23:03:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1199999999999996e-06 2024-05-11T23:03:45.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424359+0000", 2024-05-11T23:03:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045175949999999999 2024-05-11T23:03:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424433+0000", 2024-05-11T23:03:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3824000000000003e-05 2024-05-11T23:03:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424536+0000", 2024-05-11T23:03:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010252200000000001 2024-05-11T23:03:45.604 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116468 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7913472~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.164322824999999, 2024-05-11T23:03:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045525364999999998, 2024-05-11T23:03:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.605 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116468 2024-05-11T23:03:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3300000000000002e-07 2024-05-11T23:03:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379198+0000", 2024-05-11T23:03:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2670000000000001e-06 2024-05-11T23:03:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379206+0000", 2024-05-11T23:03:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7149999999999998e-06 2024-05-11T23:03:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379228+0000", 2024-05-11T23:03:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1429000000000001e-05 2024-05-11T23:03:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424574+0000", 2024-05-11T23:03:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045346209999999998 2024-05-11T23:03:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424627+0000", 2024-05-11T23:03:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3352e-05 2024-05-11T23:03:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:45.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424721+0000", 2024-05-11T23:03:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3858999999999997e-05 2024-05-11T23:03:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116469 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8077312~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.164273004999998, 2024-05-11T23:03:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045636687000000002, 2024-05-11T23:03:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.609 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116469 2024-05-11T23:03:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379246+0000", 2024-05-11T23:03:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.32e-07 2024-05-11T23:03:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379254+0000", 2024-05-11T23:03:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.807e-06 2024-05-11T23:03:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379261+0000", 2024-05-11T23:03:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.985e-06 2024-05-11T23:03:45.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379264+0000", 2024-05-11T23:03:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6480000000000001e-06 2024-05-11T23:03:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424744+0000", 2024-05-11T23:03:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045479638000000003 2024-05-11T23:03:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424797+0000", 2024-05-11T23:03:45.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2725000000000001e-05 2024-05-11T23:03:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424882+0000", 2024-05-11T23:03:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5451999999999994e-05 2024-05-11T23:03:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116470 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8118272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.164247793000001, 2024-05-11T23:03:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045755745, 2024-05-11T23:03:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.613 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116470 2024-05-11T23:03:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3999999999999997e-07 2024-05-11T23:03:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379273+0000", 2024-05-11T23:03:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-05-11T23:03:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379277+0000", 2024-05-11T23:03:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4020000000000004e-06 2024-05-11T23:03:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379280+0000", 2024-05-11T23:03:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-05-11T23:03:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424922+0000", 2024-05-11T23:03:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045642598999999999 2024-05-11T23:03:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.616 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424973+0000", 2024-05-11T23:03:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0674999999999998e-05 2024-05-11T23:03:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425026+0000", 2024-05-11T23:03:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3492000000000002e-05 2024-05-11T23:03:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116471 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8241152~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:45.617 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.164235009999999, 2024-05-11T23:03:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045867374000000002, 2024-05-11T23:03:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116471 2024-05-11T23:03:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379284+0000", 2024-05-11T23:03:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.53e-07 2024-05-11T23:03:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379285+0000", 2024-05-11T23:03:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.223e-06 2024-05-11T23:03:45.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379288+0000", 2024-05-11T23:03:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5379999999999998e-06 2024-05-11T23:03:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379291+0000", 2024-05-11T23:03:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6699999999999998e-06 2024-05-11T23:03:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425055+0000", 2024-05-11T23:03:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045763497 2024-05-11T23:03:45.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425099+0000", 2024-05-11T23:03:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3899000000000001e-05 2024-05-11T23:03:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425151+0000", 2024-05-11T23:03:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2293999999999998e-05 2024-05-11T23:03:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116472 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8282112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:45.621 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.164224011000002, 2024-05-11T23:03:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045973778999999999, 2024-05-11T23:03:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116472 2024-05-11T23:03:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381226+0000", 2024-05-11T23:03:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001931627 2024-05-11T23:03:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381231+0000", 2024-05-11T23:03:45.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8679999999999998e-06 2024-05-11T23:03:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381242+0000", 2024-05-11T23:03:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0702e-05 2024-05-11T23:03:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381249+0000", 2024-05-11T23:03:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9800000000000001e-06 2024-05-11T23:03:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425175+0000", 2024-05-11T23:03:45.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043926805999999999 2024-05-11T23:03:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425215+0000", 2024-05-11T23:03:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9737999999999998e-05 2024-05-11T23:03:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425268+0000", 2024-05-11T23:03:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3057999999999999e-05 2024-05-11T23:03:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116482 3.1b 3:db6e8843:::rbd_object_map.111e67f27da3.0000000000000028:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:45.626 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:45.626 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 70.162255467999998, 2024-05-11T23:03:45.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046526076, 2024-05-11T23:03:45.626 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.626 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.626 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:45.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.626 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.626 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116482 2024-05-11T23:03:45.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.626 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:45.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:45.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381264+0000", 2024-05-11T23:03:45.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5300000000000004e-07 2024-05-11T23:03:45.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381267+0000", 2024-05-11T23:03:45.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1999999999999999e-06 2024-05-11T23:03:45.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381274+0000", 2024-05-11T23:03:45.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7109999999999995e-06 2024-05-11T23:03:45.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381285+0000", 2024-05-11T23:03:45.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0856000000000001e-05 2024-05-11T23:03:45.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425292+0000", 2024-05-11T23:03:45.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044006376 2024-05-11T23:03:45.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:45.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425337+0000", 2024-05-11T23:03:45.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4907999999999998e-05 2024-05-11T23:03:45.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:03:45.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425507+0000", 2024-05-11T23:03:45.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017029999999999999 2024-05-11T23:03:45.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:45.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427668+0000", 2024-05-11T23:03:45.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021612430000000002 2024-05-11T23:03:45.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:45.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427683+0000", 2024-05-11T23:03:45.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5016e-05 2024-05-11T23:03:45.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:45.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427742+0000", 2024-05-11T23:03:45.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8811999999999999e-05 2024-05-11T23:03:45.630 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.630 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:45.630 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427789+0000", 2024-05-11T23:03:45.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7101000000000002e-05 2024-05-11T23:03:45.630 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.630 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.631 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.631 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.631 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.631 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:45.632 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:03:45.633 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:45.632+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:45.733 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:45.733+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:45.778 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:282741 3.1c 3.30a98c1c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e142)", 2024-05-11T23:03:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:44.877284+0000", 2024-05-11T23:03:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 0.90200024300000003, 2024-05-11T23:03:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.90202060299999998, 2024-05-11T23:03:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "waiting for peered", 2024-05-11T23:03:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 282741 2024-05-11T23:03:45.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:44.877284+0000", 2024-05-11T23:03:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:44.877284+0000", 2024-05-11T23:03:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:44.877285+0000", 2024-05-11T23:03:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.525e-06 2024-05-11T23:03:45.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:44.877291+0000", 2024-05-11T23:03:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6540000000000004e-06 2024-05-11T23:03:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:44.877304+0000", 2024-05-11T23:03:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3508e-05 2024-05-11T23:03:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:44.877313+0000", 2024-05-11T23:03:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6389999999999998e-06 2024-05-11T23:03:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:44.877375+0000", 2024-05-11T23:03:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2509999999999996e-05 2024-05-11T23:03:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:03:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:44.877384+0000", 2024-05-11T23:03:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7600000000000008e-06 2024-05-11T23:03:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: ], 2024-05-11T23:03:45.782 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 1 2024-05-11T23:03:45.783 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:45.787 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:03:45.834 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:45.834+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:45.933 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:45.933 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:45.933 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:45.933 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:45.933 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:45.934 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:45.935+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:45.942 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:03:46.035 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:46.036+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:46.079 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:03:46.089 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:46.089 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:46.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:46.089 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:46.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.090 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267820 3.16 3:6b963329:::rbd_data.111ec06c817a.000000000000000a:33 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:46.090 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:03:46.090 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.146246466999997, 2024-05-11T23:03:46.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050141218000000001, 2024-05-11T23:03:46.090 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:46.090 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:46.090 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:46.090 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:46.090 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:46.090 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:46.090 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267820 2024-05-11T23:03:46.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.090 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:46.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:46.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:03:46.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:46.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:03:46.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:46.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942754+0000", 2024-05-11T23:03:46.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8559000000000003e-05 2024-05-11T23:03:46.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:46.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942756+0000", 2024-05-11T23:03:46.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.604e-06 2024-05-11T23:03:46.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:46.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942763+0000", 2024-05-11T23:03:46.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7939999999999997e-06 2024-05-11T23:03:46.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:46.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942768+0000", 2024-05-11T23:03:46.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4410000000000001e-06 2024-05-11T23:03:46.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:46.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992601+0000", 2024-05-11T23:03:46.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049832754999999999 2024-05-11T23:03:46.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:46.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992687+0000", 2024-05-11T23:03:46.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5605000000000005e-05 2024-05-11T23:03:46.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:46.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992796+0000", 2024-05-11T23:03:46.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010946 2024-05-11T23:03:46.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.094 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.094 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267852 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:46.094 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:03:46.094 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.146125263000002, 2024-05-11T23:03:46.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050212600000000003, 2024-05-11T23:03:46.094 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:46.094 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:46.095 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:46.099 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:46.099 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:46.099 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:46.099 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267852 2024-05-11T23:03:46.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.099 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:46.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:46.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:03:46.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:46.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:03:46.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:46.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943228+0000", 2024-05-11T23:03:46.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00045180600000000002 2024-05-11T23:03:46.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:46.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943232+0000", 2024-05-11T23:03:46.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8399999999999997e-06 2024-05-11T23:03:46.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:46.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943240+0000", 2024-05-11T23:03:46.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8900000000000007e-06 2024-05-11T23:03:46.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:46.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943246+0000", 2024-05-11T23:03:46.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6239999999999996e-06 2024-05-11T23:03:46.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:46.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992828+0000", 2024-05-11T23:03:46.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049581247000000002 2024-05-11T23:03:46.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:46.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992905+0000", 2024-05-11T23:03:46.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7656000000000003e-05 2024-05-11T23:03:46.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:46.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992989+0000", 2024-05-11T23:03:46.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3536999999999997e-05 2024-05-11T23:03:46.103 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.103 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.103 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.103 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267854 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1134592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:46.103 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:03:46.103 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.145612122000003, 2024-05-11T23:03:46.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050077425000000002, 2024-05-11T23:03:46.103 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:46.103 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:46.103 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:46.103 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:46.103 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:46.103 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:46.103 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267854 2024-05-11T23:03:46.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.104 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:46.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:46.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:03:46.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:46.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:03:46.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:46.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943290+0000", 2024-05-11T23:03:46.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2300000000000002e-07 2024-05-11T23:03:46.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:46.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943291+0000", 2024-05-11T23:03:46.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6190000000000001e-06 2024-05-11T23:03:46.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:46.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943296+0000", 2024-05-11T23:03:46.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2889999999999999e-06 2024-05-11T23:03:46.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:46.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943299+0000", 2024-05-11T23:03:46.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1030000000000002e-06 2024-05-11T23:03:46.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:46.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993205+0000", 2024-05-11T23:03:46.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049906405000000001 2024-05-11T23:03:46.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:46.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993285+0000", 2024-05-11T23:03:46.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9752999999999993e-05 2024-05-11T23:03:46.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:46.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993367+0000", 2024-05-11T23:03:46.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1933000000000003e-05 2024-05-11T23:03:46.107 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.107 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.107 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.107 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267855 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1171456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:46.107 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:03:46.107 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.145595180000001, 2024-05-11T23:03:46.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050250346000000001, 2024-05-11T23:03:46.107 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:46.108 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:46.108 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:46.108 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:46.108 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:46.108 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:46.108 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267855 2024-05-11T23:03:46.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.108 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:46.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:46.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:03:46.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:46.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:03:46.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:46.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945226+0000", 2024-05-11T23:03:46.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001919235 2024-05-11T23:03:46.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:46.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945230+0000", 2024-05-11T23:03:46.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9580000000000001e-06 2024-05-11T23:03:46.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:46.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945238+0000", 2024-05-11T23:03:46.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1040000000000007e-06 2024-05-11T23:03:46.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:46.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945244+0000", 2024-05-11T23:03:46.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6919999999999996e-06 2024-05-11T23:03:46.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:46.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993399+0000", 2024-05-11T23:03:46.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048154651 2024-05-11T23:03:46.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:46.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993472+0000", 2024-05-11T23:03:46.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2780000000000005e-05 2024-05-11T23:03:46.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:46.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993557+0000", 2024-05-11T23:03:46.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4926000000000006e-05 2024-05-11T23:03:46.111 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.111 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.111 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.112 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267865 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1540096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:46.112 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:46.112 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.143443554000001, 2024-05-11T23:03:46.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050054409000000001, 2024-05-11T23:03:46.112 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:46.112 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:46.112 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:46.112 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:46.112 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:46.112 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:46.112 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267865 2024-05-11T23:03:46.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.112 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:46.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:46.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:46.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:46.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:46.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:46.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:46.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6600000000000002e-07 2024-05-11T23:03:46.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:46.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945461+0000", 2024-05-11T23:03:46.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2670000000000001e-06 2024-05-11T23:03:46.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:46.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945466+0000", 2024-05-11T23:03:46.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7200000000000003e-06 2024-05-11T23:03:46.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:46.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945471+0000", 2024-05-11T23:03:46.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9439999999999996e-06 2024-05-11T23:03:46.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:46.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995337+0000", 2024-05-11T23:03:46.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049865415000000003 2024-05-11T23:03:46.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:46.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995413+0000", 2024-05-11T23:03:46.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5989999999999996e-05 2024-05-11T23:03:46.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:46.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995512+0000", 2024-05-11T23:03:46.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9706999999999999e-05 2024-05-11T23:03:46.115 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.115 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.115 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.116 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268017 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:46.116 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947359+0000", 2024-05-11T23:03:46.116 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.141542669000003, 2024-05-11T23:03:46.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050017655000000001, 2024-05-11T23:03:46.116 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:46.116 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:46.116 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:46.116 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:46.116 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:46.116 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:46.116 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268017 2024-05-11T23:03:46.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.116 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:46.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947359+0000", 2024-05-11T23:03:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947359+0000", 2024-05-11T23:03:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947359+0000", 2024-05-11T23:03:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5000000000000003e-07 2024-05-11T23:03:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:46.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947411+0000", 2024-05-11T23:03:46.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1158e-05 2024-05-11T23:03:46.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:46.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947418+0000", 2024-05-11T23:03:46.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8269999999999998e-06 2024-05-11T23:03:46.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:46.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947424+0000", 2024-05-11T23:03:46.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6529999999999999e-06 2024-05-11T23:03:46.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:46.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997179+0000", 2024-05-11T23:03:46.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049754675999999998 2024-05-11T23:03:46.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:46.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997279+0000", 2024-05-11T23:03:46.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9789000000000003e-05 2024-05-11T23:03:46.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:46.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997377+0000", 2024-05-11T23:03:46.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8002000000000001e-05 2024-05-11T23:03:46.119 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.119 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.119 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.120 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268018 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 872448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:46.120 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:03:46.120 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.141467489, 2024-05-11T23:03:46.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050170927999999997, 2024-05-11T23:03:46.120 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:46.120 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:46.120 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:46.120 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:46.120 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:46.120 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:46.120 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268018 2024-05-11T23:03:46.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.121 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:46.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:46.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:03:46.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:46.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:03:46.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:46.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947435+0000", 2024-05-11T23:03:46.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3099999999999997e-07 2024-05-11T23:03:46.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:46.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947437+0000", 2024-05-11T23:03:46.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6340000000000001e-06 2024-05-11T23:03:46.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:46.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947443+0000", 2024-05-11T23:03:46.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7810000000000002e-06 2024-05-11T23:03:46.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:46.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947448+0000", 2024-05-11T23:03:46.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9679999999999997e-06 2024-05-11T23:03:46.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:46.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997418+0000", 2024-05-11T23:03:46.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049969587000000003 2024-05-11T23:03:46.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.123 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:46.123 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997504+0000", 2024-05-11T23:03:46.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6426000000000002e-05 2024-05-11T23:03:46.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:46.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997605+0000", 2024-05-11T23:03:46.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100901 2024-05-11T23:03:46.124 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.124 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.124 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.124 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268019 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:46.124 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:46.124 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.141446930999997, 2024-05-11T23:03:46.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050285074999999999, 2024-05-11T23:03:46.124 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:46.124 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:46.124 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:46.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:46.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:46.125 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:46.125 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268019 2024-05-11T23:03:46.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.125 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:46.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:46.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:46.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:46.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:46.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:46.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:46.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0999999999999999e-07 2024-05-11T23:03:46.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:46.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947457+0000", 2024-05-11T23:03:46.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-05-11T23:03:46.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:46.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947462+0000", 2024-05-11T23:03:46.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2410000000000003e-06 2024-05-11T23:03:46.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:46.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947467+0000", 2024-05-11T23:03:46.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8339999999999998e-06 2024-05-11T23:03:46.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:46.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997639+0000", 2024-05-11T23:03:46.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050171697000000001 2024-05-11T23:03:46.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:46.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997685+0000", 2024-05-11T23:03:46.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6146000000000002e-05 2024-05-11T23:03:46.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:46.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997740+0000", 2024-05-11T23:03:46.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4737000000000001e-05 2024-05-11T23:03:46.128 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.128 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.128 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.128 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268020 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:46.128 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:03:46.128 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.141428685999998, 2024-05-11T23:03:46.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050504089000000002, 2024-05-11T23:03:46.128 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:46.129 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:46.129 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:46.129 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:46.129 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:46.129 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:46.129 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268020 2024-05-11T23:03:46.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.129 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:46.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:46.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:03:46.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:46.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:03:46.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:46.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947591+0000", 2024-05-11T23:03:46.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011805 2024-05-11T23:03:46.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:46.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947594+0000", 2024-05-11T23:03:46.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1549999999999999e-06 2024-05-11T23:03:46.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:46.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947602+0000", 2024-05-11T23:03:46.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.174e-06 2024-05-11T23:03:46.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:46.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947608+0000", 2024-05-11T23:03:46.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.263e-06 2024-05-11T23:03:46.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:46.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997775+0000", 2024-05-11T23:03:46.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050166980999999999 2024-05-11T23:03:46.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.131 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:46.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997875+0000", 2024-05-11T23:03:46.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9844000000000002e-05 2024-05-11T23:03:46.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:46.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997977+0000", 2024-05-11T23:03:46.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101622 2024-05-11T23:03:46.132 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.132 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.132 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.132 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268021 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:46.132 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:03:46.132 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.141283225999999, 2024-05-11T23:03:46.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050498874999999999, 2024-05-11T23:03:46.132 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:46.133 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:46.133 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:46.133 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:46.133 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:46.133 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:46.133 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268021 2024-05-11T23:03:46.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.133 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:46.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:46.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:03:46.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:46.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:03:46.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:46.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947695+0000", 2024-05-11T23:03:46.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6247000000000004e-05 2024-05-11T23:03:46.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:46.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947698+0000", 2024-05-11T23:03:46.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9199999999999997e-06 2024-05-11T23:03:46.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:46.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947706+0000", 2024-05-11T23:03:46.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:03:46.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:46.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947711+0000", 2024-05-11T23:03:46.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6720000000000001e-06 2024-05-11T23:03:46.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:46.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998017+0000", 2024-05-11T23:03:46.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050305470999999997 2024-05-11T23:03:46.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:46.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998060+0000", 2024-05-11T23:03:46.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3374000000000002e-05 2024-05-11T23:03:46.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:46.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998117+0000", 2024-05-11T23:03:46.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6971000000000001e-05 2024-05-11T23:03:46.136 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.136 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.136 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.136 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268022 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:46.136 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:03:46.136 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.14118019, 2024-05-11T23:03:46.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050731361000000003, 2024-05-11T23:03:46.137 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:46.137 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:46.137 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:46.137 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:46.137 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:46.137 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:46.137 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268022 2024-05-11T23:03:46.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.137 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:46.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:46.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:03:46.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:46.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:03:46.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:46.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947722+0000", 2024-05-11T23:03:46.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8999999999999996e-07 2024-05-11T23:03:46.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:46.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947724+0000", 2024-05-11T23:03:46.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1639999999999999e-06 2024-05-11T23:03:46.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:46.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947730+0000", 2024-05-11T23:03:46.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6099999999999997e-06 2024-05-11T23:03:46.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:46.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947735+0000", 2024-05-11T23:03:46.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7189999999999998e-06 2024-05-11T23:03:46.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:46.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998140+0000", 2024-05-11T23:03:46.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050404254000000003 2024-05-11T23:03:46.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:46.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998239+0000", 2024-05-11T23:03:46.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9192999999999997e-05 2024-05-11T23:03:46.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:46.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998453+0000", 2024-05-11T23:03:46.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021383099999999999 2024-05-11T23:03:46.140 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.140 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.140 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.141 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268023 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1069056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:46.141 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:03:46.141 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.141157925000002, 2024-05-11T23:03:46.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050874509999999998, 2024-05-11T23:03:46.141 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:46.141 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:46.141 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:46.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:46.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:46.141 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:46.141 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268023 2024-05-11T23:03:46.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.141 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:46.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:46.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:03:46.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:46.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:03:46.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:46.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948934+0000", 2024-05-11T23:03:46.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011903339999999999 2024-05-11T23:03:46.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:46.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948938+0000", 2024-05-11T23:03:46.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1330000000000001e-06 2024-05-11T23:03:46.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:46.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948949+0000", 2024-05-11T23:03:46.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0872e-05 2024-05-11T23:03:46.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:46.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948956+0000", 2024-05-11T23:03:46.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.131e-06 2024-05-11T23:03:46.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:46.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998494+0000", 2024-05-11T23:03:46.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049537678000000002 2024-05-11T23:03:46.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:46.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998551+0000", 2024-05-11T23:03:46.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7256000000000002e-05 2024-05-11T23:03:46.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.144 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:46.144 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998618+0000", 2024-05-11T23:03:46.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7106000000000004e-05 2024-05-11T23:03:46.145 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.145 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.145 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.145 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268026 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1179648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:46.145 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:03:46.145 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.139880871999999, 2024-05-11T23:03:46.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050107977999999997, 2024-05-11T23:03:46.145 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:46.145 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:46.145 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:46.145 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:46.145 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:46.145 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:46.145 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268026 2024-05-11T23:03:46.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.146 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:46.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:46.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:03:46.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:46.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:03:46.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:46.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949029+0000", 2024-05-11T23:03:46.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2220000000000002e-06 2024-05-11T23:03:46.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:46.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949041+0000", 2024-05-11T23:03:46.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.241e-05 2024-05-11T23:03:46.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:46.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949049+0000", 2024-05-11T23:03:46.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.072e-06 2024-05-11T23:03:46.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:46.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949056+0000", 2024-05-11T23:03:46.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1380000000000001e-06 2024-05-11T23:03:46.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:46.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999016+0000", 2024-05-11T23:03:46.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049959212000000003 2024-05-11T23:03:46.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:46.148 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999067+0000", 2024-05-11T23:03:46.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1598000000000003e-05 2024-05-11T23:03:46.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.148 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999129+0000", 2024-05-11T23:03:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1326000000000002e-05 2024-05-11T23:03:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268027 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1216512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.139836721999998, 2024-05-11T23:03:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050189865, 2024-05-11T23:03:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:46.149 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268027 2024-05-11T23:03:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:46.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:46.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-05-11T23:03:46.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:46.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949068+0000", 2024-05-11T23:03:46.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.26e-06 2024-05-11T23:03:46.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:46.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949074+0000", 2024-05-11T23:03:46.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7050000000000002e-06 2024-05-11T23:03:46.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:46.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949080+0000", 2024-05-11T23:03:46.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3689999999999998e-06 2024-05-11T23:03:46.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:46.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999152+0000", 2024-05-11T23:03:46.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050072653000000002 2024-05-11T23:03:46.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:46.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999197+0000", 2024-05-11T23:03:46.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4972999999999998e-05 2024-05-11T23:03:46.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:46.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999255+0000", 2024-05-11T23:03:46.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7425e-05 2024-05-11T23:03:46.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.153 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.153 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.153 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268028 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1253376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:46.153 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:03:46.153 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.139815061999997, 2024-05-11T23:03:46.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050280017000000003, 2024-05-11T23:03:46.153 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:46.154 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:46.154 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:46.154 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:46.154 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:46.154 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:46.154 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268028 2024-05-11T23:03:46.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.154 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:46.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:46.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:03:46.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:46.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:03:46.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:46.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949087+0000", 2024-05-11T23:03:46.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:03:46.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:46.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949089+0000", 2024-05-11T23:03:46.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8950000000000001e-06 2024-05-11T23:03:46.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:46.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949095+0000", 2024-05-11T23:03:46.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7540000000000003e-06 2024-05-11T23:03:46.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:46.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949114+0000", 2024-05-11T23:03:46.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9738e-05 2024-05-11T23:03:46.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:46.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999278+0000", 2024-05-11T23:03:46.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050163368999999999 2024-05-11T23:03:46.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:46.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999319+0000", 2024-05-11T23:03:46.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0979000000000002e-05 2024-05-11T23:03:46.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.157 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:46.157 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999366+0000", 2024-05-11T23:03:46.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7879e-05 2024-05-11T23:03:46.157 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.157 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.157 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.157 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.157 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.157 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268029 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1290240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:46.158 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:46.158 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.139774954000004, 2024-05-11T23:03:46.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050344904000000003, 2024-05-11T23:03:46.158 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:46.158 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:46.158 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:46.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:46.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:46.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:46.158 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268029 2024-05-11T23:03:46.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.158 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:46.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:46.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:46.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:46.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:46.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:46.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:46.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:03:46.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:46.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949130+0000", 2024-05-11T23:03:46.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-05-11T23:03:46.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:46.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949139+0000", 2024-05-11T23:03:46.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9800000000000004e-06 2024-05-11T23:03:46.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:46.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949145+0000", 2024-05-11T23:03:46.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0510000000000002e-06 2024-05-11T23:03:46.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:46.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999387+0000", 2024-05-11T23:03:46.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050242167999999997 2024-05-11T23:03:46.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:46.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999427+0000", 2024-05-11T23:03:46.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9566e-05 2024-05-11T23:03:46.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:46.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999471+0000", 2024-05-11T23:03:46.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4914999999999999e-05 2024-05-11T23:03:46.162 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.162 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.162 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.162 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268030 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1327104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:46.162 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:46.162 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.13974967, 2024-05-11T23:03:46.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050447394, 2024-05-11T23:03:46.163 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:46.163 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:46.163 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:46.163 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:46.163 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:46.163 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:46.163 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268030 2024-05-11T23:03:46.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.163 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:46.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:46.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:46.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:46.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:46.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:46.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:46.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0500000000000004e-07 2024-05-11T23:03:46.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:46.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949155+0000", 2024-05-11T23:03:46.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3819999999999998e-06 2024-05-11T23:03:46.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:46.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949162+0000", 2024-05-11T23:03:46.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9609999999999999e-06 2024-05-11T23:03:46.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:46.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949188+0000", 2024-05-11T23:03:46.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5877e-05 2024-05-11T23:03:46.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:46.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999494+0000", 2024-05-11T23:03:46.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050306265000000003 2024-05-11T23:03:46.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:46.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999534+0000", 2024-05-11T23:03:46.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0447000000000001e-05 2024-05-11T23:03:46.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:46.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999599+0000", 2024-05-11T23:03:46.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4956999999999995e-05 2024-05-11T23:03:46.166 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.166 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.166 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.166 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268031 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1363968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:46.166 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:03:46.167 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.139704569999999, 2024-05-11T23:03:46.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050511525000000002, 2024-05-11T23:03:46.167 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:46.167 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:46.167 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:46.167 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:46.167 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:46.167 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:46.167 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268031 2024-05-11T23:03:46.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.167 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:46.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:46.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:03:46.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:46.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:03:46.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:46.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949198+0000", 2024-05-11T23:03:46.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-05-11T23:03:46.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:46.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949200+0000", 2024-05-11T23:03:46.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-05-11T23:03:46.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:46.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949208+0000", 2024-05-11T23:03:46.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2840000000000001e-06 2024-05-11T23:03:46.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:46.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949214+0000", 2024-05-11T23:03:46.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9869999999999996e-06 2024-05-11T23:03:46.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:46.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999619+0000", 2024-05-11T23:03:46.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050405278999999997 2024-05-11T23:03:46.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:46.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999660+0000", 2024-05-11T23:03:46.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1427999999999997e-05 2024-05-11T23:03:46.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:46.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999708+0000", 2024-05-11T23:03:46.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8229999999999997e-05 2024-05-11T23:03:46.170 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.170 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.170 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.171 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268032 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1400832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:46.171 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:03:46.171 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.13967495, 2024-05-11T23:03:46.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050611863999999999, 2024-05-11T23:03:46.171 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:46.171 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:46.171 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:46.171 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:46.171 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:46.171 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:46.171 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268032 2024-05-11T23:03:46.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.172 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:46.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:46.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:03:46.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:46.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:03:46.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:46.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949230+0000", 2024-05-11T23:03:46.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.371e-06 2024-05-11T23:03:46.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:46.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949233+0000", 2024-05-11T23:03:46.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6599999999999999e-06 2024-05-11T23:03:46.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:46.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949241+0000", 2024-05-11T23:03:46.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8939999999999993e-06 2024-05-11T23:03:46.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:46.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949246+0000", 2024-05-11T23:03:46.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6590000000000004e-06 2024-05-11T23:03:46.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:46.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999733+0000", 2024-05-11T23:03:46.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050487268000000002 2024-05-11T23:03:46.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:46.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999784+0000", 2024-05-11T23:03:46.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0562000000000001e-05 2024-05-11T23:03:46.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:46.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999838+0000", 2024-05-11T23:03:46.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000002e-05 2024-05-11T23:03:46.175 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.175 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.175 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.175 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268033 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1437696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:46.175 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:03:46.175 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 33.139647822999997, 2024-05-11T23:03:46.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050697157999999999, 2024-05-11T23:03:46.175 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:46.175 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:46.175 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:46.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:46.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:46.176 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:46.176 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268033 2024-05-11T23:03:46.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.176 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:46.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:46.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:03:46.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:46.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:03:46.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:46.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:46.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950862+0000", 2024-05-11T23:03:46.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001607975 2024-05-11T23:03:46.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:46.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950866+0000", 2024-05-11T23:03:46.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7770000000000002e-06 2024-05-11T23:03:46.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:46.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950878+0000", 2024-05-11T23:03:46.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1421e-05 2024-05-11T23:03:46.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:46.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950885+0000", 2024-05-11T23:03:46.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-05-11T23:03:46.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:46.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999859+0000", 2024-05-11T23:03:46.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048973199000000002 2024-05-11T23:03:46.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:46.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999901+0000", 2024-05-11T23:03:46.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1943000000000001e-05 2024-05-11T23:03:46.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:46.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:46.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:46.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999951+0000", 2024-05-11T23:03:46.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0243e-05 2024-05-11T23:03:46.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.179 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.179 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:46.179 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:46.179 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:46.180 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:03:46.181 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:46.180+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:46.218 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:46.218 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:46.236 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:46.253 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:46.253 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:46.253 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-05-11T23:03:46.270 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:46.271 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:46.281 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:46.281+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:46.288 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:46.288 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:46.306 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:46.306 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:46.323 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:46.323 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:46.327 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-11T23:03:46.327 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-11T23:03:46.328 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:46.328 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:46.337 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:03:46.382 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:46.382+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:46.442 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:46.443 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:46.451 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:03:46.495 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:46.496+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:46.556 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:46.557 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:46.565 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:03:46.584 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:46.584+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:46.685 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:46.686+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:46.711 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:46.711 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:46.720 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:03:46.804 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:46.805+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:46.879 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-11T23:03:46.886 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:46.887+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:46.902 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:46.902 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:46.913 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:03:46.996 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:46.997+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:47.069 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:47.069 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:47.079 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:03:47.123 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:47.123+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:47.189 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:47.189+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:47.189 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:47.189 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:47.199 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:03:47.290 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:47.290+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:47.350 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:47.350 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:47.359 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:03:47.390 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:47.391+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:47.492 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:47.492+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:47.509 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:47.509 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:47.518 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:03:47.593 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:47.594+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:47.668 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:47.669 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:47.678 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:03:47.723 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:47.723+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:47.787 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:47.787 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:47.796 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:03:47.797 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:47.796+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:47.878 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-05-11T23:03:47.897 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:47.897+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:47.957 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:47.958 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:47.967 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:03:48.012 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:48.012+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:48.081 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:48.082 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:48.090 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:03:48.131 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:48.131+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:48.200 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:48.201+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:48.243 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:48.243 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:48.253 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:03:48.302 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:48.303+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:48.328 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:03:48.374 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:48.375 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:48.385 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:03:48.430 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:48.431+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:48.470 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:48.470 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:48.489 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:48.489 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:48.495 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:48.495 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:48.504 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:48.504+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:48.506 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:03:48.507 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:48.507 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:48.508 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-05-11T23:03:48.524 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:48.524 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:48.545 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:48.545 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:48.563 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:48.563 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:48.580 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:48.584 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-11T23:03:48.584 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-11T23:03:48.604 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:48.605+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:48.657 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:48.657 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:48.657 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:48.657 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:48.657 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:48.666 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:03:48.707 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:48.708+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:48.806 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:48.806+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:48.828 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116446 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8273920~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:48.828 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.451557528999999, 2024-05-11T23:03:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045535299000000001, 2024-05-11T23:03:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116446 2024-05-11T23:03:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.829 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375547+0000", 2024-05-11T23:03:48.830 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4000000000000001e-07 2024-05-11T23:03:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375549+0000", 2024-05-11T23:03:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.368e-06 2024-05-11T23:03:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375556+0000", 2024-05-11T23:03:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1840000000000004e-06 2024-05-11T23:03:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.831 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375560+0000", 2024-05-11T23:03:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.741e-06 2024-05-11T23:03:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420896+0000", 2024-05-11T23:03:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045335435 2024-05-11T23:03:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:48.832 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420987+0000", 2024-05-11T23:03:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1033999999999996e-05 2024-05-11T23:03:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421082+0000", 2024-05-11T23:03:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4896999999999996e-05 2024-05-11T23:03:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.833 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116447 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6316032~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.451536019000002, 2024-05-11T23:03:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045745830000000001, 2024-05-11T23:03:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116447 2024-05-11T23:03:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.834 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.835 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.58e-07 2024-05-11T23:03:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375570+0000", 2024-05-11T23:03:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:03:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375576+0000", 2024-05-11T23:03:48.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7939999999999999e-06 2024-05-11T23:03:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375581+0000", 2024-05-11T23:03:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9280000000000001e-06 2024-05-11T23:03:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421127+0000", 2024-05-11T23:03:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045545785999999998 2024-05-11T23:03:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421214+0000", 2024-05-11T23:03:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7039000000000005e-05 2024-05-11T23:03:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421314+0000", 2024-05-11T23:03:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9772999999999995e-05 2024-05-11T23:03:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116448 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8314880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.451516944999995, 2024-05-11T23:03:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045941753000000002, 2024-05-11T23:03:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:48.839 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116448 2024-05-11T23:03:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:48.840 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.77e-07 2024-05-11T23:03:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375589+0000", 2024-05-11T23:03:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.091e-06 2024-05-11T23:03:48.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375596+0000", 2024-05-11T23:03:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9179999999999997e-06 2024-05-11T23:03:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375601+0000", 2024-05-11T23:03:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0030000000000002e-06 2024-05-11T23:03:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421363+0000", 2024-05-11T23:03:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045761352999999998 2024-05-11T23:03:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421440+0000", 2024-05-11T23:03:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7585999999999999e-05 2024-05-11T23:03:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421529+0000", 2024-05-11T23:03:48.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8424999999999994e-05 2024-05-11T23:03:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116449 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6438912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.451496509999998, 2024-05-11T23:03:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046082208999999999, 2024-05-11T23:03:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:48.844 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116449 2024-05-11T23:03:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.845 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377225+0000", 2024-05-11T23:03:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016177030000000001 2024-05-11T23:03:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377229+0000", 2024-05-11T23:03:48.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8e-06 2024-05-11T23:03:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377238+0000", 2024-05-11T23:03:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2229999999999996e-06 2024-05-11T23:03:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377244+0000", 2024-05-11T23:03:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4060000000000002e-06 2024-05-11T23:03:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421520+0000", 2024-05-11T23:03:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044275222000000003 2024-05-11T23:03:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421595+0000", 2024-05-11T23:03:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4838000000000004e-05 2024-05-11T23:03:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:48.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421690+0000", 2024-05-11T23:03:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5017000000000002e-05 2024-05-11T23:03:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116458 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7094272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.449682292999995, 2024-05-11T23:03:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045706877999999999, 2024-05-11T23:03:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:48.849 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116458 2024-05-11T23:03:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9400000000000005e-07 2024-05-11T23:03:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377424+0000", 2024-05-11T23:03:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1859999999999999e-06 2024-05-11T23:03:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377431+0000", 2024-05-11T23:03:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4049999999999997e-06 2024-05-11T23:03:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377436+0000", 2024-05-11T23:03:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1399999999999999e-06 2024-05-11T23:03:48.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.422945+0000", 2024-05-11T23:03:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045509326000000003 2024-05-11T23:03:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423036+0000", 2024-05-11T23:03:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0649999999999994e-05 2024-05-11T23:03:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:48.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423128+0000", 2024-05-11T23:03:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2577000000000005e-05 2024-05-11T23:03:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116459 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7258112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.449661560999999, 2024-05-11T23:03:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045831945999999998, 2024-05-11T23:03:48.854 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116459 2024-05-11T23:03:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:48.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377443+0000", 2024-05-11T23:03:48.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999999e-07 2024-05-11T23:03:48.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:48.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377445+0000", 2024-05-11T23:03:48.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0209999999999998e-06 2024-05-11T23:03:48.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:48.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377451+0000", 2024-05-11T23:03:48.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.071e-06 2024-05-11T23:03:48.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:48.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377456+0000", 2024-05-11T23:03:48.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8289999999999998e-06 2024-05-11T23:03:48.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:48.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423155+0000", 2024-05-11T23:03:48.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045699447999999997 2024-05-11T23:03:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423200+0000", 2024-05-11T23:03:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5083000000000002e-05 2024-05-11T23:03:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423274+0000", 2024-05-11T23:03:48.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4046000000000002e-05 2024-05-11T23:03:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116460 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7299072~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.449642499000007, 2024-05-11T23:03:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046010987000000003, 2024-05-11T23:03:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:48.861 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116460 2024-05-11T23:03:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:48.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:48.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377462+0000", 2024-05-11T23:03:48.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:03:48.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:48.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377464+0000", 2024-05-11T23:03:48.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1600000000000001e-06 2024-05-11T23:03:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377469+0000", 2024-05-11T23:03:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5430000000000001e-06 2024-05-11T23:03:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377474+0000", 2024-05-11T23:03:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8509999999999998e-06 2024-05-11T23:03:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423312+0000", 2024-05-11T23:03:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045838080000000003 2024-05-11T23:03:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423401+0000", 2024-05-11T23:03:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8170000000000005e-05 2024-05-11T23:03:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423472+0000", 2024-05-11T23:03:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1847000000000005e-05 2024-05-11T23:03:48.867 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.868 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.868 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.868 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116461 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7421952~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:48.868 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:48.868 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.449623728999995, 2024-05-11T23:03:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046130749999999998, 2024-05-11T23:03:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116461 2024-05-11T23:03:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:48.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4700000000000002e-07 2024-05-11T23:03:48.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377483+0000", 2024-05-11T23:03:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0219999999999999e-06 2024-05-11T23:03:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377489+0000", 2024-05-11T23:03:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4420000000000003e-06 2024-05-11T23:03:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377494+0000", 2024-05-11T23:03:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9239999999999998e-06 2024-05-11T23:03:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423500+0000", 2024-05-11T23:03:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046006054999999997 2024-05-11T23:03:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423557+0000", 2024-05-11T23:03:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7169000000000001e-05 2024-05-11T23:03:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423611+0000", 2024-05-11T23:03:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3791e-05 2024-05-11T23:03:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116462 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7462912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.449604110999999, 2024-05-11T23:03:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046254666, 2024-05-11T23:03:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:48.874 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116462 2024-05-11T23:03:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377502+0000", 2024-05-11T23:03:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-05-11T23:03:48.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377508+0000", 2024-05-11T23:03:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4500000000000003e-06 2024-05-11T23:03:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377512+0000", 2024-05-11T23:03:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7700000000000001e-06 2024-05-11T23:03:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:48.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423649+0000", 2024-05-11T23:03:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046136859000000002 2024-05-11T23:03:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423704+0000", 2024-05-11T23:03:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4938000000000002e-05 2024-05-11T23:03:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423754+0000", 2024-05-11T23:03:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0145999999999997e-05 2024-05-11T23:03:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.878 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116463 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7585792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.449585685000002, 2024-05-11T23:03:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361763, 2024-05-11T23:03:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:48.879 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116463 2024-05-11T23:03:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377519+0000", 2024-05-11T23:03:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2600000000000002e-07 2024-05-11T23:03:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377521+0000", 2024-05-11T23:03:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1710000000000001e-06 2024-05-11T23:03:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377527+0000", 2024-05-11T23:03:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9649999999999996e-06 2024-05-11T23:03:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:48.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377532+0000", 2024-05-11T23:03:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.78e-06 2024-05-11T23:03:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423779+0000", 2024-05-11T23:03:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046247706 2024-05-11T23:03:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423829+0000", 2024-05-11T23:03:48.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9188999999999999e-05 2024-05-11T23:03:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423880+0000", 2024-05-11T23:03:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1425999999999999e-05 2024-05-11T23:03:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116464 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7749632~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:48.884 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.449566449000002, 2024-05-11T23:03:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046491719000000001, 2024-05-11T23:03:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116464 2024-05-11T23:03:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:48.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377538+0000", 2024-05-11T23:03:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4499999999999998e-07 2024-05-11T23:03:48.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377540+0000", 2024-05-11T23:03:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2340000000000001e-06 2024-05-11T23:03:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377546+0000", 2024-05-11T23:03:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8289999999999996e-06 2024-05-11T23:03:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377551+0000", 2024-05-11T23:03:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8559999999999998e-06 2024-05-11T23:03:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:48.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423913+0000", 2024-05-11T23:03:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361885999999998 2024-05-11T23:03:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423958+0000", 2024-05-11T23:03:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5513999999999998e-05 2024-05-11T23:03:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424029+0000", 2024-05-11T23:03:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1055000000000003e-05 2024-05-11T23:03:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.889 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116465 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7626752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.449547300999996, 2024-05-11T23:03:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046600057, 2024-05-11T23:03:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:48.890 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116465 2024-05-11T23:03:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:48.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3599999999999999e-07 2024-05-11T23:03:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377559+0000", 2024-05-11T23:03:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9640000000000002e-06 2024-05-11T23:03:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377572+0000", 2024-05-11T23:03:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3188e-05 2024-05-11T23:03:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377578+0000", 2024-05-11T23:03:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.942e-06 2024-05-11T23:03:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424058+0000", 2024-05-11T23:03:48.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046480021000000003 2024-05-11T23:03:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424103+0000", 2024-05-11T23:03:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5021999999999997e-05 2024-05-11T23:03:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424157+0000", 2024-05-11T23:03:48.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3483999999999998e-05 2024-05-11T23:03:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116466 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7790592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.449518718999997, 2024-05-11T23:03:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046743741999999998, 2024-05-11T23:03:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:48.895 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116466 2024-05-11T23:03:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:48.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:48.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:48.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377586+0000", 2024-05-11T23:03:48.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:03:48.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:48.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377588+0000", 2024-05-11T23:03:48.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3039999999999999e-06 2024-05-11T23:03:48.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:48.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377594+0000", 2024-05-11T23:03:48.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-05-11T23:03:48.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:48.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377600+0000", 2024-05-11T23:03:48.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7699999999999998e-06 2024-05-11T23:03:48.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:48.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424185+0000", 2024-05-11T23:03:48.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046585891999999997 2024-05-11T23:03:48.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:48.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424235+0000", 2024-05-11T23:03:48.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9356e-05 2024-05-11T23:03:48.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:48.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424329+0000", 2024-05-11T23:03:48.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4036999999999994e-05 2024-05-11T23:03:48.899 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.900 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.900 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116467 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7954432~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:48.900 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:48.900 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.449498431999999, 2024-05-11T23:03:48.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046930134999999998, 2024-05-11T23:03:48.900 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:48.900 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:48.900 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:48.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:48.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:48.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:48.901 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116467 2024-05-11T23:03:48.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.901 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:48.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:48.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:48.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:48.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:48.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:48.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379161+0000", 2024-05-11T23:03:48.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015555009999999999 2024-05-11T23:03:48.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:48.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379166+0000", 2024-05-11T23:03:48.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6689999999999997e-06 2024-05-11T23:03:48.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:48.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379176+0000", 2024-05-11T23:03:48.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0549e-05 2024-05-11T23:03:48.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:48.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379183+0000", 2024-05-11T23:03:48.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1199999999999996e-06 2024-05-11T23:03:48.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:48.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424359+0000", 2024-05-11T23:03:48.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045175949999999999 2024-05-11T23:03:48.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:48.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424433+0000", 2024-05-11T23:03:48.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3824000000000003e-05 2024-05-11T23:03:48.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:48.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424536+0000", 2024-05-11T23:03:48.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010252200000000001 2024-05-11T23:03:48.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.905 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.905 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.906 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116468 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7913472~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:48.906 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:48.906 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.447908213000005, 2024-05-11T23:03:48.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045525364999999998, 2024-05-11T23:03:48.906 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:48.906 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:48.906 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:48.906 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:48.906 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:48.906 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:48.907 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116468 2024-05-11T23:03:48.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.907 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:48.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:48.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:48.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:48.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:48.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:48.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379196+0000", 2024-05-11T23:03:48.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3300000000000002e-07 2024-05-11T23:03:48.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:48.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379198+0000", 2024-05-11T23:03:48.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2670000000000001e-06 2024-05-11T23:03:48.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:48.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379206+0000", 2024-05-11T23:03:48.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7149999999999998e-06 2024-05-11T23:03:48.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:48.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379228+0000", 2024-05-11T23:03:48.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1429000000000001e-05 2024-05-11T23:03:48.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:48.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424574+0000", 2024-05-11T23:03:48.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045346209999999998 2024-05-11T23:03:48.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:48.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424627+0000", 2024-05-11T23:03:48.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3352e-05 2024-05-11T23:03:48.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:48.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424721+0000", 2024-05-11T23:03:48.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3858999999999997e-05 2024-05-11T23:03:48.910 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.911 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.911 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116469 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8077312~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:48.911 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:48.911 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.447858393000004, 2024-05-11T23:03:48.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045636687000000002, 2024-05-11T23:03:48.911 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:48.911 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:48.911 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:48.911 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:48.911 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:48.911 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:48.912 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116469 2024-05-11T23:03:48.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.912 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:48.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:48.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:48.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:48.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:48.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:48.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379246+0000", 2024-05-11T23:03:48.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.32e-07 2024-05-11T23:03:48.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:48.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379254+0000", 2024-05-11T23:03:48.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.807e-06 2024-05-11T23:03:48.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:48.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379261+0000", 2024-05-11T23:03:48.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.985e-06 2024-05-11T23:03:48.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:48.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379264+0000", 2024-05-11T23:03:48.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6480000000000001e-06 2024-05-11T23:03:48.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:48.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424744+0000", 2024-05-11T23:03:48.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045479638000000003 2024-05-11T23:03:48.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:48.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424797+0000", 2024-05-11T23:03:48.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2725000000000001e-05 2024-05-11T23:03:48.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:48.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424882+0000", 2024-05-11T23:03:48.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5451999999999994e-05 2024-05-11T23:03:48.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.915 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.916 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116470 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8118272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:48.916 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:48.916 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.447833180999993, 2024-05-11T23:03:48.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045755745, 2024-05-11T23:03:48.916 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:48.916 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:48.916 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:48.916 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:48.916 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:48.917 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:48.917 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116470 2024-05-11T23:03:48.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.917 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:48.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:48.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:48.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:48.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:48.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:48.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:48.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3999999999999997e-07 2024-05-11T23:03:48.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:48.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379273+0000", 2024-05-11T23:03:48.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-05-11T23:03:48.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:48.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379277+0000", 2024-05-11T23:03:48.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4020000000000004e-06 2024-05-11T23:03:48.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:48.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379280+0000", 2024-05-11T23:03:48.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-05-11T23:03:48.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:48.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424922+0000", 2024-05-11T23:03:48.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045642598999999999 2024-05-11T23:03:48.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:48.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424973+0000", 2024-05-11T23:03:48.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0674999999999998e-05 2024-05-11T23:03:48.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:48.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425026+0000", 2024-05-11T23:03:48.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3492000000000002e-05 2024-05-11T23:03:48.921 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.921 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.921 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.921 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116471 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8241152~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:48.921 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:48.921 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.447820398000005, 2024-05-11T23:03:48.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045867374000000002, 2024-05-11T23:03:48.922 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:48.922 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:48.922 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:48.922 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:48.922 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:48.922 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:48.922 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116471 2024-05-11T23:03:48.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.922 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:48.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:48.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:48.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:48.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:48.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:48.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379284+0000", 2024-05-11T23:03:48.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.53e-07 2024-05-11T23:03:48.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:48.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379285+0000", 2024-05-11T23:03:48.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.223e-06 2024-05-11T23:03:48.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:48.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379288+0000", 2024-05-11T23:03:48.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5379999999999998e-06 2024-05-11T23:03:48.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:48.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379291+0000", 2024-05-11T23:03:48.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6699999999999998e-06 2024-05-11T23:03:48.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:48.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425055+0000", 2024-05-11T23:03:48.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045763497 2024-05-11T23:03:48.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:48.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425099+0000", 2024-05-11T23:03:48.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3899000000000001e-05 2024-05-11T23:03:48.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:48.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425151+0000", 2024-05-11T23:03:48.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2293999999999998e-05 2024-05-11T23:03:48.926 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.926 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.926 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.927 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116472 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8282112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:48.927 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:48.927 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.447809398999993, 2024-05-11T23:03:48.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045973778999999999, 2024-05-11T23:03:48.927 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:48.927 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:48.927 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:48.927 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:48.927 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:48.927 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:48.927 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116472 2024-05-11T23:03:48.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.928 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:48.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:48.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:48.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:48.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:48.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:48.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381226+0000", 2024-05-11T23:03:48.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001931627 2024-05-11T23:03:48.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:48.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381231+0000", 2024-05-11T23:03:48.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8679999999999998e-06 2024-05-11T23:03:48.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:48.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381242+0000", 2024-05-11T23:03:48.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0702e-05 2024-05-11T23:03:48.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:48.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381249+0000", 2024-05-11T23:03:48.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9800000000000001e-06 2024-05-11T23:03:48.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:48.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425175+0000", 2024-05-11T23:03:48.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043926805999999999 2024-05-11T23:03:48.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:48.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425215+0000", 2024-05-11T23:03:48.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9737999999999998e-05 2024-05-11T23:03:48.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:48.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425268+0000", 2024-05-11T23:03:48.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3057999999999999e-05 2024-05-11T23:03:48.931 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.931 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.931 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.932 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116482 3.1b 3:db6e8843:::rbd_object_map.111e67f27da3.0000000000000028:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:48.932 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:48.932 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 73.445840856000004, 2024-05-11T23:03:48.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046526076, 2024-05-11T23:03:48.932 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:48.932 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:48.932 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:48.932 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:48.932 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:48.932 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:48.932 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116482 2024-05-11T23:03:48.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.932 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:48.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:48.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:48.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:48.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:48.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:48.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:48.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381264+0000", 2024-05-11T23:03:48.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5300000000000004e-07 2024-05-11T23:03:48.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:48.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381267+0000", 2024-05-11T23:03:48.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1999999999999999e-06 2024-05-11T23:03:48.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:48.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381274+0000", 2024-05-11T23:03:48.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7109999999999995e-06 2024-05-11T23:03:48.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:48.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381285+0000", 2024-05-11T23:03:48.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0856000000000001e-05 2024-05-11T23:03:48.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:48.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425292+0000", 2024-05-11T23:03:48.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044006376 2024-05-11T23:03:48.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:48.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425337+0000", 2024-05-11T23:03:48.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4907999999999998e-05 2024-05-11T23:03:48.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:03:48.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425507+0000", 2024-05-11T23:03:48.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017029999999999999 2024-05-11T23:03:48.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.936 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:48.936 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427668+0000", 2024-05-11T23:03:48.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021612430000000002 2024-05-11T23:03:48.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:48.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427683+0000", 2024-05-11T23:03:48.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5016e-05 2024-05-11T23:03:48.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:48.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427742+0000", 2024-05-11T23:03:48.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8811999999999999e-05 2024-05-11T23:03:48.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:48.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:48.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:48.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427789+0000", 2024-05-11T23:03:48.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7101000000000002e-05 2024-05-11T23:03:48.938 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.938 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.938 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.938 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:48.938 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:48.938 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:48.940 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:03:48.940 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:48.940+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:49.040 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:49.041+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:49.102 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:49.102 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:49.102 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:49.102 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:49.111 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:03:49.141 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:49.141+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:49.229 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:49.230 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:49.230 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:49.230 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:49.230 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:49.239 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:03:49.241 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:49.242+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:49.342 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:49.343+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:49.353 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267820 3.16 3:6b963329:::rbd_data.111ec06c817a.000000000000000a:33 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:03:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.410371323, 2024-05-11T23:03:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050141218000000001, 2024-05-11T23:03:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.354 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267820 2024-05-11T23:03:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:03:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:03:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942754+0000", 2024-05-11T23:03:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8559000000000003e-05 2024-05-11T23:03:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942756+0000", 2024-05-11T23:03:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.604e-06 2024-05-11T23:03:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942763+0000", 2024-05-11T23:03:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7939999999999997e-06 2024-05-11T23:03:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942768+0000", 2024-05-11T23:03:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4410000000000001e-06 2024-05-11T23:03:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992601+0000", 2024-05-11T23:03:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049832754999999999 2024-05-11T23:03:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992687+0000", 2024-05-11T23:03:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5605000000000005e-05 2024-05-11T23:03:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992796+0000", 2024-05-11T23:03:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010946 2024-05-11T23:03:49.358 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267852 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:03:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.410250118999997, 2024-05-11T23:03:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050212600000000003, 2024-05-11T23:03:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.359 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267852 2024-05-11T23:03:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:03:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:03:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943228+0000", 2024-05-11T23:03:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00045180600000000002 2024-05-11T23:03:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943232+0000", 2024-05-11T23:03:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8399999999999997e-06 2024-05-11T23:03:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943240+0000", 2024-05-11T23:03:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8900000000000007e-06 2024-05-11T23:03:49.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943246+0000", 2024-05-11T23:03:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6239999999999996e-06 2024-05-11T23:03:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992828+0000", 2024-05-11T23:03:49.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049581247000000002 2024-05-11T23:03:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992905+0000", 2024-05-11T23:03:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7656000000000003e-05 2024-05-11T23:03:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992989+0000", 2024-05-11T23:03:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3536999999999997e-05 2024-05-11T23:03:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267854 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1134592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:03:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.409736977999998, 2024-05-11T23:03:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050077425000000002, 2024-05-11T23:03:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267854 2024-05-11T23:03:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.364 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:03:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:03:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943290+0000", 2024-05-11T23:03:49.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2300000000000002e-07 2024-05-11T23:03:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943291+0000", 2024-05-11T23:03:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6190000000000001e-06 2024-05-11T23:03:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943296+0000", 2024-05-11T23:03:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2889999999999999e-06 2024-05-11T23:03:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943299+0000", 2024-05-11T23:03:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1030000000000002e-06 2024-05-11T23:03:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993205+0000", 2024-05-11T23:03:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049906405000000001 2024-05-11T23:03:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993285+0000", 2024-05-11T23:03:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9752999999999993e-05 2024-05-11T23:03:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993367+0000", 2024-05-11T23:03:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1933000000000003e-05 2024-05-11T23:03:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267855 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1171456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:03:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.409720036000003, 2024-05-11T23:03:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050250346000000001, 2024-05-11T23:03:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.368 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267855 2024-05-11T23:03:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:03:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:03:49.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945226+0000", 2024-05-11T23:03:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001919235 2024-05-11T23:03:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945230+0000", 2024-05-11T23:03:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9580000000000001e-06 2024-05-11T23:03:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945238+0000", 2024-05-11T23:03:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1040000000000007e-06 2024-05-11T23:03:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945244+0000", 2024-05-11T23:03:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6919999999999996e-06 2024-05-11T23:03:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993399+0000", 2024-05-11T23:03:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048154651 2024-05-11T23:03:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993472+0000", 2024-05-11T23:03:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2780000000000005e-05 2024-05-11T23:03:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993557+0000", 2024-05-11T23:03:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4926000000000006e-05 2024-05-11T23:03:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267865 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1540096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:49.372 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.407568410000003, 2024-05-11T23:03:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050054409000000001, 2024-05-11T23:03:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267865 2024-05-11T23:03:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:49.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6600000000000002e-07 2024-05-11T23:03:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945461+0000", 2024-05-11T23:03:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2670000000000001e-06 2024-05-11T23:03:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945466+0000", 2024-05-11T23:03:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7200000000000003e-06 2024-05-11T23:03:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945471+0000", 2024-05-11T23:03:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9439999999999996e-06 2024-05-11T23:03:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995337+0000", 2024-05-11T23:03:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049865415000000003 2024-05-11T23:03:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995413+0000", 2024-05-11T23:03:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5989999999999996e-05 2024-05-11T23:03:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995512+0000", 2024-05-11T23:03:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9706999999999999e-05 2024-05-11T23:03:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268017 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 835584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947359+0000", 2024-05-11T23:03:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.405667524999998, 2024-05-11T23:03:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050017655000000001, 2024-05-11T23:03:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268017 2024-05-11T23:03:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947359+0000", 2024-05-11T23:03:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947359+0000", 2024-05-11T23:03:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947359+0000", 2024-05-11T23:03:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5000000000000003e-07 2024-05-11T23:03:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947411+0000", 2024-05-11T23:03:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1158e-05 2024-05-11T23:03:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947418+0000", 2024-05-11T23:03:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8269999999999998e-06 2024-05-11T23:03:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947424+0000", 2024-05-11T23:03:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6529999999999999e-06 2024-05-11T23:03:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997179+0000", 2024-05-11T23:03:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049754675999999998 2024-05-11T23:03:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997279+0000", 2024-05-11T23:03:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9789000000000003e-05 2024-05-11T23:03:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997377+0000", 2024-05-11T23:03:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8002000000000001e-05 2024-05-11T23:03:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268018 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 872448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:03:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.405592345000002, 2024-05-11T23:03:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050170927999999997, 2024-05-11T23:03:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268018 2024-05-11T23:03:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:03:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:03:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947435+0000", 2024-05-11T23:03:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3099999999999997e-07 2024-05-11T23:03:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947437+0000", 2024-05-11T23:03:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6340000000000001e-06 2024-05-11T23:03:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947443+0000", 2024-05-11T23:03:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7810000000000002e-06 2024-05-11T23:03:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947448+0000", 2024-05-11T23:03:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9679999999999997e-06 2024-05-11T23:03:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997418+0000", 2024-05-11T23:03:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049969587000000003 2024-05-11T23:03:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997504+0000", 2024-05-11T23:03:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6426000000000002e-05 2024-05-11T23:03:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997605+0000", 2024-05-11T23:03:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100901 2024-05-11T23:03:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.384 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268019 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.405571787, 2024-05-11T23:03:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050285074999999999, 2024-05-11T23:03:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268019 2024-05-11T23:03:49.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0999999999999999e-07 2024-05-11T23:03:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947457+0000", 2024-05-11T23:03:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-05-11T23:03:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947462+0000", 2024-05-11T23:03:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2410000000000003e-06 2024-05-11T23:03:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947467+0000", 2024-05-11T23:03:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8339999999999998e-06 2024-05-11T23:03:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997639+0000", 2024-05-11T23:03:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050171697000000001 2024-05-11T23:03:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997685+0000", 2024-05-11T23:03:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6146000000000002e-05 2024-05-11T23:03:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997740+0000", 2024-05-11T23:03:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4737000000000001e-05 2024-05-11T23:03:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268020 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:03:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.405553542, 2024-05-11T23:03:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050504089000000002, 2024-05-11T23:03:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268020 2024-05-11T23:03:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:03:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:03:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947591+0000", 2024-05-11T23:03:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011805 2024-05-11T23:03:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947594+0000", 2024-05-11T23:03:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1549999999999999e-06 2024-05-11T23:03:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947602+0000", 2024-05-11T23:03:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.174e-06 2024-05-11T23:03:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947608+0000", 2024-05-11T23:03:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.263e-06 2024-05-11T23:03:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997775+0000", 2024-05-11T23:03:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050166980999999999 2024-05-11T23:03:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997875+0000", 2024-05-11T23:03:49.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9844000000000002e-05 2024-05-11T23:03:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997977+0000", 2024-05-11T23:03:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101622 2024-05-11T23:03:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268021 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:03:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.405408082000001, 2024-05-11T23:03:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050498874999999999, 2024-05-11T23:03:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.393 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.394 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.394 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.394 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.394 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.394 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268021 2024-05-11T23:03:49.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.394 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:03:49.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:03:49.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947695+0000", 2024-05-11T23:03:49.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6247000000000004e-05 2024-05-11T23:03:49.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947698+0000", 2024-05-11T23:03:49.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9199999999999997e-06 2024-05-11T23:03:49.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947706+0000", 2024-05-11T23:03:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:03:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947711+0000", 2024-05-11T23:03:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6720000000000001e-06 2024-05-11T23:03:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998017+0000", 2024-05-11T23:03:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050305470999999997 2024-05-11T23:03:49.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998060+0000", 2024-05-11T23:03:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3374000000000002e-05 2024-05-11T23:03:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998117+0000", 2024-05-11T23:03:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6971000000000001e-05 2024-05-11T23:03:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268022 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:03:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.405305046000002, 2024-05-11T23:03:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050731361000000003, 2024-05-11T23:03:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268022 2024-05-11T23:03:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:03:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:03:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947722+0000", 2024-05-11T23:03:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8999999999999996e-07 2024-05-11T23:03:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947724+0000", 2024-05-11T23:03:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1639999999999999e-06 2024-05-11T23:03:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947730+0000", 2024-05-11T23:03:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6099999999999997e-06 2024-05-11T23:03:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947735+0000", 2024-05-11T23:03:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7189999999999998e-06 2024-05-11T23:03:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998140+0000", 2024-05-11T23:03:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050404254000000003 2024-05-11T23:03:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998239+0000", 2024-05-11T23:03:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9192999999999997e-05 2024-05-11T23:03:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998453+0000", 2024-05-11T23:03:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021383099999999999 2024-05-11T23:03:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.401 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268023 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1069056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:03:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.405282780999997, 2024-05-11T23:03:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050874509999999998, 2024-05-11T23:03:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268023 2024-05-11T23:03:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.402 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:03:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:03:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948934+0000", 2024-05-11T23:03:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011903339999999999 2024-05-11T23:03:49.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948938+0000", 2024-05-11T23:03:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1330000000000001e-06 2024-05-11T23:03:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948949+0000", 2024-05-11T23:03:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0872e-05 2024-05-11T23:03:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948956+0000", 2024-05-11T23:03:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.131e-06 2024-05-11T23:03:49.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998494+0000", 2024-05-11T23:03:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049537678000000002 2024-05-11T23:03:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998551+0000", 2024-05-11T23:03:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7256000000000002e-05 2024-05-11T23:03:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998618+0000", 2024-05-11T23:03:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7106000000000004e-05 2024-05-11T23:03:49.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268026 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1179648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:03:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.404005728000001, 2024-05-11T23:03:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050107977999999997, 2024-05-11T23:03:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.406 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268026 2024-05-11T23:03:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:03:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:03:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949029+0000", 2024-05-11T23:03:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2220000000000002e-06 2024-05-11T23:03:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949041+0000", 2024-05-11T23:03:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.241e-05 2024-05-11T23:03:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949049+0000", 2024-05-11T23:03:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.072e-06 2024-05-11T23:03:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949056+0000", 2024-05-11T23:03:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1380000000000001e-06 2024-05-11T23:03:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999016+0000", 2024-05-11T23:03:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049959212000000003 2024-05-11T23:03:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999067+0000", 2024-05-11T23:03:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1598000000000003e-05 2024-05-11T23:03:49.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999129+0000", 2024-05-11T23:03:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1326000000000002e-05 2024-05-11T23:03:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268027 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1216512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.403961578000001, 2024-05-11T23:03:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050189865, 2024-05-11T23:03:49.410 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268027 2024-05-11T23:03:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-05-11T23:03:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949068+0000", 2024-05-11T23:03:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.26e-06 2024-05-11T23:03:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949074+0000", 2024-05-11T23:03:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7050000000000002e-06 2024-05-11T23:03:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949080+0000", 2024-05-11T23:03:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3689999999999998e-06 2024-05-11T23:03:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999152+0000", 2024-05-11T23:03:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050072653000000002 2024-05-11T23:03:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999197+0000", 2024-05-11T23:03:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4972999999999998e-05 2024-05-11T23:03:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999255+0000", 2024-05-11T23:03:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7425e-05 2024-05-11T23:03:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268028 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1253376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:49.414 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:03:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.403939917999999, 2024-05-11T23:03:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050280017000000003, 2024-05-11T23:03:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268028 2024-05-11T23:03:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:03:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:03:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949087+0000", 2024-05-11T23:03:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:03:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949089+0000", 2024-05-11T23:03:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8950000000000001e-06 2024-05-11T23:03:49.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949095+0000", 2024-05-11T23:03:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7540000000000003e-06 2024-05-11T23:03:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949114+0000", 2024-05-11T23:03:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9738e-05 2024-05-11T23:03:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999278+0000", 2024-05-11T23:03:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050163368999999999 2024-05-11T23:03:49.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999319+0000", 2024-05-11T23:03:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0979000000000002e-05 2024-05-11T23:03:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999366+0000", 2024-05-11T23:03:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7879e-05 2024-05-11T23:03:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268029 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1290240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.403899809999999, 2024-05-11T23:03:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050344904000000003, 2024-05-11T23:03:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268029 2024-05-11T23:03:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:49.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:03:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949130+0000", 2024-05-11T23:03:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-05-11T23:03:49.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949139+0000", 2024-05-11T23:03:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9800000000000004e-06 2024-05-11T23:03:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949145+0000", 2024-05-11T23:03:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0510000000000002e-06 2024-05-11T23:03:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999387+0000", 2024-05-11T23:03:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050242167999999997 2024-05-11T23:03:49.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999427+0000", 2024-05-11T23:03:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9566e-05 2024-05-11T23:03:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999471+0000", 2024-05-11T23:03:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4914999999999999e-05 2024-05-11T23:03:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.422 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268030 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1327104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.403874526000003, 2024-05-11T23:03:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050447394, 2024-05-11T23:03:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268030 2024-05-11T23:03:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:49.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0500000000000004e-07 2024-05-11T23:03:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949155+0000", 2024-05-11T23:03:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3819999999999998e-06 2024-05-11T23:03:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949162+0000", 2024-05-11T23:03:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9609999999999999e-06 2024-05-11T23:03:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949188+0000", 2024-05-11T23:03:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5877e-05 2024-05-11T23:03:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999494+0000", 2024-05-11T23:03:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050306265000000003 2024-05-11T23:03:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999534+0000", 2024-05-11T23:03:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0447000000000001e-05 2024-05-11T23:03:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999599+0000", 2024-05-11T23:03:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4956999999999995e-05 2024-05-11T23:03:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268031 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1363968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:03:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.403829426000001, 2024-05-11T23:03:49.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050511525000000002, 2024-05-11T23:03:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268031 2024-05-11T23:03:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:03:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:03:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949198+0000", 2024-05-11T23:03:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-05-11T23:03:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949200+0000", 2024-05-11T23:03:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-05-11T23:03:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949208+0000", 2024-05-11T23:03:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2840000000000001e-06 2024-05-11T23:03:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949214+0000", 2024-05-11T23:03:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9869999999999996e-06 2024-05-11T23:03:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999619+0000", 2024-05-11T23:03:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050405278999999997 2024-05-11T23:03:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999660+0000", 2024-05-11T23:03:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1427999999999997e-05 2024-05-11T23:03:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999708+0000", 2024-05-11T23:03:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8229999999999997e-05 2024-05-11T23:03:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268032 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1400832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:03:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.403799806000002, 2024-05-11T23:03:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050611863999999999, 2024-05-11T23:03:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268032 2024-05-11T23:03:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:03:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:03:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949230+0000", 2024-05-11T23:03:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.371e-06 2024-05-11T23:03:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949233+0000", 2024-05-11T23:03:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6599999999999999e-06 2024-05-11T23:03:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949241+0000", 2024-05-11T23:03:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8939999999999993e-06 2024-05-11T23:03:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949246+0000", 2024-05-11T23:03:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6590000000000004e-06 2024-05-11T23:03:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999733+0000", 2024-05-11T23:03:49.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050487268000000002 2024-05-11T23:03:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999784+0000", 2024-05-11T23:03:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0562000000000001e-05 2024-05-11T23:03:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999838+0000", 2024-05-11T23:03:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000002e-05 2024-05-11T23:03:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.434 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268033 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1437696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:03:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 36.403772678999999, 2024-05-11T23:03:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050697157999999999, 2024-05-11T23:03:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268033 2024-05-11T23:03:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:03:49.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:03:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950862+0000", 2024-05-11T23:03:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001607975 2024-05-11T23:03:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950866+0000", 2024-05-11T23:03:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7770000000000002e-06 2024-05-11T23:03:49.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950878+0000", 2024-05-11T23:03:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1421e-05 2024-05-11T23:03:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950885+0000", 2024-05-11T23:03:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-05-11T23:03:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999859+0000", 2024-05-11T23:03:49.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048973199000000002 2024-05-11T23:03:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999901+0000", 2024-05-11T23:03:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1943000000000001e-05 2024-05-11T23:03:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999951+0000", 2024-05-11T23:03:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0243e-05 2024-05-11T23:03:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.438 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.439 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:49.440 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:03:49.483 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:49.484+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:49.543 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:49.544+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:49.552 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:49.552 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:49.552 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:49.552 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:49.561 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:03:49.644 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:49.644+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:49.710 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:49.710 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:49.710 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:49.719 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:03:49.745 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:49.745+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:49.835 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.179981062, 2024-05-11T23:03:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:03:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.835 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:03:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.836 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:03:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:03:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:03:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.837 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:03:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:03:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:03:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:03:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:03:49.838 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:03:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:03:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:03:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:03:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:03:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.839 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.176599873, 2024-05-11T23:03:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:03:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:03:49.840 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:03:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:03:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:03:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:03:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:03:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:03:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:03:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:03:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.843 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:03:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:03:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:03:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:03:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.844 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.17658116600001, 2024-05-11T23:03:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:03:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:03:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.845 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.846 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:03:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:03:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:03:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:03:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:03:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:03:49.847 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:03:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:03:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:03:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:03:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:03:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:03:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:03:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:03:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:49.849 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.176563023, 2024-05-11T23:03:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:03:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:03:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:03:49.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:03:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:03:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:03:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:03:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:03:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:03:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:03:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:03:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:03:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:03:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:03:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:03:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.176535536, 2024-05-11T23:03:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:03:49.854 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:03:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:03:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:03:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:03:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:03:49.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:03:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:03:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:03:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:03:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:03:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:03:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:03:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:03:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:03:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:03:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.858 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.176515008, 2024-05-11T23:03:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:03:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.859 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:03:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.860 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:03:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:03:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:03:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:03:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:03:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:03:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:03:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:03:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:03:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:03:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:03:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:03:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:03:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.176495233, 2024-05-11T23:03:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:03:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.863 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:03:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:03:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:03:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:03:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:03:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:03:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:03:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:03:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:03:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:03:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:03:49.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:03:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:03:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:03:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:03:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.176476427, 2024-05-11T23:03:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:03:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.868 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:03:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:49.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:03:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:03:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:03:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:03:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:03:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:03:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:03:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:03:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:03:49.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:03:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:03:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:03:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:03:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:03:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.872 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.176457504, 2024-05-11T23:03:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:03:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.873 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:03:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:03:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:03:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:03:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:03:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:03:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:03:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:03:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:03:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:03:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:03:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:03:49.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:03:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:03:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.176438842, 2024-05-11T23:03:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:03:49.877 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:03:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:03:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:03:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:03:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:03:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:03:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:03:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:03:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:03:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:03:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:03:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:03:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:03:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:03:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.881 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.17609367999999, 2024-05-11T23:03:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:03:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.882 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:03:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:03:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:03:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:03:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:03:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:03:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:03:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:03:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:03:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:03:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:03:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:03:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:03:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:03:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:03:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.17606813, 2024-05-11T23:03:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:03:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:03:49.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:03:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:03:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:03:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:03:49.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:03:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:03:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:03:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:03:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:03:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:03:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:03:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:03:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:03:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:49.891 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.175985872, 2024-05-11T23:03:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:03:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:03:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:03:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:03:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:03:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:03:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:03:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:03:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:03:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:03:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:03:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:03:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:03:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:03:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:03:49.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.17596342500001, 2024-05-11T23:03:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:03:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.896 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:03:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:03:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:03:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:03:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:03:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:03:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:03:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:03:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:03:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:03:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:03:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:03:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:03:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:03:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:03:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.17594396699999, 2024-05-11T23:03:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:03:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.901 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:03:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:49.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:03:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:03:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:03:49.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:03:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:03:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:03:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:03:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:03:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:03:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:03:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:03:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:03:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:03:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:49.906 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.175923907, 2024-05-11T23:03:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:03:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:03:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:03:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:03:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:03:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:03:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:03:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:03:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:03:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:03:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:03:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:03:49.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:03:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:03:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:49.911 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.175904637, 2024-05-11T23:03:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:03:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:03:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:03:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:03:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:03:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:03:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:03:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:03:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:03:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:03:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:03:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:03:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:03:49.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:03:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:03:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.175358571, 2024-05-11T23:03:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:03:49.916 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:03:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:03:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:03:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:03:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:03:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:03:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:03:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:03:49.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:03:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:03:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:03:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:03:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:03:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:03:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:03:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.175326787, 2024-05-11T23:03:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:03:49.921 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:03:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:49.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:03:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:03:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:03:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:03:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:03:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:03:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:03:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:03:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:03:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:03:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:03:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:03:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:03:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:03:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.738081323, 2024-05-11T23:03:49.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:03:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:03:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:49.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:03:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:03:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:03:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:03:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:49.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:03:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:03:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:03:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:03:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:03:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:03:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:03:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:03:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:03:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:03:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:03:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:03:49.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:03:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:03:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:03:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:03:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:03:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:49.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:03:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:03:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:49.934 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:49.934 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:49.935 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 5, 7, 3, 6, 4] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-05-11T23:03:49.935 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:03:49.935 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 2, 5, 7, 3, 6, 4] 2024-05-11T23:03:49.935 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-05-11T23:03:49.935 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-11T23:03:49.937 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:49.935+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:49.938 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:03:50.037 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:50.037+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:50.056 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:50.056 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:50.056 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:50.056 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:50.066 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:03:50.137 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:50.137+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:50.226 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:50.226 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:50.226 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:50.226 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:50.226 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:50.236 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:03:50.238 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:50.238+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:50.256 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-11T23:03:50.256+0000 7fd16247e640 -1 mon.a@0(leader).osd e147 definitely_dead 0 2024-05-11T23:03:50.339 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:50.339+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:50.351 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:50.352 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:50.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:50.352 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:50.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.352 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:03:50.352 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:50.352 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 153.18765180099999, 2024-05-11T23:03:50.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:03:50.352 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:50.352 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:50.352 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:50.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:50.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:03:50.353 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:03:50.353 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:03:50.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.353 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:50.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:50.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:50.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:50.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:50.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:50.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:03:50.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:03:50.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:50.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:03:50.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:03:50.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:50.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:03:50.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:03:50.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:50.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:03:50.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:03:50.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:03:50.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:03:50.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:03:50.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:03:50.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:03:50.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:03:50.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:03:50.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:50.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:03:50.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:03:50.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:03:50.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:03:50.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:03:50.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:50.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:03:50.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:03:50.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:50.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:03:50.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:03:50.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:50.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:03:50.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:03:50.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:50.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:03:50.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:03:50.358 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.358 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.358 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.358 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78429 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:50.358 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:50.358 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.386382459000004, 2024-05-11T23:03:50.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044563497000000001, 2024-05-11T23:03:50.359 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:50.359 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:50.359 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:50.359 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:50.359 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:50.359 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:50.359 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78429 2024-05-11T23:03:50.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.359 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:50.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:50.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:50.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:50.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:50.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:50.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964872+0000", 2024-05-11T23:03:50.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035070300000000001 2024-05-11T23:03:50.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:50.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964876+0000", 2024-05-11T23:03:50.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5009999999999998e-06 2024-05-11T23:03:50.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:50.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964902+0000", 2024-05-11T23:03:50.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5715999999999999e-05 2024-05-11T23:03:50.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:50.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964910+0000", 2024-05-11T23:03:50.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3930000000000002e-06 2024-05-11T23:03:50.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008898+0000", 2024-05-11T23:03:50.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043988772000000002 2024-05-11T23:03:50.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:50.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008981+0000", 2024-05-11T23:03:50.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3140999999999996e-05 2024-05-11T23:03:50.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:50.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009085+0000", 2024-05-11T23:03:50.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010327099999999999 2024-05-11T23:03:50.362 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.362 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.362 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.362 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78431 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:50.362 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:50.362 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.385922151000003, 2024-05-11T23:03:50.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044542033000000002, 2024-05-11T23:03:50.363 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:50.363 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:50.363 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:50.363 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:50.363 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:50.363 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:50.363 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78431 2024-05-11T23:03:50.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.363 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:50.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:50.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:50.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:50.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:50.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:50.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965038+0000", 2024-05-11T23:03:50.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6993000000000001e-05 2024-05-11T23:03:50.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:50.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965043+0000", 2024-05-11T23:03:50.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7040000000000002e-06 2024-05-11T23:03:50.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:50.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965053+0000", 2024-05-11T23:03:50.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8539999999999992e-06 2024-05-11T23:03:50.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:50.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965061+0000", 2024-05-11T23:03:50.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9810000000000003e-06 2024-05-11T23:03:50.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009350+0000", 2024-05-11T23:03:50.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044288739000000001 2024-05-11T23:03:50.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:50.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009429+0000", 2024-05-11T23:03:50.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9354e-05 2024-05-11T23:03:50.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:50.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009524+0000", 2024-05-11T23:03:50.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4407999999999995e-05 2024-05-11T23:03:50.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.366 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.366 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.367 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78432 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:50.367 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:50.367 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.385831303000003, 2024-05-11T23:03:50.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044635421000000002, 2024-05-11T23:03:50.367 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:50.367 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:50.367 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:50.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:50.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:50.367 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:50.367 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78432 2024-05-11T23:03:50.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.367 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:50.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:50.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:50.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:50.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:50.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965343+0000", 2024-05-11T23:03:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027056500000000001 2024-05-11T23:03:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965347+0000", 2024-05-11T23:03:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3540000000000002e-06 2024-05-11T23:03:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965356+0000", 2024-05-11T23:03:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2140000000000002e-06 2024-05-11T23:03:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965363+0000", 2024-05-11T23:03:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4150000000000004e-06 2024-05-11T23:03:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009551+0000", 2024-05-11T23:03:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044187917 2024-05-11T23:03:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009626+0000", 2024-05-11T23:03:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5024000000000005e-05 2024-05-11T23:03:50.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009708+0000", 2024-05-11T23:03:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1932000000000001e-05 2024-05-11T23:03:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78433 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.385530415999995, 2024-05-11T23:03:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044564864000000003, 2024-05-11T23:03:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:50.371 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:50.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:50.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:50.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:50.372 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78433 2024-05-11T23:03:50.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.372 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:50.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:50.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:50.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:50.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:50.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:50.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966225+0000", 2024-05-11T23:03:50.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085220700000000001 2024-05-11T23:03:50.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:50.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966230+0000", 2024-05-11T23:03:50.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0690000000000003e-06 2024-05-11T23:03:50.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:50.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966239+0000", 2024-05-11T23:03:50.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-05-11T23:03:50.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:50.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966245+0000", 2024-05-11T23:03:50.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5520000000000001e-06 2024-05-11T23:03:50.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009743+0000", 2024-05-11T23:03:50.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043498311999999997 2024-05-11T23:03:50.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:50.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009848+0000", 2024-05-11T23:03:50.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104885 2024-05-11T23:03:50.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:50.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009938+0000", 2024-05-11T23:03:50.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9812e-05 2024-05-11T23:03:50.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.375 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.376 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78580 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 999424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:50.376 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:50.376 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.382417732999997, 2024-05-11T23:03:50.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044493076999999999, 2024-05-11T23:03:50.376 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:50.376 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:50.376 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:50.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:50.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:50.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:50.377 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78580 2024-05-11T23:03:50.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.377 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:50.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:50.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:50.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:50.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:50.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:50.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:50.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:03:50.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:50.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968489+0000", 2024-05-11T23:03:50.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.52e-06 2024-05-11T23:03:50.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:50.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968496+0000", 2024-05-11T23:03:50.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9419999999999998e-06 2024-05-11T23:03:50.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:50.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968501+0000", 2024-05-11T23:03:50.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8720000000000001e-06 2024-05-11T23:03:50.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012690+0000", 2024-05-11T23:03:50.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044189655000000001 2024-05-11T23:03:50.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:50.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012848+0000", 2024-05-11T23:03:50.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015758100000000001 2024-05-11T23:03:50.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:50.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012979+0000", 2024-05-11T23:03:50.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000131066 2024-05-11T23:03:50.380 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.380 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.380 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.380 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78581 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:50.380 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:50.380 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.382396729000007, 2024-05-11T23:03:50.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044616243, 2024-05-11T23:03:50.380 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:50.380 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:50.381 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:50.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:50.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:50.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:50.381 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78581 2024-05-11T23:03:50.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.381 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:50.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:50.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:50.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:50.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:50.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:50.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970227+0000", 2024-05-11T23:03:50.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017200049999999999 2024-05-11T23:03:50.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:50.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970232+0000", 2024-05-11T23:03:50.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000003e-06 2024-05-11T23:03:50.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:50.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970242+0000", 2024-05-11T23:03:50.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0192e-05 2024-05-11T23:03:50.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:50.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970249+0000", 2024-05-11T23:03:50.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.888e-06 2024-05-11T23:03:50.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013008+0000", 2024-05-11T23:03:50.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042759495000000002 2024-05-11T23:03:50.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:50.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013063+0000", 2024-05-11T23:03:50.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5071999999999997e-05 2024-05-11T23:03:50.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:50.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013123+0000", 2024-05-11T23:03:50.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9930999999999999e-05 2024-05-11T23:03:50.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.384 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.384 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78589 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:50.384 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:50.384 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.380477485, 2024-05-11T23:03:50.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044587697000000003, 2024-05-11T23:03:50.385 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:50.385 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:50.385 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:50.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:50.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:50.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:50.385 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78589 2024-05-11T23:03:50.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.385 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:50.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:50.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:50.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:50.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:50.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:50.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970427+0000", 2024-05-11T23:03:50.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:03:50.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:50.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970429+0000", 2024-05-11T23:03:50.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3800000000000001e-06 2024-05-11T23:03:50.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:50.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970435+0000", 2024-05-11T23:03:50.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2530000000000001e-06 2024-05-11T23:03:50.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:50.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970441+0000", 2024-05-11T23:03:50.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-05-11T23:03:50.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014822+0000", 2024-05-11T23:03:50.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04438168 2024-05-11T23:03:50.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:50.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014911+0000", 2024-05-11T23:03:50.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8882999999999996e-05 2024-05-11T23:03:50.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:50.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015014+0000", 2024-05-11T23:03:50.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010263399999999999 2024-05-11T23:03:50.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.390 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.390 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78590 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:50.390 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:50.390 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.380456686000002, 2024-05-11T23:03:50.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044880120000000003, 2024-05-11T23:03:50.391 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:50.391 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:50.391 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:50.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:50.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:50.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:50.391 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78590 2024-05-11T23:03:50.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.391 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:50.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:50.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:50.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:50.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:50.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:50.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970918+0000", 2024-05-11T23:03:50.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000470834 2024-05-11T23:03:50.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:50.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970925+0000", 2024-05-11T23:03:50.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1690000000000003e-06 2024-05-11T23:03:50.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:50.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970934+0000", 2024-05-11T23:03:50.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9099999999999994e-06 2024-05-11T23:03:50.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:50.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970941+0000", 2024-05-11T23:03:50.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7410000000000003e-06 2024-05-11T23:03:50.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015049+0000", 2024-05-11T23:03:50.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044108392000000003 2024-05-11T23:03:50.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:50.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015133+0000", 2024-05-11T23:03:50.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3568999999999999e-05 2024-05-11T23:03:50.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:50.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015327+0000", 2024-05-11T23:03:50.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019450500000000001 2024-05-11T23:03:50.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.394 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.394 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78591 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:50.395 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:50.395 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.379953270000001, 2024-05-11T23:03:50.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044618497, 2024-05-11T23:03:50.395 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:50.395 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:50.395 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:50.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:50.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:50.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:50.395 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78591 2024-05-11T23:03:50.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.395 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:50.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:50.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:50.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:50.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:50.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:50.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970951+0000", 2024-05-11T23:03:50.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:03:50.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:50.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970953+0000", 2024-05-11T23:03:50.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:03:50.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:50.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970956+0000", 2024-05-11T23:03:50.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-05-11T23:03:50.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:50.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970959+0000", 2024-05-11T23:03:50.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9409999999999999e-06 2024-05-11T23:03:50.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015367+0000", 2024-05-11T23:03:50.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044407621000000001 2024-05-11T23:03:50.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:50.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015453+0000", 2024-05-11T23:03:50.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6685000000000001e-05 2024-05-11T23:03:50.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:50.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015569+0000", 2024-05-11T23:03:50.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115396 2024-05-11T23:03:50.398 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.398 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.398 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.399 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78592 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:50.399 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:50.399 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.379940461999993, 2024-05-11T23:03:50.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044831546999999999, 2024-05-11T23:03:50.399 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:50.399 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:50.399 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:50.399 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:50.399 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:50.399 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:50.399 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78592 2024-05-11T23:03:50.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.399 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:50.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:50.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:50.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:50.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:50.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:50.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972226+0000", 2024-05-11T23:03:50.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00126246 2024-05-11T23:03:50.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:50.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972230+0000", 2024-05-11T23:03:50.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1860000000000002e-06 2024-05-11T23:03:50.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:50.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972237+0000", 2024-05-11T23:03:50.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:03:50.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:50.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972241+0000", 2024-05-11T23:03:50.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.284e-06 2024-05-11T23:03:50.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015611+0000", 2024-05-11T23:03:50.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043369461999999998 2024-05-11T23:03:50.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:50.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015695+0000", 2024-05-11T23:03:50.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4458999999999999e-05 2024-05-11T23:03:50.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:50.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015795+0000", 2024-05-11T23:03:50.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9475999999999994e-05 2024-05-11T23:03:50.402 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.402 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.402 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.403 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78593 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:50.403 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:50.403 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.378653542999999, 2024-05-11T23:03:50.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046045214000000001, 2024-05-11T23:03:50.403 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:50.403 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:50.403 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:50.403 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:50.403 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:50.403 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:50.403 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78593 2024-05-11T23:03:50.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.403 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:50.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:50.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:50.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:50.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:50.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:50.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:50.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7599999999999998e-07 2024-05-11T23:03:50.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:50.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972252+0000", 2024-05-11T23:03:50.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7349999999999999e-06 2024-05-11T23:03:50.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:50.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972256+0000", 2024-05-11T23:03:50.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5659999999999998e-06 2024-05-11T23:03:50.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:50.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972259+0000", 2024-05-11T23:03:50.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8040000000000002e-06 2024-05-11T23:03:50.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018088+0000", 2024-05-11T23:03:50.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045829316000000002 2024-05-11T23:03:50.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:50.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018179+0000", 2024-05-11T23:03:50.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1299999999999997e-05 2024-05-11T23:03:50.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:50.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018295+0000", 2024-05-11T23:03:50.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116117 2024-05-11T23:03:50.406 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.406 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.407 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.407 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78594 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:50.407 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:50.407 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.378641182999999, 2024-05-11T23:03:50.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046245204999999998, 2024-05-11T23:03:50.407 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:50.407 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:50.407 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:50.407 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:50.407 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:50.407 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:50.407 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78594 2024-05-11T23:03:50.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.408 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:50.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:50.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:50.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:50.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:50.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:50.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972263+0000", 2024-05-11T23:03:50.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:03:50.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:50.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972264+0000", 2024-05-11T23:03:50.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.206e-06 2024-05-11T23:03:50.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:50.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972267+0000", 2024-05-11T23:03:50.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0690000000000002e-06 2024-05-11T23:03:50.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:50.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972270+0000", 2024-05-11T23:03:50.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5610000000000001e-06 2024-05-11T23:03:50.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018330+0000", 2024-05-11T23:03:50.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04606085 2024-05-11T23:03:50.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:50.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018414+0000", 2024-05-11T23:03:50.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3125000000000001e-05 2024-05-11T23:03:50.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:50.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018508+0000", 2024-05-11T23:03:50.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4153000000000005e-05 2024-05-11T23:03:50.411 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.411 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.411 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.411 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78595 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:50.411 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:50.411 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.378628317999997, 2024-05-11T23:03:50.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046396698, 2024-05-11T23:03:50.411 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:50.411 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:50.411 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:50.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:50.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:50.411 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:50.412 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78595 2024-05-11T23:03:50.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.412 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:50.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:50.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:50.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:50.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:50.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:50.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972276+0000", 2024-05-11T23:03:50.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5600000000000002e-07 2024-05-11T23:03:50.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:50.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972277+0000", 2024-05-11T23:03:50.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.576e-06 2024-05-11T23:03:50.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:50.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972280+0000", 2024-05-11T23:03:50.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-05-11T23:03:50.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:50.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972283+0000", 2024-05-11T23:03:50.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:03:50.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018538+0000", 2024-05-11T23:03:50.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046255255000000002 2024-05-11T23:03:50.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:50.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018605+0000", 2024-05-11T23:03:50.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6847000000000006e-05 2024-05-11T23:03:50.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:50.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018672+0000", 2024-05-11T23:03:50.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7045000000000006e-05 2024-05-11T23:03:50.415 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.415 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.415 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.415 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78596 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:50.415 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:50.415 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.378617469000005, 2024-05-11T23:03:50.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046586806000000001, 2024-05-11T23:03:50.415 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:50.415 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:50.416 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:50.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:50.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:50.416 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:50.416 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78596 2024-05-11T23:03:50.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.416 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:50.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:50.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:50.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:50.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:50.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:50.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:50.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:03:50.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:50.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972288+0000", 2024-05-11T23:03:50.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2550000000000001e-06 2024-05-11T23:03:50.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:50.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972291+0000", 2024-05-11T23:03:50.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9490000000000001e-06 2024-05-11T23:03:50.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:50.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972293+0000", 2024-05-11T23:03:50.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5069999999999999e-06 2024-05-11T23:03:50.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018707+0000", 2024-05-11T23:03:50.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046413604999999997 2024-05-11T23:03:50.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:50.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018782+0000", 2024-05-11T23:03:50.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5556000000000006e-05 2024-05-11T23:03:50.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:50.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018873+0000", 2024-05-11T23:03:50.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0703000000000002e-05 2024-05-11T23:03:50.419 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.419 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.419 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.419 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78597 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:50.419 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:50.419 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.378607696000003, 2024-05-11T23:03:50.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046825787000000001, 2024-05-11T23:03:50.419 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:50.419 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:50.420 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:50.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:50.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:50.420 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:50.420 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78597 2024-05-11T23:03:50.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.420 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:50.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:50.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:50.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:50.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:50.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:50.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:50.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-05-11T23:03:50.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:50.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972297+0000", 2024-05-11T23:03:50.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3069999999999999e-06 2024-05-11T23:03:50.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:50.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972300+0000", 2024-05-11T23:03:50.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:03:50.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:50.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972303+0000", 2024-05-11T23:03:50.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5509999999999998e-06 2024-05-11T23:03:50.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018911+0000", 2024-05-11T23:03:50.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046608194999999998 2024-05-11T23:03:50.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:50.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018986+0000", 2024-05-11T23:03:50.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5360999999999999e-05 2024-05-11T23:03:50.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.423 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:50.423 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019122+0000", 2024-05-11T23:03:50.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013526 2024-05-11T23:03:50.423 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.423 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.423 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.423 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78598 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:50.423 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:50.423 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 94.378597795999994, 2024-05-11T23:03:50.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046998572000000002, 2024-05-11T23:03:50.423 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:50.424 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:50.424 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:50.424 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:50.424 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:50.424 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:50.424 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78598 2024-05-11T23:03:50.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.424 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:50.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:50.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:50.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:50.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:50.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:50.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:50.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:03:50.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:50.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972307+0000", 2024-05-11T23:03:50.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1799999999999999e-06 2024-05-11T23:03:50.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:50.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972310+0000", 2024-05-11T23:03:50.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:03:50.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:50.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972313+0000", 2024-05-11T23:03:50.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4760000000000001e-06 2024-05-11T23:03:50.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019149+0000", 2024-05-11T23:03:50.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046836056000000001 2024-05-11T23:03:50.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:50.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019203+0000", 2024-05-11T23:03:50.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-05 2024-05-11T23:03:50.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:50.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019304+0000", 2024-05-11T23:03:50.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101537 2024-05-11T23:03:50.427 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.427 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.427 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.427 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:105234 3.15 3:adf885e5:::rbd_object_map.111e913a615f.0000000000000025:head [writefull 0~39] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-11T23:03:50.427 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:50.427 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 80.343282086000002, 2024-05-11T23:03:50.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77797599100000003, 2024-05-11T23:03:50.427 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:50.428 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:50.428 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:50.428 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:50.428 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:50.428 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:50.428 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 105234 2024-05-11T23:03:50.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.428 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:50.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:50.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:50.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:50.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:50.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:50.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007623+0000", 2024-05-11T23:03:50.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2240000000000001e-06 2024-05-11T23:03:50.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:50.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007627+0000", 2024-05-11T23:03:50.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:03:50.429 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.429 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.429 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:50.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007636+0000", 2024-05-11T23:03:50.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2409999999999995e-06 2024-05-11T23:03:50.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:50.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007641+0000", 2024-05-11T23:03:50.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6810000000000003e-06 2024-05-11T23:03:50.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007676+0000", 2024-05-11T23:03:50.430 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5061999999999997e-05 2024-05-11T23:03:50.430 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.430 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.430 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:03:50.430 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007682+0000", 2024-05-11T23:03:50.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0399999999999998e-06 2024-05-11T23:03:50.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783880+0000", 2024-05-11T23:03:50.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77619752900000005 2024-05-11T23:03:50.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:50.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783995+0000", 2024-05-11T23:03:50.431 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011550800000000001 2024-05-11T23:03:50.431 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.431 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.431 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:03:50.431 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784082+0000", 2024-05-11T23:03:50.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6347999999999994e-05 2024-05-11T23:03:50.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:50.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784682+0000", 2024-05-11T23:03:50.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000600162 2024-05-11T23:03:50.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:50.432 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785539+0000", 2024-05-11T23:03:50.432 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085699099999999998 2024-05-11T23:03:50.432 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.432 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.432 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:50.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785572+0000", 2024-05-11T23:03:50.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3182999999999997e-05 2024-05-11T23:03:50.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.433 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:50.433 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785598+0000", 2024-05-11T23:03:50.433 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5522000000000001e-05 2024-05-11T23:03:50.433 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.433 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.433 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.433 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.433 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.433 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113761 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [call rbd.sparse_copyup in=999496b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:03:50.433 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:50.433 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 75.820595166999993, 2024-05-11T23:03:50.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048572964000000003, 2024-05-11T23:03:50.434 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:50.434 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:50.434 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:50.434 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:50.434 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:50.434 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:50.434 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113761 2024-05-11T23:03:50.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.434 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:50.434 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.434 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:50.434 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:50.434 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.434 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:50.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:50.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:50.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530309+0000", 2024-05-11T23:03:50.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:03:50.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.435 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:50.435 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531242+0000", 2024-05-11T23:03:50.435 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093316899999999997 2024-05-11T23:03:50.435 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.435 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:50.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531249+0000", 2024-05-11T23:03:50.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8140000000000003e-06 2024-05-11T23:03:50.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:50.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531253+0000", 2024-05-11T23:03:50.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4290000000000003e-06 2024-05-11T23:03:50.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.436 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.436 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570325+0000", 2024-05-11T23:03:50.436 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039071379000000003 2024-05-11T23:03:50.436 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.436 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:50.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570382+0000", 2024-05-11T23:03:50.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6988999999999999e-05 2024-05-11T23:03:50.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:03:50.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570555+0000", 2024-05-11T23:03:50.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017379799999999999 2024-05-11T23:03:50.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.437 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.437 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:50.437 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.575953+0000", 2024-05-11T23:03:50.437 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053975539999999997 2024-05-11T23:03:50.437 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:50.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578778+0000", 2024-05-11T23:03:50.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002825616 2024-05-11T23:03:50.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:50.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578837+0000", 2024-05-11T23:03:50.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9005000000000002e-05 2024-05-11T23:03:50.438 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.438 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.438 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:50.438 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578881+0000", 2024-05-11T23:03:50.438 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3970000000000001e-05 2024-05-11T23:03:50.439 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.439 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.439 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.439 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.439 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.439 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113762 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [delete] snapc 27=[27] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:03:50.439 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:50.439 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 75.819641215999994, 2024-05-11T23:03:50.439 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047916260000000002, 2024-05-11T23:03:50.439 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:50.439 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:50.439 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:50.439 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:50.439 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:50.439 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:50.439 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113762 2024-05-11T23:03:50.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.440 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:50.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:50.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:50.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:50.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:50.440 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:50.440 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.440 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.440 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:50.440 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531263+0000", 2024-05-11T23:03:50.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.85e-07 2024-05-11T23:03:50.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:50.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531265+0000", 2024-05-11T23:03:50.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.827e-06 2024-05-11T23:03:50.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:50.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531268+0000", 2024-05-11T23:03:50.441 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3780000000000001e-06 2024-05-11T23:03:50.441 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.441 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.441 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:50.441 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531271+0000", 2024-05-11T23:03:50.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8509999999999999e-06 2024-05-11T23:03:50.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:50.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572298+0000", 2024-05-11T23:03:50.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041026990999999999 2024-05-11T23:03:50.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:50.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572345+0000", 2024-05-11T23:03:50.442 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6878000000000001e-05 2024-05-11T23:03:50.442 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.442 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.442 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:03:50.442 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572443+0000", 2024-05-11T23:03:50.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8184999999999996e-05 2024-05-11T23:03:50.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:50.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578585+0000", 2024-05-11T23:03:50.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006141919 2024-05-11T23:03:50.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:50.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579126+0000", 2024-05-11T23:03:50.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00054063799999999997 2024-05-11T23:03:50.443 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.443 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.443 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:50.443 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579154+0000", 2024-05-11T23:03:50.443 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8042999999999999e-05 2024-05-11T23:03:50.444 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:50.444 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:50.444 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:50.444 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579179+0000", 2024-05-11T23:03:50.444 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4765000000000001e-05 2024-05-11T23:03:50.444 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.444 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.444 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.444 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:50.444 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:50.444 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:50.446 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:03:50.447 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:50.445+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:50.547 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:50.547+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:50.561 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:50.561 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:50.570 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:03:50.585 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:03:50.657 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:50.657+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:50.686 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:50.687 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:50.687 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:50.687 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:50.687 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:50.696 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:03:50.746 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:50.747 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:50.748 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:50.748+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:50.766 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:50.766 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:50.784 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:50.784 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:50.784 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-05-11T23:03:50.788 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-05-11T23:03:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:marked out osd.0. 2024-05-11T23:03:50.816 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:50.817 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:50.835 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:50.835 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:50.849 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:50.850+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:50.853 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:50.853 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:50.861 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:50.861 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:50.871 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:03:50.871 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:50.871 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:50.873 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-11T23:03:50.874 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-11T23:03:50.957 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:50.957+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:51.023 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:51.023 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:51.032 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:03:51.051 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:51.051+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:51.151 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:51.151+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:51.182 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:51.182 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:51.191 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:03:51.252 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:51.253+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:51.341 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:51.341 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:51.350 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:03:51.394 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:51.394+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:51.454 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:51.455+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:51.459 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:51.459 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:51.468 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:03:51.555 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:51.556+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:51.617 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:51.617 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:51.626 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:03:51.656 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:51.657+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:51.736 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:51.736 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:51.745 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:03:51.757 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:51.758+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:51.859 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:51.859+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:51.866 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:51.866 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:51.875 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:03:51.960 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:51.961+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:52.040 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:52.040 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:52.050 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:03:52.093 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:52.094+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:52.161 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:52.162+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:52.165 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:52.165 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:52.175 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:03:52.261 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:52.262+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:52.324 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:52.324 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:52.332 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:03:52.373 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:52.374+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:52.464 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:52.465+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:52.478 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:52.478 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:52.487 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:03:52.571 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:52.572+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:52.632 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:52.633 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:52.641 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:03:52.682 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:52.683+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:52.766 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:52.767+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:52.786 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:52.786 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:52.801 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:03:52.812 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-11T23:03:52.867 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:52.867+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:52.875 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:03:52.968 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:52.969+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:52.989 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:52.989 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:53.001 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:03:53.044 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:53.045 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:53.066 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:53.066 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:53.068 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:53.069+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:53.096 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:53.096 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:53.096 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-05-11T23:03:53.114 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:53.115 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:53.132 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:53.132 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:53.149 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:53.149 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:53.167 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:53.168 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:53.170 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:53.171+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:53.171 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-11T23:03:53.172 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-11T23:03:53.178 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:53.178 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:53.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 120.52356303800001, 2024-05-11T23:03:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:03:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:03:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.179 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:53.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:03:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:03:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:03:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:03:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:03:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:03:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:03:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:03:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:03:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:03:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:03:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:03:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:03:53.182 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 120.520181849, 2024-05-11T23:03:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:03:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:03:53.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.184 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:53.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:53.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:03:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:03:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:03:53.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:03:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:03:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:03:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:03:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:03:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:03:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:03:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:03:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:03:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:53.187 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 120.520163142, 2024-05-11T23:03:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:03:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:03:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:03:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:03:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:03:53.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:03:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:03:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:03:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:03:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:03:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:03:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:03:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:03:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:03:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:03:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:03:53.191 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 120.520144999, 2024-05-11T23:03:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:03:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.192 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:03:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:03:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:03:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:03:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:03:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:03:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:03:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:03:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:03:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:03:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:03:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:03:53.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:03:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:03:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 120.520117512, 2024-05-11T23:03:53.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:03:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:03:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:03:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:03:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:03:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:03:53.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:03:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:03:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:03:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:03:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:03:53.199 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:03:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:03:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:03:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:03:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:03:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:53.200 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 120.52009698400001, 2024-05-11T23:03:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:03:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:03:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:03:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:03:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:03:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:03:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:03:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:03:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:03:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:03:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:03:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:03:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:03:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:03:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:03:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 120.52007720899999, 2024-05-11T23:03:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:03:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:03:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:03:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:03:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:03:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:03:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:03:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:03:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:03:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:03:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:03:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:03:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:03:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:03:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:03:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:03:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 120.52005840299999, 2024-05-11T23:03:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:03:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.209 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:03:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:03:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:03:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:03:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:03:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:03:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:03:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:03:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:03:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:03:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:03:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:03:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:03:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:03:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:03:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:53.213 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 120.52003947999999, 2024-05-11T23:03:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:03:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:03:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:53.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:53.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:03:53.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:03:53.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:03:53.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:03:53.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:03:53.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:03:53.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:03:53.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:03:53.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:03:53.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:03:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:03:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:03:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:03:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:53.217 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 120.52002081800001, 2024-05-11T23:03:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:03:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:03:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:53.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:03:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:03:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:03:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:03:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:03:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:03:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:03:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:03:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:03:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:03:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:03:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:03:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:03:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.221 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 120.519675656, 2024-05-11T23:03:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:03:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.222 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:03:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:03:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:03:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:03:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:03:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:03:53.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:03:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:03:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:03:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:03:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:03:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:03:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:03:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:03:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:03:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:53.226 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 120.519650106, 2024-05-11T23:03:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:03:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:03:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:53.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:03:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:03:53.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:03:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:03:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:03:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:03:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:03:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:03:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:03:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:03:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:03:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:03:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:03:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.230 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 120.51956784799999, 2024-05-11T23:03:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:03:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:03:53.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:03:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:03:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:03:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:03:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:03:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:03:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:03:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:03:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:03:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:03:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:03:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:03:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:03:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 120.519545401, 2024-05-11T23:03:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:03:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.235 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:03:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:03:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:03:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:03:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:03:53.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:03:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:03:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:03:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:03:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:03:53.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:03:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:03:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:03:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:03:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:03:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.239 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 120.519525943, 2024-05-11T23:03:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:03:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:03:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:03:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:03:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:03:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:03:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:03:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:03:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:03:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:03:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:03:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:03:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:03:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:03:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:03:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 120.51950588299999, 2024-05-11T23:03:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:03:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:03:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:53.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:03:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:03:53.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:03:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:03:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:03:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:03:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:03:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:03:53.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:03:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:03:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:03:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:03:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:53.247 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 120.519486613, 2024-05-11T23:03:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:03:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:03:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:03:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:03:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:03:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:03:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:03:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:03:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:03:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:03:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:03:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:03:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:03:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:03:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:03:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.251 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 120.518940547, 2024-05-11T23:03:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:03:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:03:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:03:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:03:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:03:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:03:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:03:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:03:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:03:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:03:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.254 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:03:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:03:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:03:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:03:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:03:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:03:53.255 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 120.518908763, 2024-05-11T23:03:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:03:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.256 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:03:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.257 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:03:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:03:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:03:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:03:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:03:53.258 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:03:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:03:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:03:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:03:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.259 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:03:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:03:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:03:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:03:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:03:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 111.081663299, 2024-05-11T23:03:53.260 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:03:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:03:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.261 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:03:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:03:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:03:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:03:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.262 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:03:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:03:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:03:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:03:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:03:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:03:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.263 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:03:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:03:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:03:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:03:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:03:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:03:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:03:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.264 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:03:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:03:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:03:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:03:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:03:53.265 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:03:53.266 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.266 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.266 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.266 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.266 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.266 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:53.267 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:03:53.308 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:53.309+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:53.371 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:53.371+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:53.427 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:53.427 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:53.427 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:53.427 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:53.436 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:03:53.481 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:53.482+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:53.552 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:53.552 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:53.552 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:53.552 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:53.552 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:53.562 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:03:53.573 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:53.573+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:53.674 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:53.675+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:53.713 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:03:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 156.54876630000001, 2024-05-11T23:03:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:03:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.713 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:03:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:03:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:03:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:03:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.714 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:03:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:03:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:03:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:03:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:03:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:03:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.715 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:03:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:03:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:03:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:03:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:03:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:03:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:03:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:03:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:03:53.716 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:03:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:03:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:03:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:03:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:03:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:03:53.717 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:03:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:03:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:03:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:03:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:03:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:03:53.718 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:03:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78429 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 97.747496957999999, 2024-05-11T23:03:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044563497000000001, 2024-05-11T23:03:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.719 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78429 2024-05-11T23:03:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:03:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.720 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964872+0000", 2024-05-11T23:03:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035070300000000001 2024-05-11T23:03:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964876+0000", 2024-05-11T23:03:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5009999999999998e-06 2024-05-11T23:03:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964902+0000", 2024-05-11T23:03:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5715999999999999e-05 2024-05-11T23:03:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.721 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964910+0000", 2024-05-11T23:03:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3930000000000002e-06 2024-05-11T23:03:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008898+0000", 2024-05-11T23:03:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043988772000000002 2024-05-11T23:03:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008981+0000", 2024-05-11T23:03:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3140999999999996e-05 2024-05-11T23:03:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.722 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009085+0000", 2024-05-11T23:03:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010327099999999999 2024-05-11T23:03:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78431 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 97.747036649999998, 2024-05-11T23:03:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044542033000000002, 2024-05-11T23:03:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.723 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78431 2024-05-11T23:03:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:03:53.724 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965038+0000", 2024-05-11T23:03:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6993000000000001e-05 2024-05-11T23:03:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965043+0000", 2024-05-11T23:03:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7040000000000002e-06 2024-05-11T23:03:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.725 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965053+0000", 2024-05-11T23:03:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8539999999999992e-06 2024-05-11T23:03:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965061+0000", 2024-05-11T23:03:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9810000000000003e-06 2024-05-11T23:03:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009350+0000", 2024-05-11T23:03:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044288739000000001 2024-05-11T23:03:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.726 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009429+0000", 2024-05-11T23:03:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9354e-05 2024-05-11T23:03:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009524+0000", 2024-05-11T23:03:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4407999999999995e-05 2024-05-11T23:03:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78432 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:53.727 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 97.746945801999999, 2024-05-11T23:03:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044635421000000002, 2024-05-11T23:03:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78432 2024-05-11T23:03:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.728 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:03:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965343+0000", 2024-05-11T23:03:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027056500000000001 2024-05-11T23:03:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965347+0000", 2024-05-11T23:03:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3540000000000002e-06 2024-05-11T23:03:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.729 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965356+0000", 2024-05-11T23:03:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2140000000000002e-06 2024-05-11T23:03:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965363+0000", 2024-05-11T23:03:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4150000000000004e-06 2024-05-11T23:03:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009551+0000", 2024-05-11T23:03:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044187917 2024-05-11T23:03:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.730 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009626+0000", 2024-05-11T23:03:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5024000000000005e-05 2024-05-11T23:03:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009708+0000", 2024-05-11T23:03:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1932000000000001e-05 2024-05-11T23:03:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78433 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:53.731 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 97.746644915000005, 2024-05-11T23:03:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044564864000000003, 2024-05-11T23:03:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78433 2024-05-11T23:03:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:53.732 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:03:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966225+0000", 2024-05-11T23:03:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085220700000000001 2024-05-11T23:03:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966230+0000", 2024-05-11T23:03:53.733 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0690000000000003e-06 2024-05-11T23:03:53.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966239+0000", 2024-05-11T23:03:53.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-05-11T23:03:53.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966245+0000", 2024-05-11T23:03:53.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5520000000000001e-06 2024-05-11T23:03:53.734 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.734 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.734 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.734 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009743+0000", 2024-05-11T23:03:53.734 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043498311999999997 2024-05-11T23:03:53.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009848+0000", 2024-05-11T23:03:53.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104885 2024-05-11T23:03:53.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.735 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.735 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009938+0000", 2024-05-11T23:03:53.735 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9812e-05 2024-05-11T23:03:53.735 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.735 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.735 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.735 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.735 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.735 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78580 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 999424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:53.736 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:53.736 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 97.743532232000007, 2024-05-11T23:03:53.736 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044493076999999999, 2024-05-11T23:03:53.736 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.736 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.736 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.736 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.736 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.736 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.736 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78580 2024-05-11T23:03:53.736 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.736 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.736 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.736 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.736 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:53.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:53.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.737 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:03:53.737 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:03:53.737 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.737 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.737 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968489+0000", 2024-05-11T23:03:53.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.52e-06 2024-05-11T23:03:53.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968496+0000", 2024-05-11T23:03:53.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9419999999999998e-06 2024-05-11T23:03:53.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.738 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.738 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968501+0000", 2024-05-11T23:03:53.738 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8720000000000001e-06 2024-05-11T23:03:53.738 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.738 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012690+0000", 2024-05-11T23:03:53.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044189655000000001 2024-05-11T23:03:53.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012848+0000", 2024-05-11T23:03:53.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015758100000000001 2024-05-11T23:03:53.739 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.739 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.739 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.739 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012979+0000", 2024-05-11T23:03:53.739 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000131066 2024-05-11T23:03:53.739 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.740 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.740 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.740 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.740 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.740 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78581 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:53.740 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:53.740 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 97.743511228000003, 2024-05-11T23:03:53.740 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044616243, 2024-05-11T23:03:53.740 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.740 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.740 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.740 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.741 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78581 2024-05-11T23:03:53.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.741 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:53.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.741 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:03:53.741 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.741 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.741 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.741 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970227+0000", 2024-05-11T23:03:53.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017200049999999999 2024-05-11T23:03:53.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970232+0000", 2024-05-11T23:03:53.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000003e-06 2024-05-11T23:03:53.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.742 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970242+0000", 2024-05-11T23:03:53.742 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0192e-05 2024-05-11T23:03:53.742 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.742 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.742 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970249+0000", 2024-05-11T23:03:53.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.888e-06 2024-05-11T23:03:53.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013008+0000", 2024-05-11T23:03:53.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042759495000000002 2024-05-11T23:03:53.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.743 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013063+0000", 2024-05-11T23:03:53.743 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5071999999999997e-05 2024-05-11T23:03:53.743 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.743 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.743 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.744 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013123+0000", 2024-05-11T23:03:53.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9930999999999999e-05 2024-05-11T23:03:53.744 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.744 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.744 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.744 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.744 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78589 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:53.744 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:53.744 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 97.741591983999996, 2024-05-11T23:03:53.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044587697000000003, 2024-05-11T23:03:53.744 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.744 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.745 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78589 2024-05-11T23:03:53.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.745 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:53.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:03:53.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970427+0000", 2024-05-11T23:03:53.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:03:53.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970429+0000", 2024-05-11T23:03:53.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3800000000000001e-06 2024-05-11T23:03:53.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970435+0000", 2024-05-11T23:03:53.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2530000000000001e-06 2024-05-11T23:03:53.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970441+0000", 2024-05-11T23:03:53.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-05-11T23:03:53.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014822+0000", 2024-05-11T23:03:53.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04438168 2024-05-11T23:03:53.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014911+0000", 2024-05-11T23:03:53.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8882999999999996e-05 2024-05-11T23:03:53.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015014+0000", 2024-05-11T23:03:53.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010263399999999999 2024-05-11T23:03:53.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.748 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.748 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78590 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:53.748 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:53.748 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 97.741571184999998, 2024-05-11T23:03:53.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044880120000000003, 2024-05-11T23:03:53.748 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.748 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.748 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.749 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78590 2024-05-11T23:03:53.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.749 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:53.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:03:53.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970918+0000", 2024-05-11T23:03:53.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000470834 2024-05-11T23:03:53.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970925+0000", 2024-05-11T23:03:53.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1690000000000003e-06 2024-05-11T23:03:53.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970934+0000", 2024-05-11T23:03:53.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9099999999999994e-06 2024-05-11T23:03:53.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970941+0000", 2024-05-11T23:03:53.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7410000000000003e-06 2024-05-11T23:03:53.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015049+0000", 2024-05-11T23:03:53.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044108392000000003 2024-05-11T23:03:53.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015133+0000", 2024-05-11T23:03:53.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3568999999999999e-05 2024-05-11T23:03:53.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015327+0000", 2024-05-11T23:03:53.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019450500000000001 2024-05-11T23:03:53.752 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.752 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.752 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.752 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78591 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:53.752 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:53.752 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 97.741067768999997, 2024-05-11T23:03:53.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044618497, 2024-05-11T23:03:53.752 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.752 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.752 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.752 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.753 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.753 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.753 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78591 2024-05-11T23:03:53.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.753 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:53.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:03:53.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970951+0000", 2024-05-11T23:03:53.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:03:53.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970953+0000", 2024-05-11T23:03:53.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:03:53.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970956+0000", 2024-05-11T23:03:53.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-05-11T23:03:53.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970959+0000", 2024-05-11T23:03:53.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9409999999999999e-06 2024-05-11T23:03:53.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015367+0000", 2024-05-11T23:03:53.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044407621000000001 2024-05-11T23:03:53.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015453+0000", 2024-05-11T23:03:53.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6685000000000001e-05 2024-05-11T23:03:53.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015569+0000", 2024-05-11T23:03:53.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115396 2024-05-11T23:03:53.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.756 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.756 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78592 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:53.756 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:53.756 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 97.741054961000003, 2024-05-11T23:03:53.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044831546999999999, 2024-05-11T23:03:53.756 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.756 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.756 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.756 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.756 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78592 2024-05-11T23:03:53.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.757 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:53.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:03:53.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972226+0000", 2024-05-11T23:03:53.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00126246 2024-05-11T23:03:53.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972230+0000", 2024-05-11T23:03:53.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1860000000000002e-06 2024-05-11T23:03:53.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972237+0000", 2024-05-11T23:03:53.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:03:53.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972241+0000", 2024-05-11T23:03:53.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.284e-06 2024-05-11T23:03:53.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015611+0000", 2024-05-11T23:03:53.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043369461999999998 2024-05-11T23:03:53.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015695+0000", 2024-05-11T23:03:53.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4458999999999999e-05 2024-05-11T23:03:53.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015795+0000", 2024-05-11T23:03:53.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9475999999999994e-05 2024-05-11T23:03:53.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.760 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.760 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78593 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:53.760 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:53.760 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 97.739768041999994, 2024-05-11T23:03:53.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046045214000000001, 2024-05-11T23:03:53.760 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.760 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.760 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.760 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.760 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.761 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78593 2024-05-11T23:03:53.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.761 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:53.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.761 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:53.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:03:53.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7599999999999998e-07 2024-05-11T23:03:53.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972252+0000", 2024-05-11T23:03:53.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7349999999999999e-06 2024-05-11T23:03:53.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972256+0000", 2024-05-11T23:03:53.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5659999999999998e-06 2024-05-11T23:03:53.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972259+0000", 2024-05-11T23:03:53.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8040000000000002e-06 2024-05-11T23:03:53.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018088+0000", 2024-05-11T23:03:53.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045829316000000002 2024-05-11T23:03:53.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018179+0000", 2024-05-11T23:03:53.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1299999999999997e-05 2024-05-11T23:03:53.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018295+0000", 2024-05-11T23:03:53.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116117 2024-05-11T23:03:53.764 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.764 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.764 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.764 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78594 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:53.764 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:53.764 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 97.739755681999995, 2024-05-11T23:03:53.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046245204999999998, 2024-05-11T23:03:53.765 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.765 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.765 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.765 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.765 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.765 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.765 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78594 2024-05-11T23:03:53.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.765 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.765 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.765 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:53.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:03:53.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972263+0000", 2024-05-11T23:03:53.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:03:53.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972264+0000", 2024-05-11T23:03:53.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.206e-06 2024-05-11T23:03:53.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972267+0000", 2024-05-11T23:03:53.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0690000000000002e-06 2024-05-11T23:03:53.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972270+0000", 2024-05-11T23:03:53.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5610000000000001e-06 2024-05-11T23:03:53.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018330+0000", 2024-05-11T23:03:53.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04606085 2024-05-11T23:03:53.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018414+0000", 2024-05-11T23:03:53.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3125000000000001e-05 2024-05-11T23:03:53.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018508+0000", 2024-05-11T23:03:53.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4153000000000005e-05 2024-05-11T23:03:53.768 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.768 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.768 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.769 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78595 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:53.769 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:53.769 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 97.739742817000007, 2024-05-11T23:03:53.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046396698, 2024-05-11T23:03:53.769 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.769 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.769 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.769 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.769 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.769 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.769 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78595 2024-05-11T23:03:53.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.769 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.769 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.769 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:53.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:03:53.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972276+0000", 2024-05-11T23:03:53.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5600000000000002e-07 2024-05-11T23:03:53.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972277+0000", 2024-05-11T23:03:53.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.576e-06 2024-05-11T23:03:53.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972280+0000", 2024-05-11T23:03:53.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-05-11T23:03:53.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972283+0000", 2024-05-11T23:03:53.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:03:53.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018538+0000", 2024-05-11T23:03:53.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046255255000000002 2024-05-11T23:03:53.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018605+0000", 2024-05-11T23:03:53.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6847000000000006e-05 2024-05-11T23:03:53.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018672+0000", 2024-05-11T23:03:53.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7045000000000006e-05 2024-05-11T23:03:53.772 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.772 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.772 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.772 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78596 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:53.773 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:53.773 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 97.739731968000001, 2024-05-11T23:03:53.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046586806000000001, 2024-05-11T23:03:53.773 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.773 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.773 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.773 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.773 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.773 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.773 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78596 2024-05-11T23:03:53.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.773 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:53.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:53.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:03:53.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:03:53.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972288+0000", 2024-05-11T23:03:53.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2550000000000001e-06 2024-05-11T23:03:53.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972291+0000", 2024-05-11T23:03:53.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9490000000000001e-06 2024-05-11T23:03:53.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972293+0000", 2024-05-11T23:03:53.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5069999999999999e-06 2024-05-11T23:03:53.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018707+0000", 2024-05-11T23:03:53.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046413604999999997 2024-05-11T23:03:53.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018782+0000", 2024-05-11T23:03:53.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5556000000000006e-05 2024-05-11T23:03:53.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018873+0000", 2024-05-11T23:03:53.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0703000000000002e-05 2024-05-11T23:03:53.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.776 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.776 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.776 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78597 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:53.777 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:53.777 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 97.739722194999999, 2024-05-11T23:03:53.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046825787000000001, 2024-05-11T23:03:53.777 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.777 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.777 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.777 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.777 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78597 2024-05-11T23:03:53.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.777 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:53.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:53.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:03:53.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-05-11T23:03:53.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972297+0000", 2024-05-11T23:03:53.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3069999999999999e-06 2024-05-11T23:03:53.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972300+0000", 2024-05-11T23:03:53.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:03:53.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972303+0000", 2024-05-11T23:03:53.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5509999999999998e-06 2024-05-11T23:03:53.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018911+0000", 2024-05-11T23:03:53.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046608194999999998 2024-05-11T23:03:53.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018986+0000", 2024-05-11T23:03:53.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5360999999999999e-05 2024-05-11T23:03:53.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019122+0000", 2024-05-11T23:03:53.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013526 2024-05-11T23:03:53.780 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.780 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.780 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.780 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78598 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:03:53.780 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:53.781 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 97.739712295000004, 2024-05-11T23:03:53.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046998572000000002, 2024-05-11T23:03:53.781 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.781 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.781 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:53.781 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.781 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.781 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.781 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78598 2024-05-11T23:03:53.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.781 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:53.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:53.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:03:53.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:03:53.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972307+0000", 2024-05-11T23:03:53.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1799999999999999e-06 2024-05-11T23:03:53.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972310+0000", 2024-05-11T23:03:53.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:03:53.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972313+0000", 2024-05-11T23:03:53.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4760000000000001e-06 2024-05-11T23:03:53.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019149+0000", 2024-05-11T23:03:53.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046836056000000001 2024-05-11T23:03:53.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019203+0000", 2024-05-11T23:03:53.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-05 2024-05-11T23:03:53.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019304+0000", 2024-05-11T23:03:53.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101537 2024-05-11T23:03:53.784 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.784 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.784 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.784 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:105234 3.15 3:adf885e5:::rbd_object_map.111e913a615f.0000000000000025:head [writefull 0~39] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-11T23:03:53.784 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:53.784 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.704396584999998, 2024-05-11T23:03:53.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77797599100000003, 2024-05-11T23:03:53.784 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.784 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.785 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:53.785 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.785 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.785 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.785 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 105234 2024-05-11T23:03:53.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.785 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:53.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:03:53.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007623+0000", 2024-05-11T23:03:53.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2240000000000001e-06 2024-05-11T23:03:53.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007627+0000", 2024-05-11T23:03:53.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:03:53.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.786 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.786 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007636+0000", 2024-05-11T23:03:53.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2409999999999995e-06 2024-05-11T23:03:53.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007641+0000", 2024-05-11T23:03:53.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6810000000000003e-06 2024-05-11T23:03:53.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007676+0000", 2024-05-11T23:03:53.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5061999999999997e-05 2024-05-11T23:03:53.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:03:53.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007682+0000", 2024-05-11T23:03:53.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0399999999999998e-06 2024-05-11T23:03:53.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783880+0000", 2024-05-11T23:03:53.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77619752900000005 2024-05-11T23:03:53.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783995+0000", 2024-05-11T23:03:53.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011550800000000001 2024-05-11T23:03:53.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:03:53.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784082+0000", 2024-05-11T23:03:53.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6347999999999994e-05 2024-05-11T23:03:53.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:53.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784682+0000", 2024-05-11T23:03:53.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000600162 2024-05-11T23:03:53.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:53.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785539+0000", 2024-05-11T23:03:53.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085699099999999998 2024-05-11T23:03:53.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:53.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785572+0000", 2024-05-11T23:03:53.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3182999999999997e-05 2024-05-11T23:03:53.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.790 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.790 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785598+0000", 2024-05-11T23:03:53.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5522000000000001e-05 2024-05-11T23:03:53.790 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.790 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.790 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.790 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113761 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [call rbd.sparse_copyup in=999496b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:03:53.791 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:53.791 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.181709666000003, 2024-05-11T23:03:53.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048572964000000003, 2024-05-11T23:03:53.791 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.791 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.791 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:53.791 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.791 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.791 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.791 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113761 2024-05-11T23:03:53.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.791 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:53.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:03:53.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530309+0000", 2024-05-11T23:03:53.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:03:53.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531242+0000", 2024-05-11T23:03:53.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093316899999999997 2024-05-11T23:03:53.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531249+0000", 2024-05-11T23:03:53.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8140000000000003e-06 2024-05-11T23:03:53.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531253+0000", 2024-05-11T23:03:53.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4290000000000003e-06 2024-05-11T23:03:53.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570325+0000", 2024-05-11T23:03:53.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039071379000000003 2024-05-11T23:03:53.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570382+0000", 2024-05-11T23:03:53.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6988999999999999e-05 2024-05-11T23:03:53.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:03:53.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570555+0000", 2024-05-11T23:03:53.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017379799999999999 2024-05-11T23:03:53.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:53.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.575953+0000", 2024-05-11T23:03:53.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053975539999999997 2024-05-11T23:03:53.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:53.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578778+0000", 2024-05-11T23:03:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002825616 2024-05-11T23:03:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578837+0000", 2024-05-11T23:03:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9005000000000002e-05 2024-05-11T23:03:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578881+0000", 2024-05-11T23:03:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3970000000000001e-05 2024-05-11T23:03:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113762 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [delete] snapc 27=[27] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:03:53.795 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 79.180755715000004, 2024-05-11T23:03:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047916260000000002, 2024-05-11T23:03:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113762 2024-05-11T23:03:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:03:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531263+0000", 2024-05-11T23:03:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.85e-07 2024-05-11T23:03:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531265+0000", 2024-05-11T23:03:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.827e-06 2024-05-11T23:03:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531268+0000", 2024-05-11T23:03:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3780000000000001e-06 2024-05-11T23:03:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531271+0000", 2024-05-11T23:03:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8509999999999999e-06 2024-05-11T23:03:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572298+0000", 2024-05-11T23:03:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041026990999999999 2024-05-11T23:03:53.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572345+0000", 2024-05-11T23:03:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6878000000000001e-05 2024-05-11T23:03:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:03:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572443+0000", 2024-05-11T23:03:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8184999999999996e-05 2024-05-11T23:03:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578585+0000", 2024-05-11T23:03:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006141919 2024-05-11T23:03:53.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579126+0000", 2024-05-11T23:03:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00054063799999999997 2024-05-11T23:03:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579154+0000", 2024-05-11T23:03:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8042999999999999e-05 2024-05-11T23:03:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579179+0000", 2024-05-11T23:03:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4765000000000001e-05 2024-05-11T23:03:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.800 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.801 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:53.801 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:53.801 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-05-11T23:03:53.802 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:53.803+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:53.803 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:03:53.903 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:53.903+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:53.917 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:53.917 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:53.917 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:53.917 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:53.926 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:03:54.004 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:54.005+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:54.078 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:54.078 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:54.079 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:54.085 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:03:54.129 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:54.129+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:54.197 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:54.197 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:54.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:54.197 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:54.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.197 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226051 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6049792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:54.197 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:54.197 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.878977618, 2024-05-11T23:03:54.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050622418000000002, 2024-05-11T23:03:54.198 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.198 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.198 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.198 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.198 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.198 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.198 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226051 2024-05-11T23:03:54.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.198 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:54.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:03:54.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317156+0000", 2024-05-11T23:03:54.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:54.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317157+0000", 2024-05-11T23:03:54.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.144e-06 2024-05-11T23:03:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317160+0000", 2024-05-11T23:03:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1350000000000001e-06 2024-05-11T23:03:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317162+0000", 2024-05-11T23:03:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5050000000000002e-06 2024-05-11T23:03:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367689+0000", 2024-05-11T23:03:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526797999999998 2024-05-11T23:03:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367735+0000", 2024-05-11T23:03:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5698999999999997e-05 2024-05-11T23:03:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367778+0000", 2024-05-11T23:03:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2928e-05 2024-05-11T23:03:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226052 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6060032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.878967562, 2024-05-11T23:03:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050830110999999997, 2024-05-11T23:03:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226052 2024-05-11T23:03:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:03:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317166+0000", 2024-05-11T23:03:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:03:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317167+0000", 2024-05-11T23:03:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.155e-06 2024-05-11T23:03:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317170+0000", 2024-05-11T23:03:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.061e-06 2024-05-11T23:03:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317172+0000", 2024-05-11T23:03:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4420000000000001e-06 2024-05-11T23:03:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367816+0000", 2024-05-11T23:03:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050644013000000002 2024-05-11T23:03:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367906+0000", 2024-05-11T23:03:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9739000000000003e-05 2024-05-11T23:03:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367995+0000", 2024-05-11T23:03:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9499000000000005e-05 2024-05-11T23:03:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226053 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6070272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.878957522999997, 2024-05-11T23:03:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051106912999999997, 2024-05-11T23:03:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226053 2024-05-11T23:03:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:03:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317176+0000", 2024-05-11T23:03:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0100000000000001e-07 2024-05-11T23:03:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317179+0000", 2024-05-11T23:03:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-05-11T23:03:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317182+0000", 2024-05-11T23:03:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3589999999999999e-06 2024-05-11T23:03:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317184+0000", 2024-05-11T23:03:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4339999999999999e-06 2024-05-11T23:03:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368061+0000", 2024-05-11T23:03:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050876927000000002 2024-05-11T23:03:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368179+0000", 2024-05-11T23:03:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118001 2024-05-11T23:03:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368282+0000", 2024-05-11T23:03:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102942 2024-05-11T23:03:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226054 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6080512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.878945201000001, 2024-05-11T23:03:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051322, 2024-05-11T23:03:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226054 2024-05-11T23:03:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:03:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317189+0000", 2024-05-11T23:03:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.217e-06 2024-05-11T23:03:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317192+0000", 2024-05-11T23:03:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0699999999999998e-06 2024-05-11T23:03:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317195+0000", 2024-05-11T23:03:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3829999999999999e-06 2024-05-11T23:03:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368328+0000", 2024-05-11T23:03:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051132999999999998 2024-05-11T23:03:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368429+0000", 2024-05-11T23:03:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010168800000000001 2024-05-11T23:03:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368510+0000", 2024-05-11T23:03:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0420999999999995e-05 2024-05-11T23:03:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226055 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6090752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.878935132999999, 2024-05-11T23:03:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051562835000000001, 2024-05-11T23:03:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226055 2024-05-11T23:03:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:03:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317256+0000", 2024-05-11T23:03:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8047000000000002e-05 2024-05-11T23:03:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317259+0000", 2024-05-11T23:03:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2590000000000001e-06 2024-05-11T23:03:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317267+0000", 2024-05-11T23:03:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8769999999999993e-06 2024-05-11T23:03:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317274+0000", 2024-05-11T23:03:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.187e-06 2024-05-11T23:03:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368544+0000", 2024-05-11T23:03:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051270299999999998 2024-05-11T23:03:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368658+0000", 2024-05-11T23:03:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000113347 2024-05-11T23:03:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368761+0000", 2024-05-11T23:03:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102818 2024-05-11T23:03:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226056 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6100992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.878849287000001, 2024-05-11T23:03:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051615576000000003, 2024-05-11T23:03:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226056 2024-05-11T23:03:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:03:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318915+0000", 2024-05-11T23:03:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001631291 2024-05-11T23:03:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318918+0000", 2024-05-11T23:03:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.512e-06 2024-05-11T23:03:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318927+0000", 2024-05-11T23:03:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1960000000000008e-06 2024-05-11T23:03:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318932+0000", 2024-05-11T23:03:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:03:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368805+0000", 2024-05-11T23:03:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049872998000000002 2024-05-11T23:03:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368853+0000", 2024-05-11T23:03:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8041999999999998e-05 2024-05-11T23:03:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368899+0000", 2024-05-11T23:03:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5766000000000002e-05 2024-05-11T23:03:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226061 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6152192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.8770582, 2024-05-11T23:03:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050675395999999998, 2024-05-11T23:03:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226061 2024-05-11T23:03:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:03:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319078+0000", 2024-05-11T23:03:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5670000000000002e-06 2024-05-11T23:03:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319085+0000", 2024-05-11T23:03:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2570000000000001e-06 2024-05-11T23:03:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319100+0000", 2024-05-11T23:03:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.489e-05 2024-05-11T23:03:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369584+0000", 2024-05-11T23:03:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050484345999999999 2024-05-11T23:03:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369671+0000", 2024-05-11T23:03:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6654000000000001e-05 2024-05-11T23:03:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369750+0000", 2024-05-11T23:03:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9257000000000004e-05 2024-05-11T23:03:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226062 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6162432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.877024656000003, 2024-05-11T23:03:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050761158000000001, 2024-05-11T23:03:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226062 2024-05-11T23:03:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:03:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319109+0000", 2024-05-11T23:03:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2800000000000002e-07 2024-05-11T23:03:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319111+0000", 2024-05-11T23:03:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4980000000000001e-06 2024-05-11T23:03:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319120+0000", 2024-05-11T23:03:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3999999999999992e-06 2024-05-11T23:03:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319129+0000", 2024-05-11T23:03:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7580000000000004e-06 2024-05-11T23:03:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369775+0000", 2024-05-11T23:03:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050645580000000003 2024-05-11T23:03:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369824+0000", 2024-05-11T23:03:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9506999999999999e-05 2024-05-11T23:03:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369869+0000", 2024-05-11T23:03:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4987000000000001e-05 2024-05-11T23:03:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226063 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6172672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.876995698999998, 2024-05-11T23:03:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050939784000000002, 2024-05-11T23:03:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226063 2024-05-11T23:03:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:03:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319138+0000", 2024-05-11T23:03:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.39e-07 2024-05-11T23:03:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319140+0000", 2024-05-11T23:03:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4820000000000001e-06 2024-05-11T23:03:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319148+0000", 2024-05-11T23:03:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6599999999999995e-06 2024-05-11T23:03:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319154+0000", 2024-05-11T23:03:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4169999999999997e-06 2024-05-11T23:03:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369905+0000", 2024-05-11T23:03:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050751009999999999 2024-05-11T23:03:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369991+0000", 2024-05-11T23:03:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5909999999999996e-05 2024-05-11T23:03:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370077+0000", 2024-05-11T23:03:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5865999999999994e-05 2024-05-11T23:03:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226064 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6182912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.876969502999998, 2024-05-11T23:03:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051038241999999998, 2024-05-11T23:03:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226064 2024-05-11T23:03:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:03:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319164+0000", 2024-05-11T23:03:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9299999999999999e-07 2024-05-11T23:03:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319166+0000", 2024-05-11T23:03:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.48e-06 2024-05-11T23:03:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319176+0000", 2024-05-11T23:03:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9669999999999996e-06 2024-05-11T23:03:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319180+0000", 2024-05-11T23:03:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8020000000000002e-06 2024-05-11T23:03:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370100+0000", 2024-05-11T23:03:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050919845999999998 2024-05-11T23:03:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370155+0000", 2024-05-11T23:03:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5040000000000002e-05 2024-05-11T23:03:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370202+0000", 2024-05-11T23:03:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6714e-05 2024-05-11T23:03:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226065 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6193152~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.876946932999999, 2024-05-11T23:03:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051225087000000002, 2024-05-11T23:03:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226065 2024-05-11T23:03:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:03:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320913+0000", 2024-05-11T23:03:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001727492 2024-05-11T23:03:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320919+0000", 2024-05-11T23:03:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.964e-06 2024-05-11T23:03:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320927+0000", 2024-05-11T23:03:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2239999999999998e-06 2024-05-11T23:03:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320934+0000", 2024-05-11T23:03:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8469999999999998e-06 2024-05-11T23:03:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370240+0000", 2024-05-11T23:03:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049306562999999998 2024-05-11T23:03:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370328+0000", 2024-05-11T23:03:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8122999999999996e-05 2024-05-11T23:03:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370411+0000", 2024-05-11T23:03:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2874000000000006e-05 2024-05-11T23:03:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226073 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6275072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.875108546, 2024-05-11T23:03:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050730475999999997, 2024-05-11T23:03:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226073 2024-05-11T23:03:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:03:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322914+0000", 2024-05-11T23:03:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001889153 2024-05-11T23:03:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322917+0000", 2024-05-11T23:03:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2100000000000002e-06 2024-05-11T23:03:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322924+0000", 2024-05-11T23:03:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9929999999999998e-06 2024-05-11T23:03:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322928+0000", 2024-05-11T23:03:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5549999999999996e-06 2024-05-11T23:03:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371578+0000", 2024-05-11T23:03:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048650001999999998 2024-05-11T23:03:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371671+0000", 2024-05-11T23:03:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2230000000000003e-05 2024-05-11T23:03:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371755+0000", 2024-05-11T23:03:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4332999999999994e-05 2024-05-11T23:03:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226085 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 5338112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.873017447000002, 2024-05-11T23:03:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050727622, 2024-05-11T23:03:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226085 2024-05-11T23:03:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:03:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324914+0000", 2024-05-11T23:03:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001798298 2024-05-11T23:03:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324918+0000", 2024-05-11T23:03:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3889999999999998e-06 2024-05-11T23:03:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324928+0000", 2024-05-11T23:03:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4339999999999998e-06 2024-05-11T23:03:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324936+0000", 2024-05-11T23:03:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2770000000000005e-06 2024-05-11T23:03:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373664+0000", 2024-05-11T23:03:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048728207000000003 2024-05-11T23:03:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373757+0000", 2024-05-11T23:03:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3332999999999996e-05 2024-05-11T23:03:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373843+0000", 2024-05-11T23:03:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5684000000000002e-05 2024-05-11T23:03:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232555 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7789568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.396007122, 2024-05-11T23:03:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050583086999999999, 2024-05-11T23:03:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232555 2024-05-11T23:03:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:03:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:03:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800127+0000", 2024-05-11T23:03:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.207e-06 2024-05-11T23:03:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800132+0000", 2024-05-11T23:03:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:03:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800134+0000", 2024-05-11T23:03:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5459999999999998e-06 2024-05-11T23:03:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850548+0000", 2024-05-11T23:03:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050414182000000002 2024-05-11T23:03:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850635+0000", 2024-05-11T23:03:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6853000000000003e-05 2024-05-11T23:03:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850709+0000", 2024-05-11T23:03:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3585999999999996e-05 2024-05-11T23:03:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232556 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7799808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.395995468000002, 2024-05-11T23:03:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050749599999999999, 2024-05-11T23:03:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232556 2024-05-11T23:03:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:03:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801914+0000", 2024-05-11T23:03:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017766360000000001 2024-05-11T23:03:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801919+0000", 2024-05-11T23:03:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.42e-06 2024-05-11T23:03:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801927+0000", 2024-05-11T23:03:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1999999999999994e-06 2024-05-11T23:03:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801931+0000", 2024-05-11T23:03:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4279999999999998e-06 2024-05-11T23:03:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850740+0000", 2024-05-11T23:03:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048808658999999997 2024-05-11T23:03:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850819+0000", 2024-05-11T23:03:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9102000000000003e-05 2024-05-11T23:03:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850887+0000", 2024-05-11T23:03:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8155000000000001e-05 2024-05-11T23:03:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232578 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8025088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.392702229999998, 2024-05-11T23:03:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050679473000000003, 2024-05-11T23:03:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232578 2024-05-11T23:03:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:03:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:03:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803432+0000", 2024-05-11T23:03:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.139e-06 2024-05-11T23:03:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803435+0000", 2024-05-11T23:03:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2440000000000002e-06 2024-05-11T23:03:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803439+0000", 2024-05-11T23:03:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3129999999999999e-06 2024-05-11T23:03:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.853953+0000", 2024-05-11T23:03:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050514846000000002 2024-05-11T23:03:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854035+0000", 2024-05-11T23:03:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1148000000000003e-05 2024-05-11T23:03:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854110+0000", 2024-05-11T23:03:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5562000000000006e-05 2024-05-11T23:03:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232579 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8035328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.392689625999999, 2024-05-11T23:03:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050886656000000002, 2024-05-11T23:03:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232579 2024-05-11T23:03:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:03:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803914+0000", 2024-05-11T23:03:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00047026700000000002 2024-05-11T23:03:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803918+0000", 2024-05-11T23:03:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3460000000000004e-06 2024-05-11T23:03:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803927+0000", 2024-05-11T23:03:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5030000000000003e-06 2024-05-11T23:03:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803933+0000", 2024-05-11T23:03:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0310000000000004e-06 2024-05-11T23:03:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854153+0000", 2024-05-11T23:03:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050219321999999997 2024-05-11T23:03:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854242+0000", 2024-05-11T23:03:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9457999999999996e-05 2024-05-11T23:03:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854330+0000", 2024-05-11T23:03:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7729e-05 2024-05-11T23:03:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232581 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8055808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.392166148999998, 2024-05-11T23:03:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050664439999999998, 2024-05-11T23:03:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232581 2024-05-11T23:03:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:03:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:03:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803969+0000", 2024-05-11T23:03:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0889999999999998e-06 2024-05-11T23:03:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803978+0000", 2024-05-11T23:03:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3070000000000003e-06 2024-05-11T23:03:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803983+0000", 2024-05-11T23:03:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1599999999999997e-06 2024-05-11T23:03:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854509+0000", 2024-05-11T23:03:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526339000000003 2024-05-11T23:03:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854576+0000", 2024-05-11T23:03:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7102999999999998e-05 2024-05-11T23:03:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854631+0000", 2024-05-11T23:03:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4997999999999998e-05 2024-05-11T23:03:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232582 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8066048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.392143404000002, 2024-05-11T23:03:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050753983000000003, 2024-05-11T23:03:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232582 2024-05-11T23:03:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:03:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805917+0000", 2024-05-11T23:03:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001927432 2024-05-11T23:03:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805920+0000", 2024-05-11T23:03:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1219999999999999e-06 2024-05-11T23:03:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805933+0000", 2024-05-11T23:03:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2727e-05 2024-05-11T23:03:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805941+0000", 2024-05-11T23:03:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7090000000000002e-06 2024-05-11T23:03:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854657+0000", 2024-05-11T23:03:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048716164999999999 2024-05-11T23:03:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854699+0000", 2024-05-11T23:03:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2531999999999999e-05 2024-05-11T23:03:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854744+0000", 2024-05-11T23:03:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4295999999999997e-05 2024-05-11T23:03:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232593 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8178688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:03:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.389944010999997, 2024-05-11T23:03:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050735178999999998, 2024-05-11T23:03:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232593 2024-05-11T23:03:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:03:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807913+0000", 2024-05-11T23:03:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017240319999999999 2024-05-11T23:03:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807916+0000", 2024-05-11T23:03:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4589999999999998e-06 2024-05-11T23:03:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807924+0000", 2024-05-11T23:03:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.272e-06 2024-05-11T23:03:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807928+0000", 2024-05-11T23:03:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5369999999999999e-06 2024-05-11T23:03:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856768+0000", 2024-05-11T23:03:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048839489999999999 2024-05-11T23:03:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856845+0000", 2024-05-11T23:03:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7746999999999999e-05 2024-05-11T23:03:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856924+0000", 2024-05-11T23:03:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8641999999999997e-05 2024-05-11T23:03:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.283 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:54.285 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:03:54.285 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:54.285+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:54.385 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:54.385+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:54.436 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:54.436 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:54.436 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:54.436 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:54.447 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:03:54.487 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:54.486+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:54.557 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:54.558 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:54.566 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:03:54.586 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:54.586+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:54.677 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:54.677 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:54.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:54.677 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:54.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.677 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:64049 3.8 3:11d98727:::rbd_object_map.111e8974a681:head [read 0~38] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-05-11T23:03:54.677 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:54.677 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 113.896476916, 2024-05-11T23:03:54.677 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99884604399999999, 2024-05-11T23:03:54.677 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 64049 2024-05-11T23:03:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779883+0000", 2024-05-11T23:03:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.57e-06 2024-05-11T23:03:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779887+0000", 2024-05-11T23:03:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5709999999999999e-06 2024-05-11T23:03:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779898+0000", 2024-05-11T23:03:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.042e-05 2024-05-11T23:03:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779905+0000", 2024-05-11T23:03:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0110000000000003e-06 2024-05-11T23:03:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779962+0000", 2024-05-11T23:03:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7395999999999997e-05 2024-05-11T23:03:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779969+0000", 2024-05-11T23:03:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2459999999999997e-06 2024-05-11T23:03:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778379+0000", 2024-05-11T23:03:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99840986300000001 2024-05-11T23:03:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778564+0000", 2024-05-11T23:03:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018420900000000001 2024-05-11T23:03:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778727+0000", 2024-05-11T23:03:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016375800000000001 2024-05-11T23:03:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172773 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.072370497999998, 2024-05-11T23:03:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058721316000000003, 2024-05-11T23:03:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172773 2024-05-11T23:03:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3500000000000002e-07 2024-05-11T23:03:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603990+0000", 2024-05-11T23:03:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3549999999999999e-06 2024-05-11T23:03:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603997+0000", 2024-05-11T23:03:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3400000000000003e-06 2024-05-11T23:03:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604002+0000", 2024-05-11T23:03:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0509999999999996e-06 2024-05-11T23:03:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662697+0000", 2024-05-11T23:03:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058695487999999997 2024-05-11T23:03:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662709+0000", 2024-05-11T23:03:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1647e-05 2024-05-11T23:03:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172774 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.072349115999998, 2024-05-11T23:03:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058745040999999998, 2024-05-11T23:03:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172774 2024-05-11T23:03:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:03:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604012+0000", 2024-05-11T23:03:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2199999999999999e-06 2024-05-11T23:03:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604018+0000", 2024-05-11T23:03:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1770000000000003e-06 2024-05-11T23:03:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604023+0000", 2024-05-11T23:03:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0089999999999998e-06 2024-05-11T23:03:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662742+0000", 2024-05-11T23:03:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058719187999999999 2024-05-11T23:03:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662754+0000", 2024-05-11T23:03:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2095e-05 2024-05-11T23:03:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172775 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.072328904000003, 2024-05-11T23:03:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058764743000000001, 2024-05-11T23:03:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172775 2024-05-11T23:03:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604030+0000", 2024-05-11T23:03:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6699999999999999e-07 2024-05-11T23:03:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604032+0000", 2024-05-11T23:03:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9530000000000002e-06 2024-05-11T23:03:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604037+0000", 2024-05-11T23:03:54.693 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8590000000000001e-06 2024-05-11T23:03:54.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604042+0000", 2024-05-11T23:03:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8609999999999997e-06 2024-05-11T23:03:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662782+0000", 2024-05-11T23:03:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058739731000000003 2024-05-11T23:03:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662794+0000", 2024-05-11T23:03:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1972e-05 2024-05-11T23:03:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172776 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.072309613999998, 2024-05-11T23:03:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058778696999999998, 2024-05-11T23:03:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172776 2024-05-11T23:03:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2000000000000001e-07 2024-05-11T23:03:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604051+0000", 2024-05-11T23:03:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2879999999999999e-06 2024-05-11T23:03:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604057+0000", 2024-05-11T23:03:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:03:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604062+0000", 2024-05-11T23:03:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6800000000000001e-06 2024-05-11T23:03:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662820+0000", 2024-05-11T23:03:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058758444999999999 2024-05-11T23:03:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662827+0000", 2024-05-11T23:03:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1930000000000004e-06 2024-05-11T23:03:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172777 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.072290307999999, 2024-05-11T23:03:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058799918999999999, 2024-05-11T23:03:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172777 2024-05-11T23:03:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:03:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604075+0000", 2024-05-11T23:03:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.109e-06 2024-05-11T23:03:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604083+0000", 2024-05-11T23:03:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9170000000000006e-06 2024-05-11T23:03:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604089+0000", 2024-05-11T23:03:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3399999999999997e-06 2024-05-11T23:03:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662859+0000", 2024-05-11T23:03:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058770172000000002 2024-05-11T23:03:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662868+0000", 2024-05-11T23:03:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0289999999999994e-06 2024-05-11T23:03:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172778 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.072262217999999, 2024-05-11T23:03:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058845435000000001, 2024-05-11T23:03:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172778 2024-05-11T23:03:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:54.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.704 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:54.704 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.704 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.704 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:54.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5400000000000002e-07 2024-05-11T23:03:54.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604099+0000", 2024-05-11T23:03:54.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4650000000000001e-06 2024-05-11T23:03:54.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.705 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.705 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.705 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604105+0000", 2024-05-11T23:03:54.705 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.314e-06 2024-05-11T23:03:54.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604110+0000", 2024-05-11T23:03:54.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6310000000000002e-06 2024-05-11T23:03:54.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.706 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.706 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662931+0000", 2024-05-11T23:03:54.706 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058820960999999998 2024-05-11T23:03:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662941+0000", 2024-05-11T23:03:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.061e-05 2024-05-11T23:03:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172779 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.072242058, 2024-05-11T23:03:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058859163999999999, 2024-05-11T23:03:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172779 2024-05-11T23:03:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:03:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604119+0000", 2024-05-11T23:03:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2529999999999999e-06 2024-05-11T23:03:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604125+0000", 2024-05-11T23:03:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1670000000000004e-06 2024-05-11T23:03:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604130+0000", 2024-05-11T23:03:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.702e-06 2024-05-11T23:03:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662967+0000", 2024-05-11T23:03:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058837792999999999 2024-05-11T23:03:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662975+0000", 2024-05-11T23:03:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8490000000000006e-06 2024-05-11T23:03:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172780 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.072221908000003, 2024-05-11T23:03:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058872696000000002, 2024-05-11T23:03:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172780 2024-05-11T23:03:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604137+0000", 2024-05-11T23:03:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4400000000000001e-07 2024-05-11T23:03:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604139+0000", 2024-05-11T23:03:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3429999999999998e-06 2024-05-11T23:03:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604145+0000", 2024-05-11T23:03:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1940000000000003e-06 2024-05-11T23:03:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604150+0000", 2024-05-11T23:03:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:03:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663002+0000", 2024-05-11T23:03:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058851829000000001 2024-05-11T23:03:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663009+0000", 2024-05-11T23:03:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.199e-06 2024-05-11T23:03:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172781 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.072190067999998, 2024-05-11T23:03:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058875553999999997, 2024-05-11T23:03:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172781 2024-05-11T23:03:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604171+0000", 2024-05-11T23:03:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.21e-06 2024-05-11T23:03:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604176+0000", 2024-05-11T23:03:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2970000000000003e-06 2024-05-11T23:03:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604179+0000", 2024-05-11T23:03:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8459999999999999e-06 2024-05-11T23:03:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663033+0000", 2024-05-11T23:03:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058854565999999997 2024-05-11T23:03:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663044+0000", 2024-05-11T23:03:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0229e-05 2024-05-11T23:03:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172782 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.072175319999999, 2024-05-11T23:03:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058894375999999998, 2024-05-11T23:03:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172782 2024-05-11T23:03:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-05-11T23:03:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604184+0000", 2024-05-11T23:03:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:03:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604187+0000", 2024-05-11T23:03:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.044e-06 2024-05-11T23:03:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604190+0000", 2024-05-11T23:03:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6460000000000002e-06 2024-05-11T23:03:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663070+0000", 2024-05-11T23:03:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058879510000000003 2024-05-11T23:03:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663077+0000", 2024-05-11T23:03:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6669999999999996e-06 2024-05-11T23:03:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172783 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.072165001000002, 2024-05-11T23:03:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058913946000000002, 2024-05-11T23:03:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172783 2024-05-11T23:03:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-05-11T23:03:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604195+0000", 2024-05-11T23:03:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.19e-06 2024-05-11T23:03:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604198+0000", 2024-05-11T23:03:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:03:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604200+0000", 2024-05-11T23:03:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4719999999999998e-06 2024-05-11T23:03:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663100+0000", 2024-05-11T23:03:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058900300000000003 2024-05-11T23:03:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663107+0000", 2024-05-11T23:03:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7449999999999998e-06 2024-05-11T23:03:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172784 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.072154982000001, 2024-05-11T23:03:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933728999999997, 2024-05-11T23:03:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172784 2024-05-11T23:03:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.03e-07 2024-05-11T23:03:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604205+0000", 2024-05-11T23:03:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2300000000000001e-06 2024-05-11T23:03:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604207+0000", 2024-05-11T23:03:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8729999999999999e-06 2024-05-11T23:03:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604210+0000", 2024-05-11T23:03:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.858e-06 2024-05-11T23:03:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663130+0000", 2024-05-11T23:03:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058919314 2024-05-11T23:03:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663137+0000", 2024-05-11T23:03:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2509999999999997e-06 2024-05-11T23:03:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172785 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.072142712000002, 2024-05-11T23:03:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058951664000000001, 2024-05-11T23:03:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172785 2024-05-11T23:03:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604216+0000", 2024-05-11T23:03:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:03:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604218+0000", 2024-05-11T23:03:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7549999999999999e-06 2024-05-11T23:03:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604222+0000", 2024-05-11T23:03:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6700000000000002e-06 2024-05-11T23:03:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604227+0000", 2024-05-11T23:03:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4399999999999998e-06 2024-05-11T23:03:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663160+0000", 2024-05-11T23:03:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933119999999999 2024-05-11T23:03:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663167+0000", 2024-05-11T23:03:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1840000000000002e-06 2024-05-11T23:03:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172786 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.072125657999997, 2024-05-11T23:03:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058966991000000003, 2024-05-11T23:03:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172786 2024-05-11T23:03:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604233+0000", 2024-05-11T23:03:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6899999999999999e-07 2024-05-11T23:03:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604235+0000", 2024-05-11T23:03:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8369999999999999e-06 2024-05-11T23:03:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604240+0000", 2024-05-11T23:03:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0329999999999999e-06 2024-05-11T23:03:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604244+0000", 2024-05-11T23:03:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9859999999999996e-06 2024-05-11T23:03:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663192+0000", 2024-05-11T23:03:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058948852000000003 2024-05-11T23:03:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663199+0000", 2024-05-11T23:03:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0140000000000001e-06 2024-05-11T23:03:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172787 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.072109312000002, 2024-05-11T23:03:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058982018999999997, 2024-05-11T23:03:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172787 2024-05-11T23:03:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605916+0000", 2024-05-11T23:03:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00166752 2024-05-11T23:03:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605921+0000", 2024-05-11T23:03:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0669999999999999e-06 2024-05-11T23:03:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605934+0000", 2024-05-11T23:03:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2271e-05 2024-05-11T23:03:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605940+0000", 2024-05-11T23:03:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6880000000000002e-06 2024-05-11T23:03:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663224+0000", 2024-05-11T23:03:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057283251 2024-05-11T23:03:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663231+0000", 2024-05-11T23:03:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2219999999999996e-06 2024-05-11T23:03:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172793 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.070342465000003, 2024-05-11T23:03:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057402373999999999, 2024-05-11T23:03:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172793 2024-05-11T23:03:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3300000000000001e-07 2024-05-11T23:03:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606017+0000", 2024-05-11T23:03:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4920000000000001e-06 2024-05-11T23:03:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606020+0000", 2024-05-11T23:03:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:03:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606023+0000", 2024-05-11T23:03:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6680000000000001e-06 2024-05-11T23:03:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663411+0000", 2024-05-11T23:03:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057388206999999997 2024-05-11T23:03:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663418+0000", 2024-05-11T23:03:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.815e-06 2024-05-11T23:03:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172795 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.070301037999997, 2024-05-11T23:03:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057423015000000001, 2024-05-11T23:03:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172795 2024-05-11T23:03:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9499999999999998e-07 2024-05-11T23:03:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606059+0000", 2024-05-11T23:03:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-05-11T23:03:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606062+0000", 2024-05-11T23:03:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.977e-06 2024-05-11T23:03:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606064+0000", 2024-05-11T23:03:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7410000000000001e-06 2024-05-11T23:03:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663473+0000", 2024-05-11T23:03:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057408624999999998 2024-05-11T23:03:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663480+0000", 2024-05-11T23:03:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1049999999999997e-06 2024-05-11T23:03:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172796 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.070290475, 2024-05-11T23:03:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057449235000000001, 2024-05-11T23:03:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172796 2024-05-11T23:03:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606069+0000", 2024-05-11T23:03:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:03:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606072+0000", 2024-05-11T23:03:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8700000000000001e-06 2024-05-11T23:03:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606074+0000", 2024-05-11T23:03:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6230000000000001e-06 2024-05-11T23:03:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663510+0000", 2024-05-11T23:03:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05743529 2024-05-11T23:03:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663517+0000", 2024-05-11T23:03:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1230000000000002e-06 2024-05-11T23:03:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172797 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 56.070280443000001, 2024-05-11T23:03:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057470420000000001, 2024-05-11T23:03:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172797 2024-05-11T23:03:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607915+0000", 2024-05-11T23:03:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018375609999999999 2024-05-11T23:03:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607920+0000", 2024-05-11T23:03:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8899999999999998e-06 2024-05-11T23:03:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607930+0000", 2024-05-11T23:03:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4429999999999992e-06 2024-05-11T23:03:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607936+0000", 2024-05-11T23:03:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7089999999999996e-06 2024-05-11T23:03:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663541+0000", 2024-05-11T23:03:54.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055604537000000002 2024-05-11T23:03:54.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:54.757 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:54.757 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:54.757 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663548+0000", 2024-05-11T23:03:54.757 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2799999999999998e-06 2024-05-11T23:03:54.757 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.757 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.757 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.757 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:54.757 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:54.757 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:54.759 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:03:54.759 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:54.759+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:54.860 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:54.860+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:54.917 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:54.917 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:54.918 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:54.918 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:54.926 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:03:54.967 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:54.967+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:55.062 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:55.062+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:55.075 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:55.075 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:55.075 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:55.076 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:55.076 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:55.084 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:03:55.163 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:55.163+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:55.173 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:03:55.197 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:55.197 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:55.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:55.197 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:55.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.197 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:77128 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_get in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-11T23:03:55.197 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:55.197 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 100.179735583, 2024-05-11T23:03:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91737855899999998, 2024-05-11T23:03:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 77128 2024-05-11T23:03:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016003+0000", 2024-05-11T23:03:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5379999999999999e-06 2024-05-11T23:03:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016011+0000", 2024-05-11T23:03:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6410000000000002e-06 2024-05-11T23:03:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016026+0000", 2024-05-11T23:03:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4779e-05 2024-05-11T23:03:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016032+0000", 2024-05-11T23:03:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.066e-06 2024-05-11T23:03:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016093+0000", 2024-05-11T23:03:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0878000000000002e-05 2024-05-11T23:03:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016100+0000", 2024-05-11T23:03:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6799999999999993e-06 2024-05-11T23:03:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933302+0000", 2024-05-11T23:03:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91720135300000005 2024-05-11T23:03:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933380+0000", 2024-05-11T23:03:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8623999999999998e-05 2024-05-11T23:03:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98950 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 89.063654693999993, 2024-05-11T23:03:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021419438999999998, 2024-05-11T23:03:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98950 2024-05-11T23:03:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.34e-07 2024-05-11T23:03:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132085+0000", 2024-05-11T23:03:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0049999999999999e-06 2024-05-11T23:03:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132093+0000", 2024-05-11T23:03:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6660000000000008e-06 2024-05-11T23:03:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132097+0000", 2024-05-11T23:03:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2590000000000002e-06 2024-05-11T23:03:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135114+0000", 2024-05-11T23:03:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003017573 2024-05-11T23:03:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153416+0000", 2024-05-11T23:03:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018301884000000001 2024-05-11T23:03:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153502+0000", 2024-05-11T23:03:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5717999999999995e-05 2024-05-11T23:03:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98959 3.1e 3:7807da99:::rbd_data.111ed2934718.000000000000006d:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 89.063161957000005, 2024-05-11T23:03:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020525928999999998, 2024-05-11T23:03:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98959 2024-05-11T23:03:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:55.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132576+0000", 2024-05-11T23:03:55.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4199999999999995e-07 2024-05-11T23:03:55.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:55.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132580+0000", 2024-05-11T23:03:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2910000000000001e-06 2024-05-11T23:03:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132588+0000", 2024-05-11T23:03:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7970000000000001e-06 2024-05-11T23:03:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132602+0000", 2024-05-11T23:03:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3743999999999999e-05 2024-05-11T23:03:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132655+0000", 2024-05-11T23:03:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2714999999999999e-05 2024-05-11T23:03:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132787+0000", 2024-05-11T23:03:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013273899999999999 2024-05-11T23:03:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153101+0000", 2024-05-11T23:03:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020313900999999999 2024-05-11T23:03:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98965 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 89.060638320999999, 2024-05-11T23:03:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018608983999999999, 2024-05-11T23:03:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98965 2024-05-11T23:03:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135100+0000", 2024-05-11T23:03:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.17e-06 2024-05-11T23:03:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135105+0000", 2024-05-11T23:03:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.899e-06 2024-05-11T23:03:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135122+0000", 2024-05-11T23:03:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7346999999999998e-05 2024-05-11T23:03:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135130+0000", 2024-05-11T23:03:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0639999999999994e-06 2024-05-11T23:03:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153533+0000", 2024-05-11T23:03:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018402753000000001 2024-05-11T23:03:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153615+0000", 2024-05-11T23:03:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1720999999999999e-05 2024-05-11T23:03:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153708+0000", 2024-05-11T23:03:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3029999999999995e-05 2024-05-11T23:03:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98966 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3760128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 89.060566195000007, 2024-05-11T23:03:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018716535999999999, 2024-05-11T23:03:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98966 2024-05-11T23:03:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135172+0000", 2024-05-11T23:03:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-05-11T23:03:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135175+0000", 2024-05-11T23:03:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-05-11T23:03:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135184+0000", 2024-05-11T23:03:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1840000000000005e-06 2024-05-11T23:03:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152903+0000", 2024-05-11T23:03:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017718357000000001 2024-05-11T23:03:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153736+0000", 2024-05-11T23:03:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00083324799999999997 2024-05-11T23:03:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153809+0000", 2024-05-11T23:03:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2995000000000002e-05 2024-05-11T23:03:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153888+0000", 2024-05-11T23:03:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8695999999999993e-05 2024-05-11T23:03:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "pg_update_log_missing_reply(3.1e epoch 72/53 rep_tid 104 lcod 72'363)", 2024-05-11T23:03:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 89.060503299000004, 2024-05-11T23:03:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017871951000000001, 2024-05-11T23:03:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135235+0000", 2024-05-11T23:03:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8899999999999999e-07 2024-05-11T23:03:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135238+0000", 2024-05-11T23:03:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.27e-06 2024-05-11T23:03:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152878+0000", 2024-05-11T23:03:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01763986 2024-05-11T23:03:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152896+0000", 2024-05-11T23:03:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8573e-05 2024-05-11T23:03:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152968+0000", 2024-05-11T23:03:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1780999999999996e-05 2024-05-11T23:03:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153106+0000", 2024-05-11T23:03:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013777799999999999 2024-05-11T23:03:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:239544 3.1a 3:5fe59cad:::rbd_data.111ebabb4604.000000000000016e:head [stat,delete] snapc 2f=[2f] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 51.541146566999998, 2024-05-11T23:03:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.50514007500000002, 2024-05-11T23:03:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 239544 2024-05-11T23:03:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654592+0000", 2024-05-11T23:03:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.404e-06 2024-05-11T23:03:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654598+0000", 2024-05-11T23:03:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7660000000000004e-06 2024-05-11T23:03:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654611+0000", 2024-05-11T23:03:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2676e-05 2024-05-11T23:03:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654620+0000", 2024-05-11T23:03:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2990000000000002e-06 2024-05-11T23:03:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654698+0000", 2024-05-11T23:03:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7739999999999998e-05 2024-05-11T23:03:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654710+0000", 2024-05-11T23:03:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2648e-05 2024-05-11T23:03:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.157889+0000", 2024-05-11T23:03:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.50317850799999997 2024-05-11T23:03:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.158050+0000", 2024-05-11T23:03:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016084500000000001 2024-05-11T23:03:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.159731+0000", 2024-05-11T23:03:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001681189 2024-05-11T23:03:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262364 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2613248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.609273702000003, 2024-05-11T23:03:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016312508, 2024-05-11T23:03:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262364 2024-05-11T23:03:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.13e-07 2024-05-11T23:03:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586467+0000", 2024-05-11T23:03:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-05-11T23:03:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586474+0000", 2024-05-11T23:03:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7739999999999999e-06 2024-05-11T23:03:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586483+0000", 2024-05-11T23:03:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0340000000000002e-06 2024-05-11T23:03:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602629+0000", 2024-05-11T23:03:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016145845999999998 2024-05-11T23:03:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602703+0000", 2024-05-11T23:03:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4671000000000004e-05 2024-05-11T23:03:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602776+0000", 2024-05-11T23:03:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2793000000000006e-05 2024-05-11T23:03:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262417 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.607527402999999, 2024-05-11T23:03:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016403456, 2024-05-11T23:03:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262417 2024-05-11T23:03:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6199999999999999e-07 2024-05-11T23:03:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588212+0000", 2024-05-11T23:03:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.075e-06 2024-05-11T23:03:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588218+0000", 2024-05-11T23:03:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8150000000000002e-06 2024-05-11T23:03:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588223+0000", 2024-05-11T23:03:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7090000000000001e-06 2024-05-11T23:03:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604461+0000", 2024-05-11T23:03:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016237900999999999 2024-05-11T23:03:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604540+0000", 2024-05-11T23:03:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9376999999999996e-05 2024-05-11T23:03:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604613+0000", 2024-05-11T23:03:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3217000000000001e-05 2024-05-11T23:03:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262418 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2396160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.607507494000004, 2024-05-11T23:03:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016591180000000001, 2024-05-11T23:03:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262418 2024-05-11T23:03:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6100000000000001e-07 2024-05-11T23:03:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588232+0000", 2024-05-11T23:03:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0370000000000001e-06 2024-05-11T23:03:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588238+0000", 2024-05-11T23:03:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9340000000000003e-06 2024-05-11T23:03:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588243+0000", 2024-05-11T23:03:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6179999999999997e-06 2024-05-11T23:03:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604642+0000", 2024-05-11T23:03:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0163991 2024-05-11T23:03:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604712+0000", 2024-05-11T23:03:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9935e-05 2024-05-11T23:03:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604821+0000", 2024-05-11T23:03:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000109095 2024-05-11T23:03:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262419 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2433024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.607488175999997, 2024-05-11T23:03:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016799692000000001, 2024-05-11T23:03:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262419 2024-05-11T23:03:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588252+0000", 2024-05-11T23:03:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0990000000000001e-06 2024-05-11T23:03:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588257+0000", 2024-05-11T23:03:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8159999999999999e-06 2024-05-11T23:03:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588262+0000", 2024-05-11T23:03:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8069999999999999e-06 2024-05-11T23:03:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604868+0000", 2024-05-11T23:03:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016605880999999999 2024-05-11T23:03:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604966+0000", 2024-05-11T23:03:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7597999999999996e-05 2024-05-11T23:03:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605049+0000", 2024-05-11T23:03:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3141999999999998e-05 2024-05-11T23:03:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262420 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2469888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.607468064999999, 2024-05-11T23:03:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016972514000000001, 2024-05-11T23:03:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262420 2024-05-11T23:03:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588271+0000", 2024-05-11T23:03:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6199999999999999e-06 2024-05-11T23:03:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588273+0000", 2024-05-11T23:03:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3209999999999999e-06 2024-05-11T23:03:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588279+0000", 2024-05-11T23:03:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2519999999999996e-06 2024-05-11T23:03:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588285+0000", 2024-05-11T23:03:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1440000000000002e-06 2024-05-11T23:03:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605083+0000", 2024-05-11T23:03:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016798418999999998 2024-05-11T23:03:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605168+0000", 2024-05-11T23:03:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4851000000000005e-05 2024-05-11T23:03:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605242+0000", 2024-05-11T23:03:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3906999999999996e-05 2024-05-11T23:03:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262421 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2506752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.607445919, 2024-05-11T23:03:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017141427000000001, 2024-05-11T23:03:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262421 2024-05-11T23:03:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588292+0000", 2024-05-11T23:03:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588318+0000", 2024-05-11T23:03:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6376e-05 2024-05-11T23:03:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588326+0000", 2024-05-11T23:03:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6699999999999994e-06 2024-05-11T23:03:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588332+0000", 2024-05-11T23:03:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8540000000000002e-06 2024-05-11T23:03:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605274+0000", 2024-05-11T23:03:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016942024999999999 2024-05-11T23:03:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605360+0000", 2024-05-11T23:03:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6304999999999994e-05 2024-05-11T23:03:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605433+0000", 2024-05-11T23:03:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2772000000000001e-05 2024-05-11T23:03:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262422 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2543616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.607396694999998, 2024-05-11T23:03:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017283705, 2024-05-11T23:03:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262422 2024-05-11T23:03:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:03:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588343+0000", 2024-05-11T23:03:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.34e-06 2024-05-11T23:03:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588350+0000", 2024-05-11T23:03:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1569999999999997e-06 2024-05-11T23:03:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588354+0000", 2024-05-11T23:03:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.865e-06 2024-05-11T23:03:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605466+0000", 2024-05-11T23:03:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017111498999999999 2024-05-11T23:03:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605551+0000", 2024-05-11T23:03:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5210999999999994e-05 2024-05-11T23:03:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605624+0000", 2024-05-11T23:03:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3153999999999998e-05 2024-05-11T23:03:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262423 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2580480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.607376446000004, 2024-05-11T23:03:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017451194, 2024-05-11T23:03:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262423 2024-05-11T23:03:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588363+0000", 2024-05-11T23:03:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.855e-06 2024-05-11T23:03:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588369+0000", 2024-05-11T23:03:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1260000000000003e-06 2024-05-11T23:03:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588375+0000", 2024-05-11T23:03:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9630000000000003e-06 2024-05-11T23:03:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605656+0000", 2024-05-11T23:03:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017280865999999999 2024-05-11T23:03:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605737+0000", 2024-05-11T23:03:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0773999999999996e-05 2024-05-11T23:03:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605812+0000", 2024-05-11T23:03:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5203999999999994e-05 2024-05-11T23:03:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262424 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2617344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.607354927999999, 2024-05-11T23:03:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017617544999999998, 2024-05-11T23:03:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262424 2024-05-11T23:03:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588383+0000", 2024-05-11T23:03:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8299999999999997e-07 2024-05-11T23:03:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588385+0000", 2024-05-11T23:03:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2869999999999999e-06 2024-05-11T23:03:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588391+0000", 2024-05-11T23:03:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1689999999999997e-06 2024-05-11T23:03:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588396+0000", 2024-05-11T23:03:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5719999999999996e-06 2024-05-11T23:03:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605837+0000", 2024-05-11T23:03:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017440812999999999 2024-05-11T23:03:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605917+0000", 2024-05-11T23:03:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9924000000000003e-05 2024-05-11T23:03:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.606000+0000", 2024-05-11T23:03:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3296999999999999e-05 2024-05-11T23:03:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262896 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.536350290999998, 2024-05-11T23:03:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016295738000000001, 2024-05-11T23:03:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262896 2024-05-11T23:03:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659390+0000", 2024-05-11T23:03:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1440000000000001e-06 2024-05-11T23:03:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659397+0000", 2024-05-11T23:03:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0010000000000004e-06 2024-05-11T23:03:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659402+0000", 2024-05-11T23:03:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2179999999999998e-06 2024-05-11T23:03:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675487+0000", 2024-05-11T23:03:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016085706000000002 2024-05-11T23:03:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675580+0000", 2024-05-11T23:03:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2884e-05 2024-05-11T23:03:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675683+0000", 2024-05-11T23:03:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010236 2024-05-11T23:03:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262897 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.536314949999998, 2024-05-11T23:03:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016500932999999999, 2024-05-11T23:03:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262897 2024-05-11T23:03:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659423+0000", 2024-05-11T23:03:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9699999999999996e-07 2024-05-11T23:03:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659425+0000", 2024-05-11T23:03:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3199999999999998e-06 2024-05-11T23:03:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659433+0000", 2024-05-11T23:03:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3660000000000003e-06 2024-05-11T23:03:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659445+0000", 2024-05-11T23:03:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2102e-05 2024-05-11T23:03:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675727+0000", 2024-05-11T23:03:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016282331000000001 2024-05-11T23:03:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675815+0000", 2024-05-11T23:03:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7484999999999993e-05 2024-05-11T23:03:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675923+0000", 2024-05-11T23:03:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000108732 2024-05-11T23:03:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262898 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.536283832000002, 2024-05-11T23:03:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016685855999999999, 2024-05-11T23:03:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262898 2024-05-11T23:03:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659454+0000", 2024-05-11T23:03:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2200000000000004e-07 2024-05-11T23:03:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659457+0000", 2024-05-11T23:03:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.779e-06 2024-05-11T23:03:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659463+0000", 2024-05-11T23:03:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6660000000000002e-06 2024-05-11T23:03:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659472+0000", 2024-05-11T23:03:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2139999999999996e-06 2024-05-11T23:03:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675956+0000", 2024-05-11T23:03:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016484431000000001 2024-05-11T23:03:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676039+0000", 2024-05-11T23:03:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3358999999999999e-05 2024-05-11T23:03:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676139+0000", 2024-05-11T23:03:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9784999999999994e-05 2024-05-11T23:03:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262899 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 42.536258001, 2024-05-11T23:03:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017030087999999999, 2024-05-11T23:03:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262899 2024-05-11T23:03:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659480+0000", 2024-05-11T23:03:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7700000000000004e-07 2024-05-11T23:03:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659482+0000", 2024-05-11T23:03:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1390000000000002e-06 2024-05-11T23:03:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659489+0000", 2024-05-11T23:03:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8279999999999997e-06 2024-05-11T23:03:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659494+0000", 2024-05-11T23:03:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9019999999999998e-06 2024-05-11T23:03:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676167+0000", 2024-05-11T23:03:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016673621 2024-05-11T23:03:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676329+0000", 2024-05-11T23:03:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016206099999999999 2024-05-11T23:03:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676509+0000", 2024-05-11T23:03:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017996 2024-05-11T23:03:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:55.282 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:55.283 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:03:55.284 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:55.283+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:55.300 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:55.300 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:55.317 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:55.334 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:55.334 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:55.335 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-05-11T23:03:55.352 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:55.352 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:55.369 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:55.370 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:55.386 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:55.385+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:55.387 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:55.387 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:55.401 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:55.402 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:55.404 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:55.408 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-11T23:03:55.408 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-11T23:03:55.411 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:03:55.486 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:55.487+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:55.558 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:55.558 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:55.566 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:03:55.610 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:55.610+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:55.672 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:55.672 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:55.681 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:03:55.723 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:55.722+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:55.789 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:55.789+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:55.809 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-11T23:03:55.813 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 5, 7, 3, 6, 4] out_osds: [0] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-05-11T23:03:55.813 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:03:55.813 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-11T23:03:55.813 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.000000 2024-05-11T23:03:55.813 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 0 2024-05-11T23:03:55.832 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:55.833 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:55.842 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:03:55.896 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:55.895+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:55.949 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:55.949 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:55.958 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:03:55.991 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:55.990+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:56.064 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:56.064 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:56.073 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:03:56.117 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:56.117+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:56.179 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:56.179 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:56.187 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:03:56.228 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:56.228+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:56.292 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:56.291+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:56.334 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:56.334 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:56.342 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:03:56.394 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:56.393+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:56.447 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:56.447 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:56.456 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:03:56.499 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:56.499+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:56.561 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:56.561 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:56.570 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:03:56.611 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:56.611+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:56.697 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:56.697+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:56.715 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:56.715 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:56.724 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:03:56.799 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:56.798+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:56.818 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.4 primary-affinity to 0 (802) 2024-05-11T23:03:56.821 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-05-11T23:03:56.835 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:56.835 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:56.845 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:03:56.899 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:56.899+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:56.996 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:56.996 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:57.000 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:57.000+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:57.006 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:03:57.101 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:57.101+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:57.115 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:57.115 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:57.124 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:03:57.203 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:57.202+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:57.234 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:57.234 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:57.243 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:03:57.303 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:57.303+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:57.352 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:57.352 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:57.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:03:57.405 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:57.404+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:57.409 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:03:57.471 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:03:57.471 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:03:57.482 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:03:57.526 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:57.525+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:57.538 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:57.557 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:57.557 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:57.575 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:57.575 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:57.575 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-05-11T23:03:57.593 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:57.593 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:57.596 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:57.596 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:57.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:57.596 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:57.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.597 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:64049 3.8 3:11d98727:::rbd_object_map.111e8974a681:head [read 0~38] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-05-11T23:03:57.597 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:57.597 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 116.816140666, 2024-05-11T23:03:57.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99884604399999999, 2024-05-11T23:03:57.597 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:57.597 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:57.597 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:57.597 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:57.597 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:57.597 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:57.597 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 64049 2024-05-11T23:03:57.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.597 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:57.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:57.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:57.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:57.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:03:57.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:57.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779883+0000", 2024-05-11T23:03:57.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.57e-06 2024-05-11T23:03:57.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:57.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779887+0000", 2024-05-11T23:03:57.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5709999999999999e-06 2024-05-11T23:03:57.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:57.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779898+0000", 2024-05-11T23:03:57.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.042e-05 2024-05-11T23:03:57.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.599 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:57.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779905+0000", 2024-05-11T23:03:57.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0110000000000003e-06 2024-05-11T23:03:57.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779962+0000", 2024-05-11T23:03:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7395999999999997e-05 2024-05-11T23:03:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779969+0000", 2024-05-11T23:03:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2459999999999997e-06 2024-05-11T23:03:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778379+0000", 2024-05-11T23:03:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99840986300000001 2024-05-11T23:03:57.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778564+0000", 2024-05-11T23:03:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018420900000000001 2024-05-11T23:03:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778727+0000", 2024-05-11T23:03:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016375800000000001 2024-05-11T23:03:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.601 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172773 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.992034248000003, 2024-05-11T23:03:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058721316000000003, 2024-05-11T23:03:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172773 2024-05-11T23:03:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:57.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:03:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3500000000000002e-07 2024-05-11T23:03:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603990+0000", 2024-05-11T23:03:57.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3549999999999999e-06 2024-05-11T23:03:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603997+0000", 2024-05-11T23:03:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3400000000000003e-06 2024-05-11T23:03:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604002+0000", 2024-05-11T23:03:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0509999999999996e-06 2024-05-11T23:03:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662697+0000", 2024-05-11T23:03:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058695487999999997 2024-05-11T23:03:57.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662709+0000", 2024-05-11T23:03:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1647e-05 2024-05-11T23:03:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172774 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.992012866000003, 2024-05-11T23:03:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058745040999999998, 2024-05-11T23:03:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:57.605 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172774 2024-05-11T23:03:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:57.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:03:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:03:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604012+0000", 2024-05-11T23:03:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2199999999999999e-06 2024-05-11T23:03:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604018+0000", 2024-05-11T23:03:57.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1770000000000003e-06 2024-05-11T23:03:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604023+0000", 2024-05-11T23:03:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0089999999999998e-06 2024-05-11T23:03:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662742+0000", 2024-05-11T23:03:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058719187999999999 2024-05-11T23:03:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662754+0000", 2024-05-11T23:03:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2095e-05 2024-05-11T23:03:57.608 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172775 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.991992654000001, 2024-05-11T23:03:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058764743000000001, 2024-05-11T23:03:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:57.609 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172775 2024-05-11T23:03:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:03:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:57.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604030+0000", 2024-05-11T23:03:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6699999999999999e-07 2024-05-11T23:03:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604032+0000", 2024-05-11T23:03:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9530000000000002e-06 2024-05-11T23:03:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604037+0000", 2024-05-11T23:03:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8590000000000001e-06 2024-05-11T23:03:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:57.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604042+0000", 2024-05-11T23:03:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8609999999999997e-06 2024-05-11T23:03:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662782+0000", 2024-05-11T23:03:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058739731000000003 2024-05-11T23:03:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662794+0000", 2024-05-11T23:03:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1972e-05 2024-05-11T23:03:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172776 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.991973364000003, 2024-05-11T23:03:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058778696999999998, 2024-05-11T23:03:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172776 2024-05-11T23:03:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:03:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2000000000000001e-07 2024-05-11T23:03:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604051+0000", 2024-05-11T23:03:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2879999999999999e-06 2024-05-11T23:03:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604057+0000", 2024-05-11T23:03:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:03:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604062+0000", 2024-05-11T23:03:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6800000000000001e-06 2024-05-11T23:03:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662820+0000", 2024-05-11T23:03:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058758444999999999 2024-05-11T23:03:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662827+0000", 2024-05-11T23:03:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1930000000000004e-06 2024-05-11T23:03:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172777 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.991954057999997, 2024-05-11T23:03:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058799918999999999, 2024-05-11T23:03:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172777 2024-05-11T23:03:57.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:57.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:03:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:03:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604075+0000", 2024-05-11T23:03:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.109e-06 2024-05-11T23:03:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:57.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604083+0000", 2024-05-11T23:03:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9170000000000006e-06 2024-05-11T23:03:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604089+0000", 2024-05-11T23:03:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3399999999999997e-06 2024-05-11T23:03:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662859+0000", 2024-05-11T23:03:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058770172000000002 2024-05-11T23:03:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662868+0000", 2024-05-11T23:03:57.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0289999999999994e-06 2024-05-11T23:03:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172778 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.991925967999997, 2024-05-11T23:03:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058845435000000001, 2024-05-11T23:03:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:57.621 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172778 2024-05-11T23:03:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:03:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5400000000000002e-07 2024-05-11T23:03:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604099+0000", 2024-05-11T23:03:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4650000000000001e-06 2024-05-11T23:03:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604105+0000", 2024-05-11T23:03:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.314e-06 2024-05-11T23:03:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604110+0000", 2024-05-11T23:03:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6310000000000002e-06 2024-05-11T23:03:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662931+0000", 2024-05-11T23:03:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058820960999999998 2024-05-11T23:03:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662941+0000", 2024-05-11T23:03:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.061e-05 2024-05-11T23:03:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172779 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.991905807999999, 2024-05-11T23:03:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058859163999999999, 2024-05-11T23:03:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172779 2024-05-11T23:03:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.625 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:03:57.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:03:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604119+0000", 2024-05-11T23:03:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2529999999999999e-06 2024-05-11T23:03:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604125+0000", 2024-05-11T23:03:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1670000000000004e-06 2024-05-11T23:03:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604130+0000", 2024-05-11T23:03:57.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.702e-06 2024-05-11T23:03:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662967+0000", 2024-05-11T23:03:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058837792999999999 2024-05-11T23:03:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662975+0000", 2024-05-11T23:03:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8490000000000006e-06 2024-05-11T23:03:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172780 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.991885658000001, 2024-05-11T23:03:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058872696000000002, 2024-05-11T23:03:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172780 2024-05-11T23:03:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:57.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:03:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604137+0000", 2024-05-11T23:03:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4400000000000001e-07 2024-05-11T23:03:57.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604139+0000", 2024-05-11T23:03:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3429999999999998e-06 2024-05-11T23:03:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604145+0000", 2024-05-11T23:03:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1940000000000003e-06 2024-05-11T23:03:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604150+0000", 2024-05-11T23:03:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:03:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663002+0000", 2024-05-11T23:03:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058851829000000001 2024-05-11T23:03:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663009+0000", 2024-05-11T23:03:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.199e-06 2024-05-11T23:03:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172781 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:57.632 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.991853818000003, 2024-05-11T23:03:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058875553999999997, 2024-05-11T23:03:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172781 2024-05-11T23:03:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:57.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:03:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:57.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604171+0000", 2024-05-11T23:03:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.21e-06 2024-05-11T23:03:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604176+0000", 2024-05-11T23:03:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2970000000000003e-06 2024-05-11T23:03:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604179+0000", 2024-05-11T23:03:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8459999999999999e-06 2024-05-11T23:03:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663033+0000", 2024-05-11T23:03:57.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058854565999999997 2024-05-11T23:03:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663044+0000", 2024-05-11T23:03:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0229e-05 2024-05-11T23:03:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172782 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.991839069999997, 2024-05-11T23:03:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058894375999999998, 2024-05-11T23:03:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172782 2024-05-11T23:03:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:03:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-05-11T23:03:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604184+0000", 2024-05-11T23:03:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:03:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604187+0000", 2024-05-11T23:03:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.044e-06 2024-05-11T23:03:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604190+0000", 2024-05-11T23:03:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6460000000000002e-06 2024-05-11T23:03:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663070+0000", 2024-05-11T23:03:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058879510000000003 2024-05-11T23:03:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663077+0000", 2024-05-11T23:03:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6669999999999996e-06 2024-05-11T23:03:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172783 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.991828751, 2024-05-11T23:03:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058913946000000002, 2024-05-11T23:03:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:57.640 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172783 2024-05-11T23:03:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:03:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-05-11T23:03:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604195+0000", 2024-05-11T23:03:57.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.19e-06 2024-05-11T23:03:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604198+0000", 2024-05-11T23:03:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:03:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604200+0000", 2024-05-11T23:03:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4719999999999998e-06 2024-05-11T23:03:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663100+0000", 2024-05-11T23:03:57.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058900300000000003 2024-05-11T23:03:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663107+0000", 2024-05-11T23:03:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7449999999999998e-06 2024-05-11T23:03:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172784 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.991818731999999, 2024-05-11T23:03:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933728999999997, 2024-05-11T23:03:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:57.644 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172784 2024-05-11T23:03:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:57.645 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:03:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.03e-07 2024-05-11T23:03:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604205+0000", 2024-05-11T23:03:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2300000000000001e-06 2024-05-11T23:03:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:57.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604207+0000", 2024-05-11T23:03:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8729999999999999e-06 2024-05-11T23:03:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604210+0000", 2024-05-11T23:03:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.858e-06 2024-05-11T23:03:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663130+0000", 2024-05-11T23:03:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058919314 2024-05-11T23:03:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:57.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663137+0000", 2024-05-11T23:03:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2509999999999997e-06 2024-05-11T23:03:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172785 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.991806462, 2024-05-11T23:03:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058951664000000001, 2024-05-11T23:03:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:57.648 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172785 2024-05-11T23:03:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:03:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:57.649 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604216+0000", 2024-05-11T23:03:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:03:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604218+0000", 2024-05-11T23:03:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7549999999999999e-06 2024-05-11T23:03:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604222+0000", 2024-05-11T23:03:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6700000000000002e-06 2024-05-11T23:03:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604227+0000", 2024-05-11T23:03:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4399999999999998e-06 2024-05-11T23:03:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663160+0000", 2024-05-11T23:03:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933119999999999 2024-05-11T23:03:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663167+0000", 2024-05-11T23:03:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1840000000000002e-06 2024-05-11T23:03:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172786 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.991789408000002, 2024-05-11T23:03:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058966991000000003, 2024-05-11T23:03:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172786 2024-05-11T23:03:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:57.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:03:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604233+0000", 2024-05-11T23:03:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6899999999999999e-07 2024-05-11T23:03:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604235+0000", 2024-05-11T23:03:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8369999999999999e-06 2024-05-11T23:03:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604240+0000", 2024-05-11T23:03:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0329999999999999e-06 2024-05-11T23:03:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604244+0000", 2024-05-11T23:03:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9859999999999996e-06 2024-05-11T23:03:57.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663192+0000", 2024-05-11T23:03:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058948852000000003 2024-05-11T23:03:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663199+0000", 2024-05-11T23:03:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0140000000000001e-06 2024-05-11T23:03:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172787 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:57.655 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.991773062, 2024-05-11T23:03:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058982018999999997, 2024-05-11T23:03:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172787 2024-05-11T23:03:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:57.656 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:03:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605916+0000", 2024-05-11T23:03:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00166752 2024-05-11T23:03:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:57.657 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605921+0000", 2024-05-11T23:03:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0669999999999999e-06 2024-05-11T23:03:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605934+0000", 2024-05-11T23:03:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2271e-05 2024-05-11T23:03:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605940+0000", 2024-05-11T23:03:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6880000000000002e-06 2024-05-11T23:03:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.658 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663224+0000", 2024-05-11T23:03:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057283251 2024-05-11T23:03:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663231+0000", 2024-05-11T23:03:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2219999999999996e-06 2024-05-11T23:03:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172793 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.990006215000001, 2024-05-11T23:03:57.659 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057402373999999999, 2024-05-11T23:03:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172793 2024-05-11T23:03:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.660 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:03:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3300000000000001e-07 2024-05-11T23:03:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606017+0000", 2024-05-11T23:03:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4920000000000001e-06 2024-05-11T23:03:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.661 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606020+0000", 2024-05-11T23:03:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:03:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606023+0000", 2024-05-11T23:03:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6680000000000001e-06 2024-05-11T23:03:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663411+0000", 2024-05-11T23:03:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057388206999999997 2024-05-11T23:03:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.662 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663418+0000", 2024-05-11T23:03:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.815e-06 2024-05-11T23:03:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172795 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.989964788000002, 2024-05-11T23:03:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057423015000000001, 2024-05-11T23:03:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:57.663 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172795 2024-05-11T23:03:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:03:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9499999999999998e-07 2024-05-11T23:03:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606059+0000", 2024-05-11T23:03:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-05-11T23:03:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606062+0000", 2024-05-11T23:03:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.977e-06 2024-05-11T23:03:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606064+0000", 2024-05-11T23:03:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7410000000000001e-06 2024-05-11T23:03:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663473+0000", 2024-05-11T23:03:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057408624999999998 2024-05-11T23:03:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663480+0000", 2024-05-11T23:03:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1049999999999997e-06 2024-05-11T23:03:57.666 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172796 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.989954224999998, 2024-05-11T23:03:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057449235000000001, 2024-05-11T23:03:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:57.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172796 2024-05-11T23:03:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:57.668 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:03:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:03:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606069+0000", 2024-05-11T23:03:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:03:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606072+0000", 2024-05-11T23:03:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8700000000000001e-06 2024-05-11T23:03:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:57.669 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606074+0000", 2024-05-11T23:03:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6230000000000001e-06 2024-05-11T23:03:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663510+0000", 2024-05-11T23:03:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05743529 2024-05-11T23:03:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663517+0000", 2024-05-11T23:03:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1230000000000002e-06 2024-05-11T23:03:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172797 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:03:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 58.989944192999999, 2024-05-11T23:03:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057470420000000001, 2024-05-11T23:03:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172797 2024-05-11T23:03:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:57.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:03:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607915+0000", 2024-05-11T23:03:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018375609999999999 2024-05-11T23:03:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.672 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607920+0000", 2024-05-11T23:03:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8899999999999998e-06 2024-05-11T23:03:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607930+0000", 2024-05-11T23:03:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4429999999999992e-06 2024-05-11T23:03:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607936+0000", 2024-05-11T23:03:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7089999999999996e-06 2024-05-11T23:03:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:57.673 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663541+0000", 2024-05-11T23:03:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055604537000000002 2024-05-11T23:03:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663548+0000", 2024-05-11T23:03:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2799999999999998e-06 2024-05-11T23:03:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:57.674 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:57.674 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:57.676 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:03:57.676 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:57.676 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:57.677 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:57.677 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:57.677 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:57.678 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-11T23:03:57.678 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:57.678 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-11T23:03:57.679 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:57.675+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:57.777 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:57.776+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:57.829 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:57.829 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:57.829 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:03:57.829 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:57.839 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:03:57.877 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:57.877+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:57.979 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:57.979+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:57.992 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:57.993 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:03:57.993 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:03:57.993 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:03:57.993 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:58.002 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:03:58.080 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:58.079+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:58.112 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:03:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:03:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:03:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:03:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:77128 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_get in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-11T23:03:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 103.09610592999999, 2024-05-11T23:03:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91737855899999998, 2024-05-11T23:03:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:58.113 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 77128 2024-05-11T23:03:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:03:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016003+0000", 2024-05-11T23:03:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5379999999999999e-06 2024-05-11T23:03:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016011+0000", 2024-05-11T23:03:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6410000000000002e-06 2024-05-11T23:03:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016026+0000", 2024-05-11T23:03:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4779e-05 2024-05-11T23:03:58.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016032+0000", 2024-05-11T23:03:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.066e-06 2024-05-11T23:03:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016093+0000", 2024-05-11T23:03:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0878000000000002e-05 2024-05-11T23:03:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016100+0000", 2024-05-11T23:03:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6799999999999993e-06 2024-05-11T23:03:58.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933302+0000", 2024-05-11T23:03:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91720135300000005 2024-05-11T23:03:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933380+0000", 2024-05-11T23:03:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8623999999999998e-05 2024-05-11T23:03:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.117 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98950 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 91.980025041000005, 2024-05-11T23:03:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021419438999999998, 2024-05-11T23:03:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98950 2024-05-11T23:03:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:58.118 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:03:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.34e-07 2024-05-11T23:03:58.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132085+0000", 2024-05-11T23:03:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0049999999999999e-06 2024-05-11T23:03:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132093+0000", 2024-05-11T23:03:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6660000000000008e-06 2024-05-11T23:03:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132097+0000", 2024-05-11T23:03:58.120 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2590000000000002e-06 2024-05-11T23:03:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135114+0000", 2024-05-11T23:03:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003017573 2024-05-11T23:03:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153416+0000", 2024-05-11T23:03:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018301884000000001 2024-05-11T23:03:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.121 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153502+0000", 2024-05-11T23:03:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5717999999999995e-05 2024-05-11T23:03:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98959 3.1e 3:7807da99:::rbd_data.111ed2934718.000000000000006d:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 91.979532304000003, 2024-05-11T23:03:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020525928999999998, 2024-05-11T23:03:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98959 2024-05-11T23:03:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.123 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:03:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132576+0000", 2024-05-11T23:03:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4199999999999995e-07 2024-05-11T23:03:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132580+0000", 2024-05-11T23:03:58.124 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2910000000000001e-06 2024-05-11T23:03:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132588+0000", 2024-05-11T23:03:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7970000000000001e-06 2024-05-11T23:03:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132602+0000", 2024-05-11T23:03:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3743999999999999e-05 2024-05-11T23:03:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132655+0000", 2024-05-11T23:03:58.125 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2714999999999999e-05 2024-05-11T23:03:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132787+0000", 2024-05-11T23:03:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013273899999999999 2024-05-11T23:03:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153101+0000", 2024-05-11T23:03:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020313900999999999 2024-05-11T23:03:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.126 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98965 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 91.977008667999996, 2024-05-11T23:03:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018608983999999999, 2024-05-11T23:03:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98965 2024-05-11T23:03:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.127 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:03:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135100+0000", 2024-05-11T23:03:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.17e-06 2024-05-11T23:03:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:58.128 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135105+0000", 2024-05-11T23:03:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.899e-06 2024-05-11T23:03:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135122+0000", 2024-05-11T23:03:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7346999999999998e-05 2024-05-11T23:03:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135130+0000", 2024-05-11T23:03:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0639999999999994e-06 2024-05-11T23:03:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153533+0000", 2024-05-11T23:03:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018402753000000001 2024-05-11T23:03:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153615+0000", 2024-05-11T23:03:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1720999999999999e-05 2024-05-11T23:03:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153708+0000", 2024-05-11T23:03:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3029999999999995e-05 2024-05-11T23:03:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98966 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3760128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:03:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 91.976936542000004, 2024-05-11T23:03:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018716535999999999, 2024-05-11T23:03:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98966 2024-05-11T23:03:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.131 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:03:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135172+0000", 2024-05-11T23:03:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-05-11T23:03:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.132 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135175+0000", 2024-05-11T23:03:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-05-11T23:03:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135184+0000", 2024-05-11T23:03:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1840000000000005e-06 2024-05-11T23:03:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152903+0000", 2024-05-11T23:03:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017718357000000001 2024-05-11T23:03:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153736+0000", 2024-05-11T23:03:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00083324799999999997 2024-05-11T23:03:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153809+0000", 2024-05-11T23:03:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2995000000000002e-05 2024-05-11T23:03:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153888+0000", 2024-05-11T23:03:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8695999999999993e-05 2024-05-11T23:03:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "pg_update_log_missing_reply(3.1e epoch 72/53 rep_tid 104 lcod 72'363)", 2024-05-11T23:03:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 91.976873646000001, 2024-05-11T23:03:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017871951000000001, 2024-05-11T23:03:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:58.135 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:03:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:03:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135235+0000", 2024-05-11T23:03:58.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8899999999999999e-07 2024-05-11T23:03:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135238+0000", 2024-05-11T23:03:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.27e-06 2024-05-11T23:03:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152878+0000", 2024-05-11T23:03:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01763986 2024-05-11T23:03:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152896+0000", 2024-05-11T23:03:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8573e-05 2024-05-11T23:03:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152968+0000", 2024-05-11T23:03:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1780999999999996e-05 2024-05-11T23:03:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153106+0000", 2024-05-11T23:03:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013777799999999999 2024-05-11T23:03:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:239544 3.1a 3:5fe59cad:::rbd_data.111ebabb4604.000000000000016e:head [stat,delete] snapc 2f=[2f] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:03:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 54.457516914000003, 2024-05-11T23:03:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.50514007500000002, 2024-05-11T23:03:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 239544 2024-05-11T23:03:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:03:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:58.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654592+0000", 2024-05-11T23:03:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.404e-06 2024-05-11T23:03:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654598+0000", 2024-05-11T23:03:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7660000000000004e-06 2024-05-11T23:03:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654611+0000", 2024-05-11T23:03:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2676e-05 2024-05-11T23:03:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654620+0000", 2024-05-11T23:03:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2990000000000002e-06 2024-05-11T23:03:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654698+0000", 2024-05-11T23:03:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7739999999999998e-05 2024-05-11T23:03:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:03:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654710+0000", 2024-05-11T23:03:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2648e-05 2024-05-11T23:03:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.157889+0000", 2024-05-11T23:03:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.50317850799999997 2024-05-11T23:03:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.158050+0000", 2024-05-11T23:03:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016084500000000001 2024-05-11T23:03:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.159731+0000", 2024-05-11T23:03:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001681189 2024-05-11T23:03:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262364 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2613248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.525644049, 2024-05-11T23:03:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016312508, 2024-05-11T23:03:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262364 2024-05-11T23:03:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:03:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.13e-07 2024-05-11T23:03:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586467+0000", 2024-05-11T23:03:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-05-11T23:03:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586474+0000", 2024-05-11T23:03:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7739999999999999e-06 2024-05-11T23:03:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586483+0000", 2024-05-11T23:03:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0340000000000002e-06 2024-05-11T23:03:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602629+0000", 2024-05-11T23:03:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016145845999999998 2024-05-11T23:03:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602703+0000", 2024-05-11T23:03:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4671000000000004e-05 2024-05-11T23:03:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602776+0000", 2024-05-11T23:03:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2793000000000006e-05 2024-05-11T23:03:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262417 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.523897750000003, 2024-05-11T23:03:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016403456, 2024-05-11T23:03:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262417 2024-05-11T23:03:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:03:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6199999999999999e-07 2024-05-11T23:03:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588212+0000", 2024-05-11T23:03:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.075e-06 2024-05-11T23:03:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588218+0000", 2024-05-11T23:03:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8150000000000002e-06 2024-05-11T23:03:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588223+0000", 2024-05-11T23:03:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7090000000000001e-06 2024-05-11T23:03:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604461+0000", 2024-05-11T23:03:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016237900999999999 2024-05-11T23:03:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604540+0000", 2024-05-11T23:03:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9376999999999996e-05 2024-05-11T23:03:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604613+0000", 2024-05-11T23:03:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3217000000000001e-05 2024-05-11T23:03:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262418 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2396160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.523877841000001, 2024-05-11T23:03:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016591180000000001, 2024-05-11T23:03:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262418 2024-05-11T23:03:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:03:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6100000000000001e-07 2024-05-11T23:03:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:58.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588232+0000", 2024-05-11T23:03:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0370000000000001e-06 2024-05-11T23:03:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588238+0000", 2024-05-11T23:03:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9340000000000003e-06 2024-05-11T23:03:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588243+0000", 2024-05-11T23:03:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6179999999999997e-06 2024-05-11T23:03:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604642+0000", 2024-05-11T23:03:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0163991 2024-05-11T23:03:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604712+0000", 2024-05-11T23:03:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9935e-05 2024-05-11T23:03:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604821+0000", 2024-05-11T23:03:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000109095 2024-05-11T23:03:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262419 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2433024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.523858523000001, 2024-05-11T23:03:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016799692000000001, 2024-05-11T23:03:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262419 2024-05-11T23:03:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.157 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:58.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:03:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588252+0000", 2024-05-11T23:03:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0990000000000001e-06 2024-05-11T23:03:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588257+0000", 2024-05-11T23:03:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8159999999999999e-06 2024-05-11T23:03:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.159 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588262+0000", 2024-05-11T23:03:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8069999999999999e-06 2024-05-11T23:03:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604868+0000", 2024-05-11T23:03:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016605880999999999 2024-05-11T23:03:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604966+0000", 2024-05-11T23:03:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7597999999999996e-05 2024-05-11T23:03:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605049+0000", 2024-05-11T23:03:58.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3141999999999998e-05 2024-05-11T23:03:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262420 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2469888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.523838412000003, 2024-05-11T23:03:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016972514000000001, 2024-05-11T23:03:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.161 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262420 2024-05-11T23:03:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:03:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:58.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588271+0000", 2024-05-11T23:03:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6199999999999999e-06 2024-05-11T23:03:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588273+0000", 2024-05-11T23:03:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3209999999999999e-06 2024-05-11T23:03:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588279+0000", 2024-05-11T23:03:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2519999999999996e-06 2024-05-11T23:03:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588285+0000", 2024-05-11T23:03:58.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1440000000000002e-06 2024-05-11T23:03:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605083+0000", 2024-05-11T23:03:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016798418999999998 2024-05-11T23:03:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605168+0000", 2024-05-11T23:03:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4851000000000005e-05 2024-05-11T23:03:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605242+0000", 2024-05-11T23:03:58.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3906999999999996e-05 2024-05-11T23:03:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262421 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2506752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.523816265999997, 2024-05-11T23:03:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017141427000000001, 2024-05-11T23:03:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.165 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262421 2024-05-11T23:03:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:03:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:58.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588292+0000", 2024-05-11T23:03:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588318+0000", 2024-05-11T23:03:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6376e-05 2024-05-11T23:03:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588326+0000", 2024-05-11T23:03:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6699999999999994e-06 2024-05-11T23:03:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588332+0000", 2024-05-11T23:03:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8540000000000002e-06 2024-05-11T23:03:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605274+0000", 2024-05-11T23:03:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016942024999999999 2024-05-11T23:03:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605360+0000", 2024-05-11T23:03:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6304999999999994e-05 2024-05-11T23:03:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605433+0000", 2024-05-11T23:03:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2772000000000001e-05 2024-05-11T23:03:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262422 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2543616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.523767042000003, 2024-05-11T23:03:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017283705, 2024-05-11T23:03:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:58.169 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262422 2024-05-11T23:03:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:58.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:03:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:03:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588343+0000", 2024-05-11T23:03:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.34e-06 2024-05-11T23:03:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:58.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588350+0000", 2024-05-11T23:03:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1569999999999997e-06 2024-05-11T23:03:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588354+0000", 2024-05-11T23:03:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.865e-06 2024-05-11T23:03:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605466+0000", 2024-05-11T23:03:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017111498999999999 2024-05-11T23:03:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605551+0000", 2024-05-11T23:03:58.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5210999999999994e-05 2024-05-11T23:03:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605624+0000", 2024-05-11T23:03:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3153999999999998e-05 2024-05-11T23:03:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262423 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2580480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.523746793000001, 2024-05-11T23:03:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017451194, 2024-05-11T23:03:58.173 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262423 2024-05-11T23:03:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:03:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588363+0000", 2024-05-11T23:03:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.855e-06 2024-05-11T23:03:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:58.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588369+0000", 2024-05-11T23:03:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1260000000000003e-06 2024-05-11T23:03:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588375+0000", 2024-05-11T23:03:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9630000000000003e-06 2024-05-11T23:03:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605656+0000", 2024-05-11T23:03:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017280865999999999 2024-05-11T23:03:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605737+0000", 2024-05-11T23:03:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0773999999999996e-05 2024-05-11T23:03:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605812+0000", 2024-05-11T23:03:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5203999999999994e-05 2024-05-11T23:03:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.178 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262424 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2617344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.523725274999997, 2024-05-11T23:03:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017617544999999998, 2024-05-11T23:03:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262424 2024-05-11T23:03:58.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:03:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588383+0000", 2024-05-11T23:03:58.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8299999999999997e-07 2024-05-11T23:03:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588385+0000", 2024-05-11T23:03:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2869999999999999e-06 2024-05-11T23:03:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588391+0000", 2024-05-11T23:03:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1689999999999997e-06 2024-05-11T23:03:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588396+0000", 2024-05-11T23:03:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5719999999999996e-06 2024-05-11T23:03:58.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605837+0000", 2024-05-11T23:03:58.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017440812999999999 2024-05-11T23:03:58.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:58.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605917+0000", 2024-05-11T23:03:58.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9924000000000003e-05 2024-05-11T23:03:58.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.182 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.182 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:58.182 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.606000+0000", 2024-05-11T23:03:58.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3296999999999999e-05 2024-05-11T23:03:58.182 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.182 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.183 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.183 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262896 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:58.183 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:58.183 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.452720638000002, 2024-05-11T23:03:58.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016295738000000001, 2024-05-11T23:03:58.183 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:58.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:58.183 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:58.183 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:58.183 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:58.183 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.183 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262896 2024-05-11T23:03:58.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.184 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:58.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:58.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:58.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:58.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:58.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:58.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:03:58.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:03:58.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:58.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659390+0000", 2024-05-11T23:03:58.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1440000000000001e-06 2024-05-11T23:03:58.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:58.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659397+0000", 2024-05-11T23:03:58.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0010000000000004e-06 2024-05-11T23:03:58.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659402+0000", 2024-05-11T23:03:58.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2179999999999998e-06 2024-05-11T23:03:58.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675487+0000", 2024-05-11T23:03:58.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016085706000000002 2024-05-11T23:03:58.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:58.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675580+0000", 2024-05-11T23:03:58.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2884e-05 2024-05-11T23:03:58.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.186 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:58.186 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675683+0000", 2024-05-11T23:03:58.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010236 2024-05-11T23:03:58.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.186 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.187 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262897 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:58.187 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:58.187 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.452685297000002, 2024-05-11T23:03:58.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016500932999999999, 2024-05-11T23:03:58.187 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:58.187 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:58.187 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:58.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:58.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:58.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.187 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262897 2024-05-11T23:03:58.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.188 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:58.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:58.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:58.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:58.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:03:58.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:58.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659423+0000", 2024-05-11T23:03:58.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9699999999999996e-07 2024-05-11T23:03:58.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:58.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659425+0000", 2024-05-11T23:03:58.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3199999999999998e-06 2024-05-11T23:03:58.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:58.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659433+0000", 2024-05-11T23:03:58.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3660000000000003e-06 2024-05-11T23:03:58.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659445+0000", 2024-05-11T23:03:58.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2102e-05 2024-05-11T23:03:58.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675727+0000", 2024-05-11T23:03:58.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016282331000000001 2024-05-11T23:03:58.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:58.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675815+0000", 2024-05-11T23:03:58.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7484999999999993e-05 2024-05-11T23:03:58.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:58.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675923+0000", 2024-05-11T23:03:58.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000108732 2024-05-11T23:03:58.191 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.191 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.191 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.191 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262898 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:58.191 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:58.191 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.452654179, 2024-05-11T23:03:58.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016685855999999999, 2024-05-11T23:03:58.191 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:58.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:58.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:58.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:58.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:58.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262898 2024-05-11T23:03:58.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.192 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:58.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:58.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:58.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:58.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:03:58.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:58.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659454+0000", 2024-05-11T23:03:58.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2200000000000004e-07 2024-05-11T23:03:58.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:58.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659457+0000", 2024-05-11T23:03:58.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.779e-06 2024-05-11T23:03:58.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:58.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659463+0000", 2024-05-11T23:03:58.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6660000000000002e-06 2024-05-11T23:03:58.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659472+0000", 2024-05-11T23:03:58.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2139999999999996e-06 2024-05-11T23:03:58.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675956+0000", 2024-05-11T23:03:58.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016484431000000001 2024-05-11T23:03:58.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:58.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676039+0000", 2024-05-11T23:03:58.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3358999999999999e-05 2024-05-11T23:03:58.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.195 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:58.195 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676139+0000", 2024-05-11T23:03:58.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9784999999999994e-05 2024-05-11T23:03:58.195 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.195 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.195 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.195 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262899 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:58.195 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:58.195 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 45.452628347999998, 2024-05-11T23:03:58.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017030087999999999, 2024-05-11T23:03:58.195 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:03:58.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:03:58.195 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:03:58.196 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:03:58.196 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:03:58.196 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.196 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262899 2024-05-11T23:03:58.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.196 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:03:58.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:03:58.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:58.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:03:58.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:03:58.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:03:58.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:03:58.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659480+0000", 2024-05-11T23:03:58.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7700000000000004e-07 2024-05-11T23:03:58.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:03:58.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659482+0000", 2024-05-11T23:03:58.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1390000000000002e-06 2024-05-11T23:03:58.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:03:58.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659489+0000", 2024-05-11T23:03:58.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8279999999999997e-06 2024-05-11T23:03:58.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659494+0000", 2024-05-11T23:03:58.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9019999999999998e-06 2024-05-11T23:03:58.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:03:58.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676167+0000", 2024-05-11T23:03:58.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016673621 2024-05-11T23:03:58.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:03:58.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676329+0000", 2024-05-11T23:03:58.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016206099999999999 2024-05-11T23:03:58.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:03:58.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:03:58.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:03:58.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676509+0000", 2024-05-11T23:03:58.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017996 2024-05-11T23:03:58.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.199 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:03:58.199 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:03:58.199 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:03:58.201 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:03:58.201 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:58.200+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:58.301 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:58.301+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:03:58.309 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:58.309 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:58.309 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:58.309 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:58.318 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:03:58.402 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:58.402+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:58.423 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:58.423 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:58.423 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:58.423 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:58.423 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:58.431 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:03:58.516 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:58.515+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:58.588 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:58.588 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:58.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:58.588 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:58.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.588 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116446 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8273920~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:58.589 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:58.589 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.202313950999994, 2024-05-11T23:03:58.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045535299000000001, 2024-05-11T23:03:58.589 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:58.589 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:58.589 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:58.589 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:58.589 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:58.589 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.589 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116446 2024-05-11T23:03:58.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.589 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:58.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:58.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:58.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:58.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:03:58.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:58.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375547+0000", 2024-05-11T23:03:58.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4000000000000001e-07 2024-05-11T23:03:58.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:58.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375549+0000", 2024-05-11T23:03:58.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.368e-06 2024-05-11T23:03:58.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:58.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375556+0000", 2024-05-11T23:03:58.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1840000000000004e-06 2024-05-11T23:03:58.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375560+0000", 2024-05-11T23:03:58.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.741e-06 2024-05-11T23:03:58.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420896+0000", 2024-05-11T23:03:58.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045335435 2024-05-11T23:03:58.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:58.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420987+0000", 2024-05-11T23:03:58.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1033999999999996e-05 2024-05-11T23:03:58.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:58.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421082+0000", 2024-05-11T23:03:58.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4896999999999996e-05 2024-05-11T23:03:58.594 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.594 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.594 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.594 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116447 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6316032~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:58.595 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:58.595 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.202292440999997, 2024-05-11T23:03:58.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045745830000000001, 2024-05-11T23:03:58.595 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:58.595 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:58.595 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:58.595 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:58.595 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:58.595 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.595 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116447 2024-05-11T23:03:58.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.595 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:58.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:58.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:58.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:58.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:58.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:58.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:03:58.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.58e-07 2024-05-11T23:03:58.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:58.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375570+0000", 2024-05-11T23:03:58.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:03:58.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:58.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375576+0000", 2024-05-11T23:03:58.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7939999999999999e-06 2024-05-11T23:03:58.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375581+0000", 2024-05-11T23:03:58.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9280000000000001e-06 2024-05-11T23:03:58.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421127+0000", 2024-05-11T23:03:58.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045545785999999998 2024-05-11T23:03:58.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:58.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421214+0000", 2024-05-11T23:03:58.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7039000000000005e-05 2024-05-11T23:03:58.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:58.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421314+0000", 2024-05-11T23:03:58.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9772999999999995e-05 2024-05-11T23:03:58.598 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.598 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.598 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.599 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116448 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8314880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:58.599 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:58.599 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.202273367000004, 2024-05-11T23:03:58.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045941753000000002, 2024-05-11T23:03:58.599 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:58.599 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:58.599 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:58.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:58.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:58.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.599 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116448 2024-05-11T23:03:58.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.599 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:58.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:58.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:58.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:58.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:58.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:58.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:03:58.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.77e-07 2024-05-11T23:03:58.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:58.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375589+0000", 2024-05-11T23:03:58.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.091e-06 2024-05-11T23:03:58.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:58.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375596+0000", 2024-05-11T23:03:58.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9179999999999997e-06 2024-05-11T23:03:58.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375601+0000", 2024-05-11T23:03:58.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0030000000000002e-06 2024-05-11T23:03:58.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421363+0000", 2024-05-11T23:03:58.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045761352999999998 2024-05-11T23:03:58.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:58.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421440+0000", 2024-05-11T23:03:58.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7585999999999999e-05 2024-05-11T23:03:58.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:58.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421529+0000", 2024-05-11T23:03:58.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8424999999999994e-05 2024-05-11T23:03:58.602 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.602 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.602 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.603 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116449 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6438912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:58.603 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:58.603 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.202252931999993, 2024-05-11T23:03:58.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046082208999999999, 2024-05-11T23:03:58.603 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:58.603 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:58.603 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:58.603 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:58.603 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:58.603 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.603 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116449 2024-05-11T23:03:58.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.603 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:58.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:58.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:58.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:58.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:03:58.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:58.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377225+0000", 2024-05-11T23:03:58.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016177030000000001 2024-05-11T23:03:58.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:58.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377229+0000", 2024-05-11T23:03:58.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8e-06 2024-05-11T23:03:58.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:58.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377238+0000", 2024-05-11T23:03:58.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2229999999999996e-06 2024-05-11T23:03:58.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377244+0000", 2024-05-11T23:03:58.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4060000000000002e-06 2024-05-11T23:03:58.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421520+0000", 2024-05-11T23:03:58.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044275222000000003 2024-05-11T23:03:58.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:58.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421595+0000", 2024-05-11T23:03:58.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4838000000000004e-05 2024-05-11T23:03:58.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:58.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421690+0000", 2024-05-11T23:03:58.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5017000000000002e-05 2024-05-11T23:03:58.607 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.607 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.607 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.607 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116458 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7094272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:58.607 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:58.607 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.200438715000004, 2024-05-11T23:03:58.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045706877999999999, 2024-05-11T23:03:58.607 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:58.607 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:58.608 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:58.608 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:58.608 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:58.608 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.608 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116458 2024-05-11T23:03:58.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.608 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:58.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:58.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:58.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:58.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:58.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:58.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:03:58.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9400000000000005e-07 2024-05-11T23:03:58.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:58.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377424+0000", 2024-05-11T23:03:58.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1859999999999999e-06 2024-05-11T23:03:58.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:58.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377431+0000", 2024-05-11T23:03:58.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4049999999999997e-06 2024-05-11T23:03:58.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377436+0000", 2024-05-11T23:03:58.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1399999999999999e-06 2024-05-11T23:03:58.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.422945+0000", 2024-05-11T23:03:58.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045509326000000003 2024-05-11T23:03:58.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:58.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423036+0000", 2024-05-11T23:03:58.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0649999999999994e-05 2024-05-11T23:03:58.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:58.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423128+0000", 2024-05-11T23:03:58.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2577000000000005e-05 2024-05-11T23:03:58.611 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.611 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.611 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.611 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116459 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7258112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:58.611 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:58.611 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.200417982999994, 2024-05-11T23:03:58.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045831945999999998, 2024-05-11T23:03:58.611 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:58.612 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:58.612 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:58.612 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:58.612 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:58.612 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.612 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116459 2024-05-11T23:03:58.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.612 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:58.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:58.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:58.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:58.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:03:58.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:58.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377443+0000", 2024-05-11T23:03:58.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999999e-07 2024-05-11T23:03:58.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:58.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377445+0000", 2024-05-11T23:03:58.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0209999999999998e-06 2024-05-11T23:03:58.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:58.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377451+0000", 2024-05-11T23:03:58.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.071e-06 2024-05-11T23:03:58.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377456+0000", 2024-05-11T23:03:58.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8289999999999998e-06 2024-05-11T23:03:58.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423155+0000", 2024-05-11T23:03:58.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045699447999999997 2024-05-11T23:03:58.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:58.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423200+0000", 2024-05-11T23:03:58.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5083000000000002e-05 2024-05-11T23:03:58.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:58.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423274+0000", 2024-05-11T23:03:58.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4046000000000002e-05 2024-05-11T23:03:58.615 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.615 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.615 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.615 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116460 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7299072~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:58.615 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:58.615 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.200398921000001, 2024-05-11T23:03:58.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046010987000000003, 2024-05-11T23:03:58.616 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:58.616 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:58.616 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:58.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:58.616 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:58.618 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.618 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116460 2024-05-11T23:03:58.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.618 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:58.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:58.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:58.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:58.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:03:58.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:58.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377462+0000", 2024-05-11T23:03:58.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:03:58.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:58.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377464+0000", 2024-05-11T23:03:58.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1600000000000001e-06 2024-05-11T23:03:58.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:58.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377469+0000", 2024-05-11T23:03:58.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5430000000000001e-06 2024-05-11T23:03:58.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377474+0000", 2024-05-11T23:03:58.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8509999999999998e-06 2024-05-11T23:03:58.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423312+0000", 2024-05-11T23:03:58.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045838080000000003 2024-05-11T23:03:58.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.620 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:58.620 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423401+0000", 2024-05-11T23:03:58.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8170000000000005e-05 2024-05-11T23:03:58.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:58.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423472+0000", 2024-05-11T23:03:58.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1847000000000005e-05 2024-05-11T23:03:58.621 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.621 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.621 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.621 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116461 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7421952~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:58.621 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:58.621 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.200380151000005, 2024-05-11T23:03:58.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046130749999999998, 2024-05-11T23:03:58.621 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:58.622 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:58.622 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:58.622 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:58.622 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:58.622 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.622 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116461 2024-05-11T23:03:58.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.622 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:58.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:58.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:58.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:58.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:58.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:58.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:03:58.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4700000000000002e-07 2024-05-11T23:03:58.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:58.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377483+0000", 2024-05-11T23:03:58.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0219999999999999e-06 2024-05-11T23:03:58.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:58.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377489+0000", 2024-05-11T23:03:58.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4420000000000003e-06 2024-05-11T23:03:58.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377494+0000", 2024-05-11T23:03:58.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9239999999999998e-06 2024-05-11T23:03:58.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423500+0000", 2024-05-11T23:03:58.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046006054999999997 2024-05-11T23:03:58.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423557+0000", 2024-05-11T23:03:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7169000000000001e-05 2024-05-11T23:03:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423611+0000", 2024-05-11T23:03:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3791e-05 2024-05-11T23:03:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.629 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116462 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7462912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.200360532999994, 2024-05-11T23:03:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046254666, 2024-05-11T23:03:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:58.630 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116462 2024-05-11T23:03:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:58.631 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:03:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377502+0000", 2024-05-11T23:03:58.632 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-05-11T23:03:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377508+0000", 2024-05-11T23:03:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4500000000000003e-06 2024-05-11T23:03:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377512+0000", 2024-05-11T23:03:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7700000000000001e-06 2024-05-11T23:03:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.633 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423649+0000", 2024-05-11T23:03:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046136859000000002 2024-05-11T23:03:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423704+0000", 2024-05-11T23:03:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4938000000000002e-05 2024-05-11T23:03:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423754+0000", 2024-05-11T23:03:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0145999999999997e-05 2024-05-11T23:03:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.634 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116463 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7585792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.200342106999997, 2024-05-11T23:03:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361763, 2024-05-11T23:03:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:58.635 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116463 2024-05-11T23:03:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:03:58.636 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377519+0000", 2024-05-11T23:03:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2600000000000002e-07 2024-05-11T23:03:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377521+0000", 2024-05-11T23:03:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1710000000000001e-06 2024-05-11T23:03:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377527+0000", 2024-05-11T23:03:58.637 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9649999999999996e-06 2024-05-11T23:03:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377532+0000", 2024-05-11T23:03:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.78e-06 2024-05-11T23:03:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423779+0000", 2024-05-11T23:03:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046247706 2024-05-11T23:03:58.638 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423829+0000", 2024-05-11T23:03:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9188999999999999e-05 2024-05-11T23:03:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423880+0000", 2024-05-11T23:03:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1425999999999999e-05 2024-05-11T23:03:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.639 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116464 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7749632~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.200322870999997, 2024-05-11T23:03:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046491719000000001, 2024-05-11T23:03:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:58.640 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116464 2024-05-11T23:03:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.641 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:03:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377538+0000", 2024-05-11T23:03:58.642 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4499999999999998e-07 2024-05-11T23:03:58.643 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.643 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.643 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:58.643 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377540+0000", 2024-05-11T23:03:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2340000000000001e-06 2024-05-11T23:03:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377546+0000", 2024-05-11T23:03:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8289999999999996e-06 2024-05-11T23:03:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.644 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377551+0000", 2024-05-11T23:03:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8559999999999998e-06 2024-05-11T23:03:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423913+0000", 2024-05-11T23:03:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361885999999998 2024-05-11T23:03:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.645 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.646 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:58.646 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423958+0000", 2024-05-11T23:03:58.646 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5513999999999998e-05 2024-05-11T23:03:58.646 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.646 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424029+0000", 2024-05-11T23:03:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1055000000000003e-05 2024-05-11T23:03:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.647 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116465 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7626752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.200303723000005, 2024-05-11T23:03:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046600057, 2024-05-11T23:03:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116465 2024-05-11T23:03:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.648 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:03:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3599999999999999e-07 2024-05-11T23:03:58.649 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377559+0000", 2024-05-11T23:03:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9640000000000002e-06 2024-05-11T23:03:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377572+0000", 2024-05-11T23:03:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3188e-05 2024-05-11T23:03:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377578+0000", 2024-05-11T23:03:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.942e-06 2024-05-11T23:03:58.650 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424058+0000", 2024-05-11T23:03:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046480021000000003 2024-05-11T23:03:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424103+0000", 2024-05-11T23:03:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5021999999999997e-05 2024-05-11T23:03:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424157+0000", 2024-05-11T23:03:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3483999999999998e-05 2024-05-11T23:03:58.651 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116466 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7790592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.200275141000006, 2024-05-11T23:03:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046743741999999998, 2024-05-11T23:03:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:58.652 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116466 2024-05-11T23:03:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:58.653 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:03:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377586+0000", 2024-05-11T23:03:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:03:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.654 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377588+0000", 2024-05-11T23:03:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3039999999999999e-06 2024-05-11T23:03:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377594+0000", 2024-05-11T23:03:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-05-11T23:03:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377600+0000", 2024-05-11T23:03:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7699999999999998e-06 2024-05-11T23:03:58.655 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424185+0000", 2024-05-11T23:03:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046585891999999997 2024-05-11T23:03:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424235+0000", 2024-05-11T23:03:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9356e-05 2024-05-11T23:03:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424329+0000", 2024-05-11T23:03:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4036999999999994e-05 2024-05-11T23:03:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.656 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116467 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7954432~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.200254853999994, 2024-05-11T23:03:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046930134999999998, 2024-05-11T23:03:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116467 2024-05-11T23:03:58.657 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:03:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:58.658 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379161+0000", 2024-05-11T23:03:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015555009999999999 2024-05-11T23:03:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379166+0000", 2024-05-11T23:03:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6689999999999997e-06 2024-05-11T23:03:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379176+0000", 2024-05-11T23:03:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0549e-05 2024-05-11T23:03:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379183+0000", 2024-05-11T23:03:58.659 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1199999999999996e-06 2024-05-11T23:03:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424359+0000", 2024-05-11T23:03:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045175949999999999 2024-05-11T23:03:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424433+0000", 2024-05-11T23:03:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3824000000000003e-05 2024-05-11T23:03:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424536+0000", 2024-05-11T23:03:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010252200000000001 2024-05-11T23:03:58.660 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116469 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8077312~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.198614814999999, 2024-05-11T23:03:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045636687000000002, 2024-05-11T23:03:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:58.661 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116469 2024-05-11T23:03:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:03:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.662 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379246+0000", 2024-05-11T23:03:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.32e-07 2024-05-11T23:03:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379254+0000", 2024-05-11T23:03:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.807e-06 2024-05-11T23:03:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379261+0000", 2024-05-11T23:03:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.985e-06 2024-05-11T23:03:58.663 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379264+0000", 2024-05-11T23:03:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6480000000000001e-06 2024-05-11T23:03:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424744+0000", 2024-05-11T23:03:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045479638000000003 2024-05-11T23:03:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424797+0000", 2024-05-11T23:03:58.664 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2725000000000001e-05 2024-05-11T23:03:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424882+0000", 2024-05-11T23:03:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5451999999999994e-05 2024-05-11T23:03:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116470 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8118272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.198589603000002, 2024-05-11T23:03:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045755745, 2024-05-11T23:03:58.665 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116470 2024-05-11T23:03:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.666 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:03:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3999999999999997e-07 2024-05-11T23:03:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379273+0000", 2024-05-11T23:03:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-05-11T23:03:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.667 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379277+0000", 2024-05-11T23:03:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4020000000000004e-06 2024-05-11T23:03:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379280+0000", 2024-05-11T23:03:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-05-11T23:03:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424922+0000", 2024-05-11T23:03:58.668 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045642598999999999 2024-05-11T23:03:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424973+0000", 2024-05-11T23:03:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0674999999999998e-05 2024-05-11T23:03:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425026+0000", 2024-05-11T23:03:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3492000000000002e-05 2024-05-11T23:03:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.669 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116471 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8241152~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.19857682, 2024-05-11T23:03:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045867374000000002, 2024-05-11T23:03:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116471 2024-05-11T23:03:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:58.670 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:03:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379284+0000", 2024-05-11T23:03:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.53e-07 2024-05-11T23:03:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.671 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379285+0000", 2024-05-11T23:03:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.223e-06 2024-05-11T23:03:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379288+0000", 2024-05-11T23:03:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5379999999999998e-06 2024-05-11T23:03:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379291+0000", 2024-05-11T23:03:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6699999999999998e-06 2024-05-11T23:03:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.672 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425055+0000", 2024-05-11T23:03:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045763497 2024-05-11T23:03:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425099+0000", 2024-05-11T23:03:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3899000000000001e-05 2024-05-11T23:03:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425151+0000", 2024-05-11T23:03:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2293999999999998e-05 2024-05-11T23:03:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.673 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116472 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8282112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.198565821000003, 2024-05-11T23:03:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045973778999999999, 2024-05-11T23:03:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116472 2024-05-11T23:03:58.674 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:03:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381226+0000", 2024-05-11T23:03:58.675 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001931627 2024-05-11T23:03:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381231+0000", 2024-05-11T23:03:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8679999999999998e-06 2024-05-11T23:03:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381242+0000", 2024-05-11T23:03:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0702e-05 2024-05-11T23:03:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381249+0000", 2024-05-11T23:03:58.676 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9800000000000001e-06 2024-05-11T23:03:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425175+0000", 2024-05-11T23:03:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043926805999999999 2024-05-11T23:03:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425215+0000", 2024-05-11T23:03:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9737999999999998e-05 2024-05-11T23:03:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425268+0000", 2024-05-11T23:03:58.677 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3057999999999999e-05 2024-05-11T23:03:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116482 3.1b 3:db6e8843:::rbd_object_map.111e67f27da3.0000000000000028:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:03:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 83.196597277999999, 2024-05-11T23:03:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046526076, 2024-05-11T23:03:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:58.678 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116482 2024-05-11T23:03:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:03:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.679 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381264+0000", 2024-05-11T23:03:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5300000000000004e-07 2024-05-11T23:03:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381267+0000", 2024-05-11T23:03:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1999999999999999e-06 2024-05-11T23:03:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381274+0000", 2024-05-11T23:03:58.680 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7109999999999995e-06 2024-05-11T23:03:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381285+0000", 2024-05-11T23:03:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0856000000000001e-05 2024-05-11T23:03:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425292+0000", 2024-05-11T23:03:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044006376 2024-05-11T23:03:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425337+0000", 2024-05-11T23:03:58.681 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4907999999999998e-05 2024-05-11T23:03:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:03:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425507+0000", 2024-05-11T23:03:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017029999999999999 2024-05-11T23:03:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427668+0000", 2024-05-11T23:03:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021612430000000002 2024-05-11T23:03:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427683+0000", 2024-05-11T23:03:58.682 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5016e-05 2024-05-11T23:03:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427742+0000", 2024-05-11T23:03:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8811999999999999e-05 2024-05-11T23:03:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427789+0000", 2024-05-11T23:03:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7101000000000002e-05 2024-05-11T23:03:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.683 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:283279 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-05-11T23:03:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:53.865673+0000", 2024-05-11T23:03:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 4.7121872620000005, 2024-05-11T23:03:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.93944476799999999, 2024-05-11T23:03:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 283279 2024-05-11T23:03:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.684 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865673+0000", 2024-05-11T23:03:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865673+0000", 2024-05-11T23:03:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865674+0000", 2024-05-11T23:03:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.012e-06 2024-05-11T23:03:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.685 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865679+0000", 2024-05-11T23:03:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7740000000000004e-06 2024-05-11T23:03:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865689+0000", 2024-05-11T23:03:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0644e-05 2024-05-11T23:03:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865694+0000", 2024-05-11T23:03:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.826e-06 2024-05-11T23:03:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.686 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865757+0000", 2024-05-11T23:03:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2493e-05 2024-05-11T23:03:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:03:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865768+0000", 2024-05-11T23:03:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0777e-05 2024-05-11T23:03:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.802182+0000", 2024-05-11T23:03:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.93641407499999996 2024-05-11T23:03:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.687 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.802338+0000", 2024-05-11T23:03:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015636599999999999 2024-05-11T23:03:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 6", 2024-05-11T23:03:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.802561+0000", 2024-05-11T23:03:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000222585 2024-05-11T23:03:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.805050+0000", 2024-05-11T23:03:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024896300000000001 2024-05-11T23:03:58.688 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.805059+0000", 2024-05-11T23:03:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6829999999999997e-06 2024-05-11T23:03:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.805093+0000", 2024-05-11T23:03:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3763000000000002e-05 2024-05-11T23:03:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.805118+0000", 2024-05-11T23:03:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.514e-05 2024-05-11T23:03:58.689 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.690 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.690 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.690 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:58.690 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:58.690 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:58.690 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:03:58.691 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:58.622+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:58.719 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:58.719+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:58.797 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:58.797 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:58.797 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:58.797 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:58.806 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:03:58.820 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:58.820+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:58.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-11T23:03:58.921 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:58.920+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:58.923 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:58.923 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:58.923 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:58.923 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:58.923 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:58.933 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:03:59.023 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:59.022+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:59.044 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:59.044 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:59.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:59.044 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:59.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.044 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267820 3.16 3:6b963329:::rbd_data.111ec06c817a.000000000000000a:33 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:59.044 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:03:59.044 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.100634835000001, 2024-05-11T23:03:59.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050141218000000001, 2024-05-11T23:03:59.045 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.045 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.045 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.045 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.045 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.045 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.045 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267820 2024-05-11T23:03:59.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.045 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:03:59.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:03:59.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942754+0000", 2024-05-11T23:03:59.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8559000000000003e-05 2024-05-11T23:03:59.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942756+0000", 2024-05-11T23:03:59.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.604e-06 2024-05-11T23:03:59.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942763+0000", 2024-05-11T23:03:59.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7939999999999997e-06 2024-05-11T23:03:59.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942768+0000", 2024-05-11T23:03:59.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4410000000000001e-06 2024-05-11T23:03:59.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992601+0000", 2024-05-11T23:03:59.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049832754999999999 2024-05-11T23:03:59.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992687+0000", 2024-05-11T23:03:59.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5605000000000005e-05 2024-05-11T23:03:59.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992796+0000", 2024-05-11T23:03:59.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010946 2024-05-11T23:03:59.048 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.048 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.048 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.048 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267852 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:59.048 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:03:59.049 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.100513630999998, 2024-05-11T23:03:59.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050212600000000003, 2024-05-11T23:03:59.049 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.049 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.049 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.049 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.049 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.049 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.049 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267852 2024-05-11T23:03:59.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.049 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:03:59.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:03:59.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943228+0000", 2024-05-11T23:03:59.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00045180600000000002 2024-05-11T23:03:59.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943232+0000", 2024-05-11T23:03:59.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8399999999999997e-06 2024-05-11T23:03:59.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943240+0000", 2024-05-11T23:03:59.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8900000000000007e-06 2024-05-11T23:03:59.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943246+0000", 2024-05-11T23:03:59.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6239999999999996e-06 2024-05-11T23:03:59.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992828+0000", 2024-05-11T23:03:59.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049581247000000002 2024-05-11T23:03:59.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992905+0000", 2024-05-11T23:03:59.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7656000000000003e-05 2024-05-11T23:03:59.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992989+0000", 2024-05-11T23:03:59.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3536999999999997e-05 2024-05-11T23:03:59.052 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.052 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.052 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.052 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267854 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1134592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:59.053 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:03:59.053 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.100000489999999, 2024-05-11T23:03:59.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050077425000000002, 2024-05-11T23:03:59.053 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.053 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.053 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.053 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.053 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.053 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.053 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267854 2024-05-11T23:03:59.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.053 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:03:59.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:03:59.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943290+0000", 2024-05-11T23:03:59.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2300000000000002e-07 2024-05-11T23:03:59.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943291+0000", 2024-05-11T23:03:59.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6190000000000001e-06 2024-05-11T23:03:59.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943296+0000", 2024-05-11T23:03:59.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2889999999999999e-06 2024-05-11T23:03:59.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943299+0000", 2024-05-11T23:03:59.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1030000000000002e-06 2024-05-11T23:03:59.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993205+0000", 2024-05-11T23:03:59.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049906405000000001 2024-05-11T23:03:59.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993285+0000", 2024-05-11T23:03:59.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9752999999999993e-05 2024-05-11T23:03:59.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993367+0000", 2024-05-11T23:03:59.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1933000000000003e-05 2024-05-11T23:03:59.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.056 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.056 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.056 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267855 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1171456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:59.057 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:03:59.057 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.099983547999997, 2024-05-11T23:03:59.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050250346000000001, 2024-05-11T23:03:59.057 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.057 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.057 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.057 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.057 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.057 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.057 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267855 2024-05-11T23:03:59.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.057 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:03:59.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:03:59.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945226+0000", 2024-05-11T23:03:59.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001919235 2024-05-11T23:03:59.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945230+0000", 2024-05-11T23:03:59.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9580000000000001e-06 2024-05-11T23:03:59.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945238+0000", 2024-05-11T23:03:59.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1040000000000007e-06 2024-05-11T23:03:59.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945244+0000", 2024-05-11T23:03:59.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6919999999999996e-06 2024-05-11T23:03:59.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993399+0000", 2024-05-11T23:03:59.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048154651 2024-05-11T23:03:59.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993472+0000", 2024-05-11T23:03:59.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2780000000000005e-05 2024-05-11T23:03:59.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993557+0000", 2024-05-11T23:03:59.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4926000000000006e-05 2024-05-11T23:03:59.060 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.060 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.060 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.061 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267865 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1540096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:59.061 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:59.061 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.097831921999997, 2024-05-11T23:03:59.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050054409000000001, 2024-05-11T23:03:59.061 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.061 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.061 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.061 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.061 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.061 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.061 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267865 2024-05-11T23:03:59.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.061 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:59.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:59.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:03:59.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6600000000000002e-07 2024-05-11T23:03:59.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945461+0000", 2024-05-11T23:03:59.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2670000000000001e-06 2024-05-11T23:03:59.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945466+0000", 2024-05-11T23:03:59.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7200000000000003e-06 2024-05-11T23:03:59.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945471+0000", 2024-05-11T23:03:59.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9439999999999996e-06 2024-05-11T23:03:59.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995337+0000", 2024-05-11T23:03:59.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049865415000000003 2024-05-11T23:03:59.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995413+0000", 2024-05-11T23:03:59.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5989999999999996e-05 2024-05-11T23:03:59.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995512+0000", 2024-05-11T23:03:59.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9706999999999999e-05 2024-05-11T23:03:59.065 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.065 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.065 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.065 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268018 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 872448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:59.065 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:03:59.065 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.095855856999997, 2024-05-11T23:03:59.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050170927999999997, 2024-05-11T23:03:59.065 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.065 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.065 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.065 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.065 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.066 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.066 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268018 2024-05-11T23:03:59.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.066 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:03:59.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:03:59.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947435+0000", 2024-05-11T23:03:59.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3099999999999997e-07 2024-05-11T23:03:59.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947437+0000", 2024-05-11T23:03:59.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6340000000000001e-06 2024-05-11T23:03:59.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947443+0000", 2024-05-11T23:03:59.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7810000000000002e-06 2024-05-11T23:03:59.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947448+0000", 2024-05-11T23:03:59.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9679999999999997e-06 2024-05-11T23:03:59.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997418+0000", 2024-05-11T23:03:59.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049969587000000003 2024-05-11T23:03:59.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997504+0000", 2024-05-11T23:03:59.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6426000000000002e-05 2024-05-11T23:03:59.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997605+0000", 2024-05-11T23:03:59.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100901 2024-05-11T23:03:59.069 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.069 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.069 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.069 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268019 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:59.069 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:59.069 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.095835299000001, 2024-05-11T23:03:59.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050285074999999999, 2024-05-11T23:03:59.069 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.069 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.069 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.070 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.070 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.070 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.070 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268019 2024-05-11T23:03:59.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.070 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:59.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:59.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:03:59.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0999999999999999e-07 2024-05-11T23:03:59.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947457+0000", 2024-05-11T23:03:59.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-05-11T23:03:59.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947462+0000", 2024-05-11T23:03:59.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2410000000000003e-06 2024-05-11T23:03:59.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947467+0000", 2024-05-11T23:03:59.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8339999999999998e-06 2024-05-11T23:03:59.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997639+0000", 2024-05-11T23:03:59.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050171697000000001 2024-05-11T23:03:59.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997685+0000", 2024-05-11T23:03:59.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6146000000000002e-05 2024-05-11T23:03:59.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997740+0000", 2024-05-11T23:03:59.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4737000000000001e-05 2024-05-11T23:03:59.073 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.073 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.073 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.073 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268020 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:59.073 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:03:59.073 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.095817054000001, 2024-05-11T23:03:59.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050504089000000002, 2024-05-11T23:03:59.074 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.074 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.074 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.074 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.074 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.074 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.074 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268020 2024-05-11T23:03:59.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.074 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:03:59.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:03:59.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947591+0000", 2024-05-11T23:03:59.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011805 2024-05-11T23:03:59.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947594+0000", 2024-05-11T23:03:59.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1549999999999999e-06 2024-05-11T23:03:59.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947602+0000", 2024-05-11T23:03:59.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.174e-06 2024-05-11T23:03:59.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947608+0000", 2024-05-11T23:03:59.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.263e-06 2024-05-11T23:03:59.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997775+0000", 2024-05-11T23:03:59.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050166980999999999 2024-05-11T23:03:59.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997875+0000", 2024-05-11T23:03:59.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9844000000000002e-05 2024-05-11T23:03:59.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997977+0000", 2024-05-11T23:03:59.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101622 2024-05-11T23:03:59.077 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.077 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.077 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.078 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268021 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:59.078 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:03:59.078 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.095671594000002, 2024-05-11T23:03:59.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050498874999999999, 2024-05-11T23:03:59.078 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.078 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.078 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.078 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.078 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.078 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.078 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268021 2024-05-11T23:03:59.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.078 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:03:59.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:03:59.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947695+0000", 2024-05-11T23:03:59.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6247000000000004e-05 2024-05-11T23:03:59.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947698+0000", 2024-05-11T23:03:59.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9199999999999997e-06 2024-05-11T23:03:59.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947706+0000", 2024-05-11T23:03:59.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:03:59.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947711+0000", 2024-05-11T23:03:59.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6720000000000001e-06 2024-05-11T23:03:59.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998017+0000", 2024-05-11T23:03:59.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050305470999999997 2024-05-11T23:03:59.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998060+0000", 2024-05-11T23:03:59.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3374000000000002e-05 2024-05-11T23:03:59.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998117+0000", 2024-05-11T23:03:59.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6971000000000001e-05 2024-05-11T23:03:59.081 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.081 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.082 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.082 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268022 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:59.082 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:03:59.082 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.095568557999997, 2024-05-11T23:03:59.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050731361000000003, 2024-05-11T23:03:59.082 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.082 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.082 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.082 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.082 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268022 2024-05-11T23:03:59.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.083 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:03:59.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:03:59.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947722+0000", 2024-05-11T23:03:59.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8999999999999996e-07 2024-05-11T23:03:59.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947724+0000", 2024-05-11T23:03:59.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1639999999999999e-06 2024-05-11T23:03:59.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947730+0000", 2024-05-11T23:03:59.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6099999999999997e-06 2024-05-11T23:03:59.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947735+0000", 2024-05-11T23:03:59.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7189999999999998e-06 2024-05-11T23:03:59.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998140+0000", 2024-05-11T23:03:59.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050404254000000003 2024-05-11T23:03:59.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998239+0000", 2024-05-11T23:03:59.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9192999999999997e-05 2024-05-11T23:03:59.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998453+0000", 2024-05-11T23:03:59.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021383099999999999 2024-05-11T23:03:59.086 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.086 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.086 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.086 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268023 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1069056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:59.086 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:03:59.086 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.095546292999998, 2024-05-11T23:03:59.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050874509999999998, 2024-05-11T23:03:59.086 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.086 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.086 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.086 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.087 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.087 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.087 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268023 2024-05-11T23:03:59.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.087 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:03:59.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:03:59.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948934+0000", 2024-05-11T23:03:59.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011903339999999999 2024-05-11T23:03:59.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948938+0000", 2024-05-11T23:03:59.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1330000000000001e-06 2024-05-11T23:03:59.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948949+0000", 2024-05-11T23:03:59.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0872e-05 2024-05-11T23:03:59.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948956+0000", 2024-05-11T23:03:59.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.131e-06 2024-05-11T23:03:59.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998494+0000", 2024-05-11T23:03:59.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049537678000000002 2024-05-11T23:03:59.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998551+0000", 2024-05-11T23:03:59.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7256000000000002e-05 2024-05-11T23:03:59.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998618+0000", 2024-05-11T23:03:59.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7106000000000004e-05 2024-05-11T23:03:59.090 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.090 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.090 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.090 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268026 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1179648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:59.090 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:03:59.090 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.094269240000003, 2024-05-11T23:03:59.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050107977999999997, 2024-05-11T23:03:59.090 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.091 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.091 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.091 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.091 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.091 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.091 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268026 2024-05-11T23:03:59.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.091 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:03:59.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:03:59.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949029+0000", 2024-05-11T23:03:59.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2220000000000002e-06 2024-05-11T23:03:59.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949041+0000", 2024-05-11T23:03:59.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.241e-05 2024-05-11T23:03:59.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949049+0000", 2024-05-11T23:03:59.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.072e-06 2024-05-11T23:03:59.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949056+0000", 2024-05-11T23:03:59.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1380000000000001e-06 2024-05-11T23:03:59.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999016+0000", 2024-05-11T23:03:59.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049959212000000003 2024-05-11T23:03:59.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999067+0000", 2024-05-11T23:03:59.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1598000000000003e-05 2024-05-11T23:03:59.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999129+0000", 2024-05-11T23:03:59.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1326000000000002e-05 2024-05-11T23:03:59.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.094 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.094 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.094 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268027 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1216512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:59.094 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:59.095 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.094225090000002, 2024-05-11T23:03:59.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050189865, 2024-05-11T23:03:59.095 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.095 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.095 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.095 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.095 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.095 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.095 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268027 2024-05-11T23:03:59.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.095 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:59.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:59.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:03:59.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-05-11T23:03:59.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949068+0000", 2024-05-11T23:03:59.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.26e-06 2024-05-11T23:03:59.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949074+0000", 2024-05-11T23:03:59.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7050000000000002e-06 2024-05-11T23:03:59.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949080+0000", 2024-05-11T23:03:59.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3689999999999998e-06 2024-05-11T23:03:59.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999152+0000", 2024-05-11T23:03:59.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050072653000000002 2024-05-11T23:03:59.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999197+0000", 2024-05-11T23:03:59.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4972999999999998e-05 2024-05-11T23:03:59.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999255+0000", 2024-05-11T23:03:59.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7425e-05 2024-05-11T23:03:59.098 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.098 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.098 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.099 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268028 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1253376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:59.099 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:03:59.099 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.09420343, 2024-05-11T23:03:59.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050280017000000003, 2024-05-11T23:03:59.099 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.099 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.099 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.099 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.099 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.099 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.099 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268028 2024-05-11T23:03:59.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.099 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:03:59.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:03:59.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949087+0000", 2024-05-11T23:03:59.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:03:59.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949089+0000", 2024-05-11T23:03:59.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8950000000000001e-06 2024-05-11T23:03:59.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949095+0000", 2024-05-11T23:03:59.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7540000000000003e-06 2024-05-11T23:03:59.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949114+0000", 2024-05-11T23:03:59.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9738e-05 2024-05-11T23:03:59.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999278+0000", 2024-05-11T23:03:59.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050163368999999999 2024-05-11T23:03:59.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999319+0000", 2024-05-11T23:03:59.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0979000000000002e-05 2024-05-11T23:03:59.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999366+0000", 2024-05-11T23:03:59.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7879e-05 2024-05-11T23:03:59.102 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.102 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.102 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268029 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1290240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.094163322, 2024-05-11T23:03:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050344904000000003, 2024-05-11T23:03:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268029 2024-05-11T23:03:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:03:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:03:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949130+0000", 2024-05-11T23:03:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-05-11T23:03:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949139+0000", 2024-05-11T23:03:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9800000000000004e-06 2024-05-11T23:03:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949145+0000", 2024-05-11T23:03:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0510000000000002e-06 2024-05-11T23:03:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999387+0000", 2024-05-11T23:03:59.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050242167999999997 2024-05-11T23:03:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999427+0000", 2024-05-11T23:03:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9566e-05 2024-05-11T23:03:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999471+0000", 2024-05-11T23:03:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4914999999999999e-05 2024-05-11T23:03:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268030 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1327104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.094138037999997, 2024-05-11T23:03:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050447394, 2024-05-11T23:03:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268030 2024-05-11T23:03:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:03:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0500000000000004e-07 2024-05-11T23:03:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949155+0000", 2024-05-11T23:03:59.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3819999999999998e-06 2024-05-11T23:03:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949162+0000", 2024-05-11T23:03:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9609999999999999e-06 2024-05-11T23:03:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949188+0000", 2024-05-11T23:03:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5877e-05 2024-05-11T23:03:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999494+0000", 2024-05-11T23:03:59.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050306265000000003 2024-05-11T23:03:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999534+0000", 2024-05-11T23:03:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0447000000000001e-05 2024-05-11T23:03:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999599+0000", 2024-05-11T23:03:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4956999999999995e-05 2024-05-11T23:03:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.110 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268031 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1363968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:03:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.094092938000003, 2024-05-11T23:03:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050511525000000002, 2024-05-11T23:03:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268031 2024-05-11T23:03:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:03:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:03:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949198+0000", 2024-05-11T23:03:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-05-11T23:03:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949200+0000", 2024-05-11T23:03:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-05-11T23:03:59.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949208+0000", 2024-05-11T23:03:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2840000000000001e-06 2024-05-11T23:03:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949214+0000", 2024-05-11T23:03:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9869999999999996e-06 2024-05-11T23:03:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999619+0000", 2024-05-11T23:03:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050405278999999997 2024-05-11T23:03:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999660+0000", 2024-05-11T23:03:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1427999999999997e-05 2024-05-11T23:03:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999708+0000", 2024-05-11T23:03:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8229999999999997e-05 2024-05-11T23:03:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268032 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1400832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:59.114 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:03:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.094063318000003, 2024-05-11T23:03:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050611863999999999, 2024-05-11T23:03:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268032 2024-05-11T23:03:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:03:59.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:03:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949230+0000", 2024-05-11T23:03:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.371e-06 2024-05-11T23:03:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949233+0000", 2024-05-11T23:03:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6599999999999999e-06 2024-05-11T23:03:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949241+0000", 2024-05-11T23:03:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8939999999999993e-06 2024-05-11T23:03:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949246+0000", 2024-05-11T23:03:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6590000000000004e-06 2024-05-11T23:03:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999733+0000", 2024-05-11T23:03:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050487268000000002 2024-05-11T23:03:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999784+0000", 2024-05-11T23:03:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0562000000000001e-05 2024-05-11T23:03:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999838+0000", 2024-05-11T23:03:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000002e-05 2024-05-11T23:03:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268033 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1437696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:03:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:03:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 46.094036191000001, 2024-05-11T23:03:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050697157999999999, 2024-05-11T23:03:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268033 2024-05-11T23:03:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:03:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:03:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950862+0000", 2024-05-11T23:03:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001607975 2024-05-11T23:03:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950866+0000", 2024-05-11T23:03:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7770000000000002e-06 2024-05-11T23:03:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950878+0000", 2024-05-11T23:03:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1421e-05 2024-05-11T23:03:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950885+0000", 2024-05-11T23:03:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-05-11T23:03:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999859+0000", 2024-05-11T23:03:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048973199000000002 2024-05-11T23:03:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999901+0000", 2024-05-11T23:03:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1943000000000001e-05 2024-05-11T23:03:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999951+0000", 2024-05-11T23:03:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0243e-05 2024-05-11T23:03:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.122 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.123 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:283084 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e147)", 2024-05-11T23:03:59.123 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:49.906190+0000", 2024-05-11T23:03:59.123 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 9.1371002410000006, 2024-05-11T23:03:59.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.89363789900000001, 2024-05-11T23:03:59.123 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.123 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.123 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.123 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.123 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.123 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 283084 2024-05-11T23:03:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906190+0000", 2024-05-11T23:03:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906190+0000", 2024-05-11T23:03:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906191+0000", 2024-05-11T23:03:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6029999999999999e-06 2024-05-11T23:03:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906198+0000", 2024-05-11T23:03:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7170000000000002e-06 2024-05-11T23:03:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906214+0000", 2024-05-11T23:03:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6290000000000002e-05 2024-05-11T23:03:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906229+0000", 2024-05-11T23:03:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4735999999999999e-05 2024-05-11T23:03:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906295+0000", 2024-05-11T23:03:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5604000000000004e-05 2024-05-11T23:03:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:03:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906305+0000", 2024-05-11T23:03:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0795000000000001e-05 2024-05-11T23:03:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:50.799618+0000", 2024-05-11T23:03:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.89331292600000001 2024-05-11T23:03:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:50.799711+0000", 2024-05-11T23:03:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2406999999999998e-05 2024-05-11T23:03:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:50.799828+0000", 2024-05-11T23:03:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116821 2024-05-11T23:03:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.127 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.128 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.128 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.128 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:59.129 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:03:59.129 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:03:59.129+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:03:59.230 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:59.230+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:59.275 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:59.275 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:59.275 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:59.275 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:59.283 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:03:59.331 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:03:59.331+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:03:59.389 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:59.389 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:59.389 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:59.389 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:59.389 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:59.398 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:03:59.439 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:03:59.439+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:03:59.533 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:03:59.533+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:03:59.545 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:59.545 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:03:59.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:03:59.545 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:03:59.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.546 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:59.546 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:59.546 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 126.89099353500001, 2024-05-11T23:03:59.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:03:59.546 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.546 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.546 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.546 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.546 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.546 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.546 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:03:59.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.546 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:59.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:59.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:03:59.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:03:59.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:03:59.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:03:59.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:03:59.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:03:59.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:03:59.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:03:59.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:03:59.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:03:59.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:03:59.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:03:59.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:03:59.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:03:59.549 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.549 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.549 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.550 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:59.550 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:59.550 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 126.887612346, 2024-05-11T23:03:59.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:03:59.550 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.550 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.550 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.550 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.550 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:03:59.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.550 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:59.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:59.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:03:59.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:03:59.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:03:59.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:03:59.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:03:59.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:03:59.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:03:59.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:03:59.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:03:59.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:03:59.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:03:59.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:03:59.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:03:59.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:03:59.553 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.553 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.553 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.554 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:59.554 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:59.554 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 126.887593639, 2024-05-11T23:03:59.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:03:59.554 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.554 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.554 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.554 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.554 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:03:59.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.554 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:59.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:03:59.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:03:59.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:03:59.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:03:59.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:03:59.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:03:59.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:03:59.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:03:59.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:03:59.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:03:59.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:03:59.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:03:59.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:03:59.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:03:59.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:03:59.557 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.557 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.557 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.558 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:59.558 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:59.558 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 126.887575496, 2024-05-11T23:03:59.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:03:59.558 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.558 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.558 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.558 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.558 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.558 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.558 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:03:59.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.558 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:59.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:59.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:03:59.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:03:59.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:03:59.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:03:59.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:03:59.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:03:59.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:03:59.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:03:59.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:03:59.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:03:59.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:03:59.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:03:59.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:03:59.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:03:59.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.561 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.562 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.562 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:59.562 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:59.562 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 126.887548009, 2024-05-11T23:03:59.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:03:59.562 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.562 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.562 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.562 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:03:59.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.563 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:59.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:03:59.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:03:59.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:03:59.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:03:59.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:03:59.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:03:59.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:03:59.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:03:59.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:03:59.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:03:59.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:03:59.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:03:59.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:03:59.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:03:59.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:03:59.565 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.566 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.566 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.566 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:59.566 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:59.566 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 126.88752748100001, 2024-05-11T23:03:59.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:03:59.566 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.566 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.566 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.566 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:03:59.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.567 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:59.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:59.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:03:59.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:03:59.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:03:59.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:03:59.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:03:59.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:03:59.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:03:59.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:03:59.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:03:59.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:03:59.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:03:59.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:03:59.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:03:59.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:03:59.569 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.570 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.570 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.570 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:59.570 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:59.570 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 126.88750770599999, 2024-05-11T23:03:59.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:03:59.570 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.570 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.570 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.570 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.570 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.570 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.570 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:03:59.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.571 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:59.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:03:59.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:03:59.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:03:59.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:03:59.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:03:59.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:03:59.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:03:59.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:03:59.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:03:59.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:03:59.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:03:59.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:03:59.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:03:59.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:03:59.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:03:59.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.574 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.574 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.574 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:59.574 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:59.574 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 126.88748889999999, 2024-05-11T23:03:59.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:03:59.574 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.574 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.574 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.574 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.575 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.575 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:03:59.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.575 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:59.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:03:59.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:03:59.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:03:59.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:03:59.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:03:59.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:03:59.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:03:59.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:03:59.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:03:59.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:03:59.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:03:59.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:03:59.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:03:59.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:03:59.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:03:59.578 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.578 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.578 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.578 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:59.578 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:59.578 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 126.88746997699999, 2024-05-11T23:03:59.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:03:59.578 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.578 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.578 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.578 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.578 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.578 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.579 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:03:59.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.579 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:59.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:59.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:03:59.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:03:59.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:03:59.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:03:59.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:03:59.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:03:59.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:03:59.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:03:59.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:03:59.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:03:59.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:03:59.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:03:59.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:03:59.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:03:59.582 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.582 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.582 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.582 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:59.582 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:59.582 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 126.88745131500001, 2024-05-11T23:03:59.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:03:59.582 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.582 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.582 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.582 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.582 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.583 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.583 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:03:59.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.583 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:59.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:59.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:03:59.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:03:59.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:03:59.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:03:59.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:03:59.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:03:59.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:03:59.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:03:59.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:03:59.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:03:59.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:03:59.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:03:59.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:03:59.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:03:59.586 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.586 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.586 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.586 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:59.586 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:59.586 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 126.887106153, 2024-05-11T23:03:59.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:03:59.586 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.586 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.586 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.586 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.587 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.587 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:03:59.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.587 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:59.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:03:59.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:03:59.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:03:59.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:03:59.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:03:59.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:03:59.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:03:59.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:03:59.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:03:59.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:03:59.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:03:59.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:03:59.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:03:59.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:03:59.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:03:59.590 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.590 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.590 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.590 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:59.590 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:59.590 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 126.887080603, 2024-05-11T23:03:59.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:03:59.590 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.591 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.591 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.591 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.591 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:03:59.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.591 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:59.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:59.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:03:59.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:03:59.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:03:59.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:03:59.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:03:59.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:03:59.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:03:59.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:03:59.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:03:59.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:03:59.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:03:59.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:03:59.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:03:59.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:03:59.594 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.594 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.594 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.594 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:59.594 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:59.594 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 126.88699834499999, 2024-05-11T23:03:59.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:03:59.594 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.595 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.595 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.595 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.595 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.595 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.595 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:03:59.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.595 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:59.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:59.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:03:59.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:03:59.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:03:59.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:03:59.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:03:59.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:03:59.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:03:59.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:03:59.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:03:59.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:03:59.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:03:59.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:03:59.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.598 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.598 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:03:59.598 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:03:59.598 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.598 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.598 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.598 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.598 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.598 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:59.598 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:59.598 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 126.886975898, 2024-05-11T23:03:59.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:03:59.599 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.599 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.599 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.599 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.599 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:03:59.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.599 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.599 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.599 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:59.599 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.599 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.599 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:03:59.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:03:59.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:03:59.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.600 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:03:59.600 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:03:59.600 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.600 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.600 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:03:59.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:03:59.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:03:59.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:03:59.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:03:59.601 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:03:59.601 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.601 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.601 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.601 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:03:59.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:03:59.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.602 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.602 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:03:59.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:03:59.602 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.602 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.602 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.602 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.602 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.602 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:59.602 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:59.602 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 126.88695644000001, 2024-05-11T23:03:59.602 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:03:59.603 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.603 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.603 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.603 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.603 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.603 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.603 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:03:59.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.603 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.603 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:59.603 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.603 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.603 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.603 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:59.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:03:59.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:03:59.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:03:59.604 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:03:59.604 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.604 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.604 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.604 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:03:59.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:03:59.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:03:59.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:03:59.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.605 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:03:59.605 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:03:59.605 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.605 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.605 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:03:59.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:03:59.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.606 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.606 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:03:59.606 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:03:59.606 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.606 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.606 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.606 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.606 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.606 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:59.606 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:59.606 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 126.88693637999999, 2024-05-11T23:03:59.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:03:59.607 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.607 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.607 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.607 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.607 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.607 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.607 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:03:59.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.607 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.607 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.607 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:59.607 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.607 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.607 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:59.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:03:59.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:03:59.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:03:59.608 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:03:59.608 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.608 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.608 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.608 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:03:59.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:03:59.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:03:59.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:03:59.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:03:59.609 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:03:59.609 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.609 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.609 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.609 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:03:59.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:03:59.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.610 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.610 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:03:59.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:03:59.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.610 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.610 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.610 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.610 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.610 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:59.610 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:59.610 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 126.88691711, 2024-05-11T23:03:59.610 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:03:59.611 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.611 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.611 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.611 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.611 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:03:59.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.611 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.611 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:59.611 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.611 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.611 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.611 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:59.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:03:59.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:03:59.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:03:59.612 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:03:59.612 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.612 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.612 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.612 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:03:59.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:03:59.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:03:59.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:03:59.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:03:59.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:03:59.613 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.613 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.613 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.613 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:03:59.613 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:03:59.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.614 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.614 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:03:59.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:03:59.614 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.614 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.614 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.614 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.614 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.614 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:59.614 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:59.614 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 126.886371044, 2024-05-11T23:03:59.614 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:03:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:03:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.615 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.616 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:03:59.616 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.617 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:03:59.617 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:03:59.617 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.617 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.617 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:03:59.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:03:59.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:03:59.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:03:59.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.618 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:03:59.618 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:03:59.618 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.618 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.618 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:03:59.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:03:59.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:03:59.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:03:59.619 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.619 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.619 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.619 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:03:59.619 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:03:59.619 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.619 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.619 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.620 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:03:59.620 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:59.620 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 126.88633926, 2024-05-11T23:03:59.620 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:03:59.620 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.620 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.620 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:03:59.620 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.620 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.620 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.620 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:03:59.620 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.620 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.620 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:59.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:59.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.621 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.621 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:03:59.621 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:03:59.621 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.621 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:03:59.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:03:59.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:03:59.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:03:59.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.622 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.622 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:03:59.622 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:03:59.622 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.622 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:03:59.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:03:59.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:03:59.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:03:59.623 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.623 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.623 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.623 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:03:59.623 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:03:59.623 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.623 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.624 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.624 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.624 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:03:59.624 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:59.624 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 117.449093796, 2024-05-11T23:03:59.624 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:03:59.624 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:03:59.624 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:03:59.624 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:03:59.624 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:03:59.624 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:03:59.624 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:03:59.624 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:03:59.624 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.624 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:03:59.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:03:59.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:59.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:03:59.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:03:59.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:03:59.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.625 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.625 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:03:59.625 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:03:59.625 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:03:59.625 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:03:59.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:03:59.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:03:59.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:03:59.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:03:59.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:03:59.626 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.626 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.626 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:03:59.626 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:03:59.626 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:03:59.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:03:59.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:03:59.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:03:59.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:03:59.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:03:59.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:03:59.627 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.627 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.627 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:03:59.627 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:03:59.627 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:03:59.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:03:59.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:03:59.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:03:59.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:03:59.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:03:59.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:03:59.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.628 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.628 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:03:59.628 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:03:59.628 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:03:59.628 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:03:59.629 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:03:59.629 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:03:59.629 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:03:59.629 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:03:59.629 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.629 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.629 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.629 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:03:59.629 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:03:59.629 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:59.630 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:03:59.634 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:03:59.633+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:03:59.679 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:03:59.735 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:59.734+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:59.776 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:59.776 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:59.776 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:03:59.776 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:59.785 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:03:59.818 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:03:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:03:59.833 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-05-11T23:03:59.835 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:03:59.834+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:03:59.837 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:03:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:03:59.855 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:03:59.856 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:03:59.856 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-05-11T23:03:59.875 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:03:59.875 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:03:59.895 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:03:59.895 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:03:59.901 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:03:59.901 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:03:59.901 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:03:59.901 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:03:59.901 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:03:59.911 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:03:59.912 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:03:59.913 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:03:59.930 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:03:59.930 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:03:59.933 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-11T23:03:59.934 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-11T23:03:59.975 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:03:59.975+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:00.021 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:00.021 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:00.031 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:04:00.037 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:00.037+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:00.138 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:00.138+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:00.140 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:00.141 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:00.150 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:04:00.238 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:00.238+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:00.258 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:00.259 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:00.267 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:04:00.339 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:00.339+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:00.377 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:00.377 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:00.386 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:04:00.440 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:00.439+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:00.496 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:00.496 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:00.504 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:04:00.541 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:04:00.541+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:04:00.642 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:00.642+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:00.655 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:00.655 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:00.664 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:04:00.743 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:00.742+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:00.814 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:00.814 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:00.823 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:04:00.867 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:00.867+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:00.940 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:00.940 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:00.944 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:00.944+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:00.949 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:04:01.046 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:01.046+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:01.099 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:01.099 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:01.107 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:04:01.147 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:01.147+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:01.249 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:01.248+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:01.256 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:01.256 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:01.265 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:04:01.349 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:01.348+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:01.372 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:01.372 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:01.381 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:04:01.450 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:01.450+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:01.486 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:01.486 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:01.495 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:04:01.551 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:01.551+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:01.639 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:01.639 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:01.648 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:04:01.651 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:01.651+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:01.752 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:01.752+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:01.753 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:01.753 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:01.761 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:04:01.841 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 5, 7, 3, 6, 4] out_osds: [0] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-05-11T23:04:01.841 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:04:01.841 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-11T23:04:01.842 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.339946 2024-05-11T23:04:01.842 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0.3399456848149407 2024-05-11T23:04:01.849 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-11T23:04:01.853 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:01.852+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:01.876 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:01.877 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:01.887 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:04:01.934 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:04:01.973 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:01.973+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:02.044 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:02.044 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:02.053 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:04:02.074 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:04:02.074 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:04:02.094 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:02.094+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:02.110 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:04:02.110 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:04:02.146 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:04:02.147 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:04:02.147 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-05-11T23:04:02.155 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:02.155+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:02.166 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:04:02.166 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:04:02.183 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:04:02.184 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:04:02.200 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:02.200 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:02.202 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:04:02.202 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:04:02.209 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:04:02.220 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:04:02.220 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:04:02.223 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-11T23:04:02.224 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-11T23:04:02.256 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:02.256+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:02.354 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:02.355 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:02.355 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:04:02.355 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:04:02.355 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:02.357 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:04:02.357+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:04:02.363 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:04:02.458 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:02.458+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:02.511 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:02.511 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:04:02.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:04:02.511 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:04:02.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.512 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:02.512 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:04:02.512 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 129.85629528199999, 2024-05-11T23:04:02.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:04:02.512 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:02.512 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:02.513 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:02.513 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:02.513 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:02.513 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:02.513 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:04:02.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.513 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:02.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:02.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:04:02.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:02.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:04:02.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:02.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:04:02.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:04:02.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:02.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:04:02.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:04:02.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:02.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:04:02.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:04:02.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:02.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:04:02.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:04:02.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.517 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:02.517 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:04:02.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:04:02.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:02.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:04:02.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:04:02.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:02.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:04:02.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:04:02.518 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.518 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.518 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.518 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:02.519 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:04:02.519 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 129.85291409300001, 2024-05-11T23:04:02.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:04:02.519 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:02.519 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:02.519 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:02.520 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:02.520 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:02.520 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:02.520 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:04:02.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.520 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:02.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:02.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:04:02.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:02.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:04:02.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:02.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:04:02.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:04:02.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:02.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:04:02.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:04:02.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:02.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:04:02.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:04:02.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:02.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:04:02.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:04:02.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:02.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:04:02.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:04:02.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:02.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:04:02.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:04:02.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:02.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:04:02.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:04:02.523 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.523 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.523 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.523 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:02.523 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:04:02.523 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 129.852895386, 2024-05-11T23:04:02.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:04:02.524 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:02.524 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:02.524 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:02.524 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:02.524 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:02.524 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:02.524 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:04:02.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.524 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:02.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:02.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:04:02.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:02.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:04:02.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:02.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:04:02.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:04:02.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:02.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:04:02.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:04:02.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:02.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:04:02.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:04:02.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:02.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:04:02.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:04:02.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:02.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:04:02.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:04:02.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:02.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:04:02.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:04:02.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:02.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:04:02.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:04:02.527 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.527 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.527 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.527 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:02.528 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:04:02.528 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 129.85287724299999, 2024-05-11T23:04:02.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:04:02.528 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:02.528 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:02.528 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:02.528 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:02.528 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:02.528 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:02.528 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:04:02.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.528 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:02.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:02.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:04:02.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:02.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:04:02.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:02.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:04:02.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:04:02.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:02.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:04:02.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:04:02.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:02.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:04:02.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:04:02.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:02.530 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:04:02.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:04:02.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.530 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:02.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:04:02.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:04:02.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:02.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:04:02.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:04:02.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:02.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:04:02.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:04:02.531 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.531 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.531 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:04:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 129.85284975600001, 2024-05-11T23:04:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:04:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:04:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:04:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:04:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:04:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:04:02.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:04:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:04:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:04:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:04:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:04:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:04:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:02.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:04:02.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:04:02.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:02.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:04:02.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:04:02.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:02.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:04:02.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:04:02.535 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.535 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:04:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 129.85282922799999, 2024-05-11T23:04:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:04:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:04:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.536 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:02.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:02.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:04:02.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:02.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:04:02.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:02.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:04:02.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:04:02.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:02.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:04:02.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:04:02.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:02.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:04:02.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:04:02.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:02.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:04:02.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:04:02.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:02.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:04:02.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:04:02.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:02.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:04:02.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:04:02.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.539 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:02.539 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:04:02.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:04:02.539 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.539 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.539 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.540 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:02.540 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:04:02.540 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 129.85280945299999, 2024-05-11T23:04:02.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:04:02.540 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:02.540 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:02.540 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:02.540 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:02.540 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:02.540 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:02.540 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:04:02.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.540 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:02.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:02.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:04:02.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:02.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:04:02.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:02.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:04:02.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:04:02.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:02.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:04:02.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:04:02.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:02.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:04:02.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:04:02.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:02.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:04:02.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:04:02.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:02.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:04:02.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:04:02.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:02.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:04:02.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:04:02.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:02.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:04:02.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:04:02.543 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.543 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.544 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.544 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:02.544 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:04:02.544 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 129.85279064700001, 2024-05-11T23:04:02.544 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:04:02.544 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:02.544 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:02.544 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:02.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:02.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:02.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:02.544 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:04:02.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.545 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:02.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:02.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:04:02.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:02.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:04:02.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:02.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:04:02.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:04:02.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:02.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:04:02.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:04:02.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:02.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:04:02.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:04:02.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:02.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:04:02.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:04:02.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:02.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:04:02.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:04:02.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:02.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:04:02.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:04:02.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:02.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:04:02.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:04:02.547 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.548 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.548 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.548 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:02.548 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:04:02.548 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 129.85277172400001, 2024-05-11T23:04:02.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:04:02.548 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:02.548 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:02.548 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:02.548 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:02.548 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:02.548 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:02.548 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:04:02.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.549 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:02.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:02.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:04:02.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:02.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:04:02.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:02.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:04:02.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:04:02.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:02.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:04:02.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:04:02.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:02.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:04:02.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:04:02.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:02.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:04:02.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:04:02.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:02.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:04:02.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:04:02.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:02.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:04:02.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:04:02.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:02.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:04:02.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:04:02.551 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.552 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.552 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.552 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:02.552 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:04:02.552 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 129.85275306200001, 2024-05-11T23:04:02.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:04:02.552 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:02.552 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:02.552 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:02.552 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:02.552 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:02.552 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:02.552 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:04:02.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.553 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:02.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:02.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:04:02.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:02.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:04:02.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:02.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:04:02.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:04:02.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:02.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:04:02.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:04:02.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:02.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:04:02.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:04:02.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:02.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:04:02.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:04:02.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:02.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:04:02.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:04:02.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:02.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:04:02.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:04:02.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:02.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:04:02.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:04:02.556 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.556 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.556 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.556 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:02.556 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:04:02.556 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 129.8524079, 2024-05-11T23:04:02.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:04:02.556 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:02.556 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:02.556 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:02.556 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:02.556 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:02.556 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:02.557 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:04:02.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.557 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:02.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:02.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:04:02.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:02.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:04:02.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:02.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:04:02.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:04:02.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:02.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:04:02.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:04:02.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:02.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:04:02.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:04:02.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:04:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:04:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:04:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:04:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:04:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:04:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:04:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:04:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:04:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 129.85238235, 2024-05-11T23:04:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:04:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:02.560 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:04:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:04:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:04:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:04:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:04:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:04:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:04:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:04:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:04:02.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:04:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:04:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:04:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:04:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:04:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:04:02.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:04:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:04:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:04:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 129.85230009200001, 2024-05-11T23:04:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:04:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:02.564 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:04:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:04:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:04:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:04:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:04:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:04:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:04:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:04:02.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:04:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:04:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:04:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:04:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:04:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:02.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:04:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:04:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:04:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:04:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:04:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 129.85227764499999, 2024-05-11T23:04:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:04:02.568 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:04:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:04:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:02.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:04:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:04:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:04:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:04:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:04:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:02.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:04:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:04:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:04:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:04:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:04:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:04:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:04:02.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:04:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:04:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:04:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:04:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 129.85225818699999, 2024-05-11T23:04:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:04:02.572 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:04:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:04:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:02.573 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:04:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:04:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:04:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:04:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:04:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:02.574 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:04:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:04:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:04:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:04:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:04:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:04:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:04:02.575 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:04:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:04:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:04:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:04:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 129.85223812699999, 2024-05-11T23:04:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:04:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:02.576 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:04:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:04:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:02.577 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:04:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:04:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:04:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:04:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:04:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:04:02.578 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:04:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:04:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:04:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:04:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:04:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:04:02.579 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:04:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:04:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:04:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:04:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 129.852218857, 2024-05-11T23:04:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:04:02.580 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:04:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:04:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:02.581 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:04:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:04:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:04:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:04:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:04:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:02.582 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:04:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:04:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:04:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:04:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:04:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:04:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:04:02.583 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:04:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:04:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:04:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:04:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 129.851672791, 2024-05-11T23:04:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:04:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:02.584 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:04:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:04:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:02.585 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:04:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:04:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:04:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:04:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:04:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:04:02.586 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:04:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:04:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:04:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:04:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:04:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:04:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:04:02.587 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:04:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:04:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:04:02.588 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 129.85164100700001, 2024-05-11T23:04:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:04:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:04:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:04:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.589 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:04:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:04:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:04:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:04:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:04:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.590 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:04:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:04:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:04:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:04:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:04:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:04:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.591 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:02.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:04:02.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:04:02.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.592 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:02.592 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:04:02.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:04:02.592 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.592 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.592 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.592 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.592 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.592 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:04:02.592 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:04:02.592 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 120.414395543, 2024-05-11T23:04:02.592 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:04:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:04:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:04:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:04:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.593 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:04:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:04:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:04:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:04:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:04:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:02.594 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:04:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:04:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:04:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:04:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:04:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:04:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:04:02.595 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:04:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:04:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:04:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:04:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:04:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:04:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:04:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:04:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:04:02.596 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:04:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:04:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:04:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:04:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:04:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:04:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:02.597 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:02.597 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:02.599 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:04:02.599 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:02.598+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:02.699 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:02.699+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:02.747 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:02.747 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:02.748 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:04:02.748 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:02.755 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:04:02.838 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:02.838+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:02.845 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-05-11T23:04:02.849 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.1 primary-affinity to 0.339946 (8222782) 2024-05-11T23:04:02.900 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:02.900+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:02.915 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:02.915 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:02.915 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:04:02.915 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:04:02.915 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:02.925 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:04:03.009 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:03.009+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:03.077 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:03.077 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:04:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:04:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:04:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:04:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:04:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 165.91120958299999, 2024-05-11T23:04:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:04:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:04:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:04:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:04:03.078 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:04:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:04:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:04:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:04:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:04:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:04:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:04:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:04:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:04:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:04:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:04:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:04:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:04:03.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:04:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:04:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:04:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:04:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:04:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:04:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:04:03.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:04:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:04:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:04:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:04:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:04:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:04:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:04:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:04:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:04:03.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:04:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:04:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:04:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:04:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:04:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.084 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78429 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:04:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.109940241, 2024-05-11T23:04:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044563497000000001, 2024-05-11T23:04:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78429 2024-05-11T23:04:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:04:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:04:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964872+0000", 2024-05-11T23:04:03.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035070300000000001 2024-05-11T23:04:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964876+0000", 2024-05-11T23:04:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5009999999999998e-06 2024-05-11T23:04:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964902+0000", 2024-05-11T23:04:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5715999999999999e-05 2024-05-11T23:04:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964910+0000", 2024-05-11T23:04:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3930000000000002e-06 2024-05-11T23:04:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008898+0000", 2024-05-11T23:04:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043988772000000002 2024-05-11T23:04:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008981+0000", 2024-05-11T23:04:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3140999999999996e-05 2024-05-11T23:04:03.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009085+0000", 2024-05-11T23:04:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010327099999999999 2024-05-11T23:04:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78431 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:04:03.090 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.109479933, 2024-05-11T23:04:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044542033000000002, 2024-05-11T23:04:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78431 2024-05-11T23:04:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:03.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:04:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:04:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965038+0000", 2024-05-11T23:04:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6993000000000001e-05 2024-05-11T23:04:03.092 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965043+0000", 2024-05-11T23:04:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7040000000000002e-06 2024-05-11T23:04:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965053+0000", 2024-05-11T23:04:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8539999999999992e-06 2024-05-11T23:04:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965061+0000", 2024-05-11T23:04:03.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9810000000000003e-06 2024-05-11T23:04:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009350+0000", 2024-05-11T23:04:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044288739000000001 2024-05-11T23:04:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009429+0000", 2024-05-11T23:04:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9354e-05 2024-05-11T23:04:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:03.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009524+0000", 2024-05-11T23:04:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4407999999999995e-05 2024-05-11T23:04:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78432 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:04:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.109389085, 2024-05-11T23:04:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044635421000000002, 2024-05-11T23:04:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78432 2024-05-11T23:04:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:04:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:04:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965343+0000", 2024-05-11T23:04:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027056500000000001 2024-05-11T23:04:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965347+0000", 2024-05-11T23:04:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3540000000000002e-06 2024-05-11T23:04:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965356+0000", 2024-05-11T23:04:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2140000000000002e-06 2024-05-11T23:04:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965363+0000", 2024-05-11T23:04:03.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4150000000000004e-06 2024-05-11T23:04:03.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009551+0000", 2024-05-11T23:04:03.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044187917 2024-05-11T23:04:03.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009626+0000", 2024-05-11T23:04:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5024000000000005e-05 2024-05-11T23:04:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009708+0000", 2024-05-11T23:04:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1932000000000001e-05 2024-05-11T23:04:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78433 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:04:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.10908819799999, 2024-05-11T23:04:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044564864000000003, 2024-05-11T23:04:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78433 2024-05-11T23:04:03.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:04:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:04:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966225+0000", 2024-05-11T23:04:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085220700000000001 2024-05-11T23:04:03.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:03.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966230+0000", 2024-05-11T23:04:03.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0690000000000003e-06 2024-05-11T23:04:03.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:03.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966239+0000", 2024-05-11T23:04:03.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-05-11T23:04:03.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966245+0000", 2024-05-11T23:04:03.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5520000000000001e-06 2024-05-11T23:04:03.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009743+0000", 2024-05-11T23:04:03.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043498311999999997 2024-05-11T23:04:03.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:03.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009848+0000", 2024-05-11T23:04:03.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104885 2024-05-11T23:04:03.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:03.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009938+0000", 2024-05-11T23:04:03.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9812e-05 2024-05-11T23:04:03.104 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.104 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.104 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.105 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78580 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 999424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:03.105 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:04:03.105 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.105975515, 2024-05-11T23:04:03.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044493076999999999, 2024-05-11T23:04:03.105 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:03.105 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:03.105 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:03.105 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:03.105 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:03.105 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.105 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78580 2024-05-11T23:04:03.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.105 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:03.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:03.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:04:03.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:03.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:04:03.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.106 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.106 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:03.106 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:04:03.106 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:04:03.106 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:03.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968489+0000", 2024-05-11T23:04:03.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.52e-06 2024-05-11T23:04:03.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:03.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968496+0000", 2024-05-11T23:04:03.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9419999999999998e-06 2024-05-11T23:04:03.107 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.107 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.107 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.107 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968501+0000", 2024-05-11T23:04:03.107 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8720000000000001e-06 2024-05-11T23:04:03.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012690+0000", 2024-05-11T23:04:03.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044189655000000001 2024-05-11T23:04:03.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:03.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012848+0000", 2024-05-11T23:04:03.108 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015758100000000001 2024-05-11T23:04:03.108 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.108 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.108 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:03.108 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012979+0000", 2024-05-11T23:04:03.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000131066 2024-05-11T23:04:03.109 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.109 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.109 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.109 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.109 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.109 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78581 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:03.109 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:04:03.109 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.10595451099999, 2024-05-11T23:04:03.109 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044616243, 2024-05-11T23:04:03.109 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:03.109 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:03.109 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:03.110 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:03.110 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:03.110 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.110 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78581 2024-05-11T23:04:03.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.110 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:03.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:03.110 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:04:03.110 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.110 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.110 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.110 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:03.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:04:03.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:03.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970227+0000", 2024-05-11T23:04:03.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017200049999999999 2024-05-11T23:04:03.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.111 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.111 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:03.111 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970232+0000", 2024-05-11T23:04:03.111 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000003e-06 2024-05-11T23:04:03.111 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:03.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970242+0000", 2024-05-11T23:04:03.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0192e-05 2024-05-11T23:04:03.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970249+0000", 2024-05-11T23:04:03.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.888e-06 2024-05-11T23:04:03.112 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.112 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.112 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.112 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013008+0000", 2024-05-11T23:04:03.112 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042759495000000002 2024-05-11T23:04:03.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:03.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013063+0000", 2024-05-11T23:04:03.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5071999999999997e-05 2024-05-11T23:04:03.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.113 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.113 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:03.113 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013123+0000", 2024-05-11T23:04:03.113 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9930999999999999e-05 2024-05-11T23:04:03.113 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.113 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.113 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.113 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.114 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.114 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78589 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:03.114 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:04:03.114 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.104035267, 2024-05-11T23:04:03.114 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044587697000000003, 2024-05-11T23:04:03.114 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:03.114 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:03.114 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:03.114 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:03.114 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:03.114 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.114 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78589 2024-05-11T23:04:03.114 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.114 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:03.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:03.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:04:03.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:03.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:04:03.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.115 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.115 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.115 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:03.115 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970427+0000", 2024-05-11T23:04:03.115 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:04:03.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:03.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970429+0000", 2024-05-11T23:04:03.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3800000000000001e-06 2024-05-11T23:04:03.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:03.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970435+0000", 2024-05-11T23:04:03.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2530000000000001e-06 2024-05-11T23:04:03.116 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.116 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.116 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.116 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970441+0000", 2024-05-11T23:04:03.116 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-05-11T23:04:03.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014822+0000", 2024-05-11T23:04:03.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04438168 2024-05-11T23:04:03.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.117 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:03.117 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014911+0000", 2024-05-11T23:04:03.117 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8882999999999996e-05 2024-05-11T23:04:03.117 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.117 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.118 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:03.118 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015014+0000", 2024-05-11T23:04:03.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010263399999999999 2024-05-11T23:04:03.118 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.118 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.118 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.118 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.118 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.118 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78590 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:03.118 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:04:03.118 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.104014468, 2024-05-11T23:04:03.118 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044880120000000003, 2024-05-11T23:04:03.118 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:03.119 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:03.119 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:03.119 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:03.119 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:03.119 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.119 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78590 2024-05-11T23:04:03.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.119 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:03.119 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.119 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:03.119 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:04:03.119 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.119 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:03.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:04:03.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:03.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970918+0000", 2024-05-11T23:04:03.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000470834 2024-05-11T23:04:03.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.120 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.120 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:03.120 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970925+0000", 2024-05-11T23:04:03.120 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1690000000000003e-06 2024-05-11T23:04:03.120 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:03.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970934+0000", 2024-05-11T23:04:03.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9099999999999994e-06 2024-05-11T23:04:03.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970941+0000", 2024-05-11T23:04:03.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7410000000000003e-06 2024-05-11T23:04:03.121 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.121 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.121 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.121 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015049+0000", 2024-05-11T23:04:03.121 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044108392000000003 2024-05-11T23:04:03.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:03.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015133+0000", 2024-05-11T23:04:03.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3568999999999999e-05 2024-05-11T23:04:03.122 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.122 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.122 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:03.122 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015327+0000", 2024-05-11T23:04:03.122 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019450500000000001 2024-05-11T23:04:03.122 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.122 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.122 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78591 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:04:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.103511052, 2024-05-11T23:04:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044618497, 2024-05-11T23:04:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.123 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78591 2024-05-11T23:04:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:04:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:04:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:03.124 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970951+0000", 2024-05-11T23:04:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:04:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970953+0000", 2024-05-11T23:04:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:04:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970956+0000", 2024-05-11T23:04:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-05-11T23:04:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.125 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970959+0000", 2024-05-11T23:04:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9409999999999999e-06 2024-05-11T23:04:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015367+0000", 2024-05-11T23:04:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044407621000000001 2024-05-11T23:04:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015453+0000", 2024-05-11T23:04:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6685000000000001e-05 2024-05-11T23:04:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:03.126 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015569+0000", 2024-05-11T23:04:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115396 2024-05-11T23:04:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78592 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:04:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.10349824399999, 2024-05-11T23:04:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044831546999999999, 2024-05-11T23:04:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:03.127 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78592 2024-05-11T23:04:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:04:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:04:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.128 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972226+0000", 2024-05-11T23:04:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00126246 2024-05-11T23:04:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972230+0000", 2024-05-11T23:04:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1860000000000002e-06 2024-05-11T23:04:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972237+0000", 2024-05-11T23:04:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:04:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.129 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972241+0000", 2024-05-11T23:04:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.284e-06 2024-05-11T23:04:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015611+0000", 2024-05-11T23:04:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043369461999999998 2024-05-11T23:04:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015695+0000", 2024-05-11T23:04:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4458999999999999e-05 2024-05-11T23:04:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.130 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015795+0000", 2024-05-11T23:04:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9475999999999994e-05 2024-05-11T23:04:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78593 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:04:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.102211325, 2024-05-11T23:04:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046045214000000001, 2024-05-11T23:04:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:03.131 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78593 2024-05-11T23:04:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:04:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:04:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.132 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:04:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7599999999999998e-07 2024-05-11T23:04:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972252+0000", 2024-05-11T23:04:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7349999999999999e-06 2024-05-11T23:04:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972256+0000", 2024-05-11T23:04:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5659999999999998e-06 2024-05-11T23:04:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.133 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972259+0000", 2024-05-11T23:04:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8040000000000002e-06 2024-05-11T23:04:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018088+0000", 2024-05-11T23:04:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045829316000000002 2024-05-11T23:04:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018179+0000", 2024-05-11T23:04:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1299999999999997e-05 2024-05-11T23:04:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.134 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018295+0000", 2024-05-11T23:04:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116117 2024-05-11T23:04:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78594 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:04:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.102198965, 2024-05-11T23:04:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046245204999999998, 2024-05-11T23:04:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:03.135 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78594 2024-05-11T23:04:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:04:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.136 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:04:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972263+0000", 2024-05-11T23:04:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:04:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972264+0000", 2024-05-11T23:04:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.206e-06 2024-05-11T23:04:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.137 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972267+0000", 2024-05-11T23:04:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0690000000000002e-06 2024-05-11T23:04:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972270+0000", 2024-05-11T23:04:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5610000000000001e-06 2024-05-11T23:04:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018330+0000", 2024-05-11T23:04:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04606085 2024-05-11T23:04:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.138 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018414+0000", 2024-05-11T23:04:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3125000000000001e-05 2024-05-11T23:04:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018508+0000", 2024-05-11T23:04:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4153000000000005e-05 2024-05-11T23:04:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78595 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:04:03.139 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.1021861, 2024-05-11T23:04:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046396698, 2024-05-11T23:04:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78595 2024-05-11T23:04:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:04:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.140 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:04:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972276+0000", 2024-05-11T23:04:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5600000000000002e-07 2024-05-11T23:04:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972277+0000", 2024-05-11T23:04:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.576e-06 2024-05-11T23:04:03.141 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972280+0000", 2024-05-11T23:04:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-05-11T23:04:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972283+0000", 2024-05-11T23:04:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:04:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018538+0000", 2024-05-11T23:04:03.142 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046255255000000002 2024-05-11T23:04:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018605+0000", 2024-05-11T23:04:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6847000000000006e-05 2024-05-11T23:04:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018672+0000", 2024-05-11T23:04:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7045000000000006e-05 2024-05-11T23:04:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.143 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78596 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:04:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.10217525100001, 2024-05-11T23:04:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046586806000000001, 2024-05-11T23:04:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78596 2024-05-11T23:04:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.144 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:04:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:04:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:04:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:04:03.145 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972288+0000", 2024-05-11T23:04:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2550000000000001e-06 2024-05-11T23:04:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972291+0000", 2024-05-11T23:04:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9490000000000001e-06 2024-05-11T23:04:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972293+0000", 2024-05-11T23:04:03.146 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5069999999999999e-06 2024-05-11T23:04:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018707+0000", 2024-05-11T23:04:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046413604999999997 2024-05-11T23:04:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018782+0000", 2024-05-11T23:04:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5556000000000006e-05 2024-05-11T23:04:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018873+0000", 2024-05-11T23:04:03.147 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0703000000000002e-05 2024-05-11T23:04:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78597 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:04:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.102165478, 2024-05-11T23:04:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046825787000000001, 2024-05-11T23:04:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:03.148 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78597 2024-05-11T23:04:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:04:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:04:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.149 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:04:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-05-11T23:04:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972297+0000", 2024-05-11T23:04:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3069999999999999e-06 2024-05-11T23:04:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972300+0000", 2024-05-11T23:04:03.150 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:04:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972303+0000", 2024-05-11T23:04:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5509999999999998e-06 2024-05-11T23:04:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018911+0000", 2024-05-11T23:04:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046608194999999998 2024-05-11T23:04:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018986+0000", 2024-05-11T23:04:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5360999999999999e-05 2024-05-11T23:04:03.151 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019122+0000", 2024-05-11T23:04:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013526 2024-05-11T23:04:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78598 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:04:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 107.10215557799999, 2024-05-11T23:04:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046998572000000002, 2024-05-11T23:04:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:03.152 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78598 2024-05-11T23:04:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:04:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:03.153 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:04:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:04:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:04:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972307+0000", 2024-05-11T23:04:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1799999999999999e-06 2024-05-11T23:04:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.154 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972310+0000", 2024-05-11T23:04:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:04:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972313+0000", 2024-05-11T23:04:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4760000000000001e-06 2024-05-11T23:04:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019149+0000", 2024-05-11T23:04:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046836056000000001 2024-05-11T23:04:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.155 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019203+0000", 2024-05-11T23:04:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-05 2024-05-11T23:04:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019304+0000", 2024-05-11T23:04:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101537 2024-05-11T23:04:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:105234 3.15 3:adf885e5:::rbd_object_map.111e913a615f.0000000000000025:head [writefull 0~39] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-11T23:04:03.156 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:04:03.157 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 93.066839868000002, 2024-05-11T23:04:03.157 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77797599100000003, 2024-05-11T23:04:03.157 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:03.157 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:03.157 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:04:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 105234 2024-05-11T23:04:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:04:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.158 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:04:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007623+0000", 2024-05-11T23:04:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2240000000000001e-06 2024-05-11T23:04:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007627+0000", 2024-05-11T23:04:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:04:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:03.159 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007636+0000", 2024-05-11T23:04:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2409999999999995e-06 2024-05-11T23:04:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007641+0000", 2024-05-11T23:04:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6810000000000003e-06 2024-05-11T23:04:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007676+0000", 2024-05-11T23:04:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5061999999999997e-05 2024-05-11T23:04:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.160 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:04:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007682+0000", 2024-05-11T23:04:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0399999999999998e-06 2024-05-11T23:04:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783880+0000", 2024-05-11T23:04:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77619752900000005 2024-05-11T23:04:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783995+0000", 2024-05-11T23:04:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011550800000000001 2024-05-11T23:04:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.161 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:04:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784082+0000", 2024-05-11T23:04:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6347999999999994e-05 2024-05-11T23:04:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:04:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784682+0000", 2024-05-11T23:04:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000600162 2024-05-11T23:04:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:04:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785539+0000", 2024-05-11T23:04:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085699099999999998 2024-05-11T23:04:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.162 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:04:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785572+0000", 2024-05-11T23:04:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3182999999999997e-05 2024-05-11T23:04:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785598+0000", 2024-05-11T23:04:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5522000000000001e-05 2024-05-11T23:04:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113761 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [call rbd.sparse_copyup in=999496b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:04:03.163 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:04:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.544152948999994, 2024-05-11T23:04:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048572964000000003, 2024-05-11T23:04:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:04:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113761 2024-05-11T23:04:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:03.164 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:04:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:04:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530309+0000", 2024-05-11T23:04:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:04:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531242+0000", 2024-05-11T23:04:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093316899999999997 2024-05-11T23:04:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531249+0000", 2024-05-11T23:04:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8140000000000003e-06 2024-05-11T23:04:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531253+0000", 2024-05-11T23:04:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4290000000000003e-06 2024-05-11T23:04:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570325+0000", 2024-05-11T23:04:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039071379000000003 2024-05-11T23:04:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570382+0000", 2024-05-11T23:04:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6988999999999999e-05 2024-05-11T23:04:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:04:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570555+0000", 2024-05-11T23:04:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017379799999999999 2024-05-11T23:04:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:04:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.575953+0000", 2024-05-11T23:04:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053975539999999997 2024-05-11T23:04:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:04:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578778+0000", 2024-05-11T23:04:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002825616 2024-05-11T23:04:03.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:04:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578837+0000", 2024-05-11T23:04:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9005000000000002e-05 2024-05-11T23:04:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578881+0000", 2024-05-11T23:04:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3970000000000001e-05 2024-05-11T23:04:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113762 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [delete] snapc 27=[27] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:04:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:04:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 88.543198997999994, 2024-05-11T23:04:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047916260000000002, 2024-05-11T23:04:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:04:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113762 2024-05-11T23:04:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.170 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:04:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:04:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531263+0000", 2024-05-11T23:04:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.85e-07 2024-05-11T23:04:03.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531265+0000", 2024-05-11T23:04:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.827e-06 2024-05-11T23:04:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531268+0000", 2024-05-11T23:04:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3780000000000001e-06 2024-05-11T23:04:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531271+0000", 2024-05-11T23:04:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8509999999999999e-06 2024-05-11T23:04:03.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572298+0000", 2024-05-11T23:04:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041026990999999999 2024-05-11T23:04:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572345+0000", 2024-05-11T23:04:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6878000000000001e-05 2024-05-11T23:04:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:04:03.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572443+0000", 2024-05-11T23:04:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8184999999999996e-05 2024-05-11T23:04:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:04:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578585+0000", 2024-05-11T23:04:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006141919 2024-05-11T23:04:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:04:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579126+0000", 2024-05-11T23:04:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00054063799999999997 2024-05-11T23:04:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:04:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579154+0000", 2024-05-11T23:04:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8042999999999999e-05 2024-05-11T23:04:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579179+0000", 2024-05-11T23:04:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4765000000000001e-05 2024-05-11T23:04:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:03.175 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:03.175 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:03.177 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:03.176+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:03.177 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:04:03.277 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:03.277+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:03.292 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:03.292 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:04:03.292 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:04:03.292 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:03.301 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:04:03.378 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:03.378+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:03.450 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:04:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:04:03.450 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:04:03.450 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:03.459 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:04:03.478 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:04:03.478+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:04:03.569 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:03.569 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:04:03.569 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:04:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:04:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226051 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6049792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:04:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 61.251188194000001, 2024-05-11T23:04:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050622418000000002, 2024-05-11T23:04:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.570 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226051 2024-05-11T23:04:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:04:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:04:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.571 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317156+0000", 2024-05-11T23:04:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:04:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317157+0000", 2024-05-11T23:04:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.144e-06 2024-05-11T23:04:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317160+0000", 2024-05-11T23:04:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1350000000000001e-06 2024-05-11T23:04:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.572 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317162+0000", 2024-05-11T23:04:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5050000000000002e-06 2024-05-11T23:04:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367689+0000", 2024-05-11T23:04:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526797999999998 2024-05-11T23:04:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367735+0000", 2024-05-11T23:04:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5698999999999997e-05 2024-05-11T23:04:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:03.573 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367778+0000", 2024-05-11T23:04:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2928e-05 2024-05-11T23:04:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226052 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6060032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:04:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 61.251178138, 2024-05-11T23:04:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050830110999999997, 2024-05-11T23:04:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:03.574 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226052 2024-05-11T23:04:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:04:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:04:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317166+0000", 2024-05-11T23:04:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:04:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317167+0000", 2024-05-11T23:04:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.155e-06 2024-05-11T23:04:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317170+0000", 2024-05-11T23:04:03.576 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.061e-06 2024-05-11T23:04:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317172+0000", 2024-05-11T23:04:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4420000000000001e-06 2024-05-11T23:04:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367816+0000", 2024-05-11T23:04:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050644013000000002 2024-05-11T23:04:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367906+0000", 2024-05-11T23:04:03.577 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9739000000000003e-05 2024-05-11T23:04:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367995+0000", 2024-05-11T23:04:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9499000000000005e-05 2024-05-11T23:04:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226053 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6070272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:04:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 61.251168098999997, 2024-05-11T23:04:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051106912999999997, 2024-05-11T23:04:03.578 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226053 2024-05-11T23:04:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:04:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.579 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:04:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317176+0000", 2024-05-11T23:04:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0100000000000001e-07 2024-05-11T23:04:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317179+0000", 2024-05-11T23:04:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-05-11T23:04:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317182+0000", 2024-05-11T23:04:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3589999999999999e-06 2024-05-11T23:04:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317184+0000", 2024-05-11T23:04:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4339999999999999e-06 2024-05-11T23:04:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368061+0000", 2024-05-11T23:04:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050876927000000002 2024-05-11T23:04:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.581 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368179+0000", 2024-05-11T23:04:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118001 2024-05-11T23:04:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368282+0000", 2024-05-11T23:04:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102942 2024-05-11T23:04:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226054 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6080512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:03.582 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:04:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 61.251155777000001, 2024-05-11T23:04:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051322, 2024-05-11T23:04:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226054 2024-05-11T23:04:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:04:03.583 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:04:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:04:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:04:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317189+0000", 2024-05-11T23:04:03.584 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.217e-06 2024-05-11T23:04:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317192+0000", 2024-05-11T23:04:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0699999999999998e-06 2024-05-11T23:04:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317195+0000", 2024-05-11T23:04:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3829999999999999e-06 2024-05-11T23:04:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368328+0000", 2024-05-11T23:04:03.585 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051132999999999998 2024-05-11T23:04:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368429+0000", 2024-05-11T23:04:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010168800000000001 2024-05-11T23:04:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368510+0000", 2024-05-11T23:04:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0420999999999995e-05 2024-05-11T23:04:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226055 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6090752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:04:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 61.251145708999999, 2024-05-11T23:04:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051562835000000001, 2024-05-11T23:04:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226055 2024-05-11T23:04:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.587 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:04:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:04:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317256+0000", 2024-05-11T23:04:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8047000000000002e-05 2024-05-11T23:04:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:03.588 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317259+0000", 2024-05-11T23:04:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2590000000000001e-06 2024-05-11T23:04:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317267+0000", 2024-05-11T23:04:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8769999999999993e-06 2024-05-11T23:04:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317274+0000", 2024-05-11T23:04:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.187e-06 2024-05-11T23:04:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:03.589 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368544+0000", 2024-05-11T23:04:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051270299999999998 2024-05-11T23:04:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368658+0000", 2024-05-11T23:04:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000113347 2024-05-11T23:04:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368761+0000", 2024-05-11T23:04:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102818 2024-05-11T23:04:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226056 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6100992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:04:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 61.251059863000002, 2024-05-11T23:04:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051615576000000003, 2024-05-11T23:04:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226056 2024-05-11T23:04:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.591 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:04:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:04:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318915+0000", 2024-05-11T23:04:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001631291 2024-05-11T23:04:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.592 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318918+0000", 2024-05-11T23:04:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.512e-06 2024-05-11T23:04:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318927+0000", 2024-05-11T23:04:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1960000000000008e-06 2024-05-11T23:04:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318932+0000", 2024-05-11T23:04:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:04:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.593 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368805+0000", 2024-05-11T23:04:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049872998000000002 2024-05-11T23:04:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368853+0000", 2024-05-11T23:04:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8041999999999998e-05 2024-05-11T23:04:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368899+0000", 2024-05-11T23:04:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5766000000000002e-05 2024-05-11T23:04:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.594 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226061 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6152192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:04:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 61.249268776000001, 2024-05-11T23:04:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050675395999999998, 2024-05-11T23:04:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226061 2024-05-11T23:04:03.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:04:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:04:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:04:03.596 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:04:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319078+0000", 2024-05-11T23:04:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5670000000000002e-06 2024-05-11T23:04:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319085+0000", 2024-05-11T23:04:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2570000000000001e-06 2024-05-11T23:04:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.597 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319100+0000", 2024-05-11T23:04:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.489e-05 2024-05-11T23:04:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369584+0000", 2024-05-11T23:04:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050484345999999999 2024-05-11T23:04:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369671+0000", 2024-05-11T23:04:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6654000000000001e-05 2024-05-11T23:04:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.598 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369750+0000", 2024-05-11T23:04:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9257000000000004e-05 2024-05-11T23:04:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226062 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6162432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:04:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 61.249235231999997, 2024-05-11T23:04:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050761158000000001, 2024-05-11T23:04:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:03.599 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226062 2024-05-11T23:04:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:04:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:04:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319109+0000", 2024-05-11T23:04:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2800000000000002e-07 2024-05-11T23:04:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319111+0000", 2024-05-11T23:04:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4980000000000001e-06 2024-05-11T23:04:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319120+0000", 2024-05-11T23:04:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3999999999999992e-06 2024-05-11T23:04:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.601 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319129+0000", 2024-05-11T23:04:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7580000000000004e-06 2024-05-11T23:04:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369775+0000", 2024-05-11T23:04:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050645580000000003 2024-05-11T23:04:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369824+0000", 2024-05-11T23:04:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9506999999999999e-05 2024-05-11T23:04:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369869+0000", 2024-05-11T23:04:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4987000000000001e-05 2024-05-11T23:04:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226063 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6172672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:04:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 61.249206274999999, 2024-05-11T23:04:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050939784000000002, 2024-05-11T23:04:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:03.603 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226063 2024-05-11T23:04:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:04:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:04:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319138+0000", 2024-05-11T23:04:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.39e-07 2024-05-11T23:04:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319140+0000", 2024-05-11T23:04:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4820000000000001e-06 2024-05-11T23:04:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319148+0000", 2024-05-11T23:04:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6599999999999995e-06 2024-05-11T23:04:03.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319154+0000", 2024-05-11T23:04:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4169999999999997e-06 2024-05-11T23:04:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369905+0000", 2024-05-11T23:04:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050751009999999999 2024-05-11T23:04:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369991+0000", 2024-05-11T23:04:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5909999999999996e-05 2024-05-11T23:04:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.606 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370077+0000", 2024-05-11T23:04:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5865999999999994e-05 2024-05-11T23:04:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226064 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6182912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:04:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 61.249180078999998, 2024-05-11T23:04:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051038241999999998, 2024-05-11T23:04:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:03.607 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226064 2024-05-11T23:04:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:04:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:04:03.608 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319164+0000", 2024-05-11T23:04:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9299999999999999e-07 2024-05-11T23:04:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319166+0000", 2024-05-11T23:04:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.48e-06 2024-05-11T23:04:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319176+0000", 2024-05-11T23:04:03.609 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9669999999999996e-06 2024-05-11T23:04:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319180+0000", 2024-05-11T23:04:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8020000000000002e-06 2024-05-11T23:04:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370100+0000", 2024-05-11T23:04:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050919845999999998 2024-05-11T23:04:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370155+0000", 2024-05-11T23:04:03.610 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5040000000000002e-05 2024-05-11T23:04:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370202+0000", 2024-05-11T23:04:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6714e-05 2024-05-11T23:04:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226065 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6193152~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:04:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 61.249157509, 2024-05-11T23:04:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051225087000000002, 2024-05-11T23:04:03.611 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226065 2024-05-11T23:04:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:04:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:04:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320913+0000", 2024-05-11T23:04:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001727492 2024-05-11T23:04:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320919+0000", 2024-05-11T23:04:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.964e-06 2024-05-11T23:04:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.613 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320927+0000", 2024-05-11T23:04:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2239999999999998e-06 2024-05-11T23:04:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320934+0000", 2024-05-11T23:04:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8469999999999998e-06 2024-05-11T23:04:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370240+0000", 2024-05-11T23:04:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049306562999999998 2024-05-11T23:04:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370328+0000", 2024-05-11T23:04:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8122999999999996e-05 2024-05-11T23:04:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370411+0000", 2024-05-11T23:04:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2874000000000006e-05 2024-05-11T23:04:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226073 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6275072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:04:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 61.247319122, 2024-05-11T23:04:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050730475999999997, 2024-05-11T23:04:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226073 2024-05-11T23:04:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:04:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:04:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322914+0000", 2024-05-11T23:04:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001889153 2024-05-11T23:04:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322917+0000", 2024-05-11T23:04:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2100000000000002e-06 2024-05-11T23:04:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322924+0000", 2024-05-11T23:04:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9929999999999998e-06 2024-05-11T23:04:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322928+0000", 2024-05-11T23:04:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5549999999999996e-06 2024-05-11T23:04:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371578+0000", 2024-05-11T23:04:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048650001999999998 2024-05-11T23:04:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371671+0000", 2024-05-11T23:04:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2230000000000003e-05 2024-05-11T23:04:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371755+0000", 2024-05-11T23:04:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4332999999999994e-05 2024-05-11T23:04:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226085 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 5338112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:04:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 61.245228023000003, 2024-05-11T23:04:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050727622, 2024-05-11T23:04:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226085 2024-05-11T23:04:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:04:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:04:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324914+0000", 2024-05-11T23:04:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001798298 2024-05-11T23:04:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324918+0000", 2024-05-11T23:04:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3889999999999998e-06 2024-05-11T23:04:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324928+0000", 2024-05-11T23:04:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4339999999999998e-06 2024-05-11T23:04:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324936+0000", 2024-05-11T23:04:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2770000000000005e-06 2024-05-11T23:04:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373664+0000", 2024-05-11T23:04:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048728207000000003 2024-05-11T23:04:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373757+0000", 2024-05-11T23:04:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3332999999999996e-05 2024-05-11T23:04:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373843+0000", 2024-05-11T23:04:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5684000000000002e-05 2024-05-11T23:04:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232555 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7789568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:04:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.768217698000001, 2024-05-11T23:04:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050583086999999999, 2024-05-11T23:04:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232555 2024-05-11T23:04:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:04:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:04:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:04:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:04:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800127+0000", 2024-05-11T23:04:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.207e-06 2024-05-11T23:04:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800132+0000", 2024-05-11T23:04:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:04:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800134+0000", 2024-05-11T23:04:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5459999999999998e-06 2024-05-11T23:04:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850548+0000", 2024-05-11T23:04:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050414182000000002 2024-05-11T23:04:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850635+0000", 2024-05-11T23:04:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6853000000000003e-05 2024-05-11T23:04:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850709+0000", 2024-05-11T23:04:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3585999999999996e-05 2024-05-11T23:04:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232556 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7799808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:04:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.768206044000003, 2024-05-11T23:04:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050749599999999999, 2024-05-11T23:04:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232556 2024-05-11T23:04:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:03.628 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:04:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:04:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801914+0000", 2024-05-11T23:04:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017766360000000001 2024-05-11T23:04:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801919+0000", 2024-05-11T23:04:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.42e-06 2024-05-11T23:04:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801927+0000", 2024-05-11T23:04:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1999999999999994e-06 2024-05-11T23:04:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.630 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801931+0000", 2024-05-11T23:04:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4279999999999998e-06 2024-05-11T23:04:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850740+0000", 2024-05-11T23:04:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048808658999999997 2024-05-11T23:04:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850819+0000", 2024-05-11T23:04:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9102000000000003e-05 2024-05-11T23:04:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:03.631 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850887+0000", 2024-05-11T23:04:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8155000000000001e-05 2024-05-11T23:04:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232578 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8025088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:04:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.764912805999998, 2024-05-11T23:04:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050679473000000003, 2024-05-11T23:04:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:03.632 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232578 2024-05-11T23:04:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:04:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:04:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:04:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:04:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803432+0000", 2024-05-11T23:04:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.139e-06 2024-05-11T23:04:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803435+0000", 2024-05-11T23:04:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2440000000000002e-06 2024-05-11T23:04:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803439+0000", 2024-05-11T23:04:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3129999999999999e-06 2024-05-11T23:04:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.853953+0000", 2024-05-11T23:04:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050514846000000002 2024-05-11T23:04:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854035+0000", 2024-05-11T23:04:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1148000000000003e-05 2024-05-11T23:04:03.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854110+0000", 2024-05-11T23:04:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5562000000000006e-05 2024-05-11T23:04:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232579 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8035328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:04:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.764900202, 2024-05-11T23:04:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050886656000000002, 2024-05-11T23:04:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:03.636 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232579 2024-05-11T23:04:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:04:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.637 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:04:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803914+0000", 2024-05-11T23:04:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00047026700000000002 2024-05-11T23:04:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803918+0000", 2024-05-11T23:04:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3460000000000004e-06 2024-05-11T23:04:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.638 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803927+0000", 2024-05-11T23:04:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5030000000000003e-06 2024-05-11T23:04:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803933+0000", 2024-05-11T23:04:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0310000000000004e-06 2024-05-11T23:04:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854153+0000", 2024-05-11T23:04:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050219321999999997 2024-05-11T23:04:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.639 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854242+0000", 2024-05-11T23:04:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9457999999999996e-05 2024-05-11T23:04:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854330+0000", 2024-05-11T23:04:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7729e-05 2024-05-11T23:04:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.640 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232581 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8055808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:04:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.764376724999998, 2024-05-11T23:04:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050664439999999998, 2024-05-11T23:04:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232581 2024-05-11T23:04:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:04:03.641 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:04:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:04:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:04:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803969+0000", 2024-05-11T23:04:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0889999999999998e-06 2024-05-11T23:04:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803978+0000", 2024-05-11T23:04:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3070000000000003e-06 2024-05-11T23:04:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803983+0000", 2024-05-11T23:04:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1599999999999997e-06 2024-05-11T23:04:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854509+0000", 2024-05-11T23:04:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526339000000003 2024-05-11T23:04:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854576+0000", 2024-05-11T23:04:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7102999999999998e-05 2024-05-11T23:04:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854631+0000", 2024-05-11T23:04:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4997999999999998e-05 2024-05-11T23:04:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232582 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8066048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:04:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.764353980000003, 2024-05-11T23:04:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050753983000000003, 2024-05-11T23:04:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232582 2024-05-11T23:04:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:03.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:04:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:04:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805917+0000", 2024-05-11T23:04:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001927432 2024-05-11T23:04:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.646 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805920+0000", 2024-05-11T23:04:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1219999999999999e-06 2024-05-11T23:04:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805933+0000", 2024-05-11T23:04:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2727e-05 2024-05-11T23:04:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805941+0000", 2024-05-11T23:04:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7090000000000002e-06 2024-05-11T23:04:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.647 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854657+0000", 2024-05-11T23:04:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048716164999999999 2024-05-11T23:04:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854699+0000", 2024-05-11T23:04:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2531999999999999e-05 2024-05-11T23:04:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854744+0000", 2024-05-11T23:04:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4295999999999997e-05 2024-05-11T23:04:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.648 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232593 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8178688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:04:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 60.762154586999998, 2024-05-11T23:04:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050735178999999998, 2024-05-11T23:04:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232593 2024-05-11T23:04:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.649 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:04:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:04:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807913+0000", 2024-05-11T23:04:03.650 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017240319999999999 2024-05-11T23:04:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807916+0000", 2024-05-11T23:04:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4589999999999998e-06 2024-05-11T23:04:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807924+0000", 2024-05-11T23:04:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.272e-06 2024-05-11T23:04:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807928+0000", 2024-05-11T23:04:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5369999999999999e-06 2024-05-11T23:04:03.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856768+0000", 2024-05-11T23:04:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048839489999999999 2024-05-11T23:04:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856845+0000", 2024-05-11T23:04:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7746999999999999e-05 2024-05-11T23:04:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856924+0000", 2024-05-11T23:04:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8641999999999997e-05 2024-05-11T23:04:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.652 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.653 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.653 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:03.653 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:03.653 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:03.654 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:04:03.655 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:03.654+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:03.754 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:03.755+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:03.803 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:03.804 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:04:03.804 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:04:03.804 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:03.813 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:04:03.855 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:03.855+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:03.956 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:03.956+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:03.974 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:03.974 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:04:03.974 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:04:03.974 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:04:03.974 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:03.984 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:04:04.057 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:04:04.057+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:04:04.134 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:04.135 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:04:04.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:04:04.135 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:04:04.135 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.135 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:64049 3.8 3:11d98727:::rbd_object_map.111e8974a681:head [read 0~38] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-05-11T23:04:04.135 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:04:04.135 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 123.354133417, 2024-05-11T23:04:04.135 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99884604399999999, 2024-05-11T23:04:04.135 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:04.135 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:04.135 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:04.136 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:04.136 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:04.136 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:04.136 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 64049 2024-05-11T23:04:04.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.136 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:04.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:04.136 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:04:04.136 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.136 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:04.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:04:04.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:04.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779883+0000", 2024-05-11T23:04:04.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.57e-06 2024-05-11T23:04:04.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.137 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:04.137 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779887+0000", 2024-05-11T23:04:04.137 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5709999999999999e-06 2024-05-11T23:04:04.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:04.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779898+0000", 2024-05-11T23:04:04.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.042e-05 2024-05-11T23:04:04.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:04.138 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779905+0000", 2024-05-11T23:04:04.138 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0110000000000003e-06 2024-05-11T23:04:04.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.138 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.138 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779962+0000", 2024-05-11T23:04:04.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7395999999999997e-05 2024-05-11T23:04:04.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:04:04.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779969+0000", 2024-05-11T23:04:04.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2459999999999997e-06 2024-05-11T23:04:04.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.139 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.139 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778379+0000", 2024-05-11T23:04:04.139 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99840986300000001 2024-05-11T23:04:04.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:04.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778564+0000", 2024-05-11T23:04:04.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018420900000000001 2024-05-11T23:04:04.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.140 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.140 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:04.140 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778727+0000", 2024-05-11T23:04:04.140 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016375800000000001 2024-05-11T23:04:04.140 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.140 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.140 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.141 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.141 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172773 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:04.141 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:04:04.141 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.530026999, 2024-05-11T23:04:04.141 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058721316000000003, 2024-05-11T23:04:04.141 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:04.141 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:04.141 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:04.141 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:04.141 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:04.141 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:04.141 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172773 2024-05-11T23:04:04.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.142 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:04.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:04.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:04:04.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.142 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:04.142 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:04:04.142 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:04.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:04:04.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3500000000000002e-07 2024-05-11T23:04:04.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:04.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603990+0000", 2024-05-11T23:04:04.143 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3549999999999999e-06 2024-05-11T23:04:04.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.143 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:04.143 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603997+0000", 2024-05-11T23:04:04.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3400000000000003e-06 2024-05-11T23:04:04.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:04.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604002+0000", 2024-05-11T23:04:04.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0509999999999996e-06 2024-05-11T23:04:04.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.144 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.144 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662697+0000", 2024-05-11T23:04:04.144 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058695487999999997 2024-05-11T23:04:04.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.145 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:04.145 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662709+0000", 2024-05-11T23:04:04.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1647e-05 2024-05-11T23:04:04.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.145 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.145 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.145 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172774 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:04.145 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:04:04.145 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.530005617, 2024-05-11T23:04:04.145 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058745040999999998, 2024-05-11T23:04:04.146 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:04.146 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:04.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:04.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:04.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:04.146 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:04.146 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172774 2024-05-11T23:04:04.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.146 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:04.146 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.146 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:04.146 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:04:04.146 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:04.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:04:04.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.147 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:04.147 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:04:04.147 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:04:04.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:04.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604012+0000", 2024-05-11T23:04:04.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2199999999999999e-06 2024-05-11T23:04:04.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:04.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604018+0000", 2024-05-11T23:04:04.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1770000000000003e-06 2024-05-11T23:04:04.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.148 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:04.148 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604023+0000", 2024-05-11T23:04:04.148 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0089999999999998e-06 2024-05-11T23:04:04.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662742+0000", 2024-05-11T23:04:04.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058719187999999999 2024-05-11T23:04:04.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.149 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.149 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:04.149 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662754+0000", 2024-05-11T23:04:04.149 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2095e-05 2024-05-11T23:04:04.149 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.149 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.149 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.150 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172775 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:04.150 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:04:04.150 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.529985405000005, 2024-05-11T23:04:04.150 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058764743000000001, 2024-05-11T23:04:04.150 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:04.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:04.150 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:04.150 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:04.150 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:04.150 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:04.150 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172775 2024-05-11T23:04:04.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.151 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:04.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:04.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:04:04.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.151 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:04.151 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:04:04.151 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.151 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:04.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604030+0000", 2024-05-11T23:04:04.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6699999999999999e-07 2024-05-11T23:04:04.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:04.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604032+0000", 2024-05-11T23:04:04.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9530000000000002e-06 2024-05-11T23:04:04.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.152 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.152 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:04.152 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604037+0000", 2024-05-11T23:04:04.152 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8590000000000001e-06 2024-05-11T23:04:04.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:04.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604042+0000", 2024-05-11T23:04:04.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8609999999999997e-06 2024-05-11T23:04:04.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.153 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.153 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662782+0000", 2024-05-11T23:04:04.153 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058739731000000003 2024-05-11T23:04:04.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.154 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:04.154 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662794+0000", 2024-05-11T23:04:04.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1972e-05 2024-05-11T23:04:04.154 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.154 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.154 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.154 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172776 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:04.154 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:04:04.154 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.529966114999993, 2024-05-11T23:04:04.154 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058778696999999998, 2024-05-11T23:04:04.155 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:04.155 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:04.155 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:04.155 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:04.155 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:04.155 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:04.155 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172776 2024-05-11T23:04:04.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.155 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:04.155 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.155 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:04.155 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:04:04.155 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:04.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:04:04.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:04.156 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:04:04.156 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2000000000000001e-07 2024-05-11T23:04:04.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.156 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.156 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:04.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604051+0000", 2024-05-11T23:04:04.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2879999999999999e-06 2024-05-11T23:04:04.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:04.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604057+0000", 2024-05-11T23:04:04.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:04:04.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.157 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.157 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:04.157 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604062+0000", 2024-05-11T23:04:04.157 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6800000000000001e-06 2024-05-11T23:04:04.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662820+0000", 2024-05-11T23:04:04.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058758444999999999 2024-05-11T23:04:04.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.158 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.158 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:04.158 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662827+0000", 2024-05-11T23:04:04.158 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1930000000000004e-06 2024-05-11T23:04:04.158 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.158 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.159 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.159 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.159 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172777 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:04.159 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:04:04.159 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.529946808999995, 2024-05-11T23:04:04.159 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058799918999999999, 2024-05-11T23:04:04.159 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:04.159 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:04.159 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:04.159 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:04.159 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:04.159 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:04.160 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172777 2024-05-11T23:04:04.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.160 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:04.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:04.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:04:04.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.160 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.160 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:04.160 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:04:04.160 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:04.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:04:04.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:04:04.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.161 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:04.161 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604075+0000", 2024-05-11T23:04:04.161 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.109e-06 2024-05-11T23:04:04.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.161 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:04.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604083+0000", 2024-05-11T23:04:04.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9170000000000006e-06 2024-05-11T23:04:04.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:04.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604089+0000", 2024-05-11T23:04:04.162 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3399999999999997e-06 2024-05-11T23:04:04.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.162 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.162 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662859+0000", 2024-05-11T23:04:04.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058770172000000002 2024-05-11T23:04:04.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.163 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:04.163 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662868+0000", 2024-05-11T23:04:04.163 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0289999999999994e-06 2024-05-11T23:04:04.163 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.163 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.163 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.163 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172778 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:04.163 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:04:04.164 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.529918718999994, 2024-05-11T23:04:04.164 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058845435000000001, 2024-05-11T23:04:04.164 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:04.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:04.164 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:04.164 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:04.164 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:04.164 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:04.164 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172778 2024-05-11T23:04:04.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.164 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:04.164 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.164 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:04.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:04:04.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:04.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:04:04.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.165 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:04.165 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:04:04.165 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5400000000000002e-07 2024-05-11T23:04:04.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:04.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604099+0000", 2024-05-11T23:04:04.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4650000000000001e-06 2024-05-11T23:04:04.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:04.166 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604105+0000", 2024-05-11T23:04:04.166 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.314e-06 2024-05-11T23:04:04.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.166 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.166 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:04.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604110+0000", 2024-05-11T23:04:04.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6310000000000002e-06 2024-05-11T23:04:04.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662931+0000", 2024-05-11T23:04:04.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058820960999999998 2024-05-11T23:04:04.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.167 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.167 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:04.167 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662941+0000", 2024-05-11T23:04:04.167 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.061e-05 2024-05-11T23:04:04.167 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.168 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.168 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172779 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:04.168 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:04:04.168 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.529898559000003, 2024-05-11T23:04:04.168 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058859163999999999, 2024-05-11T23:04:04.168 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:04.168 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:04.168 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:04.168 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:04.169 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:04.169 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:04.169 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172779 2024-05-11T23:04:04.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.169 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:04.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:04.169 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:04:04.169 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.169 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:04.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:04:04.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:04.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:04:04.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:04:04.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.170 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:04.170 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604119+0000", 2024-05-11T23:04:04.170 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2529999999999999e-06 2024-05-11T23:04:04.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:04.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604125+0000", 2024-05-11T23:04:04.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1670000000000004e-06 2024-05-11T23:04:04.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.171 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:04.171 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604130+0000", 2024-05-11T23:04:04.171 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.702e-06 2024-05-11T23:04:04.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662967+0000", 2024-05-11T23:04:04.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058837792999999999 2024-05-11T23:04:04.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.172 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:04.172 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662975+0000", 2024-05-11T23:04:04.172 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8490000000000006e-06 2024-05-11T23:04:04.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.172 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.173 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172780 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:04.173 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:04:04.173 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.529878409000005, 2024-05-11T23:04:04.173 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058872696000000002, 2024-05-11T23:04:04.173 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:04.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:04.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:04.173 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:04.173 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:04.173 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:04.173 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172780 2024-05-11T23:04:04.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.174 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:04.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:04.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:04:04.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.174 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:04.174 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:04:04.174 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.174 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:04.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604137+0000", 2024-05-11T23:04:04.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4400000000000001e-07 2024-05-11T23:04:04.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:04.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604139+0000", 2024-05-11T23:04:04.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3429999999999998e-06 2024-05-11T23:04:04.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.175 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:04.175 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604145+0000", 2024-05-11T23:04:04.175 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1940000000000003e-06 2024-05-11T23:04:04.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:04.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604150+0000", 2024-05-11T23:04:04.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:04:04.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.176 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663002+0000", 2024-05-11T23:04:04.176 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058851829000000001 2024-05-11T23:04:04.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.176 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:04.177 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663009+0000", 2024-05-11T23:04:04.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.199e-06 2024-05-11T23:04:04.177 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.177 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.177 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.177 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.177 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172781 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:04.177 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:04:04.177 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.529846569, 2024-05-11T23:04:04.177 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058875553999999997, 2024-05-11T23:04:04.177 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:04.178 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:04.178 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:04.178 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:04.178 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:04.178 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:04.178 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172781 2024-05-11T23:04:04.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.178 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:04.178 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.178 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:04.178 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:04:04.178 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:04.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:04:04.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:04.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:04:04.179 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:04:04.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.179 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.179 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:04.179 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604171+0000", 2024-05-11T23:04:04.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.21e-06 2024-05-11T23:04:04.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:04.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604176+0000", 2024-05-11T23:04:04.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2970000000000003e-06 2024-05-11T23:04:04.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.180 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:04.180 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604179+0000", 2024-05-11T23:04:04.180 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8459999999999999e-06 2024-05-11T23:04:04.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.180 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663033+0000", 2024-05-11T23:04:04.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058854565999999997 2024-05-11T23:04:04.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.181 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:04.181 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663044+0000", 2024-05-11T23:04:04.181 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0229e-05 2024-05-11T23:04:04.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.181 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.181 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172782 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:04.182 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:04:04.182 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.529831821000002, 2024-05-11T23:04:04.182 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058894375999999998, 2024-05-11T23:04:04.182 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:04.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:04.182 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:04.182 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:04.182 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:04.182 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:04.182 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172782 2024-05-11T23:04:04.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.183 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:04.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:04.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:04:04.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:04.183 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:04:04.183 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.183 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.183 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:04.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:04:04.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-05-11T23:04:04.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:04.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604184+0000", 2024-05-11T23:04:04.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:04:04.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.184 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:04.184 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604187+0000", 2024-05-11T23:04:04.184 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.044e-06 2024-05-11T23:04:04.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:04.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604190+0000", 2024-05-11T23:04:04.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6460000000000002e-06 2024-05-11T23:04:04.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663070+0000", 2024-05-11T23:04:04.185 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058879510000000003 2024-05-11T23:04:04.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.185 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.185 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:04.185 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663077+0000", 2024-05-11T23:04:04.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6669999999999996e-06 2024-05-11T23:04:04.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.186 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.186 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172783 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:04.186 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:04:04.186 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.529821502000004, 2024-05-11T23:04:04.186 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058913946000000002, 2024-05-11T23:04:04.186 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:04.186 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:04.187 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:04.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:04.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:04.187 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:04.187 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172783 2024-05-11T23:04:04.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.187 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:04.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:04.187 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:04:04.187 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.187 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.187 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:04.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:04:04.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:04.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:04:04.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-05-11T23:04:04.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.188 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:04.188 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604195+0000", 2024-05-11T23:04:04.188 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.19e-06 2024-05-11T23:04:04.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:04.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604198+0000", 2024-05-11T23:04:04.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:04:04.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:04.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604200+0000", 2024-05-11T23:04:04.189 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4719999999999998e-06 2024-05-11T23:04:04.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.189 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.189 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663100+0000", 2024-05-11T23:04:04.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058900300000000003 2024-05-11T23:04:04.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.190 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:04.190 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663107+0000", 2024-05-11T23:04:04.190 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7449999999999998e-06 2024-05-11T23:04:04.190 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.190 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.190 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.190 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.190 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172784 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:04.190 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:04:04.191 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.529811483000003, 2024-05-11T23:04:04.191 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933728999999997, 2024-05-11T23:04:04.191 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:04.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:04.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:04.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:04.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:04.191 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:04.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172784 2024-05-11T23:04:04.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.191 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:04.191 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:04.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:04:04.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:04.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:04:04.192 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.192 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.192 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:04.192 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:04:04.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.03e-07 2024-05-11T23:04:04.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:04.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604205+0000", 2024-05-11T23:04:04.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2300000000000001e-06 2024-05-11T23:04:04.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.193 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:04.193 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604207+0000", 2024-05-11T23:04:04.193 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8729999999999999e-06 2024-05-11T23:04:04.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.193 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:04.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604210+0000", 2024-05-11T23:04:04.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.858e-06 2024-05-11T23:04:04.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663130+0000", 2024-05-11T23:04:04.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058919314 2024-05-11T23:04:04.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.194 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.194 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:04.194 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663137+0000", 2024-05-11T23:04:04.194 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2509999999999997e-06 2024-05-11T23:04:04.195 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.195 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.195 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.195 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172785 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:04.195 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:04:04.195 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.529799213000004, 2024-05-11T23:04:04.195 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058951664000000001, 2024-05-11T23:04:04.195 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:04.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:04.195 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:04.195 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:04.196 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:04.196 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:04.196 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172785 2024-05-11T23:04:04.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.196 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:04.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:04.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:04:04.196 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.196 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.196 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:04.196 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:04:04.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:04.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604216+0000", 2024-05-11T23:04:04.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:04:04.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.197 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:04.197 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604218+0000", 2024-05-11T23:04:04.197 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7549999999999999e-06 2024-05-11T23:04:04.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.197 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:04.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604222+0000", 2024-05-11T23:04:04.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6700000000000002e-06 2024-05-11T23:04:04.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:04.198 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604227+0000", 2024-05-11T23:04:04.198 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4399999999999998e-06 2024-05-11T23:04:04.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.198 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.198 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663160+0000", 2024-05-11T23:04:04.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933119999999999 2024-05-11T23:04:04.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.199 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:04.199 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663167+0000", 2024-05-11T23:04:04.199 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1840000000000002e-06 2024-05-11T23:04:04.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.199 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.199 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172786 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:04.200 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:04:04.200 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.529782159000007, 2024-05-11T23:04:04.200 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058966991000000003, 2024-05-11T23:04:04.200 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:04.200 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:04.200 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:04.200 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:04.200 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:04.200 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:04.200 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172786 2024-05-11T23:04:04.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.200 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:04.200 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:04.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:04:04.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:04.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:04:04.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.201 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.201 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:04.201 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604233+0000", 2024-05-11T23:04:04.201 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6899999999999999e-07 2024-05-11T23:04:04.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:04.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604235+0000", 2024-05-11T23:04:04.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8369999999999999e-06 2024-05-11T23:04:04.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:04.202 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604240+0000", 2024-05-11T23:04:04.202 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0329999999999999e-06 2024-05-11T23:04:04.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.202 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.202 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:04.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604244+0000", 2024-05-11T23:04:04.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9859999999999996e-06 2024-05-11T23:04:04.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663192+0000", 2024-05-11T23:04:04.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058948852000000003 2024-05-11T23:04:04.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.203 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:04.203 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663199+0000", 2024-05-11T23:04:04.203 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0140000000000001e-06 2024-05-11T23:04:04.203 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.204 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.204 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.204 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172787 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:04.204 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:04:04.204 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.529765812999997, 2024-05-11T23:04:04.204 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058982018999999997, 2024-05-11T23:04:04.204 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:04.204 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:04.204 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:04.204 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:04.204 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:04.205 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:04.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172787 2024-05-11T23:04:04.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.205 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:04.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:04.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:04:04.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.205 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.205 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:04.205 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:04:04.205 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:04.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605916+0000", 2024-05-11T23:04:04.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00166752 2024-05-11T23:04:04.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.206 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:04.206 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605921+0000", 2024-05-11T23:04:04.206 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0669999999999999e-06 2024-05-11T23:04:04.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.206 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:04.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605934+0000", 2024-05-11T23:04:04.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2271e-05 2024-05-11T23:04:04.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:04.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605940+0000", 2024-05-11T23:04:04.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6880000000000002e-06 2024-05-11T23:04:04.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.207 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.207 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.207 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663224+0000", 2024-05-11T23:04:04.207 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057283251 2024-05-11T23:04:04.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.208 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:04.208 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663231+0000", 2024-05-11T23:04:04.208 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2219999999999996e-06 2024-05-11T23:04:04.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.208 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.208 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.208 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172793 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:04.208 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:04:04.209 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.527998965999998, 2024-05-11T23:04:04.209 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057402373999999999, 2024-05-11T23:04:04.209 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:04.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:04.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:04.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:04.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:04.209 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:04.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172793 2024-05-11T23:04:04.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.209 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:04.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.209 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:04.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:04:04.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:04.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:04:04.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.210 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:04.210 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:04:04.210 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3300000000000001e-07 2024-05-11T23:04:04.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:04.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606017+0000", 2024-05-11T23:04:04.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4920000000000001e-06 2024-05-11T23:04:04.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:04.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606020+0000", 2024-05-11T23:04:04.211 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:04:04.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.211 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:04.211 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606023+0000", 2024-05-11T23:04:04.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6680000000000001e-06 2024-05-11T23:04:04.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663411+0000", 2024-05-11T23:04:04.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057388206999999997 2024-05-11T23:04:04.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.212 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.212 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:04.212 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663418+0000", 2024-05-11T23:04:04.212 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.815e-06 2024-05-11T23:04:04.212 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.212 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.213 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.213 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.213 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172795 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:04.213 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:04:04.213 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.527957538999999, 2024-05-11T23:04:04.213 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057423015000000001, 2024-05-11T23:04:04.213 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:04.213 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:04.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:04.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:04.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:04.213 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:04.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172795 2024-05-11T23:04:04.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.214 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:04.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:04.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:04:04.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.214 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.214 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:04.214 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:04:04.214 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:04.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:04:04.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9499999999999998e-07 2024-05-11T23:04:04.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:04.215 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606059+0000", 2024-05-11T23:04:04.215 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-05-11T23:04:04.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.215 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:04.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606062+0000", 2024-05-11T23:04:04.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.977e-06 2024-05-11T23:04:04.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:04.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606064+0000", 2024-05-11T23:04:04.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7410000000000001e-06 2024-05-11T23:04:04.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.216 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.216 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663473+0000", 2024-05-11T23:04:04.216 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057408624999999998 2024-05-11T23:04:04.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.217 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:04.217 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663480+0000", 2024-05-11T23:04:04.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1049999999999997e-06 2024-05-11T23:04:04.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.217 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.217 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.217 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172796 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:04.217 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:04:04.217 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.527946975999996, 2024-05-11T23:04:04.217 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057449235000000001, 2024-05-11T23:04:04.218 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:04.218 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:04.218 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:04.218 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:04.218 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:04.218 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:04.218 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172796 2024-05-11T23:04:04.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.218 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:04.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.218 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:04.218 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:04:04.218 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:04.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:04:04.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.219 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:04.219 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:04:04.219 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:04:04.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.219 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:04.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606069+0000", 2024-05-11T23:04:04.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:04:04.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:04.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606072+0000", 2024-05-11T23:04:04.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8700000000000001e-06 2024-05-11T23:04:04.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.220 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.220 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:04.220 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606074+0000", 2024-05-11T23:04:04.220 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6230000000000001e-06 2024-05-11T23:04:04.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663510+0000", 2024-05-11T23:04:04.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05743529 2024-05-11T23:04:04.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.221 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.221 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:04.221 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663517+0000", 2024-05-11T23:04:04.221 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1230000000000002e-06 2024-05-11T23:04:04.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.221 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.221 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.222 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.222 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172797 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:04.222 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:04:04.222 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 65.527936944000004, 2024-05-11T23:04:04.222 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057470420000000001, 2024-05-11T23:04:04.222 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:04.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:04.222 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:04.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:04.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:04.222 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:04.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172797 2024-05-11T23:04:04.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.223 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:04.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:04.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:04:04.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.223 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:04.223 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:04:04.223 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:04.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.223 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:04.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607915+0000", 2024-05-11T23:04:04.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018375609999999999 2024-05-11T23:04:04.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:04.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607920+0000", 2024-05-11T23:04:04.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8899999999999998e-06 2024-05-11T23:04:04.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.224 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.224 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:04.224 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607930+0000", 2024-05-11T23:04:04.224 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4429999999999992e-06 2024-05-11T23:04:04.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:04.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607936+0000", 2024-05-11T23:04:04.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7089999999999996e-06 2024-05-11T23:04:04.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:04.225 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663541+0000", 2024-05-11T23:04:04.225 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055604537000000002 2024-05-11T23:04:04.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:04.225 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:04.225 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:04.226 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663548+0000", 2024-05-11T23:04:04.226 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2799999999999998e-06 2024-05-11T23:04:04.226 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.226 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.226 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.226 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:04.226 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:04.226 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:04.227 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:04:04.228 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:04:04.229 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:04.227+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:04.328 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:04.328+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:04.380 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:04.380 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:04:04.380 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:04:04.381 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:04.390 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:04:04.395 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:04:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:04:04.414 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:04:04.414 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:04:04.429 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:04.429+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:04.431 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:04:04.431 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:04:04.432 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-05-11T23:04:04.450 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:04:04.450 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:04:04.469 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:04:04.469 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:04:04.486 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:04:04.486 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:04:04.504 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:04:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:04:04.505 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:04.506 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:04.508 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-11T23:04:04.509 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-11T23:04:04.515 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:04:04.530 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:04:04.530+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:04:04.631 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:04.631+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:04.665 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:04.666 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:04.675 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:04:04.732 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:04.732+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:04.784 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:04.785 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:04.793 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:04:04.833 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:04.833+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:04.874 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-11T23:04:04.934 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:04:04.934+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:04:04.958 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:04.958 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:04.968 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:04:05.035 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:05.035+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:05.113 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:05.113 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:05.123 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:04:05.167 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:05.167+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:05.229 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:05.229 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:05.237 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:05.236+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:05.238 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:04:05.337 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:05.337+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:05.384 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:05.384 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:05.393 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:04:05.478 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:05.478+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:05.539 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:04:05.538+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:04:05.540 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:05.540 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:05.550 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:04:05.640 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:05.639+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:05.695 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:05.696 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:05.704 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:04:05.749 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:05.748+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:05.810 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:05.810 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:05.818 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:04:05.860 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:04:05.859+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:04:05.868 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-05-11T23:04:05.943 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:05.943+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:05.977 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:05.978 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:05.987 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:04:06.044 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:06.043+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:06.093 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:06.093 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:06.101 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:04:06.144 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:06.144+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:06.246 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:06.246 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:06.246 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:06.246+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:06.255 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:04:06.347 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:06.346+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:06.359 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:06.360 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:06.368 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:04:06.448 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:06.448+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:06.491 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:06.491 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:06.500 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:04:06.510 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:04:06.584 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:06.584+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:06.649 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:06.649+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:06.654 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:06.654 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:06.664 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:04:06.670 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:04:06.671 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:04:06.689 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:04:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:04:06.706 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:04:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:04:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-05-11T23:04:06.725 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:04:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:04:06.743 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:04:06.743 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:04:06.750 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:06.749+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:06.760 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:04:06.761 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:04:06.778 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:04:06.779 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:04:06.782 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-11T23:04:06.782 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:06.783 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:04:06.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:04:06.783 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:04:06.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.783 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226051 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6049792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:06.783 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:04:06.783 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.464278570000005, 2024-05-11T23:04:06.783 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050622418000000002, 2024-05-11T23:04:06.784 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:06.784 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:06.784 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:06.784 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:06.784 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:06.784 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:06.784 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226051 2024-05-11T23:04:06.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.784 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:06.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.784 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:06.784 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:04:06.784 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:06.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:04:06.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:06.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317156+0000", 2024-05-11T23:04:06.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:04:06.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.785 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.785 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:06.785 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317157+0000", 2024-05-11T23:04:06.785 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.144e-06 2024-05-11T23:04:06.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:06.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317160+0000", 2024-05-11T23:04:06.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1350000000000001e-06 2024-05-11T23:04:06.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:06.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317162+0000", 2024-05-11T23:04:06.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5050000000000002e-06 2024-05-11T23:04:06.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.786 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:06.786 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367689+0000", 2024-05-11T23:04:06.786 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526797999999998 2024-05-11T23:04:06.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:06.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367735+0000", 2024-05-11T23:04:06.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5698999999999997e-05 2024-05-11T23:04:06.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.787 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.787 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:06.787 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367778+0000", 2024-05-11T23:04:06.787 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2928e-05 2024-05-11T23:04:06.787 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.787 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.787 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.788 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226052 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6060032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:06.788 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:04:06.788 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.464268513999997, 2024-05-11T23:04:06.788 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050830110999999997, 2024-05-11T23:04:06.788 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:06.788 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:06.789 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:06.789 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:06.789 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:06.789 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:06.789 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226052 2024-05-11T23:04:06.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.789 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:06.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.789 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:06.789 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:04:06.789 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:06.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:04:06.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:06.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317166+0000", 2024-05-11T23:04:06.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:04:06.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.790 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.790 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:06.790 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317167+0000", 2024-05-11T23:04:06.790 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.155e-06 2024-05-11T23:04:06.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:06.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317170+0000", 2024-05-11T23:04:06.791 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.061e-06 2024-05-11T23:04:06.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.791 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.791 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:06.791 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317172+0000", 2024-05-11T23:04:06.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4420000000000001e-06 2024-05-11T23:04:06.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.792 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.792 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:06.792 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367816+0000", 2024-05-11T23:04:06.792 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050644013000000002 2024-05-11T23:04:06.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.793 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.793 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:06.793 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367906+0000", 2024-05-11T23:04:06.793 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9739000000000003e-05 2024-05-11T23:04:06.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.794 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:06.794 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367995+0000", 2024-05-11T23:04:06.794 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9499000000000005e-05 2024-05-11T23:04:06.794 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.794 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.794 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.794 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.794 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226053 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6070272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:06.794 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:04:06.795 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.464258474999994, 2024-05-11T23:04:06.795 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051106912999999997, 2024-05-11T23:04:06.795 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:06.795 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:06.795 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:06.795 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:06.795 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:06.795 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:06.795 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226053 2024-05-11T23:04:06.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.795 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:06.795 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:06.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:04:06.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:06.796 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:04:06.796 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.796 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.796 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:06.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317176+0000", 2024-05-11T23:04:06.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0100000000000001e-07 2024-05-11T23:04:06.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:06.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317179+0000", 2024-05-11T23:04:06.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-05-11T23:04:06.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.797 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:06.797 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317182+0000", 2024-05-11T23:04:06.797 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3589999999999999e-06 2024-05-11T23:04:06.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:06.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317184+0000", 2024-05-11T23:04:06.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4339999999999999e-06 2024-05-11T23:04:06.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:06.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368061+0000", 2024-05-11T23:04:06.798 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050876927000000002 2024-05-11T23:04:06.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.798 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:06.798 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368179+0000", 2024-05-11T23:04:06.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118001 2024-05-11T23:04:06.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.799 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:06.799 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368282+0000", 2024-05-11T23:04:06.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102942 2024-05-11T23:04:06.799 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.799 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.799 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.799 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226054 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6080512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:06.799 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:04:06.799 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.464246153000005, 2024-05-11T23:04:06.799 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051322, 2024-05-11T23:04:06.800 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:06.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:06.800 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:06.800 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:06.800 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:06.800 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:06.800 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226054 2024-05-11T23:04:06.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.800 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:06.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.800 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:06.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:04:06.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:06.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:04:06.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:06.801 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:04:06.801 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:04:06.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.801 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.801 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:06.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317189+0000", 2024-05-11T23:04:06.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.217e-06 2024-05-11T23:04:06.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:06.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317192+0000", 2024-05-11T23:04:06.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0699999999999998e-06 2024-05-11T23:04:06.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:06.802 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317195+0000", 2024-05-11T23:04:06.802 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3829999999999999e-06 2024-05-11T23:04:06.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.802 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.802 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:06.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368328+0000", 2024-05-11T23:04:06.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051132999999999998 2024-05-11T23:04:06.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:06.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368429+0000", 2024-05-11T23:04:06.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010168800000000001 2024-05-11T23:04:06.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.803 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.803 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:06.803 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368510+0000", 2024-05-11T23:04:06.803 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0420999999999995e-05 2024-05-11T23:04:06.803 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.804 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.804 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.804 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.804 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226055 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6090752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:06.804 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:04:06.804 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.464236084999996, 2024-05-11T23:04:06.804 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051562835000000001, 2024-05-11T23:04:06.804 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:06.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:06.804 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:06.804 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:06.805 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:06.805 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:06.805 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226055 2024-05-11T23:04:06.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.805 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:06.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:06.805 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:04:06.805 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.805 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.805 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:06.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:04:06.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:06.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317256+0000", 2024-05-11T23:04:06.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8047000000000002e-05 2024-05-11T23:04:06.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.806 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:06.806 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317259+0000", 2024-05-11T23:04:06.806 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2590000000000001e-06 2024-05-11T23:04:06.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.806 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317267+0000", 2024-05-11T23:04:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8769999999999993e-06 2024-05-11T23:04:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317274+0000", 2024-05-11T23:04:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.187e-06 2024-05-11T23:04:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368544+0000", 2024-05-11T23:04:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051270299999999998 2024-05-11T23:04:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.807 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:06.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368658+0000", 2024-05-11T23:04:06.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000113347 2024-05-11T23:04:06.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.808 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:06.808 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368761+0000", 2024-05-11T23:04:06.808 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102818 2024-05-11T23:04:06.808 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.808 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.808 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.808 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226056 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6100992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:06.808 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:04:06.808 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.464150239000006, 2024-05-11T23:04:06.809 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051615576000000003, 2024-05-11T23:04:06.809 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:06.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:06.809 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:06.809 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:06.809 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:06.809 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:06.809 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226056 2024-05-11T23:04:06.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.809 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:06.809 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.809 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:06.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:04:06.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:06.810 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:04:06.810 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.810 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.810 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:06.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318915+0000", 2024-05-11T23:04:06.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001631291 2024-05-11T23:04:06.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:06.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318918+0000", 2024-05-11T23:04:06.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.512e-06 2024-05-11T23:04:06.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:06.811 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318927+0000", 2024-05-11T23:04:06.811 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1960000000000008e-06 2024-05-11T23:04:06.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.811 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.811 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:06.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318932+0000", 2024-05-11T23:04:06.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:04:06.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:06.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368805+0000", 2024-05-11T23:04:06.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049872998000000002 2024-05-11T23:04:06.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:06.812 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368853+0000", 2024-05-11T23:04:06.812 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8041999999999998e-05 2024-05-11T23:04:06.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.812 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:06.813 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368899+0000", 2024-05-11T23:04:06.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5766000000000002e-05 2024-05-11T23:04:06.813 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.813 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.813 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.813 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226061 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6152192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:06.813 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:04:06.813 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.462359152000005, 2024-05-11T23:04:06.813 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050675395999999998, 2024-05-11T23:04:06.813 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:06.813 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:06.813 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:06.813 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:06.813 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:06.814 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:06.814 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226061 2024-05-11T23:04:06.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.814 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:06.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:06.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:04:06.814 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.814 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.814 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:06.814 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:04:06.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:06.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:04:06.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:04:06.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:06.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319078+0000", 2024-05-11T23:04:06.815 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5670000000000002e-06 2024-05-11T23:04:06.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.815 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.815 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:06.815 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319085+0000", 2024-05-11T23:04:06.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2570000000000001e-06 2024-05-11T23:04:06.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:06.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319100+0000", 2024-05-11T23:04:06.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.489e-05 2024-05-11T23:04:06.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:06.816 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369584+0000", 2024-05-11T23:04:06.816 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050484345999999999 2024-05-11T23:04:06.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.816 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.816 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:06.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369671+0000", 2024-05-11T23:04:06.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6654000000000001e-05 2024-05-11T23:04:06.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.817 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:06.817 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369750+0000", 2024-05-11T23:04:06.817 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9257000000000004e-05 2024-05-11T23:04:06.817 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.817 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.817 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.817 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.817 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226062 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6162432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:06.817 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:04:06.817 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.462325608, 2024-05-11T23:04:06.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050761158000000001, 2024-05-11T23:04:06.818 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:06.818 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:06.818 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:06.818 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:06.818 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:06.818 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:06.818 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226062 2024-05-11T23:04:06.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.818 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:06.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.818 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:06.818 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:04:06.818 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:06.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:04:06.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:06.819 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319109+0000", 2024-05-11T23:04:06.819 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2800000000000002e-07 2024-05-11T23:04:06.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.819 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:06.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319111+0000", 2024-05-11T23:04:06.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4980000000000001e-06 2024-05-11T23:04:06.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:06.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319120+0000", 2024-05-11T23:04:06.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3999999999999992e-06 2024-05-11T23:04:06.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.820 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.820 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:06.820 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319129+0000", 2024-05-11T23:04:06.820 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7580000000000004e-06 2024-05-11T23:04:06.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:06.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369775+0000", 2024-05-11T23:04:06.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050645580000000003 2024-05-11T23:04:06.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:06.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369824+0000", 2024-05-11T23:04:06.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9506999999999999e-05 2024-05-11T23:04:06.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.821 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:06.821 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369869+0000", 2024-05-11T23:04:06.821 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4987000000000001e-05 2024-05-11T23:04:06.822 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.822 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.822 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.822 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.822 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226063 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6172672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:06.822 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:04:06.822 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.462296651000003, 2024-05-11T23:04:06.822 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050939784000000002, 2024-05-11T23:04:06.822 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:06.822 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:06.822 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:06.822 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:06.823 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:06.823 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:06.823 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226063 2024-05-11T23:04:06.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.823 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:06.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:06.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:04:06.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.823 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:06.823 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:04:06.823 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:06.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319138+0000", 2024-05-11T23:04:06.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.39e-07 2024-05-11T23:04:06.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:06.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319140+0000", 2024-05-11T23:04:06.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4820000000000001e-06 2024-05-11T23:04:06.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.824 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.824 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:06.824 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319148+0000", 2024-05-11T23:04:06.824 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6599999999999995e-06 2024-05-11T23:04:06.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:06.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319154+0000", 2024-05-11T23:04:06.825 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4169999999999997e-06 2024-05-11T23:04:06.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.825 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.825 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:06.825 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369905+0000", 2024-05-11T23:04:06.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050751009999999999 2024-05-11T23:04:06.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.826 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:06.826 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369991+0000", 2024-05-11T23:04:06.826 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5909999999999996e-05 2024-05-11T23:04:06.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.826 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.827 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:06.827 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370077+0000", 2024-05-11T23:04:06.827 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5865999999999994e-05 2024-05-11T23:04:06.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.827 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.827 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226064 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6182912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:06.828 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:04:06.828 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.462270454999995, 2024-05-11T23:04:06.828 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051038241999999998, 2024-05-11T23:04:06.828 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:06.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:06.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:06.828 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:06.828 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:06.828 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:06.829 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226064 2024-05-11T23:04:06.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.829 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:06.829 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.829 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:06.829 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:04:06.829 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.830 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.830 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:06.830 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:04:06.830 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:06.831 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319164+0000", 2024-05-11T23:04:06.831 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9299999999999999e-07 2024-05-11T23:04:06.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.831 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:06.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319166+0000", 2024-05-11T23:04:06.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.48e-06 2024-05-11T23:04:06.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.832 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:06.832 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319176+0000", 2024-05-11T23:04:06.832 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9669999999999996e-06 2024-05-11T23:04:06.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:06.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319180+0000", 2024-05-11T23:04:06.833 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8020000000000002e-06 2024-05-11T23:04:06.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.833 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.833 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:06.833 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370100+0000", 2024-05-11T23:04:06.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050919845999999998 2024-05-11T23:04:06.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.834 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:06.834 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370155+0000", 2024-05-11T23:04:06.834 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5040000000000002e-05 2024-05-11T23:04:06.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.835 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:06.835 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370202+0000", 2024-05-11T23:04:06.835 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6714e-05 2024-05-11T23:04:06.835 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.835 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.836 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.836 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.836 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226065 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6193152~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:06.836 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:04:06.836 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.462247884999996, 2024-05-11T23:04:06.837 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051225087000000002, 2024-05-11T23:04:06.837 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:06.837 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:06.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:06.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:06.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:06.837 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:06.837 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226065 2024-05-11T23:04:06.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.837 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:06.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:06.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:04:06.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:06.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:04:06.838 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.838 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:06.838 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320913+0000", 2024-05-11T23:04:06.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001727492 2024-05-11T23:04:06.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:06.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320919+0000", 2024-05-11T23:04:06.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.964e-06 2024-05-11T23:04:06.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:06.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320927+0000", 2024-05-11T23:04:06.839 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2239999999999998e-06 2024-05-11T23:04:06.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.839 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.839 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:06.839 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320934+0000", 2024-05-11T23:04:06.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8469999999999998e-06 2024-05-11T23:04:06.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:06.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370240+0000", 2024-05-11T23:04:06.840 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049306562999999998 2024-05-11T23:04:06.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.840 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.840 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:06.840 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370328+0000", 2024-05-11T23:04:06.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8122999999999996e-05 2024-05-11T23:04:06.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.841 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:06.841 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370411+0000", 2024-05-11T23:04:06.841 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2874000000000006e-05 2024-05-11T23:04:06.841 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.841 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.841 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.841 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226073 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6275072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:06.842 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:04:06.842 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.460409498000004, 2024-05-11T23:04:06.842 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050730475999999997, 2024-05-11T23:04:06.842 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:06.842 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:06.842 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:06.842 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:06.842 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:06.842 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:06.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226073 2024-05-11T23:04:06.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.842 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:06.842 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:06.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:04:06.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:06.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:04:06.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.843 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.843 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:06.843 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322914+0000", 2024-05-11T23:04:06.843 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001889153 2024-05-11T23:04:06.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:06.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322917+0000", 2024-05-11T23:04:06.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2100000000000002e-06 2024-05-11T23:04:06.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.844 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:06.844 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322924+0000", 2024-05-11T23:04:06.844 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9929999999999998e-06 2024-05-11T23:04:06.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:06.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322928+0000", 2024-05-11T23:04:06.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5549999999999996e-06 2024-05-11T23:04:06.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:06.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371578+0000", 2024-05-11T23:04:06.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048650001999999998 2024-05-11T23:04:06.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.845 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:06.845 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371671+0000", 2024-05-11T23:04:06.845 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2230000000000003e-05 2024-05-11T23:04:06.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.845 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.846 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:06.846 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371755+0000", 2024-05-11T23:04:06.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4332999999999994e-05 2024-05-11T23:04:06.846 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.846 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.846 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.846 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226085 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 5338112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:06.846 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:04:06.846 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.458318399000007, 2024-05-11T23:04:06.846 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050727622, 2024-05-11T23:04:06.846 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:06.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:06.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:06.847 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:06.847 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:06.847 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:06.847 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226085 2024-05-11T23:04:06.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.847 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:06.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:06.847 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:04:06.847 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.847 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.847 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:06.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:04:06.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:06.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324914+0000", 2024-05-11T23:04:06.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001798298 2024-05-11T23:04:06.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.848 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:06.848 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324918+0000", 2024-05-11T23:04:06.848 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3889999999999998e-06 2024-05-11T23:04:06.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:06.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324928+0000", 2024-05-11T23:04:06.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4339999999999998e-06 2024-05-11T23:04:06.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:06.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324936+0000", 2024-05-11T23:04:06.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2770000000000005e-06 2024-05-11T23:04:06.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:06.849 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373664+0000", 2024-05-11T23:04:06.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048728207000000003 2024-05-11T23:04:06.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.849 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:06.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373757+0000", 2024-05-11T23:04:06.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3332999999999996e-05 2024-05-11T23:04:06.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.850 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:06.850 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373843+0000", 2024-05-11T23:04:06.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5684000000000002e-05 2024-05-11T23:04:06.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.850 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.851 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232555 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7789568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:06.851 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:04:06.851 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 63.981308073999998, 2024-05-11T23:04:06.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050583086999999999, 2024-05-11T23:04:06.851 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:06.851 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:06.851 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:06.851 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:06.851 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:06.851 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:06.851 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232555 2024-05-11T23:04:06.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.851 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:06.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:06.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:04:06.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:06.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:04:06.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:06.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:04:06.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:04:06.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:06.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800127+0000", 2024-05-11T23:04:06.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.207e-06 2024-05-11T23:04:06.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:06.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800132+0000", 2024-05-11T23:04:06.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:04:06.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:06.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800134+0000", 2024-05-11T23:04:06.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5459999999999998e-06 2024-05-11T23:04:06.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:06.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850548+0000", 2024-05-11T23:04:06.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050414182000000002 2024-05-11T23:04:06.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:06.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850635+0000", 2024-05-11T23:04:06.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6853000000000003e-05 2024-05-11T23:04:06.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:06.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850709+0000", 2024-05-11T23:04:06.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3585999999999996e-05 2024-05-11T23:04:06.854 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.855 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.855 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.855 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232556 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7799808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:06.855 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:04:06.855 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 63.98129642, 2024-05-11T23:04:06.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050749599999999999, 2024-05-11T23:04:06.855 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:06.855 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:06.855 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:06.855 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:06.856 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:06.856 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:06.856 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232556 2024-05-11T23:04:06.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.856 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:06.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:06.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:04:06.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:06.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:04:06.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:06.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801914+0000", 2024-05-11T23:04:06.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017766360000000001 2024-05-11T23:04:06.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:06.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801919+0000", 2024-05-11T23:04:06.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.42e-06 2024-05-11T23:04:06.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:06.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801927+0000", 2024-05-11T23:04:06.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1999999999999994e-06 2024-05-11T23:04:06.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:06.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801931+0000", 2024-05-11T23:04:06.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4279999999999998e-06 2024-05-11T23:04:06.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:06.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850740+0000", 2024-05-11T23:04:06.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048808658999999997 2024-05-11T23:04:06.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:06.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850819+0000", 2024-05-11T23:04:06.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9102000000000003e-05 2024-05-11T23:04:06.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.859 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:06.859 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850887+0000", 2024-05-11T23:04:06.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8155000000000001e-05 2024-05-11T23:04:06.859 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.859 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.859 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.860 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232578 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8025088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:06.860 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:04:06.860 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 63.978003182000002, 2024-05-11T23:04:06.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050679473000000003, 2024-05-11T23:04:06.860 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:06.860 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:06.860 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:06.860 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:06.860 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:06.860 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:06.860 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232578 2024-05-11T23:04:06.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.860 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:06.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:06.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:04:06.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:06.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:04:06.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:06.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:04:06.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:04:06.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:06.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803432+0000", 2024-05-11T23:04:06.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.139e-06 2024-05-11T23:04:06.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:06.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803435+0000", 2024-05-11T23:04:06.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2440000000000002e-06 2024-05-11T23:04:06.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:06.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803439+0000", 2024-05-11T23:04:06.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3129999999999999e-06 2024-05-11T23:04:06.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:06.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.853953+0000", 2024-05-11T23:04:06.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050514846000000002 2024-05-11T23:04:06.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.863 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:06.863 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854035+0000", 2024-05-11T23:04:06.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1148000000000003e-05 2024-05-11T23:04:06.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:06.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854110+0000", 2024-05-11T23:04:06.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5562000000000006e-05 2024-05-11T23:04:06.864 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.864 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.864 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.864 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232579 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8035328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:06.864 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:04:06.864 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 63.977990578000004, 2024-05-11T23:04:06.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050886656000000002, 2024-05-11T23:04:06.865 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:06.865 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:06.865 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:06.865 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:06.865 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:06.865 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:06.865 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232579 2024-05-11T23:04:06.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.865 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:06.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:06.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:04:06.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:06.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:04:06.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:06.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803914+0000", 2024-05-11T23:04:06.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00047026700000000002 2024-05-11T23:04:06.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:06.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803918+0000", 2024-05-11T23:04:06.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3460000000000004e-06 2024-05-11T23:04:06.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:06.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803927+0000", 2024-05-11T23:04:06.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5030000000000003e-06 2024-05-11T23:04:06.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:06.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803933+0000", 2024-05-11T23:04:06.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0310000000000004e-06 2024-05-11T23:04:06.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:06.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854153+0000", 2024-05-11T23:04:06.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050219321999999997 2024-05-11T23:04:06.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:06.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854242+0000", 2024-05-11T23:04:06.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9457999999999996e-05 2024-05-11T23:04:06.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:06.868 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854330+0000", 2024-05-11T23:04:06.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7729e-05 2024-05-11T23:04:06.868 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.869 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.869 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.869 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232581 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8055808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:06.869 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:04:06.869 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 63.977467101000002, 2024-05-11T23:04:06.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050664439999999998, 2024-05-11T23:04:06.869 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:06.869 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:06.869 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:06.869 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:06.869 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:06.870 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:06.870 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232581 2024-05-11T23:04:06.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.870 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:06.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:06.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:04:06.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:06.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:04:06.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:06.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:04:06.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:04:06.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:06.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803969+0000", 2024-05-11T23:04:06.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0889999999999998e-06 2024-05-11T23:04:06.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:06.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803978+0000", 2024-05-11T23:04:06.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3070000000000003e-06 2024-05-11T23:04:06.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:06.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803983+0000", 2024-05-11T23:04:06.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1599999999999997e-06 2024-05-11T23:04:06.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:06.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854509+0000", 2024-05-11T23:04:06.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526339000000003 2024-05-11T23:04:06.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.872 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:06.872 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854576+0000", 2024-05-11T23:04:06.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7102999999999998e-05 2024-05-11T23:04:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854631+0000", 2024-05-11T23:04:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4997999999999998e-05 2024-05-11T23:04:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232582 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8066048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:04:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 63.977444355999999, 2024-05-11T23:04:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050753983000000003, 2024-05-11T23:04:06.873 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:06.874 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:06.874 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:06.874 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:06.874 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:06.874 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:06.874 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232582 2024-05-11T23:04:06.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.874 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:06.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:06.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:04:06.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:06.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:04:06.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:06.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805917+0000", 2024-05-11T23:04:06.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001927432 2024-05-11T23:04:06.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:06.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805920+0000", 2024-05-11T23:04:06.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1219999999999999e-06 2024-05-11T23:04:06.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:06.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805933+0000", 2024-05-11T23:04:06.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2727e-05 2024-05-11T23:04:06.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:06.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805941+0000", 2024-05-11T23:04:06.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7090000000000002e-06 2024-05-11T23:04:06.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:06.876 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854657+0000", 2024-05-11T23:04:06.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048716164999999999 2024-05-11T23:04:06.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.876 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:06.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854699+0000", 2024-05-11T23:04:06.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2531999999999999e-05 2024-05-11T23:04:06.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:06.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854744+0000", 2024-05-11T23:04:06.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4295999999999997e-05 2024-05-11T23:04:06.877 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.877 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.877 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.877 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232593 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8178688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:06.877 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:04:06.877 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 63.975244963000002, 2024-05-11T23:04:06.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050735178999999998, 2024-05-11T23:04:06.878 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:06.878 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:06.878 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:06.878 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:06.878 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:06.878 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:06.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232593 2024-05-11T23:04:06.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.878 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:06.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:06.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:04:06.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:06.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:04:06.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:06.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:06.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807913+0000", 2024-05-11T23:04:06.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017240319999999999 2024-05-11T23:04:06.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:06.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807916+0000", 2024-05-11T23:04:06.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4589999999999998e-06 2024-05-11T23:04:06.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:06.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807924+0000", 2024-05-11T23:04:06.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.272e-06 2024-05-11T23:04:06.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:06.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807928+0000", 2024-05-11T23:04:06.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5369999999999999e-06 2024-05-11T23:04:06.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:06.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856768+0000", 2024-05-11T23:04:06.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048839489999999999 2024-05-11T23:04:06.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:06.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856845+0000", 2024-05-11T23:04:06.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7746999999999999e-05 2024-05-11T23:04:06.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:06.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:06.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:06.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856924+0000", 2024-05-11T23:04:06.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8641999999999997e-05 2024-05-11T23:04:06.881 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.881 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.881 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.881 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:06.881 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:06.881 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:06.882 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-11T23:04:06.883 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:04:06.884 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:04:06.884+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:04:06.986 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:06.986+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:07.040 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:07.040 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:04:07.040 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:04:07.040 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:07.048 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:04:07.090 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:07.090+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:07.187 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:07.188+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:07.199 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:07.199 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:04:07.199 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:04:07.199 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:04:07.199 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:07.208 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:04:07.288 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:07.288+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:07.359 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:07.359 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:04:07.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:04:07.360 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:04:07.360 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.360 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:64049 3.8 3:11d98727:::rbd_object_map.111e8974a681:head [read 0~38] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-05-11T23:04:07.360 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:04:07.360 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 126.57893092400001, 2024-05-11T23:04:07.360 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99884604399999999, 2024-05-11T23:04:07.360 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.360 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.360 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:07.361 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.361 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.361 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 64049 2024-05-11T23:04:07.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.361 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:04:07.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.361 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.361 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.361 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:04:07.361 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779883+0000", 2024-05-11T23:04:07.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.57e-06 2024-05-11T23:04:07.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.362 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.362 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.362 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779887+0000", 2024-05-11T23:04:07.362 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5709999999999999e-06 2024-05-11T23:04:07.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779898+0000", 2024-05-11T23:04:07.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.042e-05 2024-05-11T23:04:07.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.363 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779905+0000", 2024-05-11T23:04:07.363 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0110000000000003e-06 2024-05-11T23:04:07.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.363 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779962+0000", 2024-05-11T23:04:07.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7395999999999997e-05 2024-05-11T23:04:07.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:04:07.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779969+0000", 2024-05-11T23:04:07.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2459999999999997e-06 2024-05-11T23:04:07.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.364 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.364 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778379+0000", 2024-05-11T23:04:07.364 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99840986300000001 2024-05-11T23:04:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778564+0000", 2024-05-11T23:04:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018420900000000001 2024-05-11T23:04:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.365 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778727+0000", 2024-05-11T23:04:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016375800000000001 2024-05-11T23:04:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172773 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:04:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.754824506000006, 2024-05-11T23:04:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058721316000000003, 2024-05-11T23:04:07.366 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172773 2024-05-11T23:04:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:04:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.367 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:04:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:04:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3500000000000002e-07 2024-05-11T23:04:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603990+0000", 2024-05-11T23:04:07.368 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3549999999999999e-06 2024-05-11T23:04:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603997+0000", 2024-05-11T23:04:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3400000000000003e-06 2024-05-11T23:04:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604002+0000", 2024-05-11T23:04:07.369 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0509999999999996e-06 2024-05-11T23:04:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662697+0000", 2024-05-11T23:04:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058695487999999997 2024-05-11T23:04:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.370 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662709+0000", 2024-05-11T23:04:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1647e-05 2024-05-11T23:04:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.371 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172774 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:04:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.754803124000006, 2024-05-11T23:04:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058745040999999998, 2024-05-11T23:04:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.372 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172774 2024-05-11T23:04:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:04:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.374 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:04:07.374 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.374 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:04:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:04:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604012+0000", 2024-05-11T23:04:07.375 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2199999999999999e-06 2024-05-11T23:04:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604018+0000", 2024-05-11T23:04:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1770000000000003e-06 2024-05-11T23:04:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.376 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604023+0000", 2024-05-11T23:04:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0089999999999998e-06 2024-05-11T23:04:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.377 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662742+0000", 2024-05-11T23:04:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058719187999999999 2024-05-11T23:04:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662754+0000", 2024-05-11T23:04:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2095e-05 2024-05-11T23:04:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.378 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172775 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:04:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.754782911999996, 2024-05-11T23:04:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058764743000000001, 2024-05-11T23:04:07.379 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.380 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172775 2024-05-11T23:04:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:04:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:04:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.382 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604030+0000", 2024-05-11T23:04:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6699999999999999e-07 2024-05-11T23:04:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604032+0000", 2024-05-11T23:04:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9530000000000002e-06 2024-05-11T23:04:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.383 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604037+0000", 2024-05-11T23:04:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8590000000000001e-06 2024-05-11T23:04:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604042+0000", 2024-05-11T23:04:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8609999999999997e-06 2024-05-11T23:04:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662782+0000", 2024-05-11T23:04:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058739731000000003 2024-05-11T23:04:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662794+0000", 2024-05-11T23:04:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1972e-05 2024-05-11T23:04:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172776 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:04:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.754763621999999, 2024-05-11T23:04:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058778696999999998, 2024-05-11T23:04:07.385 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172776 2024-05-11T23:04:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:04:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:04:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:04:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2000000000000001e-07 2024-05-11T23:04:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604051+0000", 2024-05-11T23:04:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2879999999999999e-06 2024-05-11T23:04:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604057+0000", 2024-05-11T23:04:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:04:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604062+0000", 2024-05-11T23:04:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6800000000000001e-06 2024-05-11T23:04:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662820+0000", 2024-05-11T23:04:07.388 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058758444999999999 2024-05-11T23:04:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662827+0000", 2024-05-11T23:04:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1930000000000004e-06 2024-05-11T23:04:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172777 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:04:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.754744316, 2024-05-11T23:04:07.389 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058799918999999999, 2024-05-11T23:04:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172777 2024-05-11T23:04:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.390 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:04:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:04:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.391 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:04:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:04:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604075+0000", 2024-05-11T23:04:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.109e-06 2024-05-11T23:04:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604083+0000", 2024-05-11T23:04:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9170000000000006e-06 2024-05-11T23:04:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.392 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604089+0000", 2024-05-11T23:04:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3399999999999997e-06 2024-05-11T23:04:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662859+0000", 2024-05-11T23:04:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058770172000000002 2024-05-11T23:04:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662868+0000", 2024-05-11T23:04:07.393 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0289999999999994e-06 2024-05-11T23:04:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172778 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:04:07.394 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.754716225999999, 2024-05-11T23:04:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058845435000000001, 2024-05-11T23:04:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172778 2024-05-11T23:04:07.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:04:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.396 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:04:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:04:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5400000000000002e-07 2024-05-11T23:04:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.397 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604099+0000", 2024-05-11T23:04:07.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4650000000000001e-06 2024-05-11T23:04:07.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.398 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.398 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.398 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604105+0000", 2024-05-11T23:04:07.398 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.314e-06 2024-05-11T23:04:07.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604110+0000", 2024-05-11T23:04:07.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6310000000000002e-06 2024-05-11T23:04:07.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.399 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.399 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.399 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662931+0000", 2024-05-11T23:04:07.399 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058820960999999998 2024-05-11T23:04:07.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.400 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.400 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.400 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662941+0000", 2024-05-11T23:04:07.400 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.061e-05 2024-05-11T23:04:07.400 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.400 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.400 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172779 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:04:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.754696065999994, 2024-05-11T23:04:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058859163999999999, 2024-05-11T23:04:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172779 2024-05-11T23:04:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:04:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:04:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:04:07.405 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:04:07.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.405 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.405 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.405 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604119+0000", 2024-05-11T23:04:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2529999999999999e-06 2024-05-11T23:04:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604125+0000", 2024-05-11T23:04:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1670000000000004e-06 2024-05-11T23:04:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.406 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604130+0000", 2024-05-11T23:04:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.702e-06 2024-05-11T23:04:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662967+0000", 2024-05-11T23:04:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058837792999999999 2024-05-11T23:04:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.407 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662975+0000", 2024-05-11T23:04:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8490000000000006e-06 2024-05-11T23:04:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172780 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:04:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.754675915999997, 2024-05-11T23:04:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058872696000000002, 2024-05-11T23:04:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.408 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172780 2024-05-11T23:04:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:04:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:04:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.409 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604137+0000", 2024-05-11T23:04:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4400000000000001e-07 2024-05-11T23:04:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604139+0000", 2024-05-11T23:04:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3429999999999998e-06 2024-05-11T23:04:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604145+0000", 2024-05-11T23:04:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1940000000000003e-06 2024-05-11T23:04:07.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604150+0000", 2024-05-11T23:04:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:04:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663002+0000", 2024-05-11T23:04:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058851829000000001 2024-05-11T23:04:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663009+0000", 2024-05-11T23:04:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.199e-06 2024-05-11T23:04:07.411 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172781 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:04:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.754644076000005, 2024-05-11T23:04:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058875553999999997, 2024-05-11T23:04:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172781 2024-05-11T23:04:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:04:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:04:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.413 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:04:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:04:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604171+0000", 2024-05-11T23:04:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.21e-06 2024-05-11T23:04:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604176+0000", 2024-05-11T23:04:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2970000000000003e-06 2024-05-11T23:04:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.414 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604179+0000", 2024-05-11T23:04:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8459999999999999e-06 2024-05-11T23:04:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663033+0000", 2024-05-11T23:04:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058854565999999997 2024-05-11T23:04:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663044+0000", 2024-05-11T23:04:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0229e-05 2024-05-11T23:04:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.415 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172782 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:04:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.754629327999993, 2024-05-11T23:04:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058894375999999998, 2024-05-11T23:04:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172782 2024-05-11T23:04:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.416 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:04:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:04:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:04:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-05-11T23:04:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.417 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604184+0000", 2024-05-11T23:04:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:04:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604187+0000", 2024-05-11T23:04:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.044e-06 2024-05-11T23:04:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604190+0000", 2024-05-11T23:04:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6460000000000002e-06 2024-05-11T23:04:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.418 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663070+0000", 2024-05-11T23:04:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058879510000000003 2024-05-11T23:04:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663077+0000", 2024-05-11T23:04:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6669999999999996e-06 2024-05-11T23:04:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.419 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172783 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:04:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.754619008999995, 2024-05-11T23:04:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058913946000000002, 2024-05-11T23:04:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172783 2024-05-11T23:04:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.420 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:04:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:04:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:04:07.421 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-05-11T23:04:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604195+0000", 2024-05-11T23:04:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.19e-06 2024-05-11T23:04:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604198+0000", 2024-05-11T23:04:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:04:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604200+0000", 2024-05-11T23:04:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4719999999999998e-06 2024-05-11T23:04:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663100+0000", 2024-05-11T23:04:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058900300000000003 2024-05-11T23:04:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.423 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663107+0000", 2024-05-11T23:04:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7449999999999998e-06 2024-05-11T23:04:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172784 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:04:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.754608989999994, 2024-05-11T23:04:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933728999999997, 2024-05-11T23:04:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.424 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172784 2024-05-11T23:04:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:04:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:04:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.425 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:04:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.03e-07 2024-05-11T23:04:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604205+0000", 2024-05-11T23:04:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2300000000000001e-06 2024-05-11T23:04:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604207+0000", 2024-05-11T23:04:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8729999999999999e-06 2024-05-11T23:04:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.426 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604210+0000", 2024-05-11T23:04:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.858e-06 2024-05-11T23:04:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663130+0000", 2024-05-11T23:04:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058919314 2024-05-11T23:04:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663137+0000", 2024-05-11T23:04:07.427 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2509999999999997e-06 2024-05-11T23:04:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172785 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:04:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.754596719999995, 2024-05-11T23:04:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058951664000000001, 2024-05-11T23:04:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.428 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172785 2024-05-11T23:04:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:04:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:04:07.429 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604216+0000", 2024-05-11T23:04:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:04:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604218+0000", 2024-05-11T23:04:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7549999999999999e-06 2024-05-11T23:04:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604222+0000", 2024-05-11T23:04:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6700000000000002e-06 2024-05-11T23:04:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604227+0000", 2024-05-11T23:04:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4399999999999998e-06 2024-05-11T23:04:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663160+0000", 2024-05-11T23:04:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933119999999999 2024-05-11T23:04:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.431 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663167+0000", 2024-05-11T23:04:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1840000000000002e-06 2024-05-11T23:04:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172786 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:04:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.754579665999998, 2024-05-11T23:04:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058966991000000003, 2024-05-11T23:04:07.432 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172786 2024-05-11T23:04:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:04:07.433 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:04:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604233+0000", 2024-05-11T23:04:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6899999999999999e-07 2024-05-11T23:04:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.434 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604235+0000", 2024-05-11T23:04:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8369999999999999e-06 2024-05-11T23:04:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604240+0000", 2024-05-11T23:04:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0329999999999999e-06 2024-05-11T23:04:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604244+0000", 2024-05-11T23:04:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9859999999999996e-06 2024-05-11T23:04:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.435 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663192+0000", 2024-05-11T23:04:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058948852000000003 2024-05-11T23:04:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663199+0000", 2024-05-11T23:04:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0140000000000001e-06 2024-05-11T23:04:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172787 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:07.436 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:04:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.754563320000003, 2024-05-11T23:04:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058982018999999997, 2024-05-11T23:04:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172787 2024-05-11T23:04:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:04:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:04:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.438 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605916+0000", 2024-05-11T23:04:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00166752 2024-05-11T23:04:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605921+0000", 2024-05-11T23:04:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0669999999999999e-06 2024-05-11T23:04:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605934+0000", 2024-05-11T23:04:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2271e-05 2024-05-11T23:04:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.439 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605940+0000", 2024-05-11T23:04:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6880000000000002e-06 2024-05-11T23:04:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663224+0000", 2024-05-11T23:04:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057283251 2024-05-11T23:04:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663231+0000", 2024-05-11T23:04:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2219999999999996e-06 2024-05-11T23:04:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.440 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172793 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:04:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.752796473000004, 2024-05-11T23:04:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057402373999999999, 2024-05-11T23:04:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172793 2024-05-11T23:04:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.441 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:04:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:04:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:04:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3300000000000001e-07 2024-05-11T23:04:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606017+0000", 2024-05-11T23:04:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4920000000000001e-06 2024-05-11T23:04:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606020+0000", 2024-05-11T23:04:07.443 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:04:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606023+0000", 2024-05-11T23:04:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6680000000000001e-06 2024-05-11T23:04:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663411+0000", 2024-05-11T23:04:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057388206999999997 2024-05-11T23:04:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663418+0000", 2024-05-11T23:04:07.444 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.815e-06 2024-05-11T23:04:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172795 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:04:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.752755046000004, 2024-05-11T23:04:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057423015000000001, 2024-05-11T23:04:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.445 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172795 2024-05-11T23:04:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:04:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:04:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:04:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9499999999999998e-07 2024-05-11T23:04:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606059+0000", 2024-05-11T23:04:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-05-11T23:04:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606062+0000", 2024-05-11T23:04:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.977e-06 2024-05-11T23:04:07.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606064+0000", 2024-05-11T23:04:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7410000000000001e-06 2024-05-11T23:04:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663473+0000", 2024-05-11T23:04:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057408624999999998 2024-05-11T23:04:07.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663480+0000", 2024-05-11T23:04:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1049999999999997e-06 2024-05-11T23:04:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172796 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:04:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.752744483000001, 2024-05-11T23:04:07.449 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057449235000000001, 2024-05-11T23:04:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172796 2024-05-11T23:04:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:04:07.450 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:04:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:04:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:04:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606069+0000", 2024-05-11T23:04:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:04:07.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606072+0000", 2024-05-11T23:04:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8700000000000001e-06 2024-05-11T23:04:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606074+0000", 2024-05-11T23:04:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6230000000000001e-06 2024-05-11T23:04:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663510+0000", 2024-05-11T23:04:07.452 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05743529 2024-05-11T23:04:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663517+0000", 2024-05-11T23:04:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1230000000000002e-06 2024-05-11T23:04:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.453 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172797 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:04:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 68.752734450999995, 2024-05-11T23:04:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057470420000000001, 2024-05-11T23:04:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.454 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172797 2024-05-11T23:04:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:04:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:04:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607915+0000", 2024-05-11T23:04:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018375609999999999 2024-05-11T23:04:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607920+0000", 2024-05-11T23:04:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8899999999999998e-06 2024-05-11T23:04:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607930+0000", 2024-05-11T23:04:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4429999999999992e-06 2024-05-11T23:04:07.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607936+0000", 2024-05-11T23:04:07.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7089999999999996e-06 2024-05-11T23:04:07.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663541+0000", 2024-05-11T23:04:07.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055604537000000002 2024-05-11T23:04:07.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663548+0000", 2024-05-11T23:04:07.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2799999999999998e-06 2024-05-11T23:04:07.458 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.458 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.458 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.458 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.458 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.458 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:07.460 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:04:07.460 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:07.459+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:07.560 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:04:07.560+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:04:07.610 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:07.610 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:04:07.610 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:04:07.611 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:07.619 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:04:07.661 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:04:07.660+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:04:07.729 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:07.729 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:04:07.729 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:04:07.729 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:04:07.729 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:07.738 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:04:07.762 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:07.761+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:07.849 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:07.849 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:04:07.849 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:04:07.849 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:04:07.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.849 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:77128 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_get in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-11T23:04:07.850 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:04:07.850 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 112.832467647, 2024-05-11T23:04:07.850 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91737855899999998, 2024-05-11T23:04:07.850 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.850 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.850 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.850 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.850 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.850 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.850 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 77128 2024-05-11T23:04:07.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.850 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:04:07.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:04:07.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.851 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.851 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016003+0000", 2024-05-11T23:04:07.851 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5379999999999999e-06 2024-05-11T23:04:07.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016011+0000", 2024-05-11T23:04:07.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6410000000000002e-06 2024-05-11T23:04:07.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016026+0000", 2024-05-11T23:04:07.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4779e-05 2024-05-11T23:04:07.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.852 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.852 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016032+0000", 2024-05-11T23:04:07.852 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.066e-06 2024-05-11T23:04:07.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016093+0000", 2024-05-11T23:04:07.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0878000000000002e-05 2024-05-11T23:04:07.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:04:07.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016100+0000", 2024-05-11T23:04:07.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6799999999999993e-06 2024-05-11T23:04:07.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.853 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.853 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933302+0000", 2024-05-11T23:04:07.853 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91720135300000005 2024-05-11T23:04:07.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.854 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.854 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933380+0000", 2024-05-11T23:04:07.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8623999999999998e-05 2024-05-11T23:04:07.854 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.854 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.854 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.854 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98950 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:04:07.854 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:04:07.854 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.716386758, 2024-05-11T23:04:07.854 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021419438999999998, 2024-05-11T23:04:07.854 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.854 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:07.855 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.855 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.855 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.855 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98950 2024-05-11T23:04:07.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.855 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:04:07.855 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.855 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.855 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:04:07.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:04:07.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.34e-07 2024-05-11T23:04:07.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.856 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.856 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132085+0000", 2024-05-11T23:04:07.856 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0049999999999999e-06 2024-05-11T23:04:07.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132093+0000", 2024-05-11T23:04:07.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6660000000000008e-06 2024-05-11T23:04:07.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.857 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132097+0000", 2024-05-11T23:04:07.857 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2590000000000002e-06 2024-05-11T23:04:07.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.857 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135114+0000", 2024-05-11T23:04:07.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003017573 2024-05-11T23:04:07.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:07.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153416+0000", 2024-05-11T23:04:07.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018301884000000001 2024-05-11T23:04:07.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.858 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.858 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153502+0000", 2024-05-11T23:04:07.858 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5717999999999995e-05 2024-05-11T23:04:07.858 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.858 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.859 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.859 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98959 3.1e 3:7807da99:::rbd_data.111ed2934718.000000000000006d:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:04:07.859 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:04:07.859 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.715894021, 2024-05-11T23:04:07.859 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020525928999999998, 2024-05-11T23:04:07.859 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.859 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:07.859 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.859 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.859 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.859 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98959 2024-05-11T23:04:07.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.860 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:04:07.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:04:07.860 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.860 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.860 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132576+0000", 2024-05-11T23:04:07.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4199999999999995e-07 2024-05-11T23:04:07.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132580+0000", 2024-05-11T23:04:07.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2910000000000001e-06 2024-05-11T23:04:07.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132588+0000", 2024-05-11T23:04:07.861 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7970000000000001e-06 2024-05-11T23:04:07.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.861 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.861 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132602+0000", 2024-05-11T23:04:07.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3743999999999999e-05 2024-05-11T23:04:07.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132655+0000", 2024-05-11T23:04:07.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2714999999999999e-05 2024-05-11T23:04:07.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:07.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132787+0000", 2024-05-11T23:04:07.862 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013273899999999999 2024-05-11T23:04:07.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.862 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.862 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153101+0000", 2024-05-11T23:04:07.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020313900999999999 2024-05-11T23:04:07.863 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.863 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.863 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.863 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98965 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:04:07.863 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:04:07.863 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.713370385, 2024-05-11T23:04:07.863 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018608983999999999, 2024-05-11T23:04:07.863 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.863 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.864 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:07.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.864 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98965 2024-05-11T23:04:07.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.864 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:04:07.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.864 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.864 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:04:07.864 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135100+0000", 2024-05-11T23:04:07.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.17e-06 2024-05-11T23:04:07.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135105+0000", 2024-05-11T23:04:07.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.899e-06 2024-05-11T23:04:07.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.865 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.865 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135122+0000", 2024-05-11T23:04:07.865 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7346999999999998e-05 2024-05-11T23:04:07.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135130+0000", 2024-05-11T23:04:07.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0639999999999994e-06 2024-05-11T23:04:07.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153533+0000", 2024-05-11T23:04:07.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018402753000000001 2024-05-11T23:04:07.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.866 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.866 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:07.866 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153615+0000", 2024-05-11T23:04:07.866 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1720999999999999e-05 2024-05-11T23:04:07.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.867 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.867 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153708+0000", 2024-05-11T23:04:07.867 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3029999999999995e-05 2024-05-11T23:04:07.867 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.867 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.867 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.867 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.867 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98966 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3760128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:04:07.867 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:04:07.867 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.713298259, 2024-05-11T23:04:07.868 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018716535999999999, 2024-05-11T23:04:07.868 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.868 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.868 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:07.868 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.868 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.868 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.868 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98966 2024-05-11T23:04:07.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.868 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.868 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:04:07.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:04:07.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.869 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.869 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135172+0000", 2024-05-11T23:04:07.869 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-05-11T23:04:07.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.869 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135175+0000", 2024-05-11T23:04:07.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-05-11T23:04:07.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135184+0000", 2024-05-11T23:04:07.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1840000000000005e-06 2024-05-11T23:04:07.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.870 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.870 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152903+0000", 2024-05-11T23:04:07.870 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017718357000000001 2024-05-11T23:04:07.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.870 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153736+0000", 2024-05-11T23:04:07.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00083324799999999997 2024-05-11T23:04:07.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:07.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153809+0000", 2024-05-11T23:04:07.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2995000000000002e-05 2024-05-11T23:04:07.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.871 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.871 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.871 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153888+0000", 2024-05-11T23:04:07.871 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8695999999999993e-05 2024-05-11T23:04:07.871 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.872 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.872 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.872 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "pg_update_log_missing_reply(3.1e epoch 72/53 rep_tid 104 lcod 72'363)", 2024-05-11T23:04:07.872 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:04:07.872 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 101.713235363, 2024-05-11T23:04:07.872 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017871951000000001, 2024-05-11T23:04:07.872 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.872 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.872 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:07.872 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.872 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:04:07.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:04:07.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.873 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.873 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135235+0000", 2024-05-11T23:04:07.873 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8899999999999999e-07 2024-05-11T23:04:07.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135238+0000", 2024-05-11T23:04:07.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.27e-06 2024-05-11T23:04:07.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.874 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152878+0000", 2024-05-11T23:04:07.874 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01763986 2024-05-11T23:04:07.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.874 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152896+0000", 2024-05-11T23:04:07.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8573e-05 2024-05-11T23:04:07.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152968+0000", 2024-05-11T23:04:07.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1780999999999996e-05 2024-05-11T23:04:07.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.875 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.875 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.875 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153106+0000", 2024-05-11T23:04:07.875 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013777799999999999 2024-05-11T23:04:07.875 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.876 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.876 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.876 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.876 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:239544 3.1a 3:5fe59cad:::rbd_data.111ebabb4604.000000000000016e:head [stat,delete] snapc 2f=[2f] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:04:07.876 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:04:07.876 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.193878631000004, 2024-05-11T23:04:07.876 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.50514007500000002, 2024-05-11T23:04:07.876 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.876 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.876 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:07.876 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.876 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.876 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.876 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 239544 2024-05-11T23:04:07.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.877 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.877 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.877 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.877 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:04:07.877 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:04:07.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.878 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.878 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.878 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654592+0000", 2024-05-11T23:04:07.878 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.404e-06 2024-05-11T23:04:07.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654598+0000", 2024-05-11T23:04:07.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7660000000000004e-06 2024-05-11T23:04:07.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654611+0000", 2024-05-11T23:04:07.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2676e-05 2024-05-11T23:04:07.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.879 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.879 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654620+0000", 2024-05-11T23:04:07.879 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2990000000000002e-06 2024-05-11T23:04:07.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654698+0000", 2024-05-11T23:04:07.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7739999999999998e-05 2024-05-11T23:04:07.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.880 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:04:07.880 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654710+0000", 2024-05-11T23:04:07.880 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2648e-05 2024-05-11T23:04:07.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.157889+0000", 2024-05-11T23:04:07.881 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.50317850799999997 2024-05-11T23:04:07.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.881 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.881 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:07.881 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.158050+0000", 2024-05-11T23:04:07.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016084500000000001 2024-05-11T23:04:07.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.882 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.882 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.159731+0000", 2024-05-11T23:04:07.882 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001681189 2024-05-11T23:04:07.882 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.882 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.882 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.882 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262364 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2613248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:07.883 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:04:07.883 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.262005766000001, 2024-05-11T23:04:07.883 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016312508, 2024-05-11T23:04:07.883 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.883 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.883 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:07.883 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.883 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.883 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.883 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262364 2024-05-11T23:04:07.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.883 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:04:07.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:04:07.884 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.884 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.884 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.884 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:04:07.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.13e-07 2024-05-11T23:04:07.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.885 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.885 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586467+0000", 2024-05-11T23:04:07.885 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-05-11T23:04:07.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586474+0000", 2024-05-11T23:04:07.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7739999999999999e-06 2024-05-11T23:04:07.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.886 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586483+0000", 2024-05-11T23:04:07.886 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0340000000000002e-06 2024-05-11T23:04:07.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.886 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602629+0000", 2024-05-11T23:04:07.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016145845999999998 2024-05-11T23:04:07.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:07.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602703+0000", 2024-05-11T23:04:07.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4671000000000004e-05 2024-05-11T23:04:07.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.887 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.887 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.887 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602776+0000", 2024-05-11T23:04:07.887 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2793000000000006e-05 2024-05-11T23:04:07.887 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.887 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.888 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.888 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.888 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262417 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:07.888 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:04:07.888 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.260259466999997, 2024-05-11T23:04:07.888 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016403456, 2024-05-11T23:04:07.888 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.888 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.888 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:07.888 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.888 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.888 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.888 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262417 2024-05-11T23:04:07.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.889 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:04:07.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:04:07.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.889 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.889 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:04:07.889 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6199999999999999e-07 2024-05-11T23:04:07.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588212+0000", 2024-05-11T23:04:07.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.075e-06 2024-05-11T23:04:07.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.890 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588218+0000", 2024-05-11T23:04:07.890 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8150000000000002e-06 2024-05-11T23:04:07.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.890 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.890 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588223+0000", 2024-05-11T23:04:07.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7090000000000001e-06 2024-05-11T23:04:07.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604461+0000", 2024-05-11T23:04:07.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016237900999999999 2024-05-11T23:04:07.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:07.891 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604540+0000", 2024-05-11T23:04:07.891 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9376999999999996e-05 2024-05-11T23:04:07.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.891 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.891 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.892 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604613+0000", 2024-05-11T23:04:07.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3217000000000001e-05 2024-05-11T23:04:07.892 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.892 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.892 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.892 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.892 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262418 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2396160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:07.892 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:04:07.892 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.260239558000002, 2024-05-11T23:04:07.892 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016591180000000001, 2024-05-11T23:04:07.892 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.892 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.892 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:07.892 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.893 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.893 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.893 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262418 2024-05-11T23:04:07.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.893 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.893 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.893 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.893 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:04:07.893 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:04:07.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:04:07.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6100000000000001e-07 2024-05-11T23:04:07.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.894 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.894 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588232+0000", 2024-05-11T23:04:07.894 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0370000000000001e-06 2024-05-11T23:04:07.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588238+0000", 2024-05-11T23:04:07.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9340000000000003e-06 2024-05-11T23:04:07.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588243+0000", 2024-05-11T23:04:07.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6179999999999997e-06 2024-05-11T23:04:07.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.895 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.895 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604642+0000", 2024-05-11T23:04:07.895 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0163991 2024-05-11T23:04:07.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:07.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604712+0000", 2024-05-11T23:04:07.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9935e-05 2024-05-11T23:04:07.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.896 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.896 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604821+0000", 2024-05-11T23:04:07.896 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000109095 2024-05-11T23:04:07.896 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.896 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.896 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.897 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262419 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2433024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:07.897 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:04:07.897 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.260220240000002, 2024-05-11T23:04:07.897 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016799692000000001, 2024-05-11T23:04:07.897 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.897 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.897 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:07.897 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.897 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.897 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262419 2024-05-11T23:04:07.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.897 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.897 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.897 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:04:07.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:04:07.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.898 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:04:07.898 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:04:07.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.898 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588252+0000", 2024-05-11T23:04:07.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0990000000000001e-06 2024-05-11T23:04:07.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588257+0000", 2024-05-11T23:04:07.899 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8159999999999999e-06 2024-05-11T23:04:07.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.899 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.899 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.899 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588262+0000", 2024-05-11T23:04:07.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8069999999999999e-06 2024-05-11T23:04:07.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604868+0000", 2024-05-11T23:04:07.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016605880999999999 2024-05-11T23:04:07.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.900 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:07.900 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604966+0000", 2024-05-11T23:04:07.900 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7597999999999996e-05 2024-05-11T23:04:07.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.901 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.901 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605049+0000", 2024-05-11T23:04:07.901 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3141999999999998e-05 2024-05-11T23:04:07.901 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.901 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.901 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.901 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.901 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262420 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2469888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:07.901 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:04:07.901 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.260200128999998, 2024-05-11T23:04:07.902 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016972514000000001, 2024-05-11T23:04:07.902 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.902 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.902 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:07.902 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.902 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.902 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.902 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262420 2024-05-11T23:04:07.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.902 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.902 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:04:07.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:04:07.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.903 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.903 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.903 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588271+0000", 2024-05-11T23:04:07.903 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6199999999999999e-06 2024-05-11T23:04:07.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588273+0000", 2024-05-11T23:04:07.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3209999999999999e-06 2024-05-11T23:04:07.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.904 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.904 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588279+0000", 2024-05-11T23:04:07.904 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2519999999999996e-06 2024-05-11T23:04:07.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588285+0000", 2024-05-11T23:04:07.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1440000000000002e-06 2024-05-11T23:04:07.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605083+0000", 2024-05-11T23:04:07.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016798418999999998 2024-05-11T23:04:07.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.905 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:07.905 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605168+0000", 2024-05-11T23:04:07.905 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4851000000000005e-05 2024-05-11T23:04:07.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.905 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.906 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.906 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605242+0000", 2024-05-11T23:04:07.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3906999999999996e-05 2024-05-11T23:04:07.906 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.906 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.906 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.906 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262421 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2506752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:07.906 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:04:07.906 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.260177982999998, 2024-05-11T23:04:07.906 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017141427000000001, 2024-05-11T23:04:07.906 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.906 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.907 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:07.907 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.907 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.907 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262421 2024-05-11T23:04:07.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.907 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:04:07.907 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.907 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.907 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:04:07.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588292+0000", 2024-05-11T23:04:07.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:04:07.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.908 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588318+0000", 2024-05-11T23:04:07.908 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6376e-05 2024-05-11T23:04:07.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.908 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588326+0000", 2024-05-11T23:04:07.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6699999999999994e-06 2024-05-11T23:04:07.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588332+0000", 2024-05-11T23:04:07.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8540000000000002e-06 2024-05-11T23:04:07.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.909 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605274+0000", 2024-05-11T23:04:07.909 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016942024999999999 2024-05-11T23:04:07.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.909 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:07.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605360+0000", 2024-05-11T23:04:07.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6304999999999994e-05 2024-05-11T23:04:07.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.910 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.910 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605433+0000", 2024-05-11T23:04:07.910 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2772000000000001e-05 2024-05-11T23:04:07.910 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.910 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.911 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.911 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262422 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2543616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:07.911 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:04:07.911 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.260128758999997, 2024-05-11T23:04:07.911 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017283705, 2024-05-11T23:04:07.911 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.911 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.911 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:07.911 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.911 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.911 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.911 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262422 2024-05-11T23:04:07.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.912 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:04:07.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.912 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.912 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:04:07.912 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:04:07.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:04:07.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.913 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.913 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588343+0000", 2024-05-11T23:04:07.913 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.34e-06 2024-05-11T23:04:07.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588350+0000", 2024-05-11T23:04:07.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1569999999999997e-06 2024-05-11T23:04:07.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.914 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.914 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588354+0000", 2024-05-11T23:04:07.914 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.865e-06 2024-05-11T23:04:07.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605466+0000", 2024-05-11T23:04:07.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017111498999999999 2024-05-11T23:04:07.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:07.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605551+0000", 2024-05-11T23:04:07.915 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5210999999999994e-05 2024-05-11T23:04:07.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.915 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.915 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605624+0000", 2024-05-11T23:04:07.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3153999999999998e-05 2024-05-11T23:04:07.916 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.916 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.916 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.916 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.916 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262423 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2580480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:07.916 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:04:07.916 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.260108510000002, 2024-05-11T23:04:07.916 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017451194, 2024-05-11T23:04:07.916 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.917 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.917 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:07.917 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.917 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.917 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.917 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262423 2024-05-11T23:04:07.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.917 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.917 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.917 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.917 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:04:07.917 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:04:07.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.918 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:04:07.918 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:04:07.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.918 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.918 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588363+0000", 2024-05-11T23:04:07.919 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.855e-06 2024-05-11T23:04:07.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.919 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.919 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.919 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588369+0000", 2024-05-11T23:04:07.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1260000000000003e-06 2024-05-11T23:04:07.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.920 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.920 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588375+0000", 2024-05-11T23:04:07.920 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9630000000000003e-06 2024-05-11T23:04:07.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605656+0000", 2024-05-11T23:04:07.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017280865999999999 2024-05-11T23:04:07.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.921 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:07.921 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605737+0000", 2024-05-11T23:04:07.921 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0773999999999996e-05 2024-05-11T23:04:07.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.922 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.922 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605812+0000", 2024-05-11T23:04:07.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5203999999999994e-05 2024-05-11T23:04:07.922 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.922 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.922 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.922 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262424 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2617344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:07.922 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:04:07.922 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.260086991999998, 2024-05-11T23:04:07.922 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017617544999999998, 2024-05-11T23:04:07.922 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.922 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.922 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:07.923 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.923 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.923 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.923 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262424 2024-05-11T23:04:07.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.923 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:04:07.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.923 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.923 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:04:07.923 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588383+0000", 2024-05-11T23:04:07.924 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8299999999999997e-07 2024-05-11T23:04:07.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.924 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.924 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.924 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588385+0000", 2024-05-11T23:04:07.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2869999999999999e-06 2024-05-11T23:04:07.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.925 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.925 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.925 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588391+0000", 2024-05-11T23:04:07.925 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1689999999999997e-06 2024-05-11T23:04:07.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588396+0000", 2024-05-11T23:04:07.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5719999999999996e-06 2024-05-11T23:04:07.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.926 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605837+0000", 2024-05-11T23:04:07.926 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017440812999999999 2024-05-11T23:04:07.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.926 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:07.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605917+0000", 2024-05-11T23:04:07.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9924000000000003e-05 2024-05-11T23:04:07.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.927 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.927 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.606000+0000", 2024-05-11T23:04:07.927 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3296999999999999e-05 2024-05-11T23:04:07.927 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.927 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.927 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.927 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.927 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262896 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:07.927 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:04:07.928 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.189082354999996, 2024-05-11T23:04:07.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016295738000000001, 2024-05-11T23:04:07.928 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.928 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.928 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:07.928 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.928 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.928 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.928 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262896 2024-05-11T23:04:07.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.928 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.928 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.928 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:04:07.928 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:04:07.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:04:07.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:04:07.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.929 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.929 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.929 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659390+0000", 2024-05-11T23:04:07.929 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1440000000000001e-06 2024-05-11T23:04:07.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659397+0000", 2024-05-11T23:04:07.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0010000000000004e-06 2024-05-11T23:04:07.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659402+0000", 2024-05-11T23:04:07.930 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2179999999999998e-06 2024-05-11T23:04:07.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.930 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.930 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675487+0000", 2024-05-11T23:04:07.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016085706000000002 2024-05-11T23:04:07.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:07.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675580+0000", 2024-05-11T23:04:07.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2884e-05 2024-05-11T23:04:07.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.931 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.931 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.931 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675683+0000", 2024-05-11T23:04:07.931 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010236 2024-05-11T23:04:07.931 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.931 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.931 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.932 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262897 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:07.932 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:04:07.932 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.189047014000003, 2024-05-11T23:04:07.932 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016500932999999999, 2024-05-11T23:04:07.932 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.932 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.932 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:07.932 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.932 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.932 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.932 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262897 2024-05-11T23:04:07.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.932 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:04:07.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:04:07.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.933 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.933 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659423+0000", 2024-05-11T23:04:07.933 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9699999999999996e-07 2024-05-11T23:04:07.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659425+0000", 2024-05-11T23:04:07.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3199999999999998e-06 2024-05-11T23:04:07.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659433+0000", 2024-05-11T23:04:07.934 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3660000000000003e-06 2024-05-11T23:04:07.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.934 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.934 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.934 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659445+0000", 2024-05-11T23:04:07.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2102e-05 2024-05-11T23:04:07.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675727+0000", 2024-05-11T23:04:07.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016282331000000001 2024-05-11T23:04:07.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.935 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.935 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:07.935 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675815+0000", 2024-05-11T23:04:07.935 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7484999999999993e-05 2024-05-11T23:04:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675923+0000", 2024-05-11T23:04:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000108732 2024-05-11T23:04:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262898 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:07.936 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:04:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.189015896000001, 2024-05-11T23:04:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016685855999999999, 2024-05-11T23:04:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262898 2024-05-11T23:04:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.937 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:04:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:04:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659454+0000", 2024-05-11T23:04:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2200000000000004e-07 2024-05-11T23:04:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.938 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659457+0000", 2024-05-11T23:04:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.779e-06 2024-05-11T23:04:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659463+0000", 2024-05-11T23:04:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6660000000000002e-06 2024-05-11T23:04:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659472+0000", 2024-05-11T23:04:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2139999999999996e-06 2024-05-11T23:04:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.939 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675956+0000", 2024-05-11T23:04:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016484431000000001 2024-05-11T23:04:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676039+0000", 2024-05-11T23:04:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3358999999999999e-05 2024-05-11T23:04:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676139+0000", 2024-05-11T23:04:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9784999999999994e-05 2024-05-11T23:04:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262899 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:04:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 55.188990064999999, 2024-05-11T23:04:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017030087999999999, 2024-05-11T23:04:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262899 2024-05-11T23:04:07.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:04:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:04:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:07.942 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659480+0000", 2024-05-11T23:04:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7700000000000004e-07 2024-05-11T23:04:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659482+0000", 2024-05-11T23:04:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1390000000000002e-06 2024-05-11T23:04:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659489+0000", 2024-05-11T23:04:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8279999999999997e-06 2024-05-11T23:04:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:07.943 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659494+0000", 2024-05-11T23:04:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9019999999999998e-06 2024-05-11T23:04:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676167+0000", 2024-05-11T23:04:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016673621 2024-05-11T23:04:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676329+0000", 2024-05-11T23:04:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016206099999999999 2024-05-11T23:04:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676509+0000", 2024-05-11T23:04:07.944 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017996 2024-05-11T23:04:07.945 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.945 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.945 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.945 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:07.945 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:07.945 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:07.945 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 5, 7, 3, 6, 4] out_osds: [0] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-05-11T23:04:07.945 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:04:07.946 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-05-11T23:04:07.946 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-11T23:04:07.946 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-11T23:04:07.946 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-05-11T23:04:07.947 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-11T23:04:07.948 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:07.946+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:07.949 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:04:08.047 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:08.047+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:08.064 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:08.065 INFO:teuthology.orchestra.run.smithi115.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-11T23:04:08.065 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:08.065 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:08.066 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:08.066 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:04:08.066 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:08.075 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:04:08.148 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:08.147+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:08.183 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:08.183 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:08.183 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:04:08.183 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:04:08.183 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:08.191 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:04:08.276 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:08.276+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:08.338 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:08.338 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:04:08.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:04:08.338 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:04:08.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116446 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8273920~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:04:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.961750706999993, 2024-05-11T23:04:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045535299000000001, 2024-05-11T23:04:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116446 2024-05-11T23:04:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:04:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:04:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375547+0000", 2024-05-11T23:04:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4000000000000001e-07 2024-05-11T23:04:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375549+0000", 2024-05-11T23:04:08.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.368e-06 2024-05-11T23:04:08.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375556+0000", 2024-05-11T23:04:08.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1840000000000004e-06 2024-05-11T23:04:08.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375560+0000", 2024-05-11T23:04:08.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.741e-06 2024-05-11T23:04:08.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420896+0000", 2024-05-11T23:04:08.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045335435 2024-05-11T23:04:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420987+0000", 2024-05-11T23:04:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1033999999999996e-05 2024-05-11T23:04:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421082+0000", 2024-05-11T23:04:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4896999999999996e-05 2024-05-11T23:04:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116447 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6316032~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:04:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.961729196999997, 2024-05-11T23:04:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045745830000000001, 2024-05-11T23:04:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116447 2024-05-11T23:04:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:04:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:04:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:04:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.58e-07 2024-05-11T23:04:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375570+0000", 2024-05-11T23:04:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:04:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375576+0000", 2024-05-11T23:04:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7939999999999999e-06 2024-05-11T23:04:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375581+0000", 2024-05-11T23:04:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9280000000000001e-06 2024-05-11T23:04:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421127+0000", 2024-05-11T23:04:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045545785999999998 2024-05-11T23:04:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421214+0000", 2024-05-11T23:04:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7039000000000005e-05 2024-05-11T23:04:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421314+0000", 2024-05-11T23:04:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9772999999999995e-05 2024-05-11T23:04:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.346 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116448 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8314880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:04:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.961710123000003, 2024-05-11T23:04:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045941753000000002, 2024-05-11T23:04:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116448 2024-05-11T23:04:08.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:04:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:04:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:04:08.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.77e-07 2024-05-11T23:04:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375589+0000", 2024-05-11T23:04:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.091e-06 2024-05-11T23:04:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375596+0000", 2024-05-11T23:04:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9179999999999997e-06 2024-05-11T23:04:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375601+0000", 2024-05-11T23:04:08.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0030000000000002e-06 2024-05-11T23:04:08.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421363+0000", 2024-05-11T23:04:08.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045761352999999998 2024-05-11T23:04:08.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421440+0000", 2024-05-11T23:04:08.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7585999999999999e-05 2024-05-11T23:04:08.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421529+0000", 2024-05-11T23:04:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8424999999999994e-05 2024-05-11T23:04:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116449 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6438912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:04:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.961689688000007, 2024-05-11T23:04:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046082208999999999, 2024-05-11T23:04:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116449 2024-05-11T23:04:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:04:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:04:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377225+0000", 2024-05-11T23:04:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016177030000000001 2024-05-11T23:04:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377229+0000", 2024-05-11T23:04:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8e-06 2024-05-11T23:04:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377238+0000", 2024-05-11T23:04:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2229999999999996e-06 2024-05-11T23:04:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377244+0000", 2024-05-11T23:04:08.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4060000000000002e-06 2024-05-11T23:04:08.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421520+0000", 2024-05-11T23:04:08.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044275222000000003 2024-05-11T23:04:08.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421595+0000", 2024-05-11T23:04:08.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4838000000000004e-05 2024-05-11T23:04:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421690+0000", 2024-05-11T23:04:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5017000000000002e-05 2024-05-11T23:04:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116458 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7094272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:04:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.959875471000004, 2024-05-11T23:04:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045706877999999999, 2024-05-11T23:04:08.355 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116458 2024-05-11T23:04:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:04:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:04:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:04:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9400000000000005e-07 2024-05-11T23:04:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377424+0000", 2024-05-11T23:04:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1859999999999999e-06 2024-05-11T23:04:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377431+0000", 2024-05-11T23:04:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4049999999999997e-06 2024-05-11T23:04:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377436+0000", 2024-05-11T23:04:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1399999999999999e-06 2024-05-11T23:04:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.422945+0000", 2024-05-11T23:04:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045509326000000003 2024-05-11T23:04:08.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423036+0000", 2024-05-11T23:04:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0649999999999994e-05 2024-05-11T23:04:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423128+0000", 2024-05-11T23:04:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2577000000000005e-05 2024-05-11T23:04:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116459 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7258112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:04:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.959854738999994, 2024-05-11T23:04:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045831945999999998, 2024-05-11T23:04:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116459 2024-05-11T23:04:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:04:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:04:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377443+0000", 2024-05-11T23:04:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999999e-07 2024-05-11T23:04:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377445+0000", 2024-05-11T23:04:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0209999999999998e-06 2024-05-11T23:04:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377451+0000", 2024-05-11T23:04:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.071e-06 2024-05-11T23:04:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377456+0000", 2024-05-11T23:04:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8289999999999998e-06 2024-05-11T23:04:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423155+0000", 2024-05-11T23:04:08.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045699447999999997 2024-05-11T23:04:08.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423200+0000", 2024-05-11T23:04:08.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5083000000000002e-05 2024-05-11T23:04:08.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423274+0000", 2024-05-11T23:04:08.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4046000000000002e-05 2024-05-11T23:04:08.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.363 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.364 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.364 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116460 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7299072~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:08.364 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:04:08.364 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.959835677000001, 2024-05-11T23:04:08.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046010987000000003, 2024-05-11T23:04:08.364 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.364 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.364 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.364 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116460 2024-05-11T23:04:08.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.365 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:04:08.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:04:08.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377462+0000", 2024-05-11T23:04:08.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:04:08.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377464+0000", 2024-05-11T23:04:08.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1600000000000001e-06 2024-05-11T23:04:08.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377469+0000", 2024-05-11T23:04:08.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5430000000000001e-06 2024-05-11T23:04:08.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377474+0000", 2024-05-11T23:04:08.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8509999999999998e-06 2024-05-11T23:04:08.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423312+0000", 2024-05-11T23:04:08.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045838080000000003 2024-05-11T23:04:08.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423401+0000", 2024-05-11T23:04:08.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8170000000000005e-05 2024-05-11T23:04:08.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423472+0000", 2024-05-11T23:04:08.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1847000000000005e-05 2024-05-11T23:04:08.368 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.368 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.368 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.368 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116461 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7421952~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:08.368 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:04:08.368 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.959816907000004, 2024-05-11T23:04:08.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046130749999999998, 2024-05-11T23:04:08.368 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.368 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.368 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.369 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.369 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116461 2024-05-11T23:04:08.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.369 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:04:08.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:04:08.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:04:08.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4700000000000002e-07 2024-05-11T23:04:08.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377483+0000", 2024-05-11T23:04:08.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0219999999999999e-06 2024-05-11T23:04:08.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377489+0000", 2024-05-11T23:04:08.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4420000000000003e-06 2024-05-11T23:04:08.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377494+0000", 2024-05-11T23:04:08.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9239999999999998e-06 2024-05-11T23:04:08.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423500+0000", 2024-05-11T23:04:08.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046006054999999997 2024-05-11T23:04:08.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423557+0000", 2024-05-11T23:04:08.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7169000000000001e-05 2024-05-11T23:04:08.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423611+0000", 2024-05-11T23:04:08.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3791e-05 2024-05-11T23:04:08.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.372 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.372 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116462 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7462912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:08.372 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:04:08.372 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.959797288999994, 2024-05-11T23:04:08.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046254666, 2024-05-11T23:04:08.373 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.373 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.373 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.373 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.373 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116462 2024-05-11T23:04:08.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.373 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:04:08.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:04:08.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:04:08.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:04:08.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377502+0000", 2024-05-11T23:04:08.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-05-11T23:04:08.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377508+0000", 2024-05-11T23:04:08.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4500000000000003e-06 2024-05-11T23:04:08.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377512+0000", 2024-05-11T23:04:08.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7700000000000001e-06 2024-05-11T23:04:08.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423649+0000", 2024-05-11T23:04:08.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046136859000000002 2024-05-11T23:04:08.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423704+0000", 2024-05-11T23:04:08.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4938000000000002e-05 2024-05-11T23:04:08.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.376 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.376 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423754+0000", 2024-05-11T23:04:08.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0145999999999997e-05 2024-05-11T23:04:08.376 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.376 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.376 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.377 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116463 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7585792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:08.377 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:04:08.377 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.959778862999997, 2024-05-11T23:04:08.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361763, 2024-05-11T23:04:08.377 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.377 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.377 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.377 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.377 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116463 2024-05-11T23:04:08.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.377 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:04:08.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:04:08.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377519+0000", 2024-05-11T23:04:08.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2600000000000002e-07 2024-05-11T23:04:08.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377521+0000", 2024-05-11T23:04:08.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1710000000000001e-06 2024-05-11T23:04:08.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377527+0000", 2024-05-11T23:04:08.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9649999999999996e-06 2024-05-11T23:04:08.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377532+0000", 2024-05-11T23:04:08.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.78e-06 2024-05-11T23:04:08.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423779+0000", 2024-05-11T23:04:08.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046247706 2024-05-11T23:04:08.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423829+0000", 2024-05-11T23:04:08.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9188999999999999e-05 2024-05-11T23:04:08.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423880+0000", 2024-05-11T23:04:08.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1425999999999999e-05 2024-05-11T23:04:08.381 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.381 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.381 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.381 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116464 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7749632~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:08.381 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:04:08.381 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.959759626999997, 2024-05-11T23:04:08.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046491719000000001, 2024-05-11T23:04:08.381 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.381 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.381 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.382 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116464 2024-05-11T23:04:08.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.382 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:04:08.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:04:08.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377538+0000", 2024-05-11T23:04:08.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4499999999999998e-07 2024-05-11T23:04:08.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377540+0000", 2024-05-11T23:04:08.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2340000000000001e-06 2024-05-11T23:04:08.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377546+0000", 2024-05-11T23:04:08.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8289999999999996e-06 2024-05-11T23:04:08.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377551+0000", 2024-05-11T23:04:08.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8559999999999998e-06 2024-05-11T23:04:08.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423913+0000", 2024-05-11T23:04:08.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361885999999998 2024-05-11T23:04:08.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423958+0000", 2024-05-11T23:04:08.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5513999999999998e-05 2024-05-11T23:04:08.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424029+0000", 2024-05-11T23:04:08.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1055000000000003e-05 2024-05-11T23:04:08.385 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.385 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.385 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.385 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116465 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7626752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:08.385 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:04:08.385 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.959740479000004, 2024-05-11T23:04:08.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046600057, 2024-05-11T23:04:08.386 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.386 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.386 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.386 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.386 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116465 2024-05-11T23:04:08.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.386 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:04:08.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:04:08.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:04:08.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3599999999999999e-07 2024-05-11T23:04:08.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377559+0000", 2024-05-11T23:04:08.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9640000000000002e-06 2024-05-11T23:04:08.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377572+0000", 2024-05-11T23:04:08.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3188e-05 2024-05-11T23:04:08.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377578+0000", 2024-05-11T23:04:08.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.942e-06 2024-05-11T23:04:08.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424058+0000", 2024-05-11T23:04:08.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046480021000000003 2024-05-11T23:04:08.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424103+0000", 2024-05-11T23:04:08.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5021999999999997e-05 2024-05-11T23:04:08.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424157+0000", 2024-05-11T23:04:08.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3483999999999998e-05 2024-05-11T23:04:08.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.390 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.390 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.390 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116466 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7790592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:08.390 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:04:08.390 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.959711897000005, 2024-05-11T23:04:08.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046743741999999998, 2024-05-11T23:04:08.390 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.390 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.390 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.391 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.391 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116466 2024-05-11T23:04:08.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.391 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:04:08.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:04:08.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377586+0000", 2024-05-11T23:04:08.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:04:08.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377588+0000", 2024-05-11T23:04:08.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3039999999999999e-06 2024-05-11T23:04:08.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377594+0000", 2024-05-11T23:04:08.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-05-11T23:04:08.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377600+0000", 2024-05-11T23:04:08.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7699999999999998e-06 2024-05-11T23:04:08.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424185+0000", 2024-05-11T23:04:08.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046585891999999997 2024-05-11T23:04:08.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424235+0000", 2024-05-11T23:04:08.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9356e-05 2024-05-11T23:04:08.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424329+0000", 2024-05-11T23:04:08.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4036999999999994e-05 2024-05-11T23:04:08.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.394 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.394 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.394 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116467 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7954432~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:08.394 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:04:08.394 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.959691609999993, 2024-05-11T23:04:08.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046930134999999998, 2024-05-11T23:04:08.394 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.394 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.394 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.395 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.395 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116467 2024-05-11T23:04:08.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.395 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:04:08.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:04:08.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379161+0000", 2024-05-11T23:04:08.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015555009999999999 2024-05-11T23:04:08.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379166+0000", 2024-05-11T23:04:08.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6689999999999997e-06 2024-05-11T23:04:08.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379176+0000", 2024-05-11T23:04:08.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0549e-05 2024-05-11T23:04:08.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379183+0000", 2024-05-11T23:04:08.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1199999999999996e-06 2024-05-11T23:04:08.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424359+0000", 2024-05-11T23:04:08.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045175949999999999 2024-05-11T23:04:08.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424433+0000", 2024-05-11T23:04:08.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3824000000000003e-05 2024-05-11T23:04:08.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424536+0000", 2024-05-11T23:04:08.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010252200000000001 2024-05-11T23:04:08.398 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.398 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.398 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.399 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116469 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8077312~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:08.399 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:04:08.399 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.958051570999999, 2024-05-11T23:04:08.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045636687000000002, 2024-05-11T23:04:08.399 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.399 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.399 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.399 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.399 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.399 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.399 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116469 2024-05-11T23:04:08.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.399 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:04:08.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:04:08.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379246+0000", 2024-05-11T23:04:08.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.32e-07 2024-05-11T23:04:08.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379254+0000", 2024-05-11T23:04:08.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.807e-06 2024-05-11T23:04:08.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379261+0000", 2024-05-11T23:04:08.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.985e-06 2024-05-11T23:04:08.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.401 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379264+0000", 2024-05-11T23:04:08.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6480000000000001e-06 2024-05-11T23:04:08.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424744+0000", 2024-05-11T23:04:08.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045479638000000003 2024-05-11T23:04:08.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424797+0000", 2024-05-11T23:04:08.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2725000000000001e-05 2024-05-11T23:04:08.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424882+0000", 2024-05-11T23:04:08.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5451999999999994e-05 2024-05-11T23:04:08.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.403 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.403 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.403 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116470 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8118272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:08.403 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:04:08.405 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.958026359000002, 2024-05-11T23:04:08.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045755745, 2024-05-11T23:04:08.405 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.405 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.405 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.405 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.405 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.405 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.406 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116470 2024-05-11T23:04:08.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.406 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:04:08.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.406 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:04:08.406 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.406 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.406 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.406 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:04:08.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3999999999999997e-07 2024-05-11T23:04:08.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379273+0000", 2024-05-11T23:04:08.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-05-11T23:04:08.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.407 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379277+0000", 2024-05-11T23:04:08.407 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4020000000000004e-06 2024-05-11T23:04:08.407 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.407 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.407 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379280+0000", 2024-05-11T23:04:08.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-05-11T23:04:08.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424922+0000", 2024-05-11T23:04:08.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045642598999999999 2024-05-11T23:04:08.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.408 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.408 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424973+0000", 2024-05-11T23:04:08.408 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0674999999999998e-05 2024-05-11T23:04:08.408 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.408 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.409 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.409 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425026+0000", 2024-05-11T23:04:08.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3492000000000002e-05 2024-05-11T23:04:08.409 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.409 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.409 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.409 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.409 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.409 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116471 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8241152~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:08.409 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:04:08.409 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.958013575999999, 2024-05-11T23:04:08.409 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045867374000000002, 2024-05-11T23:04:08.409 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.409 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.409 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.409 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.410 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.410 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.410 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116471 2024-05-11T23:04:08.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.410 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:04:08.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.410 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.410 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.410 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.410 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:04:08.410 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379284+0000", 2024-05-11T23:04:08.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.53e-07 2024-05-11T23:04:08.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379285+0000", 2024-05-11T23:04:08.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.223e-06 2024-05-11T23:04:08.411 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.411 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.411 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.411 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379288+0000", 2024-05-11T23:04:08.411 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5379999999999998e-06 2024-05-11T23:04:08.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379291+0000", 2024-05-11T23:04:08.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6699999999999998e-06 2024-05-11T23:04:08.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425055+0000", 2024-05-11T23:04:08.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045763497 2024-05-11T23:04:08.412 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.412 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.412 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.412 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425099+0000", 2024-05-11T23:04:08.412 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3899000000000001e-05 2024-05-11T23:04:08.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.413 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.413 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425151+0000", 2024-05-11T23:04:08.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2293999999999998e-05 2024-05-11T23:04:08.413 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.413 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.413 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.413 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.413 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.413 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116472 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8282112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:08.413 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:04:08.413 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.958002577000002, 2024-05-11T23:04:08.413 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045973778999999999, 2024-05-11T23:04:08.413 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.414 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.414 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.414 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.414 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.414 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.414 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116472 2024-05-11T23:04:08.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.414 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.414 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:04:08.414 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.414 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.414 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.414 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:04:08.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381226+0000", 2024-05-11T23:04:08.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001931627 2024-05-11T23:04:08.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.415 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381231+0000", 2024-05-11T23:04:08.415 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8679999999999998e-06 2024-05-11T23:04:08.415 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.415 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.415 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381242+0000", 2024-05-11T23:04:08.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0702e-05 2024-05-11T23:04:08.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381249+0000", 2024-05-11T23:04:08.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9800000000000001e-06 2024-05-11T23:04:08.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425175+0000", 2024-05-11T23:04:08.416 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043926805999999999 2024-05-11T23:04:08.416 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.416 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.416 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.416 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425215+0000", 2024-05-11T23:04:08.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9737999999999998e-05 2024-05-11T23:04:08.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.417 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.417 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425268+0000", 2024-05-11T23:04:08.417 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3057999999999999e-05 2024-05-11T23:04:08.417 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.417 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.417 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.417 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.417 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.417 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116482 3.1b 3:db6e8843:::rbd_object_map.111e67f27da3.0000000000000028:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:08.417 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:04:08.417 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 92.956034033999998, 2024-05-11T23:04:08.418 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046526076, 2024-05-11T23:04:08.418 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.418 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.418 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:04:08.418 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.418 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.418 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.418 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116482 2024-05-11T23:04:08.418 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.418 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.418 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.418 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.418 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:04:08.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:04:08.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.419 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381264+0000", 2024-05-11T23:04:08.419 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5300000000000004e-07 2024-05-11T23:04:08.419 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.419 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.419 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381267+0000", 2024-05-11T23:04:08.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1999999999999999e-06 2024-05-11T23:04:08.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381274+0000", 2024-05-11T23:04:08.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7109999999999995e-06 2024-05-11T23:04:08.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.420 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381285+0000", 2024-05-11T23:04:08.420 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0856000000000001e-05 2024-05-11T23:04:08.420 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.420 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.420 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425292+0000", 2024-05-11T23:04:08.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044006376 2024-05-11T23:04:08.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425337+0000", 2024-05-11T23:04:08.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4907999999999998e-05 2024-05-11T23:04:08.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:04:08.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425507+0000", 2024-05-11T23:04:08.421 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017029999999999999 2024-05-11T23:04:08.421 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.421 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.421 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:04:08.421 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427668+0000", 2024-05-11T23:04:08.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021612430000000002 2024-05-11T23:04:08.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:04:08.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427683+0000", 2024-05-11T23:04:08.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5016e-05 2024-05-11T23:04:08.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:04:08.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427742+0000", 2024-05-11T23:04:08.422 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8811999999999999e-05 2024-05-11T23:04:08.422 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.422 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.422 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.422 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427789+0000", 2024-05-11T23:04:08.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7101000000000002e-05 2024-05-11T23:04:08.423 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.423 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.423 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.423 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.423 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.423 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:283279 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-05-11T23:04:08.423 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:53.865673+0000", 2024-05-11T23:04:08.423 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 14.471624018, 2024-05-11T23:04:08.423 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.93944476799999999, 2024-05-11T23:04:08.423 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.423 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.423 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:04:08.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.423 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.424 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.424 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 283279 2024-05-11T23:04:08.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.424 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865673+0000", 2024-05-11T23:04:08.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.424 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.424 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865673+0000", 2024-05-11T23:04:08.424 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.424 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.424 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865674+0000", 2024-05-11T23:04:08.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.012e-06 2024-05-11T23:04:08.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865679+0000", 2024-05-11T23:04:08.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7740000000000004e-06 2024-05-11T23:04:08.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.425 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.425 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865689+0000", 2024-05-11T23:04:08.425 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0644e-05 2024-05-11T23:04:08.425 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.425 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865694+0000", 2024-05-11T23:04:08.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.826e-06 2024-05-11T23:04:08.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865757+0000", 2024-05-11T23:04:08.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2493e-05 2024-05-11T23:04:08.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.426 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:04:08.426 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865768+0000", 2024-05-11T23:04:08.426 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0777e-05 2024-05-11T23:04:08.426 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.426 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.802182+0000", 2024-05-11T23:04:08.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.93641407499999996 2024-05-11T23:04:08.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.802338+0000", 2024-05-11T23:04:08.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015636599999999999 2024-05-11T23:04:08.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.427 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 6", 2024-05-11T23:04:08.427 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.802561+0000", 2024-05-11T23:04:08.427 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000222585 2024-05-11T23:04:08.427 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.427 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:04:08.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.805050+0000", 2024-05-11T23:04:08.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024896300000000001 2024-05-11T23:04:08.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:04:08.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.805059+0000", 2024-05-11T23:04:08.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6829999999999997e-06 2024-05-11T23:04:08.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:04:08.428 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.805093+0000", 2024-05-11T23:04:08.428 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3763000000000002e-05 2024-05-11T23:04:08.428 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.428 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.428 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.429 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.805118+0000", 2024-05-11T23:04:08.429 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.514e-05 2024-05-11T23:04:08.429 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.429 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.429 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.429 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.429 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.429 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:08.430 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:04:08.431 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:08.430+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:08.531 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:08.531+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:08.576 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:08.576 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:08.576 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:04:08.576 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:08.585 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:04:08.632 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:08.632+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:08.731 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:08.731 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:08.731 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:04:08.731 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:04:08.731 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:08.733 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:08.733+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:08.739 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:04:08.834 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:08.834+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:08.847 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:08.847 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:04:08.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:04:08.847 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:04:08.847 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.847 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267820 3.16 3:6b963329:::rbd_data.111ec06c817a.000000000000000a:33 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:08.847 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:04:08.847 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.903626330999998, 2024-05-11T23:04:08.847 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050141218000000001, 2024-05-11T23:04:08.847 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.847 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.848 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.848 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.848 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.848 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.848 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267820 2024-05-11T23:04:08.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.848 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.848 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:04:08.848 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.848 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.848 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.848 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:04:08.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942754+0000", 2024-05-11T23:04:08.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8559000000000003e-05 2024-05-11T23:04:08.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.849 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942756+0000", 2024-05-11T23:04:08.849 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.604e-06 2024-05-11T23:04:08.849 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.849 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.849 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942763+0000", 2024-05-11T23:04:08.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7939999999999997e-06 2024-05-11T23:04:08.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942768+0000", 2024-05-11T23:04:08.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4410000000000001e-06 2024-05-11T23:04:08.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.850 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.850 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992601+0000", 2024-05-11T23:04:08.850 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049832754999999999 2024-05-11T23:04:08.850 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.850 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992687+0000", 2024-05-11T23:04:08.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5605000000000005e-05 2024-05-11T23:04:08.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.851 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.851 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992796+0000", 2024-05-11T23:04:08.851 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010946 2024-05-11T23:04:08.851 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.851 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.851 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.851 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.851 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.851 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267852 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:08.852 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:04:08.852 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.903505127000003, 2024-05-11T23:04:08.852 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050212600000000003, 2024-05-11T23:04:08.852 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.852 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.852 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.852 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.852 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.852 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.852 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267852 2024-05-11T23:04:08.852 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.852 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.852 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.852 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:04:08.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:04:08.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943228+0000", 2024-05-11T23:04:08.853 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00045180600000000002 2024-05-11T23:04:08.853 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.853 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.853 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.853 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943232+0000", 2024-05-11T23:04:08.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8399999999999997e-06 2024-05-11T23:04:08.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943240+0000", 2024-05-11T23:04:08.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8900000000000007e-06 2024-05-11T23:04:08.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.854 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943246+0000", 2024-05-11T23:04:08.854 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6239999999999996e-06 2024-05-11T23:04:08.854 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.854 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.854 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992828+0000", 2024-05-11T23:04:08.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049581247000000002 2024-05-11T23:04:08.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992905+0000", 2024-05-11T23:04:08.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7656000000000003e-05 2024-05-11T23:04:08.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.855 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.855 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.855 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992989+0000", 2024-05-11T23:04:08.855 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3536999999999997e-05 2024-05-11T23:04:08.855 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.855 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.855 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.855 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.856 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267854 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1134592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:08.856 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:04:08.856 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.902991986000004, 2024-05-11T23:04:08.856 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050077425000000002, 2024-05-11T23:04:08.856 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.856 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.856 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.856 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.856 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.856 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.856 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267854 2024-05-11T23:04:08.856 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.856 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.856 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:04:08.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:04:08.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.857 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943290+0000", 2024-05-11T23:04:08.857 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2300000000000002e-07 2024-05-11T23:04:08.857 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.857 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.857 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943291+0000", 2024-05-11T23:04:08.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6190000000000001e-06 2024-05-11T23:04:08.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943296+0000", 2024-05-11T23:04:08.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2889999999999999e-06 2024-05-11T23:04:08.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.858 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.858 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943299+0000", 2024-05-11T23:04:08.858 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1030000000000002e-06 2024-05-11T23:04:08.858 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.858 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993205+0000", 2024-05-11T23:04:08.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049906405000000001 2024-05-11T23:04:08.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993285+0000", 2024-05-11T23:04:08.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9752999999999993e-05 2024-05-11T23:04:08.859 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.859 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.859 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.859 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993367+0000", 2024-05-11T23:04:08.859 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1933000000000003e-05 2024-05-11T23:04:08.859 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.859 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.860 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.860 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.860 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267855 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1171456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:08.860 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:04:08.860 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.902975044000002, 2024-05-11T23:04:08.860 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050250346000000001, 2024-05-11T23:04:08.860 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.860 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.860 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.860 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.860 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.860 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.860 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267855 2024-05-11T23:04:08.860 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.861 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:04:08.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:04:08.861 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.861 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.861 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.861 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.861 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945226+0000", 2024-05-11T23:04:08.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001919235 2024-05-11T23:04:08.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945230+0000", 2024-05-11T23:04:08.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9580000000000001e-06 2024-05-11T23:04:08.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.862 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945238+0000", 2024-05-11T23:04:08.862 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1040000000000007e-06 2024-05-11T23:04:08.862 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.862 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.862 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945244+0000", 2024-05-11T23:04:08.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6919999999999996e-06 2024-05-11T23:04:08.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993399+0000", 2024-05-11T23:04:08.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048154651 2024-05-11T23:04:08.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.863 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.863 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993472+0000", 2024-05-11T23:04:08.863 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2780000000000005e-05 2024-05-11T23:04:08.863 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.863 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.864 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.864 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993557+0000", 2024-05-11T23:04:08.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4926000000000006e-05 2024-05-11T23:04:08.864 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.864 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.864 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.864 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.864 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.864 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267865 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1540096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:08.864 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:04:08.864 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.900823418000002, 2024-05-11T23:04:08.864 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050054409000000001, 2024-05-11T23:04:08.864 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.864 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.864 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.865 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.865 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267865 2024-05-11T23:04:08.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.865 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:04:08.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.865 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.865 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.865 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.865 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:04:08.865 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:04:08.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6600000000000002e-07 2024-05-11T23:04:08.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.866 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945461+0000", 2024-05-11T23:04:08.866 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2670000000000001e-06 2024-05-11T23:04:08.866 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.866 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.866 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945466+0000", 2024-05-11T23:04:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7200000000000003e-06 2024-05-11T23:04:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945471+0000", 2024-05-11T23:04:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9439999999999996e-06 2024-05-11T23:04:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995337+0000", 2024-05-11T23:04:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049865415000000003 2024-05-11T23:04:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.867 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995413+0000", 2024-05-11T23:04:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5989999999999996e-05 2024-05-11T23:04:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995512+0000", 2024-05-11T23:04:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9706999999999999e-05 2024-05-11T23:04:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268018 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 872448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:08.868 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:04:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.898847353000001, 2024-05-11T23:04:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050170927999999997, 2024-05-11T23:04:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268018 2024-05-11T23:04:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:04:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.869 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:04:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947435+0000", 2024-05-11T23:04:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3099999999999997e-07 2024-05-11T23:04:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947437+0000", 2024-05-11T23:04:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6340000000000001e-06 2024-05-11T23:04:08.870 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947443+0000", 2024-05-11T23:04:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7810000000000002e-06 2024-05-11T23:04:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947448+0000", 2024-05-11T23:04:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9679999999999997e-06 2024-05-11T23:04:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997418+0000", 2024-05-11T23:04:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049969587000000003 2024-05-11T23:04:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.871 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997504+0000", 2024-05-11T23:04:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6426000000000002e-05 2024-05-11T23:04:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997605+0000", 2024-05-11T23:04:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100901 2024-05-11T23:04:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268019 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:04:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.898826794999998, 2024-05-11T23:04:08.872 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050285074999999999, 2024-05-11T23:04:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268019 2024-05-11T23:04:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:04:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.873 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:04:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:04:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0999999999999999e-07 2024-05-11T23:04:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947457+0000", 2024-05-11T23:04:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-05-11T23:04:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.874 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947462+0000", 2024-05-11T23:04:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2410000000000003e-06 2024-05-11T23:04:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947467+0000", 2024-05-11T23:04:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8339999999999998e-06 2024-05-11T23:04:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997639+0000", 2024-05-11T23:04:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050171697000000001 2024-05-11T23:04:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.875 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997685+0000", 2024-05-11T23:04:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6146000000000002e-05 2024-05-11T23:04:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997740+0000", 2024-05-11T23:04:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4737000000000001e-05 2024-05-11T23:04:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268020 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:04:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.898808549999998, 2024-05-11T23:04:08.876 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050504089000000002, 2024-05-11T23:04:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268020 2024-05-11T23:04:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:04:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.877 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:04:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947591+0000", 2024-05-11T23:04:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011805 2024-05-11T23:04:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947594+0000", 2024-05-11T23:04:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1549999999999999e-06 2024-05-11T23:04:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.878 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947602+0000", 2024-05-11T23:04:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.174e-06 2024-05-11T23:04:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947608+0000", 2024-05-11T23:04:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.263e-06 2024-05-11T23:04:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997775+0000", 2024-05-11T23:04:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050166980999999999 2024-05-11T23:04:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.879 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997875+0000", 2024-05-11T23:04:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9844000000000002e-05 2024-05-11T23:04:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997977+0000", 2024-05-11T23:04:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101622 2024-05-11T23:04:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268021 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:04:08.880 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.898663089999999, 2024-05-11T23:04:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050498874999999999, 2024-05-11T23:04:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268021 2024-05-11T23:04:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:04:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.881 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:04:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947695+0000", 2024-05-11T23:04:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6247000000000004e-05 2024-05-11T23:04:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947698+0000", 2024-05-11T23:04:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9199999999999997e-06 2024-05-11T23:04:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.882 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947706+0000", 2024-05-11T23:04:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:04:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947711+0000", 2024-05-11T23:04:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6720000000000001e-06 2024-05-11T23:04:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998017+0000", 2024-05-11T23:04:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050305470999999997 2024-05-11T23:04:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.883 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998060+0000", 2024-05-11T23:04:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3374000000000002e-05 2024-05-11T23:04:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998117+0000", 2024-05-11T23:04:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6971000000000001e-05 2024-05-11T23:04:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268022 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:04:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.898560054000001, 2024-05-11T23:04:08.884 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050731361000000003, 2024-05-11T23:04:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268022 2024-05-11T23:04:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:04:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.885 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:04:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947722+0000", 2024-05-11T23:04:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8999999999999996e-07 2024-05-11T23:04:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947724+0000", 2024-05-11T23:04:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1639999999999999e-06 2024-05-11T23:04:08.886 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947730+0000", 2024-05-11T23:04:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6099999999999997e-06 2024-05-11T23:04:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947735+0000", 2024-05-11T23:04:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7189999999999998e-06 2024-05-11T23:04:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998140+0000", 2024-05-11T23:04:08.887 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050404254000000003 2024-05-11T23:04:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998239+0000", 2024-05-11T23:04:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9192999999999997e-05 2024-05-11T23:04:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998453+0000", 2024-05-11T23:04:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021383099999999999 2024-05-11T23:04:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.888 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268023 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1069056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:04:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.898537789000002, 2024-05-11T23:04:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050874509999999998, 2024-05-11T23:04:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268023 2024-05-11T23:04:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:04:08.889 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:04:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948934+0000", 2024-05-11T23:04:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011903339999999999 2024-05-11T23:04:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.890 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948938+0000", 2024-05-11T23:04:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1330000000000001e-06 2024-05-11T23:04:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948949+0000", 2024-05-11T23:04:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0872e-05 2024-05-11T23:04:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948956+0000", 2024-05-11T23:04:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.131e-06 2024-05-11T23:04:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998494+0000", 2024-05-11T23:04:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049537678000000002 2024-05-11T23:04:08.891 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998551+0000", 2024-05-11T23:04:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7256000000000002e-05 2024-05-11T23:04:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998618+0000", 2024-05-11T23:04:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7106000000000004e-05 2024-05-11T23:04:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268026 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1179648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:08.892 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:04:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.897260736, 2024-05-11T23:04:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050107977999999997, 2024-05-11T23:04:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268026 2024-05-11T23:04:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:04:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:04:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949029+0000", 2024-05-11T23:04:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2220000000000002e-06 2024-05-11T23:04:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949041+0000", 2024-05-11T23:04:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.241e-05 2024-05-11T23:04:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949049+0000", 2024-05-11T23:04:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.072e-06 2024-05-11T23:04:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949056+0000", 2024-05-11T23:04:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1380000000000001e-06 2024-05-11T23:04:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999016+0000", 2024-05-11T23:04:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049959212000000003 2024-05-11T23:04:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999067+0000", 2024-05-11T23:04:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1598000000000003e-05 2024-05-11T23:04:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999129+0000", 2024-05-11T23:04:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1326000000000002e-05 2024-05-11T23:04:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268027 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1216512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:04:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.897216585999999, 2024-05-11T23:04:08.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050189865, 2024-05-11T23:04:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268027 2024-05-11T23:04:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:04:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:04:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:04:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-05-11T23:04:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949068+0000", 2024-05-11T23:04:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.26e-06 2024-05-11T23:04:08.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949074+0000", 2024-05-11T23:04:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7050000000000002e-06 2024-05-11T23:04:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949080+0000", 2024-05-11T23:04:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3689999999999998e-06 2024-05-11T23:04:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999152+0000", 2024-05-11T23:04:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050072653000000002 2024-05-11T23:04:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999197+0000", 2024-05-11T23:04:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4972999999999998e-05 2024-05-11T23:04:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999255+0000", 2024-05-11T23:04:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7425e-05 2024-05-11T23:04:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268028 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1253376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:04:08.900 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.897194925999997, 2024-05-11T23:04:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050280017000000003, 2024-05-11T23:04:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268028 2024-05-11T23:04:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:04:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:04:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949087+0000", 2024-05-11T23:04:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:04:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949089+0000", 2024-05-11T23:04:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8950000000000001e-06 2024-05-11T23:04:08.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949095+0000", 2024-05-11T23:04:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7540000000000003e-06 2024-05-11T23:04:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949114+0000", 2024-05-11T23:04:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9738e-05 2024-05-11T23:04:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999278+0000", 2024-05-11T23:04:08.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050163368999999999 2024-05-11T23:04:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999319+0000", 2024-05-11T23:04:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0979000000000002e-05 2024-05-11T23:04:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999366+0000", 2024-05-11T23:04:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7879e-05 2024-05-11T23:04:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.904 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268029 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1290240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:04:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.897154817999997, 2024-05-11T23:04:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050344904000000003, 2024-05-11T23:04:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268029 2024-05-11T23:04:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:04:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:04:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:04:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:04:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949130+0000", 2024-05-11T23:04:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-05-11T23:04:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949139+0000", 2024-05-11T23:04:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9800000000000004e-06 2024-05-11T23:04:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949145+0000", 2024-05-11T23:04:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0510000000000002e-06 2024-05-11T23:04:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999387+0000", 2024-05-11T23:04:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050242167999999997 2024-05-11T23:04:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999427+0000", 2024-05-11T23:04:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9566e-05 2024-05-11T23:04:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999471+0000", 2024-05-11T23:04:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4914999999999999e-05 2024-05-11T23:04:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.908 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268030 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1327104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:04:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.897129534000001, 2024-05-11T23:04:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050447394, 2024-05-11T23:04:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268030 2024-05-11T23:04:08.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:04:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:04:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:04:08.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0500000000000004e-07 2024-05-11T23:04:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949155+0000", 2024-05-11T23:04:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3819999999999998e-06 2024-05-11T23:04:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949162+0000", 2024-05-11T23:04:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9609999999999999e-06 2024-05-11T23:04:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949188+0000", 2024-05-11T23:04:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5877e-05 2024-05-11T23:04:08.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999494+0000", 2024-05-11T23:04:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050306265000000003 2024-05-11T23:04:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999534+0000", 2024-05-11T23:04:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0447000000000001e-05 2024-05-11T23:04:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999599+0000", 2024-05-11T23:04:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4956999999999995e-05 2024-05-11T23:04:08.912 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268031 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1363968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:04:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.897084434, 2024-05-11T23:04:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050511525000000002, 2024-05-11T23:04:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.913 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268031 2024-05-11T23:04:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:04:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:04:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949198+0000", 2024-05-11T23:04:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-05-11T23:04:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949200+0000", 2024-05-11T23:04:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-05-11T23:04:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949208+0000", 2024-05-11T23:04:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2840000000000001e-06 2024-05-11T23:04:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949214+0000", 2024-05-11T23:04:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9869999999999996e-06 2024-05-11T23:04:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999619+0000", 2024-05-11T23:04:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050405278999999997 2024-05-11T23:04:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999660+0000", 2024-05-11T23:04:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1427999999999997e-05 2024-05-11T23:04:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999708+0000", 2024-05-11T23:04:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8229999999999997e-05 2024-05-11T23:04:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268032 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1400832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:04:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.897054814000001, 2024-05-11T23:04:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050611863999999999, 2024-05-11T23:04:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.917 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268032 2024-05-11T23:04:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:04:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:04:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949230+0000", 2024-05-11T23:04:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.371e-06 2024-05-11T23:04:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949233+0000", 2024-05-11T23:04:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6599999999999999e-06 2024-05-11T23:04:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949241+0000", 2024-05-11T23:04:08.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8939999999999993e-06 2024-05-11T23:04:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949246+0000", 2024-05-11T23:04:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6590000000000004e-06 2024-05-11T23:04:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999733+0000", 2024-05-11T23:04:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050487268000000002 2024-05-11T23:04:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999784+0000", 2024-05-11T23:04:08.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0562000000000001e-05 2024-05-11T23:04:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999838+0000", 2024-05-11T23:04:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000002e-05 2024-05-11T23:04:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268033 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1437696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:04:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 55.897027686999998, 2024-05-11T23:04:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050697157999999999, 2024-05-11T23:04:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.921 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.922 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.922 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.922 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.922 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.922 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268033 2024-05-11T23:04:08.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.922 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:04:08.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:04:08.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950862+0000", 2024-05-11T23:04:08.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001607975 2024-05-11T23:04:08.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950866+0000", 2024-05-11T23:04:08.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7770000000000002e-06 2024-05-11T23:04:08.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950878+0000", 2024-05-11T23:04:08.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1421e-05 2024-05-11T23:04:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950885+0000", 2024-05-11T23:04:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-05-11T23:04:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999859+0000", 2024-05-11T23:04:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048973199000000002 2024-05-11T23:04:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999901+0000", 2024-05-11T23:04:08.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1943000000000001e-05 2024-05-11T23:04:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999951+0000", 2024-05-11T23:04:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0243e-05 2024-05-11T23:04:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:283084 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e147)", 2024-05-11T23:04:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:49.906190+0000", 2024-05-11T23:04:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 18.940091736999999, 2024-05-11T23:04:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.89363789900000001, 2024-05-11T23:04:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:08.925 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 283084 2024-05-11T23:04:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906190+0000", 2024-05-11T23:04:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:08.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906190+0000", 2024-05-11T23:04:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906191+0000", 2024-05-11T23:04:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6029999999999999e-06 2024-05-11T23:04:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906198+0000", 2024-05-11T23:04:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7170000000000002e-06 2024-05-11T23:04:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906214+0000", 2024-05-11T23:04:08.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6290000000000002e-05 2024-05-11T23:04:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906229+0000", 2024-05-11T23:04:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4735999999999999e-05 2024-05-11T23:04:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906295+0000", 2024-05-11T23:04:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5604000000000004e-05 2024-05-11T23:04:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:04:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906305+0000", 2024-05-11T23:04:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0795000000000001e-05 2024-05-11T23:04:08.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:50.799618+0000", 2024-05-11T23:04:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.89331292600000001 2024-05-11T23:04:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:50.799711+0000", 2024-05-11T23:04:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2406999999999998e-05 2024-05-11T23:04:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:50.799828+0000", 2024-05-11T23:04:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116821 2024-05-11T23:04:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.929 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.930 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:08.930 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:08.930 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:08.930 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:04:08.932 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-05-11T23:04:08.932 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:04:08.974 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:08.974+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:09.035 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:09.035+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:09.043 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:09.043 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:09.043 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:04:09.043 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:09.052 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:04:09.102 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:04:09.102 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:04:09.120 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:04:09.120 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:04:09.136 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:04:09.136+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:04:09.157 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:09.157 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:09.157 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:04:09.157 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:04:09.157 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:09.165 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:04:09.238 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:09.238+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:09.312 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:09.312 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:04:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:04:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:04:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:04:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.65805999299999, 2024-05-11T23:04:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:04:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:04:09.313 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:04:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:04:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:09.314 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:04:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:04:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:04:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:04:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:04:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:04:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.315 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:04:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:04:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:04:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:04:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:04:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:04:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:09.316 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:04:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:04:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:04:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.65467880400001, 2024-05-11T23:04:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:04:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:09.317 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:04:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:04:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:04:09.318 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:04:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:04:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:04:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:04:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:04:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:04:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.319 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:04:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:04:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:04:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:04:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:04:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:04:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:04:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:04:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:04:09.321 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.654660097, 2024-05-11T23:04:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:04:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:04:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:04:09.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:04:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:04:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:04:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:04:09.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:04:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:04:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:04:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:04:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:04:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:04:09.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:04:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:04:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:04:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:04:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:04:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:04:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.654641954, 2024-05-11T23:04:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:04:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:04:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.326 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:04:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:04:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:04:09.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:04:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:04:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:04:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:04:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:04:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:04:09.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:04:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:04:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:04:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:04:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:04:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:04:09.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:04:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:04:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.65461446699999, 2024-05-11T23:04:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:04:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:09.330 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:04:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:04:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:04:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:04:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:04:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:04:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:04:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:04:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:04:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:04:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:04:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:04:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:04:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:04:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:04:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:04:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:04:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:04:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.65459393899999, 2024-05-11T23:04:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:04:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:09.334 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:04:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:04:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:04:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:04:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:04:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:04:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:04:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:04:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:04:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:04:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:04:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:04:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:04:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:04:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:04:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:04:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:04:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:04:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.654574164, 2024-05-11T23:04:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:04:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:09.338 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:04:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:04:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:04:09.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:04:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:04:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:04:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:04:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:04:09.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:04:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:04:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:04:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:04:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:04:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:04:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:04:09.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:04:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:04:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:04:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.65455535800001, 2024-05-11T23:04:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:04:09.342 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:04:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:04:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:04:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:04:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:04:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:04:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:04:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:09.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:04:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:04:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:04:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:04:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:04:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:04:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:09.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:04:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:04:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:04:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:04:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:04:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.65453643500001, 2024-05-11T23:04:09.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:04:09.347 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:09.347 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:09.347 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:09.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:09.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:09.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:09.347 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:04:09.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.347 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:09.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:09.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:04:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:04:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:04:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:04:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:09.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:04:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:04:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:04:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:04:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:04:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:04:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:09.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:04:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:04:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:04:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:04:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:04:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:04:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:04:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.65451777300001, 2024-05-11T23:04:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:04:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:04:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.351 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:04:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:04:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:04:09.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:04:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:04:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:04:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:04:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:04:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:09.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:04:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:04:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:04:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:04:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:04:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:04:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:09.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:04:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:04:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:04:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.65417261100001, 2024-05-11T23:04:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:04:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:09.355 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:04:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:04:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:04:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:04:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:04:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:04:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:04:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:04:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:04:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:04:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:04:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:04:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:04:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:04:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:04:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:04:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:04:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:04:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.654147061, 2024-05-11T23:04:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:04:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:09.359 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:04:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:04:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:04:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:04:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:04:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:04:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:04:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:04:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:04:09.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:04:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:04:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:04:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:04:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:04:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:04:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:04:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:04:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:04:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.65406480300001, 2024-05-11T23:04:09.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:04:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:04:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:04:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:04:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:04:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:04:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:04:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:04:09.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:09.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:04:09.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:04:09.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:09.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:04:09.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:04:09.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:09.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:04:09.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:04:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:04:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:04:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:04:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:04:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:04:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.65404235599999, 2024-05-11T23:04:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:04:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:04:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:04:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:04:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:04:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:04:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:04:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:04:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:04:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:04:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:04:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:04:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:04:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:04:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:04:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:04:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:04:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:04:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.371 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:04:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.65402289799999, 2024-05-11T23:04:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:04:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:04:09.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:04:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:04:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:09.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:04:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:04:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:04:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:04:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:04:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:04:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:09.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:04:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:04:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:04:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:04:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:04:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:04:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:09.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:04:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:04:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:04:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.654002838, 2024-05-11T23:04:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:04:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:09.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:04:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:04:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:04:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:04:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:04:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:04:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:04:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:04:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:04:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:04:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:04:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:04:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:04:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:04:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:04:09.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:09.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:04:09.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:04:09.380 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.380 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.380 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.380 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:09.380 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:04:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.653983568, 2024-05-11T23:04:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:04:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:04:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:04:09.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:04:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:04:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:04:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:09.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:04:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:04:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:04:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:04:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:04:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:04:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:09.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:04:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:04:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:04:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:04:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:04:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:04:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:04:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.653437502, 2024-05-11T23:04:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:04:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:04:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:09.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:04:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:04:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:04:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:04:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:04:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:04:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:04:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:04:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:04:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:04:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:04:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:04:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:04:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:04:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:04:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:04:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:04:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.65340571799999, 2024-05-11T23:04:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:04:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:04:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:09.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:04:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:04:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:04:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:04:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:04:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:04:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:04:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:04:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:04:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:04:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:04:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:04:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:04:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:04:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:04:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:04:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.392 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:04:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:04:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 127.216160254, 2024-05-11T23:04:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:04:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:04:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:04:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.393 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:04:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:04:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:04:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:04:09.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:04:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:04:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:04:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:04:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:04:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:04:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:04:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:04:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:04:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:04:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:04:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:04:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:04:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:04:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:04:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:04:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:04:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:04:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:04:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:04:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:04:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:04:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:09.397 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:04:09.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:04:09.398 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.398 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.398 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.398 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:09.398 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:09.398 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:09.399 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:04:09.399 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:04:09.400 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-05-11T23:04:09.400 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:04:09.400 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:04:09.401 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:04:09.401 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:04:09.401 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:04:09.402 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:04:09.402 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:04:09.402 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:09.399+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:09.411 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:04:09.412 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:04:09.415 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-11T23:04:09.415 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-11T23:04:09.500 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:09.500+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:09.508 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:09.508 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:09.517 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:04:09.601 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:09.601+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:09.661 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:09.661 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:09.670 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:04:09.711 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:09.711+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:09.803 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:09.803+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:09.815 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:09.815 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:09.824 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:04:09.903 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:09.903+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:09.945 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:09.945 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:09.955 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:04:10.004 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:10.004+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:10.105 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:10.105+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:10.106 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:10.106 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:10.115 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:04:10.206 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:10.206+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:10.265 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:10.266 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:10.275 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:04:10.307 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:10.307+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:10.408 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:10.408+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:10.426 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:10.426 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:10.435 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:04:10.508 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:04:10.509+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:04:10.584 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:10.584 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:10.594 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:04:10.609 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:10.609+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:10.704 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:10.704 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:10.710 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:10.710+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:10.712 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:04:10.811 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:10.811+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:10.823 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:10.823 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:10.832 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:04:10.916 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:10.916+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:10.933 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-11T23:04:10.992 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:10.992 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:11.002 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:04:11.013 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:11.012+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:11.114 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:11.113+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:11.115 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:11.115 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:11.125 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:04:11.215 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:11.215+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:11.231 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:11.231 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:11.240 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:04:11.324 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:04:11.324+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:04:11.386 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:11.386 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:11.395 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:04:11.416 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:04:11.418 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:11.418+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:11.518 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:11.518+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:11.541 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:11.541 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:11.543 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:04:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:04:11.549 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:04:11.561 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:04:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:04:11.577 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:04:11.578 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:04:11.578 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-05-11T23:04:11.595 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:04:11.595 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:04:11.612 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:04:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:04:11.631 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:04:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:04:11.633 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:11.633+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:11.647 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:04:11.647 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:04:11.650 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-11T23:04:11.651 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-11T23:04:11.695 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:11.695 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:11.695 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:04:11.695 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:11.703 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:04:11.720 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:11.720+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:11.821 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:11.821+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:11.848 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:11.848 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:11.849 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:04:11.849 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:04:11.849 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:11.857 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:04:11.898 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-05-11T23:04:11.941 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:11.941+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:12.021 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:12.021 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:04:12.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:04:12.021 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:04:12.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.021 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267820 3.16 3:6b963329:::rbd_data.111ec06c817a.000000000000000a:33 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:12.022 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:04:12.022 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.077714266999998, 2024-05-11T23:04:12.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050141218000000001, 2024-05-11T23:04:12.022 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.022 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.022 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.022 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.022 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.022 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.022 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267820 2024-05-11T23:04:12.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.022 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:04:12.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:04:12.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942754+0000", 2024-05-11T23:04:12.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8559000000000003e-05 2024-05-11T23:04:12.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942756+0000", 2024-05-11T23:04:12.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.604e-06 2024-05-11T23:04:12.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942763+0000", 2024-05-11T23:04:12.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7939999999999997e-06 2024-05-11T23:04:12.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942768+0000", 2024-05-11T23:04:12.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4410000000000001e-06 2024-05-11T23:04:12.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992601+0000", 2024-05-11T23:04:12.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049832754999999999 2024-05-11T23:04:12.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992687+0000", 2024-05-11T23:04:12.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5605000000000005e-05 2024-05-11T23:04:12.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.025 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.025 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992796+0000", 2024-05-11T23:04:12.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010946 2024-05-11T23:04:12.025 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.025 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.025 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.026 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267852 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:12.026 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:04:12.026 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.077593063000002, 2024-05-11T23:04:12.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050212600000000003, 2024-05-11T23:04:12.026 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.026 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.026 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.026 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267852 2024-05-11T23:04:12.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.026 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:04:12.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:04:12.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943228+0000", 2024-05-11T23:04:12.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00045180600000000002 2024-05-11T23:04:12.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943232+0000", 2024-05-11T23:04:12.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8399999999999997e-06 2024-05-11T23:04:12.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943240+0000", 2024-05-11T23:04:12.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8900000000000007e-06 2024-05-11T23:04:12.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943246+0000", 2024-05-11T23:04:12.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6239999999999996e-06 2024-05-11T23:04:12.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992828+0000", 2024-05-11T23:04:12.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049581247000000002 2024-05-11T23:04:12.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992905+0000", 2024-05-11T23:04:12.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7656000000000003e-05 2024-05-11T23:04:12.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992989+0000", 2024-05-11T23:04:12.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3536999999999997e-05 2024-05-11T23:04:12.030 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.030 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.030 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.030 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267854 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1134592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:12.031 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:04:12.031 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.077079922000003, 2024-05-11T23:04:12.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050077425000000002, 2024-05-11T23:04:12.031 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.031 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.031 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.031 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.031 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267854 2024-05-11T23:04:12.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.031 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:04:12.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:04:12.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943290+0000", 2024-05-11T23:04:12.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2300000000000002e-07 2024-05-11T23:04:12.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943291+0000", 2024-05-11T23:04:12.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6190000000000001e-06 2024-05-11T23:04:12.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943296+0000", 2024-05-11T23:04:12.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2889999999999999e-06 2024-05-11T23:04:12.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943299+0000", 2024-05-11T23:04:12.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1030000000000002e-06 2024-05-11T23:04:12.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993205+0000", 2024-05-11T23:04:12.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049906405000000001 2024-05-11T23:04:12.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993285+0000", 2024-05-11T23:04:12.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9752999999999993e-05 2024-05-11T23:04:12.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.034 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.034 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993367+0000", 2024-05-11T23:04:12.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1933000000000003e-05 2024-05-11T23:04:12.034 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.034 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.034 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.034 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267855 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1171456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:12.035 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:04:12.035 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.077062980000001, 2024-05-11T23:04:12.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050250346000000001, 2024-05-11T23:04:12.035 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.035 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.035 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.035 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.035 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267855 2024-05-11T23:04:12.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.035 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:04:12.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:04:12.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945226+0000", 2024-05-11T23:04:12.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001919235 2024-05-11T23:04:12.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945230+0000", 2024-05-11T23:04:12.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9580000000000001e-06 2024-05-11T23:04:12.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945238+0000", 2024-05-11T23:04:12.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1040000000000007e-06 2024-05-11T23:04:12.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945244+0000", 2024-05-11T23:04:12.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6919999999999996e-06 2024-05-11T23:04:12.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993399+0000", 2024-05-11T23:04:12.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048154651 2024-05-11T23:04:12.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993472+0000", 2024-05-11T23:04:12.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2780000000000005e-05 2024-05-11T23:04:12.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993557+0000", 2024-05-11T23:04:12.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4926000000000006e-05 2024-05-11T23:04:12.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.038 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.038 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.039 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267865 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1540096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:12.039 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:04:12.039 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.074911354000001, 2024-05-11T23:04:12.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050054409000000001, 2024-05-11T23:04:12.039 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.039 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.039 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.039 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.039 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267865 2024-05-11T23:04:12.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.039 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:04:12.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:04:12.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.040 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:04:12.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6600000000000002e-07 2024-05-11T23:04:12.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.040 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945461+0000", 2024-05-11T23:04:12.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2670000000000001e-06 2024-05-11T23:04:12.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945466+0000", 2024-05-11T23:04:12.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7200000000000003e-06 2024-05-11T23:04:12.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945471+0000", 2024-05-11T23:04:12.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9439999999999996e-06 2024-05-11T23:04:12.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995337+0000", 2024-05-11T23:04:12.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049865415000000003 2024-05-11T23:04:12.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995413+0000", 2024-05-11T23:04:12.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5989999999999996e-05 2024-05-11T23:04:12.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995512+0000", 2024-05-11T23:04:12.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9706999999999999e-05 2024-05-11T23:04:12.043 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.043 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.043 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.043 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268018 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 872448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:12.043 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:04:12.043 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.072935289, 2024-05-11T23:04:12.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050170927999999997, 2024-05-11T23:04:12.043 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.043 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.043 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.043 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.044 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268018 2024-05-11T23:04:12.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.044 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:04:12.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:04:12.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947435+0000", 2024-05-11T23:04:12.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3099999999999997e-07 2024-05-11T23:04:12.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947437+0000", 2024-05-11T23:04:12.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6340000000000001e-06 2024-05-11T23:04:12.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947443+0000", 2024-05-11T23:04:12.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7810000000000002e-06 2024-05-11T23:04:12.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947448+0000", 2024-05-11T23:04:12.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9679999999999997e-06 2024-05-11T23:04:12.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997418+0000", 2024-05-11T23:04:12.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049969587000000003 2024-05-11T23:04:12.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997504+0000", 2024-05-11T23:04:12.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6426000000000002e-05 2024-05-11T23:04:12.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.046 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.046 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997605+0000", 2024-05-11T23:04:12.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100901 2024-05-11T23:04:12.047 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.047 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.047 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.047 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268019 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:12.047 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:04:12.047 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.072914730999997, 2024-05-11T23:04:12.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050285074999999999, 2024-05-11T23:04:12.047 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.047 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.047 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.047 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.047 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.048 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.048 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268019 2024-05-11T23:04:12.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.048 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:04:12.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:04:12.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:04:12.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0999999999999999e-07 2024-05-11T23:04:12.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947457+0000", 2024-05-11T23:04:12.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-05-11T23:04:12.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947462+0000", 2024-05-11T23:04:12.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2410000000000003e-06 2024-05-11T23:04:12.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947467+0000", 2024-05-11T23:04:12.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8339999999999998e-06 2024-05-11T23:04:12.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997639+0000", 2024-05-11T23:04:12.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050171697000000001 2024-05-11T23:04:12.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997685+0000", 2024-05-11T23:04:12.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6146000000000002e-05 2024-05-11T23:04:12.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997740+0000", 2024-05-11T23:04:12.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4737000000000001e-05 2024-05-11T23:04:12.051 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.051 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.051 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.051 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268020 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:12.051 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:04:12.051 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.072896485999998, 2024-05-11T23:04:12.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050504089000000002, 2024-05-11T23:04:12.051 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.051 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.051 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.051 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.052 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.052 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.052 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268020 2024-05-11T23:04:12.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.052 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:04:12.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.052 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.052 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.052 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:04:12.052 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.052 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947591+0000", 2024-05-11T23:04:12.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011805 2024-05-11T23:04:12.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947594+0000", 2024-05-11T23:04:12.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1549999999999999e-06 2024-05-11T23:04:12.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.053 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.053 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947602+0000", 2024-05-11T23:04:12.053 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.174e-06 2024-05-11T23:04:12.053 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.053 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947608+0000", 2024-05-11T23:04:12.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.263e-06 2024-05-11T23:04:12.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997775+0000", 2024-05-11T23:04:12.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050166980999999999 2024-05-11T23:04:12.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.054 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.054 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997875+0000", 2024-05-11T23:04:12.054 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9844000000000002e-05 2024-05-11T23:04:12.054 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.054 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.055 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.055 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997977+0000", 2024-05-11T23:04:12.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101622 2024-05-11T23:04:12.055 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.055 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.055 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.055 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.055 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.055 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268021 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:12.055 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:04:12.055 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.072751025999999, 2024-05-11T23:04:12.055 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050498874999999999, 2024-05-11T23:04:12.055 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.055 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.055 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.055 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.056 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.056 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.056 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268021 2024-05-11T23:04:12.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.056 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:04:12.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.056 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.056 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.056 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:04:12.056 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.056 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947695+0000", 2024-05-11T23:04:12.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6247000000000004e-05 2024-05-11T23:04:12.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947698+0000", 2024-05-11T23:04:12.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9199999999999997e-06 2024-05-11T23:04:12.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.057 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.057 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.057 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947706+0000", 2024-05-11T23:04:12.057 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:04:12.057 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947711+0000", 2024-05-11T23:04:12.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6720000000000001e-06 2024-05-11T23:04:12.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998017+0000", 2024-05-11T23:04:12.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050305470999999997 2024-05-11T23:04:12.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.058 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.058 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.058 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998060+0000", 2024-05-11T23:04:12.058 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3374000000000002e-05 2024-05-11T23:04:12.058 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.059 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.059 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998117+0000", 2024-05-11T23:04:12.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6971000000000001e-05 2024-05-11T23:04:12.059 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.059 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.059 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.059 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.059 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.059 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268022 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:12.059 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:04:12.059 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.07264799, 2024-05-11T23:04:12.059 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050731361000000003, 2024-05-11T23:04:12.059 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.059 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.059 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.060 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.060 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.060 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.060 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268022 2024-05-11T23:04:12.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.060 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:04:12.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.060 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.060 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.060 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:04:12.060 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.060 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947722+0000", 2024-05-11T23:04:12.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8999999999999996e-07 2024-05-11T23:04:12.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947724+0000", 2024-05-11T23:04:12.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1639999999999999e-06 2024-05-11T23:04:12.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.061 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.061 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947730+0000", 2024-05-11T23:04:12.061 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6099999999999997e-06 2024-05-11T23:04:12.061 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.061 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947735+0000", 2024-05-11T23:04:12.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7189999999999998e-06 2024-05-11T23:04:12.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998140+0000", 2024-05-11T23:04:12.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050404254000000003 2024-05-11T23:04:12.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.062 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.062 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998239+0000", 2024-05-11T23:04:12.062 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9192999999999997e-05 2024-05-11T23:04:12.062 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.062 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.063 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.063 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998453+0000", 2024-05-11T23:04:12.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021383099999999999 2024-05-11T23:04:12.063 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.063 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.063 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.063 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.063 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.063 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268023 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1069056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:12.063 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:04:12.063 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.072625725000002, 2024-05-11T23:04:12.063 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050874509999999998, 2024-05-11T23:04:12.063 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.063 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.063 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.064 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.064 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.064 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.064 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268023 2024-05-11T23:04:12.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.064 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:04:12.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.064 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.064 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:04:12.064 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.064 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.064 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948934+0000", 2024-05-11T23:04:12.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011903339999999999 2024-05-11T23:04:12.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948938+0000", 2024-05-11T23:04:12.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1330000000000001e-06 2024-05-11T23:04:12.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.065 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.065 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948949+0000", 2024-05-11T23:04:12.065 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0872e-05 2024-05-11T23:04:12.065 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.065 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948956+0000", 2024-05-11T23:04:12.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.131e-06 2024-05-11T23:04:12.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998494+0000", 2024-05-11T23:04:12.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049537678000000002 2024-05-11T23:04:12.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.066 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.066 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998551+0000", 2024-05-11T23:04:12.066 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7256000000000002e-05 2024-05-11T23:04:12.066 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.066 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.067 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.067 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998618+0000", 2024-05-11T23:04:12.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7106000000000004e-05 2024-05-11T23:04:12.067 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.067 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.067 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.067 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.067 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.067 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268026 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1179648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:12.067 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:04:12.067 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.071348671999999, 2024-05-11T23:04:12.067 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050107977999999997, 2024-05-11T23:04:12.067 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.067 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.067 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.068 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.068 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.068 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.068 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268026 2024-05-11T23:04:12.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.068 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:04:12.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.068 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.068 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.068 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.068 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:04:12.068 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949029+0000", 2024-05-11T23:04:12.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2220000000000002e-06 2024-05-11T23:04:12.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949041+0000", 2024-05-11T23:04:12.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.241e-05 2024-05-11T23:04:12.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.069 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.069 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.069 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949049+0000", 2024-05-11T23:04:12.069 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.072e-06 2024-05-11T23:04:12.069 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949056+0000", 2024-05-11T23:04:12.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1380000000000001e-06 2024-05-11T23:04:12.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999016+0000", 2024-05-11T23:04:12.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049959212000000003 2024-05-11T23:04:12.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.070 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.070 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.070 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999067+0000", 2024-05-11T23:04:12.070 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1598000000000003e-05 2024-05-11T23:04:12.070 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.071 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.071 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999129+0000", 2024-05-11T23:04:12.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1326000000000002e-05 2024-05-11T23:04:12.071 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.071 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.071 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.071 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.071 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.071 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268027 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1216512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:12.071 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:04:12.071 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.071304521999998, 2024-05-11T23:04:12.071 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050189865, 2024-05-11T23:04:12.071 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.071 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.071 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.072 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.072 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.072 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.072 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268027 2024-05-11T23:04:12.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.072 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:04:12.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.072 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.072 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.072 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.072 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:04:12.072 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:04:12.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-05-11T23:04:12.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949068+0000", 2024-05-11T23:04:12.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.26e-06 2024-05-11T23:04:12.073 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.073 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.073 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.073 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949074+0000", 2024-05-11T23:04:12.073 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7050000000000002e-06 2024-05-11T23:04:12.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949080+0000", 2024-05-11T23:04:12.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3689999999999998e-06 2024-05-11T23:04:12.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999152+0000", 2024-05-11T23:04:12.074 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050072653000000002 2024-05-11T23:04:12.074 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.074 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.074 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.074 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999197+0000", 2024-05-11T23:04:12.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4972999999999998e-05 2024-05-11T23:04:12.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.075 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.075 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999255+0000", 2024-05-11T23:04:12.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7425e-05 2024-05-11T23:04:12.075 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.075 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.075 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.075 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.075 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.075 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268028 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1253376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:12.075 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:04:12.075 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.071282861999997, 2024-05-11T23:04:12.075 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050280017000000003, 2024-05-11T23:04:12.075 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.076 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.076 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.076 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.076 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.076 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.076 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268028 2024-05-11T23:04:12.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.076 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.076 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:04:12.076 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.076 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.076 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.076 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:04:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949087+0000", 2024-05-11T23:04:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:04:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949089+0000", 2024-05-11T23:04:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8950000000000001e-06 2024-05-11T23:04:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.077 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949095+0000", 2024-05-11T23:04:12.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7540000000000003e-06 2024-05-11T23:04:12.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949114+0000", 2024-05-11T23:04:12.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9738e-05 2024-05-11T23:04:12.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.078 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999278+0000", 2024-05-11T23:04:12.078 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050163368999999999 2024-05-11T23:04:12.078 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.078 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.078 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999319+0000", 2024-05-11T23:04:12.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0979000000000002e-05 2024-05-11T23:04:12.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.079 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.079 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999366+0000", 2024-05-11T23:04:12.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7879e-05 2024-05-11T23:04:12.079 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.079 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.079 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.079 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.079 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.079 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268029 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1290240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:12.079 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:04:12.079 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.071242753999996, 2024-05-11T23:04:12.079 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050344904000000003, 2024-05-11T23:04:12.080 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.080 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.080 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.080 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.080 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.080 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.080 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268029 2024-05-11T23:04:12.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.080 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.080 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:04:12.080 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.080 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.080 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.080 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:04:12.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:04:12.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:04:12.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.081 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949130+0000", 2024-05-11T23:04:12.081 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-05-11T23:04:12.081 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.081 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.081 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949139+0000", 2024-05-11T23:04:12.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9800000000000004e-06 2024-05-11T23:04:12.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949145+0000", 2024-05-11T23:04:12.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0510000000000002e-06 2024-05-11T23:04:12.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.082 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999387+0000", 2024-05-11T23:04:12.082 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050242167999999997 2024-05-11T23:04:12.082 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.082 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.082 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999427+0000", 2024-05-11T23:04:12.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9566e-05 2024-05-11T23:04:12.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.083 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.083 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999471+0000", 2024-05-11T23:04:12.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4914999999999999e-05 2024-05-11T23:04:12.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.083 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.083 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.083 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.083 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.083 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268030 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1327104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:12.083 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:04:12.083 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.071217470000001, 2024-05-11T23:04:12.083 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050447394, 2024-05-11T23:04:12.084 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.084 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.084 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.084 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.084 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268030 2024-05-11T23:04:12.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.084 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.084 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.084 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:04:12.084 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.084 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.084 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:04:12.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:04:12.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0500000000000004e-07 2024-05-11T23:04:12.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.085 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.085 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949155+0000", 2024-05-11T23:04:12.085 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3819999999999998e-06 2024-05-11T23:04:12.085 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.085 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949162+0000", 2024-05-11T23:04:12.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9609999999999999e-06 2024-05-11T23:04:12.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949188+0000", 2024-05-11T23:04:12.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5877e-05 2024-05-11T23:04:12.086 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.086 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.086 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.086 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999494+0000", 2024-05-11T23:04:12.086 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050306265000000003 2024-05-11T23:04:12.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999534+0000", 2024-05-11T23:04:12.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0447000000000001e-05 2024-05-11T23:04:12.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.087 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.087 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999599+0000", 2024-05-11T23:04:12.087 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4956999999999995e-05 2024-05-11T23:04:12.087 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.087 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.087 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.087 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.087 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.088 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268031 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1363968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:12.088 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:04:12.088 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.071172369999999, 2024-05-11T23:04:12.088 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050511525000000002, 2024-05-11T23:04:12.088 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.088 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.088 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.088 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.088 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.088 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.088 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268031 2024-05-11T23:04:12.088 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.088 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.088 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.088 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:04:12.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:04:12.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949198+0000", 2024-05-11T23:04:12.089 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-05-11T23:04:12.089 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.089 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.089 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.089 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949200+0000", 2024-05-11T23:04:12.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-05-11T23:04:12.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949208+0000", 2024-05-11T23:04:12.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2840000000000001e-06 2024-05-11T23:04:12.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949214+0000", 2024-05-11T23:04:12.090 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9869999999999996e-06 2024-05-11T23:04:12.090 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.090 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.090 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.090 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999619+0000", 2024-05-11T23:04:12.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050405278999999997 2024-05-11T23:04:12.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999660+0000", 2024-05-11T23:04:12.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1427999999999997e-05 2024-05-11T23:04:12.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.091 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.091 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.091 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999708+0000", 2024-05-11T23:04:12.091 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8229999999999997e-05 2024-05-11T23:04:12.091 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.091 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.091 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.091 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.092 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.093 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268032 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1400832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:12.093 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:04:12.093 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.07114275, 2024-05-11T23:04:12.093 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050611863999999999, 2024-05-11T23:04:12.093 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.093 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.093 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.093 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.093 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.094 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.094 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268032 2024-05-11T23:04:12.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.094 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:04:12.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.094 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.094 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:04:12.094 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.094 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.094 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949230+0000", 2024-05-11T23:04:12.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.371e-06 2024-05-11T23:04:12.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949233+0000", 2024-05-11T23:04:12.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6599999999999999e-06 2024-05-11T23:04:12.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.095 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.095 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949241+0000", 2024-05-11T23:04:12.095 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8939999999999993e-06 2024-05-11T23:04:12.095 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.095 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949246+0000", 2024-05-11T23:04:12.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6590000000000004e-06 2024-05-11T23:04:12.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999733+0000", 2024-05-11T23:04:12.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050487268000000002 2024-05-11T23:04:12.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.096 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999784+0000", 2024-05-11T23:04:12.096 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0562000000000001e-05 2024-05-11T23:04:12.096 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.096 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.096 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.097 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999838+0000", 2024-05-11T23:04:12.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000002e-05 2024-05-11T23:04:12.097 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.097 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.097 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.097 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.097 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.097 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268033 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1437696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:12.097 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:04:12.097 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 59.071115622999997, 2024-05-11T23:04:12.097 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050697157999999999, 2024-05-11T23:04:12.097 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.097 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.097 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.097 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.098 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.098 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.098 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268033 2024-05-11T23:04:12.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.098 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:04:12.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.098 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.098 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.098 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:04:12.098 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.098 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950862+0000", 2024-05-11T23:04:12.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001607975 2024-05-11T23:04:12.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950866+0000", 2024-05-11T23:04:12.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7770000000000002e-06 2024-05-11T23:04:12.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.099 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950878+0000", 2024-05-11T23:04:12.099 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1421e-05 2024-05-11T23:04:12.099 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.099 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.099 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950885+0000", 2024-05-11T23:04:12.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-05-11T23:04:12.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999859+0000", 2024-05-11T23:04:12.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048973199000000002 2024-05-11T23:04:12.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999901+0000", 2024-05-11T23:04:12.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1943000000000001e-05 2024-05-11T23:04:12.100 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.100 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.100 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.100 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999951+0000", 2024-05-11T23:04:12.100 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0243e-05 2024-05-11T23:04:12.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.101 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.101 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.101 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.101 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:283084 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e147)", 2024-05-11T23:04:12.101 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:49.906190+0000", 2024-05-11T23:04:12.101 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 22.114179672999999, 2024-05-11T23:04:12.101 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.89363789900000001, 2024-05-11T23:04:12.101 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.101 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.101 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.101 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.101 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.101 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.101 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 283084 2024-05-11T23:04:12.101 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.102 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906190+0000", 2024-05-11T23:04:12.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906190+0000", 2024-05-11T23:04:12.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.102 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.102 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.102 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.102 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906191+0000", 2024-05-11T23:04:12.102 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6029999999999999e-06 2024-05-11T23:04:12.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906198+0000", 2024-05-11T23:04:12.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7170000000000002e-06 2024-05-11T23:04:12.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906214+0000", 2024-05-11T23:04:12.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6290000000000002e-05 2024-05-11T23:04:12.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.103 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.103 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906229+0000", 2024-05-11T23:04:12.103 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4735999999999999e-05 2024-05-11T23:04:12.103 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.103 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906295+0000", 2024-05-11T23:04:12.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5604000000000004e-05 2024-05-11T23:04:12.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:04:12.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906305+0000", 2024-05-11T23:04:12.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0795000000000001e-05 2024-05-11T23:04:12.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.104 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:50.799618+0000", 2024-05-11T23:04:12.104 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.89331292600000001 2024-05-11T23:04:12.104 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.104 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.104 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:50.799711+0000", 2024-05-11T23:04:12.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2406999999999998e-05 2024-05-11T23:04:12.105 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.105 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.105 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.105 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:50.799828+0000", 2024-05-11T23:04:12.105 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116821 2024-05-11T23:04:12.105 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.105 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.105 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.105 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.105 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.105 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:12.107 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:04:12.107 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:12.106+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:12.207 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:12.207+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:12.253 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:12.253 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:12.253 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:04:12.253 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:12.262 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:04:12.308 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:12.308+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:12.367 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:12.367 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:12.367 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:04:12.367 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:04:12.367 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:12.375 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:04:12.409 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:12.409+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:12.481 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:04:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:04:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:04:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:04:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 139.82720179200001, 2024-05-11T23:04:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:04:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.482 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:04:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:04:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.483 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:04:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:04:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:04:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:04:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:04:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:04:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:04:12.484 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:04:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:04:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:04:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:04:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:04:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:04:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.485 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:04:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:04:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:04:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 139.823820603, 2024-05-11T23:04:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:04:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.486 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:04:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:04:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:04:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.487 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:04:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:04:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:04:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:04:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:04:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:04:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.488 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:04:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:04:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:04:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:04:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:04:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:04:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:04:12.489 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:04:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:04:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 139.82380189599999, 2024-05-11T23:04:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:04:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.490 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:04:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:04:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:04:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.491 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:04:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:04:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:04:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:04:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:04:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:04:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.492 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:04:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:04:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:04:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:04:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:04:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:04:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.493 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:04:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:04:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:04:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 139.82378375299999, 2024-05-11T23:04:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:04:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.494 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:04:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:04:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:04:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.495 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:04:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:04:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:04:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:04:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:04:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:04:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.496 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:04:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:04:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:04:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:04:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:04:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:04:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.497 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:04:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:04:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:04:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 139.823756266, 2024-05-11T23:04:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:04:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.498 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:04:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:04:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:04:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.499 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:04:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:04:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:04:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:04:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:04:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:04:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.500 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:04:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:04:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:04:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:04:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:04:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:04:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:04:12.501 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:04:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:04:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 139.82373573800001, 2024-05-11T23:04:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:04:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.502 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:04:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:04:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:04:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:04:12.503 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:04:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:04:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:04:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:04:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:04:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:04:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:04:12.504 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:04:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:04:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:04:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:04:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:04:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:04:12.505 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:04:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 139.82371596300001, 2024-05-11T23:04:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:04:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.506 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:04:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:04:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:04:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:04:12.507 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:04:12.508 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:04:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:04:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.509 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:04:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:04:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:04:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:04:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:04:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:04:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.510 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:04:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:04:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:04:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:04:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.511 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:04:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 139.823697157, 2024-05-11T23:04:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:04:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.512 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:04:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:04:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.513 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:04:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:04:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:04:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:04:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:04:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.514 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:04:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:04:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:04:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:04:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.515 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:04:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:04:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:04:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:04:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:04:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:04:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.516 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:04:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 139.823678234, 2024-05-11T23:04:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:04:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:04:12.517 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:04:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:04:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.518 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:04:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:04:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:04:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:04:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:04:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:04:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.519 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:04:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:04:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:04:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:04:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:04:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:04:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.520 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:04:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:04:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:04:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 139.823659572, 2024-05-11T23:04:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:04:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.521 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:04:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:04:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.522 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:04:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:04:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:04:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:04:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:04:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.523 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:04:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:04:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:04:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:04:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:04:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:04:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.524 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:04:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:04:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:04:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:04:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:04:12.525 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 139.82331440999999, 2024-05-11T23:04:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:04:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:04:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:04:12.526 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:04:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:04:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:04:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.527 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:04:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:04:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:04:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:04:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:04:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:04:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.528 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:04:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:04:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:04:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:04:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:04:12.529 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:04:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:04:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 139.82328885999999, 2024-05-11T23:04:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:04:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.530 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:04:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:04:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.531 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:04:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:04:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:04:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:04:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:04:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.532 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:04:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:04:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:04:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:04:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:04:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:04:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.533 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:04:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:04:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:04:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:04:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:12.534 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:04:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 139.823206602, 2024-05-11T23:04:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:04:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:04:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:04:12.535 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:04:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:04:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:04:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:04:12.536 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:04:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:04:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:04:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:04:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:04:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.537 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:04:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:04:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:04:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:04:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:04:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:04:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.538 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:04:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 139.82318415500001, 2024-05-11T23:04:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:04:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:04:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.539 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:04:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:04:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:04:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:04:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.540 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:04:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:04:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:04:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:04:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.541 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:04:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:04:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:04:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:04:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:04:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:04:12.542 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:04:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:04:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:04:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 139.82316469700001, 2024-05-11T23:04:12.543 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:04:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:04:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.544 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:04:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:04:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:04:12.545 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:04:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:04:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:04:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:04:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:04:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.546 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:04:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:04:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:04:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:04:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:04:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:04:12.547 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:04:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:04:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:04:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 139.82314463700001, 2024-05-11T23:04:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:04:12.548 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:04:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:04:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.549 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:04:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:04:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:04:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:04:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:04:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.550 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:04:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:04:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:04:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:04:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:04:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:04:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.551 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:04:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:04:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:04:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:04:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:04:12.552 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 139.82312536699999, 2024-05-11T23:04:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:04:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:04:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:04:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.553 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:04:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:04:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:04:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:04:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:04:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.554 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:04:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:04:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:04:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:04:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:04:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:04:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.555 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:04:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:04:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:04:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:04:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:12.556 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:04:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 139.82257930099999, 2024-05-11T23:04:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:04:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:04:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:04:12.557 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:04:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:04:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:04:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.558 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:04:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:04:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:04:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:04:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.559 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:04:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:04:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:04:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:04:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:04:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:04:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:04:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:04:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:04:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 139.822547517, 2024-05-11T23:04:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:04:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.561 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:04:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:04:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.562 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:04:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:04:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:04:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:04:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:04:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.563 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:04:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:04:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:04:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:04:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:04:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:04:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.564 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:04:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:04:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:04:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:04:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:04:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:04:12.565 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 130.385302053, 2024-05-11T23:04:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:04:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:04:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:04:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:04:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.566 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:04:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:04:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:04:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:04:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:04:12.567 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:04:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:04:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:04:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:04:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:04:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:04:12.568 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:04:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:04:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:04:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:04:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:04:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:04:12.569 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:04:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:04:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:04:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:04:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:04:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:04:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:04:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:04:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.570 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:04:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:04:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.571 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.571 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:12.573 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:04:12.573 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:12.572+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:12.674 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:12.673+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:12.677 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:12.677 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:12.678 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:04:12.678 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:12.686 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:04:12.775 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:12.774+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:12.832 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:12.832 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:12.832 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:04:12.832 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:04:12.833 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:12.841 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:04:12.876 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:12.876+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:12.956 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:12.956 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:04:12.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:04:12.957 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:04:12.957 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.957 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:04:12.957 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:04:12.957 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 175.792026432, 2024-05-11T23:04:12.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:04:12.957 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.957 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.957 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:04:12.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:04:12.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:04:12.957 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:04:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:04:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:04:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:04:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:04:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:04:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:04:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:04:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:04:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:04:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:04:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:04:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:04:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:04:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:04:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:04:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:04:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:04:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:04:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:04:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:04:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:04:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:04:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.961 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:04:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:04:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:04:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:04:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:04:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:04:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:04:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:04:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:04:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:04:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:04:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78429 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:04:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 116.99075709, 2024-05-11T23:04:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044563497000000001, 2024-05-11T23:04:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.963 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78429 2024-05-11T23:04:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:04:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:04:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964872+0000", 2024-05-11T23:04:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035070300000000001 2024-05-11T23:04:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964876+0000", 2024-05-11T23:04:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5009999999999998e-06 2024-05-11T23:04:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964902+0000", 2024-05-11T23:04:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5715999999999999e-05 2024-05-11T23:04:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964910+0000", 2024-05-11T23:04:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3930000000000002e-06 2024-05-11T23:04:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008898+0000", 2024-05-11T23:04:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043988772000000002 2024-05-11T23:04:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008981+0000", 2024-05-11T23:04:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3140999999999996e-05 2024-05-11T23:04:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009085+0000", 2024-05-11T23:04:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010327099999999999 2024-05-11T23:04:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78431 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:04:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 116.990296782, 2024-05-11T23:04:12.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044542033000000002, 2024-05-11T23:04:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78431 2024-05-11T23:04:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:04:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:04:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965038+0000", 2024-05-11T23:04:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6993000000000001e-05 2024-05-11T23:04:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965043+0000", 2024-05-11T23:04:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7040000000000002e-06 2024-05-11T23:04:12.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965053+0000", 2024-05-11T23:04:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8539999999999992e-06 2024-05-11T23:04:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965061+0000", 2024-05-11T23:04:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9810000000000003e-06 2024-05-11T23:04:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009350+0000", 2024-05-11T23:04:12.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044288739000000001 2024-05-11T23:04:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009429+0000", 2024-05-11T23:04:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9354e-05 2024-05-11T23:04:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009524+0000", 2024-05-11T23:04:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4407999999999995e-05 2024-05-11T23:04:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78432 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:04:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 116.990205934, 2024-05-11T23:04:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044635421000000002, 2024-05-11T23:04:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78432 2024-05-11T23:04:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:04:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:04:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965343+0000", 2024-05-11T23:04:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027056500000000001 2024-05-11T23:04:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965347+0000", 2024-05-11T23:04:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3540000000000002e-06 2024-05-11T23:04:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965356+0000", 2024-05-11T23:04:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2140000000000002e-06 2024-05-11T23:04:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965363+0000", 2024-05-11T23:04:12.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4150000000000004e-06 2024-05-11T23:04:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009551+0000", 2024-05-11T23:04:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044187917 2024-05-11T23:04:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009626+0000", 2024-05-11T23:04:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5024000000000005e-05 2024-05-11T23:04:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009708+0000", 2024-05-11T23:04:12.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1932000000000001e-05 2024-05-11T23:04:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78433 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:04:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 116.98990504699999, 2024-05-11T23:04:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044564864000000003, 2024-05-11T23:04:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.976 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78433 2024-05-11T23:04:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:04:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:04:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.977 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966225+0000", 2024-05-11T23:04:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085220700000000001 2024-05-11T23:04:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966230+0000", 2024-05-11T23:04:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0690000000000003e-06 2024-05-11T23:04:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966239+0000", 2024-05-11T23:04:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-05-11T23:04:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.978 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966245+0000", 2024-05-11T23:04:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5520000000000001e-06 2024-05-11T23:04:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009743+0000", 2024-05-11T23:04:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043498311999999997 2024-05-11T23:04:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009848+0000", 2024-05-11T23:04:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104885 2024-05-11T23:04:12.979 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.980 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.980 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009938+0000", 2024-05-11T23:04:12.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9812e-05 2024-05-11T23:04:12.980 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.980 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.980 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.980 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.980 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.980 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78580 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 999424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:12.980 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:04:12.980 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 116.986792364, 2024-05-11T23:04:12.980 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044493076999999999, 2024-05-11T23:04:12.980 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78580 2024-05-11T23:04:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:04:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.981 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:04:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:04:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:04:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968489+0000", 2024-05-11T23:04:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.52e-06 2024-05-11T23:04:12.982 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968496+0000", 2024-05-11T23:04:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9419999999999998e-06 2024-05-11T23:04:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968501+0000", 2024-05-11T23:04:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8720000000000001e-06 2024-05-11T23:04:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012690+0000", 2024-05-11T23:04:12.983 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044189655000000001 2024-05-11T23:04:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012848+0000", 2024-05-11T23:04:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015758100000000001 2024-05-11T23:04:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012979+0000", 2024-05-11T23:04:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000131066 2024-05-11T23:04:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.984 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78581 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:04:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 116.98677136000001, 2024-05-11T23:04:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044616243, 2024-05-11T23:04:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.985 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78581 2024-05-11T23:04:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:04:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:04:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.986 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970227+0000", 2024-05-11T23:04:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017200049999999999 2024-05-11T23:04:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970232+0000", 2024-05-11T23:04:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000003e-06 2024-05-11T23:04:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970242+0000", 2024-05-11T23:04:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0192e-05 2024-05-11T23:04:12.987 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970249+0000", 2024-05-11T23:04:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.888e-06 2024-05-11T23:04:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013008+0000", 2024-05-11T23:04:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042759495000000002 2024-05-11T23:04:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013063+0000", 2024-05-11T23:04:12.988 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5071999999999997e-05 2024-05-11T23:04:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013123+0000", 2024-05-11T23:04:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9930999999999999e-05 2024-05-11T23:04:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78589 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:04:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 116.984852116, 2024-05-11T23:04:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044587697000000003, 2024-05-11T23:04:12.989 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78589 2024-05-11T23:04:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:04:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.990 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:04:12.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970427+0000", 2024-05-11T23:04:12.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:04:12.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.991 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.991 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.991 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970429+0000", 2024-05-11T23:04:12.991 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3800000000000001e-06 2024-05-11T23:04:12.991 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970435+0000", 2024-05-11T23:04:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2530000000000001e-06 2024-05-11T23:04:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970441+0000", 2024-05-11T23:04:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-05-11T23:04:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014822+0000", 2024-05-11T23:04:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04438168 2024-05-11T23:04:12.992 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014911+0000", 2024-05-11T23:04:12.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8882999999999996e-05 2024-05-11T23:04:12.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.993 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.993 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015014+0000", 2024-05-11T23:04:12.993 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010263399999999999 2024-05-11T23:04:12.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.993 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.993 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.993 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.993 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.994 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78590 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:12.994 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:04:12.994 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 116.984831317, 2024-05-11T23:04:12.994 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044880120000000003, 2024-05-11T23:04:12.994 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.994 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.994 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.994 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.994 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78590 2024-05-11T23:04:12.994 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.994 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:04:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:04:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970918+0000", 2024-05-11T23:04:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000470834 2024-05-11T23:04:12.995 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:12.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970925+0000", 2024-05-11T23:04:12.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1690000000000003e-06 2024-05-11T23:04:12.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:12.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970934+0000", 2024-05-11T23:04:12.996 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9099999999999994e-06 2024-05-11T23:04:12.996 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.996 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.996 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:12.996 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970941+0000", 2024-05-11T23:04:12.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7410000000000003e-06 2024-05-11T23:04:12.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:12.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015049+0000", 2024-05-11T23:04:12.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044108392000000003 2024-05-11T23:04:12.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:12.997 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015133+0000", 2024-05-11T23:04:12.997 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3568999999999999e-05 2024-05-11T23:04:12.997 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.997 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.997 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015327+0000", 2024-05-11T23:04:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019450500000000001 2024-05-11T23:04:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78591 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:04:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 116.984327901, 2024-05-11T23:04:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044618497, 2024-05-11T23:04:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:12.998 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:12.999 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:12.999 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:12.999 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:12.999 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78591 2024-05-11T23:04:12.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.999 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:12.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:12.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:04:12.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:12.999 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:12.999 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:12.999 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:12.999 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:04:12.999 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970951+0000", 2024-05-11T23:04:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:04:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970953+0000", 2024-05-11T23:04:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:04:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970956+0000", 2024-05-11T23:04:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-05-11T23:04:13.000 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970959+0000", 2024-05-11T23:04:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9409999999999999e-06 2024-05-11T23:04:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015367+0000", 2024-05-11T23:04:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044407621000000001 2024-05-11T23:04:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015453+0000", 2024-05-11T23:04:13.001 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6685000000000001e-05 2024-05-11T23:04:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015569+0000", 2024-05-11T23:04:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115396 2024-05-11T23:04:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78592 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:04:13.002 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 116.98431509300001, 2024-05-11T23:04:13.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044831546999999999, 2024-05-11T23:04:13.003 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:13.003 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:13.003 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:13.003 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:13.003 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:13.003 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.003 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78592 2024-05-11T23:04:13.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.003 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:13.003 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.003 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:13.003 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:04:13.003 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.003 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:04:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972226+0000", 2024-05-11T23:04:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00126246 2024-05-11T23:04:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972230+0000", 2024-05-11T23:04:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1860000000000002e-06 2024-05-11T23:04:13.004 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972237+0000", 2024-05-11T23:04:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:04:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972241+0000", 2024-05-11T23:04:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.284e-06 2024-05-11T23:04:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015611+0000", 2024-05-11T23:04:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043369461999999998 2024-05-11T23:04:13.005 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:13.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015695+0000", 2024-05-11T23:04:13.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4458999999999999e-05 2024-05-11T23:04:13.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.006 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:13.006 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015795+0000", 2024-05-11T23:04:13.006 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9475999999999994e-05 2024-05-11T23:04:13.006 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.006 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:13.006 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.006 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.006 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.007 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78593 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:13.007 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:04:13.007 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 116.983028174, 2024-05-11T23:04:13.007 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046045214000000001, 2024-05-11T23:04:13.007 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:13.007 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:13.007 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:13.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:13.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:13.007 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.007 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78593 2024-05-11T23:04:13.007 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.007 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:13.007 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.007 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:04:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:04:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:04:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7599999999999998e-07 2024-05-11T23:04:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.008 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:13.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972252+0000", 2024-05-11T23:04:13.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7349999999999999e-06 2024-05-11T23:04:13.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:13.009 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972256+0000", 2024-05-11T23:04:13.009 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5659999999999998e-06 2024-05-11T23:04:13.009 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.009 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.009 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972259+0000", 2024-05-11T23:04:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8040000000000002e-06 2024-05-11T23:04:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018088+0000", 2024-05-11T23:04:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045829316000000002 2024-05-11T23:04:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018179+0000", 2024-05-11T23:04:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1299999999999997e-05 2024-05-11T23:04:13.010 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018295+0000", 2024-05-11T23:04:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116117 2024-05-11T23:04:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78594 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:04:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 116.983015814, 2024-05-11T23:04:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046245204999999998, 2024-05-11T23:04:13.011 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78594 2024-05-11T23:04:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:04:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.012 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:13.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:04:13.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:13.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972263+0000", 2024-05-11T23:04:13.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:04:13.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.013 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.013 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:13.013 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972264+0000", 2024-05-11T23:04:13.013 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.206e-06 2024-05-11T23:04:13.013 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:13.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972267+0000", 2024-05-11T23:04:13.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0690000000000002e-06 2024-05-11T23:04:13.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.014 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.014 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.014 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972270+0000", 2024-05-11T23:04:13.014 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5610000000000001e-06 2024-05-11T23:04:13.014 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018330+0000", 2024-05-11T23:04:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04606085 2024-05-11T23:04:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018414+0000", 2024-05-11T23:04:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3125000000000001e-05 2024-05-11T23:04:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018508+0000", 2024-05-11T23:04:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4153000000000005e-05 2024-05-11T23:04:13.015 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.016 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:13.016 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.016 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.016 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.016 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78595 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:13.016 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:04:13.016 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 116.983002949, 2024-05-11T23:04:13.016 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046396698, 2024-05-11T23:04:13.016 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:13.016 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:13.016 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:13.016 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:13.016 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78595 2024-05-11T23:04:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:04:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:04:13.017 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972276+0000", 2024-05-11T23:04:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5600000000000002e-07 2024-05-11T23:04:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972277+0000", 2024-05-11T23:04:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.576e-06 2024-05-11T23:04:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.018 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972280+0000", 2024-05-11T23:04:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-05-11T23:04:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972283+0000", 2024-05-11T23:04:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:04:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018538+0000", 2024-05-11T23:04:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046255255000000002 2024-05-11T23:04:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.019 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018605+0000", 2024-05-11T23:04:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6847000000000006e-05 2024-05-11T23:04:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018672+0000", 2024-05-11T23:04:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7045000000000006e-05 2024-05-11T23:04:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.020 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78596 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:04:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 116.9829921, 2024-05-11T23:04:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046586806000000001, 2024-05-11T23:04:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78596 2024-05-11T23:04:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:13.021 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:04:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:04:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:04:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:04:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.022 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972288+0000", 2024-05-11T23:04:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2550000000000001e-06 2024-05-11T23:04:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972291+0000", 2024-05-11T23:04:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9490000000000001e-06 2024-05-11T23:04:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972293+0000", 2024-05-11T23:04:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5069999999999999e-06 2024-05-11T23:04:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.023 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018707+0000", 2024-05-11T23:04:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046413604999999997 2024-05-11T23:04:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018782+0000", 2024-05-11T23:04:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5556000000000006e-05 2024-05-11T23:04:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018873+0000", 2024-05-11T23:04:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0703000000000002e-05 2024-05-11T23:04:13.024 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78597 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:04:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 116.982982327, 2024-05-11T23:04:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046825787000000001, 2024-05-11T23:04:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:13.025 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78597 2024-05-11T23:04:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:04:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:04:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.026 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:04:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-05-11T23:04:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972297+0000", 2024-05-11T23:04:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3069999999999999e-06 2024-05-11T23:04:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972300+0000", 2024-05-11T23:04:13.027 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:04:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972303+0000", 2024-05-11T23:04:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5509999999999998e-06 2024-05-11T23:04:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018911+0000", 2024-05-11T23:04:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046608194999999998 2024-05-11T23:04:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018986+0000", 2024-05-11T23:04:13.028 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5360999999999999e-05 2024-05-11T23:04:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019122+0000", 2024-05-11T23:04:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013526 2024-05-11T23:04:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78598 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:04:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 116.98297242699999, 2024-05-11T23:04:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046998572000000002, 2024-05-11T23:04:13.029 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78598 2024-05-11T23:04:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:04:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.030 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:04:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:04:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:04:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:13.031 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972307+0000", 2024-05-11T23:04:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1799999999999999e-06 2024-05-11T23:04:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972310+0000", 2024-05-11T23:04:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:04:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972313+0000", 2024-05-11T23:04:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4760000000000001e-06 2024-05-11T23:04:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:13.032 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019149+0000", 2024-05-11T23:04:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046836056000000001 2024-05-11T23:04:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019203+0000", 2024-05-11T23:04:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-05 2024-05-11T23:04:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019304+0000", 2024-05-11T23:04:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101537 2024-05-11T23:04:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.033 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:105234 3.15 3:adf885e5:::rbd_object_map.111e913a615f.0000000000000025:head [writefull 0~39] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-11T23:04:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:04:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 102.947656717, 2024-05-11T23:04:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77797599100000003, 2024-05-11T23:04:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:04:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 105234 2024-05-11T23:04:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:13.034 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:04:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:04:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007623+0000", 2024-05-11T23:04:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2240000000000001e-06 2024-05-11T23:04:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.035 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:13.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007627+0000", 2024-05-11T23:04:13.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:04:13.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:13.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007636+0000", 2024-05-11T23:04:13.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2409999999999995e-06 2024-05-11T23:04:13.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.036 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.036 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007641+0000", 2024-05-11T23:04:13.036 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6810000000000003e-06 2024-05-11T23:04:13.036 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.036 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:13.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007676+0000", 2024-05-11T23:04:13.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5061999999999997e-05 2024-05-11T23:04:13.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:04:13.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007682+0000", 2024-05-11T23:04:13.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0399999999999998e-06 2024-05-11T23:04:13.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.037 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:13.037 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783880+0000", 2024-05-11T23:04:13.037 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77619752900000005 2024-05-11T23:04:13.037 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.037 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:13.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783995+0000", 2024-05-11T23:04:13.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011550800000000001 2024-05-11T23:04:13.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:04:13.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784082+0000", 2024-05-11T23:04:13.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6347999999999994e-05 2024-05-11T23:04:13.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.038 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.038 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:04:13.038 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784682+0000", 2024-05-11T23:04:13.038 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000600162 2024-05-11T23:04:13.038 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:04:13.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785539+0000", 2024-05-11T23:04:13.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085699099999999998 2024-05-11T23:04:13.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:04:13.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785572+0000", 2024-05-11T23:04:13.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3182999999999997e-05 2024-05-11T23:04:13.039 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.039 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.039 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:13.039 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785598+0000", 2024-05-11T23:04:13.039 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5522000000000001e-05 2024-05-11T23:04:13.039 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.040 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:13.040 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.040 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.040 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.040 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113761 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [call rbd.sparse_copyup in=999496b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:04:13.040 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:04:13.040 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.424969798000006, 2024-05-11T23:04:13.040 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048572964000000003, 2024-05-11T23:04:13.040 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:13.040 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:13.040 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:04:13.040 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:13.040 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:13.040 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.041 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113761 2024-05-11T23:04:13.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.041 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:13.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:13.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:04:13.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.041 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:13.041 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:04:13.041 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.041 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.041 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:13.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530309+0000", 2024-05-11T23:04:13.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:04:13.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:13.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531242+0000", 2024-05-11T23:04:13.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093316899999999997 2024-05-11T23:04:13.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.042 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:13.042 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531249+0000", 2024-05-11T23:04:13.042 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8140000000000003e-06 2024-05-11T23:04:13.042 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.042 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531253+0000", 2024-05-11T23:04:13.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4290000000000003e-06 2024-05-11T23:04:13.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:13.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570325+0000", 2024-05-11T23:04:13.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039071379000000003 2024-05-11T23:04:13.043 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.043 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.043 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:13.043 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570382+0000", 2024-05-11T23:04:13.043 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6988999999999999e-05 2024-05-11T23:04:13.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:04:13.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570555+0000", 2024-05-11T23:04:13.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017379799999999999 2024-05-11T23:04:13.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:04:13.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.575953+0000", 2024-05-11T23:04:13.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053975539999999997 2024-05-11T23:04:13.044 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.044 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.044 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:04:13.044 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578778+0000", 2024-05-11T23:04:13.044 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002825616 2024-05-11T23:04:13.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:04:13.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578837+0000", 2024-05-11T23:04:13.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9005000000000002e-05 2024-05-11T23:04:13.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.045 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.045 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:13.045 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578881+0000", 2024-05-11T23:04:13.045 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3970000000000001e-05 2024-05-11T23:04:13.045 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.045 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:13.045 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.045 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.046 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113762 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [delete] snapc 27=[27] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:04:13.046 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:04:13.046 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 98.424015847000007, 2024-05-11T23:04:13.046 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047916260000000002, 2024-05-11T23:04:13.046 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:13.046 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:13.046 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:04:13.046 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:13.046 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:13.046 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.046 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113762 2024-05-11T23:04:13.046 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.046 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:13.046 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:13.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:04:13.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:13.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:04:13.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:13.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.047 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:13.047 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531263+0000", 2024-05-11T23:04:13.047 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.85e-07 2024-05-11T23:04:13.047 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.047 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:13.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531265+0000", 2024-05-11T23:04:13.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.827e-06 2024-05-11T23:04:13.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:13.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531268+0000", 2024-05-11T23:04:13.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3780000000000001e-06 2024-05-11T23:04:13.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.048 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.048 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.048 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531271+0000", 2024-05-11T23:04:13.048 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8509999999999999e-06 2024-05-11T23:04:13.048 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:13.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572298+0000", 2024-05-11T23:04:13.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041026990999999999 2024-05-11T23:04:13.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:13.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572345+0000", 2024-05-11T23:04:13.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6878000000000001e-05 2024-05-11T23:04:13.049 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.049 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.049 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:04:13.049 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572443+0000", 2024-05-11T23:04:13.049 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8184999999999996e-05 2024-05-11T23:04:13.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:04:13.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578585+0000", 2024-05-11T23:04:13.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006141919 2024-05-11T23:04:13.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:04:13.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579126+0000", 2024-05-11T23:04:13.050 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00054063799999999997 2024-05-11T23:04:13.050 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.050 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.050 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:04:13.050 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579154+0000", 2024-05-11T23:04:13.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8042999999999999e-05 2024-05-11T23:04:13.051 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:13.051 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:13.051 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:13.051 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579179+0000", 2024-05-11T23:04:13.051 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4765000000000001e-05 2024-05-11T23:04:13.051 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.051 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:13.051 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.051 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:13.051 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:13.051 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:13.053 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:04:13.053 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:13.052+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:13.075 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 5, 7, 3, 6, 4] out_osds: [0] dead_osds: [3] live_osds: [0, 1, 2, 4, 5, 6, 7] 2024-05-11T23:04:13.075 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-11T23:04:13.075 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-11T23:04:13.075 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 0.822876 2024-05-11T23:04:13.076 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 0.822876151500306 2024-05-11T23:04:13.154 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:13.154+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:13.172 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:13.172 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:04:13.172 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:04:13.172 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:13.182 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:04:13.255 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:13.255+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:13.334 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:13.334 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:04:13.334 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:04:13.334 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:04:13.334 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:13.343 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:04:13.387 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:13.387+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:13.456 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:04:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:04:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:04:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226051 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6049792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:04:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 71.138358332999999, 2024-05-11T23:04:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050622418000000002, 2024-05-11T23:04:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:13.457 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226051 2024-05-11T23:04:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:04:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:04:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317156+0000", 2024-05-11T23:04:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:04:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317157+0000", 2024-05-11T23:04:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.144e-06 2024-05-11T23:04:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317160+0000", 2024-05-11T23:04:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1350000000000001e-06 2024-05-11T23:04:13.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317162+0000", 2024-05-11T23:04:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5050000000000002e-06 2024-05-11T23:04:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367689+0000", 2024-05-11T23:04:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526797999999998 2024-05-11T23:04:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367735+0000", 2024-05-11T23:04:13.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5698999999999997e-05 2024-05-11T23:04:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367778+0000", 2024-05-11T23:04:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2928e-05 2024-05-11T23:04:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226052 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6060032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:04:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 71.138348277000006, 2024-05-11T23:04:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050830110999999997, 2024-05-11T23:04:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226052 2024-05-11T23:04:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.462 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:04:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:04:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317166+0000", 2024-05-11T23:04:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:04:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317167+0000", 2024-05-11T23:04:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.155e-06 2024-05-11T23:04:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317170+0000", 2024-05-11T23:04:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.061e-06 2024-05-11T23:04:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317172+0000", 2024-05-11T23:04:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4420000000000001e-06 2024-05-11T23:04:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367816+0000", 2024-05-11T23:04:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050644013000000002 2024-05-11T23:04:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367906+0000", 2024-05-11T23:04:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9739000000000003e-05 2024-05-11T23:04:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367995+0000", 2024-05-11T23:04:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9499000000000005e-05 2024-05-11T23:04:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.465 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226053 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6070272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:04:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 71.138338238000003, 2024-05-11T23:04:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051106912999999997, 2024-05-11T23:04:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.466 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226053 2024-05-11T23:04:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:04:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:04:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:13.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317176+0000", 2024-05-11T23:04:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0100000000000001e-07 2024-05-11T23:04:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317179+0000", 2024-05-11T23:04:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-05-11T23:04:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317182+0000", 2024-05-11T23:04:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3589999999999999e-06 2024-05-11T23:04:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317184+0000", 2024-05-11T23:04:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4339999999999999e-06 2024-05-11T23:04:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368061+0000", 2024-05-11T23:04:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050876927000000002 2024-05-11T23:04:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368179+0000", 2024-05-11T23:04:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118001 2024-05-11T23:04:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:13.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368282+0000", 2024-05-11T23:04:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102942 2024-05-11T23:04:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226054 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6080512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:04:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 71.138325915999999, 2024-05-11T23:04:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051322, 2024-05-11T23:04:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:13.470 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226054 2024-05-11T23:04:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:04:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:04:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:04:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:04:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317189+0000", 2024-05-11T23:04:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.217e-06 2024-05-11T23:04:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317192+0000", 2024-05-11T23:04:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0699999999999998e-06 2024-05-11T23:04:13.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317195+0000", 2024-05-11T23:04:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3829999999999999e-06 2024-05-11T23:04:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368328+0000", 2024-05-11T23:04:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051132999999999998 2024-05-11T23:04:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368429+0000", 2024-05-11T23:04:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010168800000000001 2024-05-11T23:04:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368510+0000", 2024-05-11T23:04:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0420999999999995e-05 2024-05-11T23:04:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226055 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6090752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:04:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 71.138315848000005, 2024-05-11T23:04:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051562835000000001, 2024-05-11T23:04:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:13.475 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226055 2024-05-11T23:04:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:04:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:04:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317256+0000", 2024-05-11T23:04:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8047000000000002e-05 2024-05-11T23:04:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317259+0000", 2024-05-11T23:04:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2590000000000001e-06 2024-05-11T23:04:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317267+0000", 2024-05-11T23:04:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8769999999999993e-06 2024-05-11T23:04:13.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317274+0000", 2024-05-11T23:04:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.187e-06 2024-05-11T23:04:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368544+0000", 2024-05-11T23:04:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051270299999999998 2024-05-11T23:04:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368658+0000", 2024-05-11T23:04:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000113347 2024-05-11T23:04:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368761+0000", 2024-05-11T23:04:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102818 2024-05-11T23:04:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226056 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6100992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:04:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 71.138230002, 2024-05-11T23:04:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051615576000000003, 2024-05-11T23:04:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226056 2024-05-11T23:04:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:13.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:04:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:04:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318915+0000", 2024-05-11T23:04:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001631291 2024-05-11T23:04:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318918+0000", 2024-05-11T23:04:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.512e-06 2024-05-11T23:04:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318927+0000", 2024-05-11T23:04:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1960000000000008e-06 2024-05-11T23:04:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318932+0000", 2024-05-11T23:04:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:04:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368805+0000", 2024-05-11T23:04:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049872998000000002 2024-05-11T23:04:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368853+0000", 2024-05-11T23:04:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8041999999999998e-05 2024-05-11T23:04:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368899+0000", 2024-05-11T23:04:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5766000000000002e-05 2024-05-11T23:04:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226061 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6152192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:04:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 71.136438914999999, 2024-05-11T23:04:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050675395999999998, 2024-05-11T23:04:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:13.484 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226061 2024-05-11T23:04:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:04:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:04:13.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:04:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:04:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319078+0000", 2024-05-11T23:04:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5670000000000002e-06 2024-05-11T23:04:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319085+0000", 2024-05-11T23:04:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2570000000000001e-06 2024-05-11T23:04:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319100+0000", 2024-05-11T23:04:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.489e-05 2024-05-11T23:04:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369584+0000", 2024-05-11T23:04:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050484345999999999 2024-05-11T23:04:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369671+0000", 2024-05-11T23:04:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6654000000000001e-05 2024-05-11T23:04:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369750+0000", 2024-05-11T23:04:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9257000000000004e-05 2024-05-11T23:04:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226062 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6162432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:04:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 71.136405370999995, 2024-05-11T23:04:13.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050761158000000001, 2024-05-11T23:04:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226062 2024-05-11T23:04:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:04:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:04:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319109+0000", 2024-05-11T23:04:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2800000000000002e-07 2024-05-11T23:04:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319111+0000", 2024-05-11T23:04:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4980000000000001e-06 2024-05-11T23:04:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319120+0000", 2024-05-11T23:04:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3999999999999992e-06 2024-05-11T23:04:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319129+0000", 2024-05-11T23:04:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7580000000000004e-06 2024-05-11T23:04:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369775+0000", 2024-05-11T23:04:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050645580000000003 2024-05-11T23:04:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:13.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369824+0000", 2024-05-11T23:04:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9506999999999999e-05 2024-05-11T23:04:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369869+0000", 2024-05-11T23:04:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4987000000000001e-05 2024-05-11T23:04:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226063 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6172672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:04:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 71.136376413999997, 2024-05-11T23:04:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050939784000000002, 2024-05-11T23:04:13.492 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226063 2024-05-11T23:04:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:04:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:13.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:04:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319138+0000", 2024-05-11T23:04:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.39e-07 2024-05-11T23:04:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319140+0000", 2024-05-11T23:04:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4820000000000001e-06 2024-05-11T23:04:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319148+0000", 2024-05-11T23:04:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6599999999999995e-06 2024-05-11T23:04:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319154+0000", 2024-05-11T23:04:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4169999999999997e-06 2024-05-11T23:04:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369905+0000", 2024-05-11T23:04:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050751009999999999 2024-05-11T23:04:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369991+0000", 2024-05-11T23:04:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5909999999999996e-05 2024-05-11T23:04:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370077+0000", 2024-05-11T23:04:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5865999999999994e-05 2024-05-11T23:04:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226064 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6182912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:04:13.496 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 71.136350218000004, 2024-05-11T23:04:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051038241999999998, 2024-05-11T23:04:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226064 2024-05-11T23:04:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:04:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:04:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319164+0000", 2024-05-11T23:04:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9299999999999999e-07 2024-05-11T23:04:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319166+0000", 2024-05-11T23:04:13.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.48e-06 2024-05-11T23:04:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319176+0000", 2024-05-11T23:04:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9669999999999996e-06 2024-05-11T23:04:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319180+0000", 2024-05-11T23:04:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8020000000000002e-06 2024-05-11T23:04:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370100+0000", 2024-05-11T23:04:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050919845999999998 2024-05-11T23:04:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370155+0000", 2024-05-11T23:04:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5040000000000002e-05 2024-05-11T23:04:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370202+0000", 2024-05-11T23:04:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6714e-05 2024-05-11T23:04:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226065 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6193152~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:04:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 71.136327648000005, 2024-05-11T23:04:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051225087000000002, 2024-05-11T23:04:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:13.501 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226065 2024-05-11T23:04:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:04:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:04:13.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320913+0000", 2024-05-11T23:04:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001727492 2024-05-11T23:04:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320919+0000", 2024-05-11T23:04:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.964e-06 2024-05-11T23:04:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320927+0000", 2024-05-11T23:04:13.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2239999999999998e-06 2024-05-11T23:04:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320934+0000", 2024-05-11T23:04:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8469999999999998e-06 2024-05-11T23:04:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370240+0000", 2024-05-11T23:04:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049306562999999998 2024-05-11T23:04:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370328+0000", 2024-05-11T23:04:13.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8122999999999996e-05 2024-05-11T23:04:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370411+0000", 2024-05-11T23:04:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2874000000000006e-05 2024-05-11T23:04:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226073 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6275072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:04:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 71.134489260999999, 2024-05-11T23:04:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050730475999999997, 2024-05-11T23:04:13.505 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226073 2024-05-11T23:04:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:04:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:04:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322914+0000", 2024-05-11T23:04:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001889153 2024-05-11T23:04:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322917+0000", 2024-05-11T23:04:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2100000000000002e-06 2024-05-11T23:04:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322924+0000", 2024-05-11T23:04:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9929999999999998e-06 2024-05-11T23:04:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322928+0000", 2024-05-11T23:04:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5549999999999996e-06 2024-05-11T23:04:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371578+0000", 2024-05-11T23:04:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048650001999999998 2024-05-11T23:04:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371671+0000", 2024-05-11T23:04:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2230000000000003e-05 2024-05-11T23:04:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371755+0000", 2024-05-11T23:04:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4332999999999994e-05 2024-05-11T23:04:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226085 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 5338112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:13.509 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:04:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 71.132398162000001, 2024-05-11T23:04:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050727622, 2024-05-11T23:04:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226085 2024-05-11T23:04:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:04:13.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:04:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324914+0000", 2024-05-11T23:04:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001798298 2024-05-11T23:04:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324918+0000", 2024-05-11T23:04:13.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3889999999999998e-06 2024-05-11T23:04:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324928+0000", 2024-05-11T23:04:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4339999999999998e-06 2024-05-11T23:04:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324936+0000", 2024-05-11T23:04:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2770000000000005e-06 2024-05-11T23:04:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:13.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373664+0000", 2024-05-11T23:04:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048728207000000003 2024-05-11T23:04:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373757+0000", 2024-05-11T23:04:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3332999999999996e-05 2024-05-11T23:04:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373843+0000", 2024-05-11T23:04:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5684000000000002e-05 2024-05-11T23:04:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232555 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7789568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:04:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 70.655387837000006, 2024-05-11T23:04:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050583086999999999, 2024-05-11T23:04:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232555 2024-05-11T23:04:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.514 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:04:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:04:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:04:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:04:13.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800127+0000", 2024-05-11T23:04:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.207e-06 2024-05-11T23:04:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800132+0000", 2024-05-11T23:04:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:04:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800134+0000", 2024-05-11T23:04:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5459999999999998e-06 2024-05-11T23:04:13.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850548+0000", 2024-05-11T23:04:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050414182000000002 2024-05-11T23:04:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850635+0000", 2024-05-11T23:04:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6853000000000003e-05 2024-05-11T23:04:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850709+0000", 2024-05-11T23:04:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3585999999999996e-05 2024-05-11T23:04:13.517 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232556 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7799808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:04:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 70.655376183000001, 2024-05-11T23:04:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050749599999999999, 2024-05-11T23:04:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:13.518 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232556 2024-05-11T23:04:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:04:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:04:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801914+0000", 2024-05-11T23:04:13.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017766360000000001 2024-05-11T23:04:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801919+0000", 2024-05-11T23:04:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.42e-06 2024-05-11T23:04:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801927+0000", 2024-05-11T23:04:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1999999999999994e-06 2024-05-11T23:04:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801931+0000", 2024-05-11T23:04:13.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4279999999999998e-06 2024-05-11T23:04:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850740+0000", 2024-05-11T23:04:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048808658999999997 2024-05-11T23:04:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850819+0000", 2024-05-11T23:04:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9102000000000003e-05 2024-05-11T23:04:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850887+0000", 2024-05-11T23:04:13.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8155000000000001e-05 2024-05-11T23:04:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232578 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8025088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:04:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 70.652082945000004, 2024-05-11T23:04:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050679473000000003, 2024-05-11T23:04:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:13.522 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232578 2024-05-11T23:04:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:04:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:04:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:04:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:04:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803432+0000", 2024-05-11T23:04:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.139e-06 2024-05-11T23:04:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803435+0000", 2024-05-11T23:04:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2440000000000002e-06 2024-05-11T23:04:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803439+0000", 2024-05-11T23:04:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3129999999999999e-06 2024-05-11T23:04:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.853953+0000", 2024-05-11T23:04:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050514846000000002 2024-05-11T23:04:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854035+0000", 2024-05-11T23:04:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1148000000000003e-05 2024-05-11T23:04:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854110+0000", 2024-05-11T23:04:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5562000000000006e-05 2024-05-11T23:04:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232579 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8035328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:04:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 70.652070340999998, 2024-05-11T23:04:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050886656000000002, 2024-05-11T23:04:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:13.526 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232579 2024-05-11T23:04:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:04:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:04:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803914+0000", 2024-05-11T23:04:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00047026700000000002 2024-05-11T23:04:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803918+0000", 2024-05-11T23:04:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3460000000000004e-06 2024-05-11T23:04:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803927+0000", 2024-05-11T23:04:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5030000000000003e-06 2024-05-11T23:04:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803933+0000", 2024-05-11T23:04:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0310000000000004e-06 2024-05-11T23:04:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854153+0000", 2024-05-11T23:04:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050219321999999997 2024-05-11T23:04:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854242+0000", 2024-05-11T23:04:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9457999999999996e-05 2024-05-11T23:04:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854330+0000", 2024-05-11T23:04:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7729e-05 2024-05-11T23:04:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232581 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8055808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:13.530 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:04:13.531 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 70.651546863999997, 2024-05-11T23:04:13.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050664439999999998, 2024-05-11T23:04:13.531 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:13.531 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:13.531 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232581 2024-05-11T23:04:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:13.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:04:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:04:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:04:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:04:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803969+0000", 2024-05-11T23:04:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0889999999999998e-06 2024-05-11T23:04:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803978+0000", 2024-05-11T23:04:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3070000000000003e-06 2024-05-11T23:04:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803983+0000", 2024-05-11T23:04:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1599999999999997e-06 2024-05-11T23:04:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:13.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854509+0000", 2024-05-11T23:04:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526339000000003 2024-05-11T23:04:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854576+0000", 2024-05-11T23:04:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7102999999999998e-05 2024-05-11T23:04:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854631+0000", 2024-05-11T23:04:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4997999999999998e-05 2024-05-11T23:04:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232582 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8066048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:04:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 70.651524119000001, 2024-05-11T23:04:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050753983000000003, 2024-05-11T23:04:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232582 2024-05-11T23:04:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:13.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:04:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:04:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805917+0000", 2024-05-11T23:04:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001927432 2024-05-11T23:04:13.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805920+0000", 2024-05-11T23:04:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1219999999999999e-06 2024-05-11T23:04:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805933+0000", 2024-05-11T23:04:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2727e-05 2024-05-11T23:04:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805941+0000", 2024-05-11T23:04:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7090000000000002e-06 2024-05-11T23:04:13.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854657+0000", 2024-05-11T23:04:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048716164999999999 2024-05-11T23:04:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854699+0000", 2024-05-11T23:04:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2531999999999999e-05 2024-05-11T23:04:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854744+0000", 2024-05-11T23:04:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4295999999999997e-05 2024-05-11T23:04:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.540 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232593 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8178688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:04:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 70.649324726000003, 2024-05-11T23:04:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050735178999999998, 2024-05-11T23:04:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232593 2024-05-11T23:04:13.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:04:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:04:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807913+0000", 2024-05-11T23:04:13.542 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017240319999999999 2024-05-11T23:04:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807916+0000", 2024-05-11T23:04:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4589999999999998e-06 2024-05-11T23:04:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807924+0000", 2024-05-11T23:04:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.272e-06 2024-05-11T23:04:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807928+0000", 2024-05-11T23:04:13.543 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5369999999999999e-06 2024-05-11T23:04:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856768+0000", 2024-05-11T23:04:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048839489999999999 2024-05-11T23:04:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856845+0000", 2024-05-11T23:04:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7746999999999999e-05 2024-05-11T23:04:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856924+0000", 2024-05-11T23:04:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8641999999999997e-05 2024-05-11T23:04:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.544 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.545 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.545 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:13.545 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:13.545 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:13.545 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:13.456+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:13.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:04:13.587 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:13.587+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:13.652 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-11T23:04:13.697 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:13.697 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:04:13.697 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:04:13.697 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:13.697 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:13.697+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:13.706 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:04:13.759 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:13.758+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:13.779 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:04:13.780 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:04:13.798 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:04:13.798 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:04:13.815 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:04:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:04:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-05-11T23:04:13.834 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:04:13.834 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:04:13.852 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:04:13.853 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:04:13.860 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:13.860+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:13.900 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:13.901 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:13.901 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:04:13.901 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:04:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:04:13.902 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:04:13.903 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-11T23:04:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:04:13.903 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-11T23:04:13.907 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.3 primary-affinity to 0.822876 (8539282) 2024-05-11T23:04:13.916 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-11T23:04:13.966 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:13.966+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:14.024 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:14.025 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:14.035 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:04:14.061 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:14.061+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:14.162 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:14.162+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:14.181 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:14.181 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:14.190 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:04:14.275 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:14.275+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:14.340 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:14.341 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:14.350 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:04:14.391 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:14.391+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:14.466 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:14.466+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:14.499 INFO:teuthology.orchestra.run.smithi005.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:14.499 INFO:teuthology.orchestra.run.smithi005.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:14.508 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:04:14.566 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:14.566+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:14.621 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:14.621 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:14.629 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:04:14.673 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:14.673+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:14.735 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:14.735 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:14.743 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:04:14.784 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:14.784+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:14.868 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:14.868+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:14.888 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:14.888 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:14.896 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:04:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is set 2024-05-11T23:04:14.970 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:14.970+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:15.017 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:15.017 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:15.026 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:04:15.071 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:15.071+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:15.171 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:15.171 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:15.172 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:15.172+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:15.180 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:04:15.273 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:15.273+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:15.284 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:15.284 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:15.294 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:04:15.374 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:15.374+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:15.398 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:15.399 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:15.407 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:04:15.475 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:15.475+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:15.513 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:15.513 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:15.521 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:04:15.576 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:15.576+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:15.626 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:15.626 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:15.635 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:04:15.678 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:15.678+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:15.740 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:15.740 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:15.748 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:04:15.789 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:15.789+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:15.881 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:15.880+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:15.893 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:15.893 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:15.902 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:04:15.904 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-11T23:04:15.982 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:15.981+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:16.052 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0: {} 2024-05-11T23:04:16.052 INFO:teuthology.orchestra.run.smithi005.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-11T23:04:16.056 INFO:teuthology.orchestra.run.smithi115.stdout:ERROR: (22) Invalid argument 2024-05-11T23:04:16.056 INFO:teuthology.orchestra.run.smithi115.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-11T23:04:16.065 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-11T23:04:16.070 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1: {} 2024-05-11T23:04:16.070 INFO:teuthology.orchestra.run.smithi005.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-11T23:04:16.083 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:16.083+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:16.088 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2: {} 2024-05-11T23:04:16.088 INFO:teuthology.orchestra.run.smithi005.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-11T23:04:16.088 INFO:teuthology.orchestra.run.smithi005.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-05-11T23:04:16.106 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4: {} 2024-05-11T23:04:16.106 INFO:teuthology.orchestra.run.smithi005.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-11T23:04:16.123 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5: {} 2024-05-11T23:04:16.124 INFO:teuthology.orchestra.run.smithi005.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-11T23:04:16.141 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6: {} 2024-05-11T23:04:16.142 INFO:teuthology.orchestra.run.smithi005.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-11T23:04:16.159 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7: {} 2024-05-11T23:04:16.159 INFO:teuthology.orchestra.run.smithi005.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-11T23:04:16.162 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-11T23:04:16.163 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-11T23:04:16.177 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:04:16.178 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:04:16.178 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:16.184 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:04:16.184+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:04:16.186 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-11T23:04:16.285 INFO:tasks.ceph.osd.7.smithi115.stderr:2024-05-11T23:04:16.285+0000 7fe981ac3640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33181) UID: 0 2024-05-11T23:04:16.337 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:16.337 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:04:16.337 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:04:16.337 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:04:16.337 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:16.345 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-11T23:04:16.389 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:16.389+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:16.456 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:16.456 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:04:16.456 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:04:16.456 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:04:16.456 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.456 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226051 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6049792~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:16.456 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:04:16.456 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.138043009, 2024-05-11T23:04:16.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050622418000000002, 2024-05-11T23:04:16.457 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.457 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.457 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.457 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.457 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.457 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.457 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226051 2024-05-11T23:04:16.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.457 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.457 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.457 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.457 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:04:16.457 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317155+0000", 2024-05-11T23:04:16.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317156+0000", 2024-05-11T23:04:16.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:04:16.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.458 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.458 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317157+0000", 2024-05-11T23:04:16.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.144e-06 2024-05-11T23:04:16.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317160+0000", 2024-05-11T23:04:16.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1350000000000001e-06 2024-05-11T23:04:16.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317162+0000", 2024-05-11T23:04:16.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5050000000000002e-06 2024-05-11T23:04:16.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367689+0000", 2024-05-11T23:04:16.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526797999999998 2024-05-11T23:04:16.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367735+0000", 2024-05-11T23:04:16.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5698999999999997e-05 2024-05-11T23:04:16.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367778+0000", 2024-05-11T23:04:16.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2928e-05 2024-05-11T23:04:16.460 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.460 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.460 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.460 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226052 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6060032~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:16.461 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:04:16.461 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.138032953000007, 2024-05-11T23:04:16.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050830110999999997, 2024-05-11T23:04:16.461 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.461 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.461 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.461 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.461 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.461 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.461 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226052 2024-05-11T23:04:16.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.461 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:04:16.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317165+0000", 2024-05-11T23:04:16.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317166+0000", 2024-05-11T23:04:16.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0200000000000001e-07 2024-05-11T23:04:16.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317167+0000", 2024-05-11T23:04:16.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.155e-06 2024-05-11T23:04:16.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317170+0000", 2024-05-11T23:04:16.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.061e-06 2024-05-11T23:04:16.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317172+0000", 2024-05-11T23:04:16.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4420000000000001e-06 2024-05-11T23:04:16.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.463 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.463 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367816+0000", 2024-05-11T23:04:16.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050644013000000002 2024-05-11T23:04:16.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367906+0000", 2024-05-11T23:04:16.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9739000000000003e-05 2024-05-11T23:04:16.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.367995+0000", 2024-05-11T23:04:16.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9499000000000005e-05 2024-05-11T23:04:16.464 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.464 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.464 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.464 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226053 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6070272~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:16.465 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:04:16.465 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.138022914000004, 2024-05-11T23:04:16.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051106912999999997, 2024-05-11T23:04:16.465 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.465 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.465 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.465 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.465 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.465 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.465 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226053 2024-05-11T23:04:16.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.465 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:04:16.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317175+0000", 2024-05-11T23:04:16.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317176+0000", 2024-05-11T23:04:16.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0100000000000001e-07 2024-05-11T23:04:16.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317179+0000", 2024-05-11T23:04:16.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0489999999999999e-06 2024-05-11T23:04:16.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317182+0000", 2024-05-11T23:04:16.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3589999999999999e-06 2024-05-11T23:04:16.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317184+0000", 2024-05-11T23:04:16.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4339999999999999e-06 2024-05-11T23:04:16.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.467 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.467 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368061+0000", 2024-05-11T23:04:16.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050876927000000002 2024-05-11T23:04:16.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368179+0000", 2024-05-11T23:04:16.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000118001 2024-05-11T23:04:16.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368282+0000", 2024-05-11T23:04:16.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102942 2024-05-11T23:04:16.468 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.468 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.468 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.469 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226054 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6080512~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:16.469 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:04:16.469 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.138010592000001, 2024-05-11T23:04:16.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051322, 2024-05-11T23:04:16.469 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.469 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.469 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.469 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.469 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.469 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.469 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226054 2024-05-11T23:04:16.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.469 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:04:16.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:04:16.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317188+0000", 2024-05-11T23:04:16.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:04:16.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317189+0000", 2024-05-11T23:04:16.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.217e-06 2024-05-11T23:04:16.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317192+0000", 2024-05-11T23:04:16.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.0699999999999998e-06 2024-05-11T23:04:16.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317195+0000", 2024-05-11T23:04:16.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3829999999999999e-06 2024-05-11T23:04:16.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368328+0000", 2024-05-11T23:04:16.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051132999999999998 2024-05-11T23:04:16.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368429+0000", 2024-05-11T23:04:16.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010168800000000001 2024-05-11T23:04:16.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368510+0000", 2024-05-11T23:04:16.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0420999999999995e-05 2024-05-11T23:04:16.472 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.473 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.473 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.473 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226055 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6090752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:16.473 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:04:16.474 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.138000524000006, 2024-05-11T23:04:16.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051562835000000001, 2024-05-11T23:04:16.474 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.474 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.474 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.474 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.474 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.474 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.474 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226055 2024-05-11T23:04:16.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.474 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:04:16.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317198+0000", 2024-05-11T23:04:16.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317256+0000", 2024-05-11T23:04:16.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8047000000000002e-05 2024-05-11T23:04:16.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317259+0000", 2024-05-11T23:04:16.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2590000000000001e-06 2024-05-11T23:04:16.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317267+0000", 2024-05-11T23:04:16.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8769999999999993e-06 2024-05-11T23:04:16.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.476 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317274+0000", 2024-05-11T23:04:16.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.187e-06 2024-05-11T23:04:16.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.476 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368544+0000", 2024-05-11T23:04:16.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051270299999999998 2024-05-11T23:04:16.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368658+0000", 2024-05-11T23:04:16.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000113347 2024-05-11T23:04:16.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368761+0000", 2024-05-11T23:04:16.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000102818 2024-05-11T23:04:16.477 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.477 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.478 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.478 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226056 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6100992~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:16.478 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:04:16.478 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.137914678000001, 2024-05-11T23:04:16.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051615576000000003, 2024-05-11T23:04:16.478 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.478 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.478 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.478 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.478 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.478 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.478 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226056 2024-05-11T23:04:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:04:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.317284+0000", 2024-05-11T23:04:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318915+0000", 2024-05-11T23:04:16.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001631291 2024-05-11T23:04:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318918+0000", 2024-05-11T23:04:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.512e-06 2024-05-11T23:04:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318927+0000", 2024-05-11T23:04:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1960000000000008e-06 2024-05-11T23:04:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.318932+0000", 2024-05-11T23:04:16.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:04:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368805+0000", 2024-05-11T23:04:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049872998000000002 2024-05-11T23:04:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368853+0000", 2024-05-11T23:04:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8041999999999998e-05 2024-05-11T23:04:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.368899+0000", 2024-05-11T23:04:16.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5766000000000002e-05 2024-05-11T23:04:16.482 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.482 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.482 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.482 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226061 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6152192~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:16.482 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:04:16.482 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.136123591, 2024-05-11T23:04:16.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050675395999999998, 2024-05-11T23:04:16.482 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.482 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.482 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.482 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.482 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.482 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.483 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226061 2024-05-11T23:04:16.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.483 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:04:16.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:04:16.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319075+0000", 2024-05-11T23:04:16.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:04:16.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319078+0000", 2024-05-11T23:04:16.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5670000000000002e-06 2024-05-11T23:04:16.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319085+0000", 2024-05-11T23:04:16.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2570000000000001e-06 2024-05-11T23:04:16.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319100+0000", 2024-05-11T23:04:16.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.489e-05 2024-05-11T23:04:16.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369584+0000", 2024-05-11T23:04:16.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050484345999999999 2024-05-11T23:04:16.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369671+0000", 2024-05-11T23:04:16.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6654000000000001e-05 2024-05-11T23:04:16.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369750+0000", 2024-05-11T23:04:16.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9257000000000004e-05 2024-05-11T23:04:16.486 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.486 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.486 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.486 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226062 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6162432~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:16.486 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:04:16.486 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.136090046999996, 2024-05-11T23:04:16.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050761158000000001, 2024-05-11T23:04:16.486 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.486 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.487 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.487 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.487 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.487 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.487 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226062 2024-05-11T23:04:16.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.487 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:04:16.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319108+0000", 2024-05-11T23:04:16.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319109+0000", 2024-05-11T23:04:16.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2800000000000002e-07 2024-05-11T23:04:16.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319111+0000", 2024-05-11T23:04:16.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4980000000000001e-06 2024-05-11T23:04:16.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319120+0000", 2024-05-11T23:04:16.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3999999999999992e-06 2024-05-11T23:04:16.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319129+0000", 2024-05-11T23:04:16.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7580000000000004e-06 2024-05-11T23:04:16.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369775+0000", 2024-05-11T23:04:16.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050645580000000003 2024-05-11T23:04:16.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369824+0000", 2024-05-11T23:04:16.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9506999999999999e-05 2024-05-11T23:04:16.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369869+0000", 2024-05-11T23:04:16.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4987000000000001e-05 2024-05-11T23:04:16.490 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.490 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.490 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.490 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226063 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6172672~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:16.490 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:04:16.490 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.136061089999998, 2024-05-11T23:04:16.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050939784000000002, 2024-05-11T23:04:16.491 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.491 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.491 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.491 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.491 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.491 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.491 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226063 2024-05-11T23:04:16.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.491 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:04:16.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319137+0000", 2024-05-11T23:04:16.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319138+0000", 2024-05-11T23:04:16.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.39e-07 2024-05-11T23:04:16.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319140+0000", 2024-05-11T23:04:16.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4820000000000001e-06 2024-05-11T23:04:16.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319148+0000", 2024-05-11T23:04:16.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6599999999999995e-06 2024-05-11T23:04:16.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319154+0000", 2024-05-11T23:04:16.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.4169999999999997e-06 2024-05-11T23:04:16.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369905+0000", 2024-05-11T23:04:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050751009999999999 2024-05-11T23:04:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.369991+0000", 2024-05-11T23:04:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5909999999999996e-05 2024-05-11T23:04:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370077+0000", 2024-05-11T23:04:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5865999999999994e-05 2024-05-11T23:04:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.494 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226064 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6182912~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:04:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.136034894000005, 2024-05-11T23:04:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051038241999999998, 2024-05-11T23:04:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226064 2024-05-11T23:04:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.495 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:04:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319163+0000", 2024-05-11T23:04:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319164+0000", 2024-05-11T23:04:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9299999999999999e-07 2024-05-11T23:04:16.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319166+0000", 2024-05-11T23:04:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.48e-06 2024-05-11T23:04:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319176+0000", 2024-05-11T23:04:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9669999999999996e-06 2024-05-11T23:04:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319180+0000", 2024-05-11T23:04:16.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.8020000000000002e-06 2024-05-11T23:04:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370100+0000", 2024-05-11T23:04:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050919845999999998 2024-05-11T23:04:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370155+0000", 2024-05-11T23:04:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.5040000000000002e-05 2024-05-11T23:04:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370202+0000", 2024-05-11T23:04:16.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6714e-05 2024-05-11T23:04:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226065 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6193152~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:04:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.136012324000006, 2024-05-11T23:04:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.051225087000000002, 2024-05-11T23:04:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.499 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226065 2024-05-11T23:04:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:04:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.319186+0000", 2024-05-11T23:04:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320913+0000", 2024-05-11T23:04:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001727492 2024-05-11T23:04:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320919+0000", 2024-05-11T23:04:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.964e-06 2024-05-11T23:04:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320927+0000", 2024-05-11T23:04:16.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2239999999999998e-06 2024-05-11T23:04:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.320934+0000", 2024-05-11T23:04:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8469999999999998e-06 2024-05-11T23:04:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370240+0000", 2024-05-11T23:04:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.049306562999999998 2024-05-11T23:04:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370328+0000", 2024-05-11T23:04:16.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.8122999999999996e-05 2024-05-11T23:04:16.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.370411+0000", 2024-05-11T23:04:16.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2874000000000006e-05 2024-05-11T23:04:16.503 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.503 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.503 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.503 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226073 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 6275072~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:16.503 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:04:16.503 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.134173937, 2024-05-11T23:04:16.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050730475999999997, 2024-05-11T23:04:16.504 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.504 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.504 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.504 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.504 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.504 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226073 2024-05-11T23:04:16.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.504 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:04:16.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.321024+0000", 2024-05-11T23:04:16.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322914+0000", 2024-05-11T23:04:16.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001889153 2024-05-11T23:04:16.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.505 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322917+0000", 2024-05-11T23:04:16.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2100000000000002e-06 2024-05-11T23:04:16.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322924+0000", 2024-05-11T23:04:16.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9929999999999998e-06 2024-05-11T23:04:16.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.322928+0000", 2024-05-11T23:04:16.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5549999999999996e-06 2024-05-11T23:04:16.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371578+0000", 2024-05-11T23:04:16.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048650001999999998 2024-05-11T23:04:16.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371671+0000", 2024-05-11T23:04:16.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2230000000000003e-05 2024-05-11T23:04:16.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.371755+0000", 2024-05-11T23:04:16.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4332999999999994e-05 2024-05-11T23:04:16.507 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.507 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.507 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.508 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:226085 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 5338112~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:16.508 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:04:16.508 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 74.132082838000002, 2024-05-11T23:04:16.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050727622, 2024-05-11T23:04:16.508 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.508 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.508 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.508 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.508 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.508 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 226085 2024-05-11T23:04:16.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.508 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:04:16.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.323115+0000", 2024-05-11T23:04:16.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.509 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324914+0000", 2024-05-11T23:04:16.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001798298 2024-05-11T23:04:16.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.509 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324918+0000", 2024-05-11T23:04:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3889999999999998e-06 2024-05-11T23:04:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324928+0000", 2024-05-11T23:04:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4339999999999998e-06 2024-05-11T23:04:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.324936+0000", 2024-05-11T23:04:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2770000000000005e-06 2024-05-11T23:04:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373664+0000", 2024-05-11T23:04:16.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048728207000000003 2024-05-11T23:04:16.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373757+0000", 2024-05-11T23:04:16.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3332999999999996e-05 2024-05-11T23:04:16.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.373843+0000", 2024-05-11T23:04:16.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5684000000000002e-05 2024-05-11T23:04:16.511 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.511 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.511 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.512 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232555 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7789568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:16.512 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:04:16.512 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 73.655072512999993, 2024-05-11T23:04:16.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050583086999999999, 2024-05-11T23:04:16.512 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.512 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.512 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.512 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.512 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.512 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.512 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232555 2024-05-11T23:04:16.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.512 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:04:16.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:04:16.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.513 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.513 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800126+0000", 2024-05-11T23:04:16.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1299999999999999e-07 2024-05-11T23:04:16.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800127+0000", 2024-05-11T23:04:16.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.207e-06 2024-05-11T23:04:16.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800132+0000", 2024-05-11T23:04:16.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:04:16.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800134+0000", 2024-05-11T23:04:16.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.5459999999999998e-06 2024-05-11T23:04:16.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850548+0000", 2024-05-11T23:04:16.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050414182000000002 2024-05-11T23:04:16.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850635+0000", 2024-05-11T23:04:16.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6853000000000003e-05 2024-05-11T23:04:16.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850709+0000", 2024-05-11T23:04:16.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3585999999999996e-05 2024-05-11T23:04:16.516 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.516 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.516 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.516 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232556 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 7799808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:16.516 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:04:16.516 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 73.655060859000002, 2024-05-11T23:04:16.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050749599999999999, 2024-05-11T23:04:16.516 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.516 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.516 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.516 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.516 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.516 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.517 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232556 2024-05-11T23:04:16.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.517 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:04:16.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.800137+0000", 2024-05-11T23:04:16.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.517 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801914+0000", 2024-05-11T23:04:16.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017766360000000001 2024-05-11T23:04:16.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801919+0000", 2024-05-11T23:04:16.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.42e-06 2024-05-11T23:04:16.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801927+0000", 2024-05-11T23:04:16.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1999999999999994e-06 2024-05-11T23:04:16.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.801931+0000", 2024-05-11T23:04:16.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4279999999999998e-06 2024-05-11T23:04:16.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850740+0000", 2024-05-11T23:04:16.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048808658999999997 2024-05-11T23:04:16.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850819+0000", 2024-05-11T23:04:16.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9102000000000003e-05 2024-05-11T23:04:16.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.850887+0000", 2024-05-11T23:04:16.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8155000000000001e-05 2024-05-11T23:04:16.520 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.520 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.520 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.520 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232578 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8025088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:16.520 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:04:16.520 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 73.651767621000005, 2024-05-11T23:04:16.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050679473000000003, 2024-05-11T23:04:16.520 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.520 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.520 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.521 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.521 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.521 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.521 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232578 2024-05-11T23:04:16.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.521 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:04:16.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.521 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:04:16.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803431+0000", 2024-05-11T23:04:16.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2100000000000001e-07 2024-05-11T23:04:16.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803432+0000", 2024-05-11T23:04:16.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.139e-06 2024-05-11T23:04:16.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803435+0000", 2024-05-11T23:04:16.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2440000000000002e-06 2024-05-11T23:04:16.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803439+0000", 2024-05-11T23:04:16.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.3129999999999999e-06 2024-05-11T23:04:16.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.853953+0000", 2024-05-11T23:04:16.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050514846000000002 2024-05-11T23:04:16.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854035+0000", 2024-05-11T23:04:16.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1148000000000003e-05 2024-05-11T23:04:16.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854110+0000", 2024-05-11T23:04:16.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5562000000000006e-05 2024-05-11T23:04:16.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.524 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.524 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232579 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8035328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:16.524 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:04:16.524 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 73.651755016999999, 2024-05-11T23:04:16.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050886656000000002, 2024-05-11T23:04:16.524 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.525 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.525 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.525 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.525 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.525 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.525 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232579 2024-05-11T23:04:16.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.525 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:04:16.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803443+0000", 2024-05-11T23:04:16.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803914+0000", 2024-05-11T23:04:16.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00047026700000000002 2024-05-11T23:04:16.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803918+0000", 2024-05-11T23:04:16.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3460000000000004e-06 2024-05-11T23:04:16.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803927+0000", 2024-05-11T23:04:16.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.5030000000000003e-06 2024-05-11T23:04:16.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803933+0000", 2024-05-11T23:04:16.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0310000000000004e-06 2024-05-11T23:04:16.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854153+0000", 2024-05-11T23:04:16.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050219321999999997 2024-05-11T23:04:16.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854242+0000", 2024-05-11T23:04:16.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.9457999999999996e-05 2024-05-11T23:04:16.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854330+0000", 2024-05-11T23:04:16.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7729e-05 2024-05-11T23:04:16.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.528 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.528 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232581 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8055808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:16.528 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:04:16.528 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 73.651231539999998, 2024-05-11T23:04:16.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050664439999999998, 2024-05-11T23:04:16.529 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.529 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.529 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.529 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232581 2024-05-11T23:04:16.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.529 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:04:16.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:04:16.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803967+0000", 2024-05-11T23:04:16.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:04:16.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803969+0000", 2024-05-11T23:04:16.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0889999999999998e-06 2024-05-11T23:04:16.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803978+0000", 2024-05-11T23:04:16.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3070000000000003e-06 2024-05-11T23:04:16.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803983+0000", 2024-05-11T23:04:16.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1599999999999997e-06 2024-05-11T23:04:16.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854509+0000", 2024-05-11T23:04:16.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050526339000000003 2024-05-11T23:04:16.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854576+0000", 2024-05-11T23:04:16.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7102999999999998e-05 2024-05-11T23:04:16.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854631+0000", 2024-05-11T23:04:16.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.4997999999999998e-05 2024-05-11T23:04:16.532 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.532 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.532 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.532 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232582 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8066048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:16.533 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:04:16.533 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 73.651208795000002, 2024-05-11T23:04:16.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050753983000000003, 2024-05-11T23:04:16.533 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.533 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.533 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.533 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.533 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.533 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232582 2024-05-11T23:04:16.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.533 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:04:16.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.803990+0000", 2024-05-11T23:04:16.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805917+0000", 2024-05-11T23:04:16.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001927432 2024-05-11T23:04:16.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805920+0000", 2024-05-11T23:04:16.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.1219999999999999e-06 2024-05-11T23:04:16.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805933+0000", 2024-05-11T23:04:16.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2727e-05 2024-05-11T23:04:16.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.805941+0000", 2024-05-11T23:04:16.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7090000000000002e-06 2024-05-11T23:04:16.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854657+0000", 2024-05-11T23:04:16.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048716164999999999 2024-05-11T23:04:16.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854699+0000", 2024-05-11T23:04:16.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2531999999999999e-05 2024-05-11T23:04:16.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.854744+0000", 2024-05-11T23:04:16.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4295999999999997e-05 2024-05-11T23:04:16.536 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.536 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.536 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.537 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:232593 3.1 3:80b65986:::rbd_data.111ebf0f641b.0000000000000005:2e [read 8178688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102)", 2024-05-11T23:04:16.537 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:04:16.537 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 73.649009402000004, 2024-05-11T23:04:16.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.050735178999999998, 2024-05-11T23:04:16.537 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.537 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.537 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.537 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.537 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.537 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.537 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 232593 2024-05-11T23:04:16.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.537 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:04:16.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.806189+0000", 2024-05-11T23:04:16.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807913+0000", 2024-05-11T23:04:16.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0017240319999999999 2024-05-11T23:04:16.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807916+0000", 2024-05-11T23:04:16.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4589999999999998e-06 2024-05-11T23:04:16.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807924+0000", 2024-05-11T23:04:16.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.272e-06 2024-05-11T23:04:16.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.807928+0000", 2024-05-11T23:04:16.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5369999999999999e-06 2024-05-11T23:04:16.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856768+0000", 2024-05-11T23:04:16.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.048839489999999999 2024-05-11T23:04:16.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856845+0000", 2024-05-11T23:04:16.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7746999999999999e-05 2024-05-11T23:04:16.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:02.856924+0000", 2024-05-11T23:04:16.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8641999999999997e-05 2024-05-11T23:04:16.540 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.541 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.541 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.541 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:16.542 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-11T23:04:16.543 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:16.542+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:16.643 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:16.643+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:16.693 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:16.693 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:04:16.693 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:04:16.693 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:16.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-11T23:04:16.743 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:16.744+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:16.844 INFO:tasks.ceph.osd.6.smithi115.stderr:2024-05-11T23:04:16.844+0000 7fba06079640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33177) UID: 0 2024-05-11T23:04:16.852 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:16.852 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:04:16.852 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:04:16.852 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:04:16.852 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:16.861 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-11T23:04:16.930 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-11T23:04:16.945 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:16.945+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:16.983 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:16.983 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:04:16.983 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:04:16.983 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:04:16.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.983 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:64049 3.8 3:11d98727:::rbd_object_map.111e8974a681:head [read 0~38] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-05-11T23:04:16.983 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:04:16.983 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 136.202342324, 2024-05-11T23:04:16.984 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99884604399999999, 2024-05-11T23:04:16.984 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.984 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.984 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:16.984 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.984 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.984 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.984 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 64049 2024-05-11T23:04:16.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.984 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.984 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.984 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:04:16.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779881+0000", 2024-05-11T23:04:16.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779883+0000", 2024-05-11T23:04:16.985 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.57e-06 2024-05-11T23:04:16.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.985 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.985 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.985 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779887+0000", 2024-05-11T23:04:16.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5709999999999999e-06 2024-05-11T23:04:16.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779898+0000", 2024-05-11T23:04:16.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.042e-05 2024-05-11T23:04:16.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.986 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779905+0000", 2024-05-11T23:04:16.986 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0110000000000003e-06 2024-05-11T23:04:16.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.986 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779962+0000", 2024-05-11T23:04:16.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7395999999999997e-05 2024-05-11T23:04:16.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:04:16.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:00.779969+0000", 2024-05-11T23:04:16.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2459999999999997e-06 2024-05-11T23:04:16.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.987 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778379+0000", 2024-05-11T23:04:16.987 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.99840986300000001 2024-05-11T23:04:16.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.987 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.987 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:16.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778564+0000", 2024-05-11T23:04:16.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00018420900000000001 2024-05-11T23:04:16.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.988 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.988 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:01.778727+0000", 2024-05-11T23:04:16.988 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016375800000000001 2024-05-11T23:04:16.988 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.988 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.988 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.988 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172773 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:16.988 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:04:16.988 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.378235906, 2024-05-11T23:04:16.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058721316000000003, 2024-05-11T23:04:16.989 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.989 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.989 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:16.989 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.989 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.989 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.989 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172773 2024-05-11T23:04:16.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.989 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.989 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.989 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:04:16.989 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:04:16.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603988+0000", 2024-05-11T23:04:16.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.3500000000000002e-07 2024-05-11T23:04:16.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.990 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.990 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603990+0000", 2024-05-11T23:04:16.990 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3549999999999999e-06 2024-05-11T23:04:16.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.603997+0000", 2024-05-11T23:04:16.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.3400000000000003e-06 2024-05-11T23:04:16.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604002+0000", 2024-05-11T23:04:16.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0509999999999996e-06 2024-05-11T23:04:16.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.991 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.991 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662697+0000", 2024-05-11T23:04:16.991 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058695487999999997 2024-05-11T23:04:16.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.992 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.992 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662709+0000", 2024-05-11T23:04:16.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1647e-05 2024-05-11T23:04:16.992 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.992 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.992 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.992 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.992 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172774 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:16.992 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:04:16.992 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.378214524000001, 2024-05-11T23:04:16.992 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058745040999999998, 2024-05-11T23:04:16.992 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.992 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.993 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:16.993 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.993 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.993 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.993 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172774 2024-05-11T23:04:16.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.993 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:04:16.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.993 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.993 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.993 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:04:16.993 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604009+0000", 2024-05-11T23:04:16.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:04:16.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604012+0000", 2024-05-11T23:04:16.994 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2199999999999999e-06 2024-05-11T23:04:16.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.994 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.994 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604018+0000", 2024-05-11T23:04:16.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1770000000000003e-06 2024-05-11T23:04:16.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604023+0000", 2024-05-11T23:04:16.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0089999999999998e-06 2024-05-11T23:04:16.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662742+0000", 2024-05-11T23:04:16.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058719187999999999 2024-05-11T23:04:16.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.995 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.995 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662754+0000", 2024-05-11T23:04:16.995 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2095e-05 2024-05-11T23:04:16.996 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.996 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:16.996 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:16.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.996 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172775 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:16.996 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:04:16.996 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.378194312000005, 2024-05-11T23:04:16.996 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058764743000000001, 2024-05-11T23:04:16.996 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:16.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:16.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:16.996 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:16.996 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:16.997 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:16.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172775 2024-05-11T23:04:16.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.997 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:16.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:16.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:04:16.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.997 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:16.997 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604029+0000", 2024-05-11T23:04:16.997 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:16.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:16.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604030+0000", 2024-05-11T23:04:16.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6699999999999999e-07 2024-05-11T23:04:16.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:16.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604032+0000", 2024-05-11T23:04:16.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.9530000000000002e-06 2024-05-11T23:04:16.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.998 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.998 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:16.998 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604037+0000", 2024-05-11T23:04:16.998 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8590000000000001e-06 2024-05-11T23:04:16.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:16.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604042+0000", 2024-05-11T23:04:16.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8609999999999997e-06 2024-05-11T23:04:16.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:16.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662782+0000", 2024-05-11T23:04:16.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058739731000000003 2024-05-11T23:04:16.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:16.999 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:16.999 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:16.999 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662794+0000", 2024-05-11T23:04:16.999 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1972e-05 2024-05-11T23:04:17.000 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.000 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.000 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.000 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.000 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172776 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:17.000 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:04:17.000 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.378175021999994, 2024-05-11T23:04:17.000 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058778696999999998, 2024-05-11T23:04:17.000 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.000 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.000 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:17.000 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.000 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.000 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172776 2024-05-11T23:04:17.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.001 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:04:17.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.001 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:04:17.001 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.001 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604049+0000", 2024-05-11T23:04:17.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.2000000000000001e-07 2024-05-11T23:04:17.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604051+0000", 2024-05-11T23:04:17.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2879999999999999e-06 2024-05-11T23:04:17.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.002 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604057+0000", 2024-05-11T23:04:17.002 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7710000000000003e-06 2024-05-11T23:04:17.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.002 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604062+0000", 2024-05-11T23:04:17.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6800000000000001e-06 2024-05-11T23:04:17.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662820+0000", 2024-05-11T23:04:17.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058758444999999999 2024-05-11T23:04:17.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.003 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.003 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662827+0000", 2024-05-11T23:04:17.003 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1930000000000004e-06 2024-05-11T23:04:17.003 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.003 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.003 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.004 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.004 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172777 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:17.004 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:04:17.004 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.378155715999995, 2024-05-11T23:04:17.004 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058799918999999999, 2024-05-11T23:04:17.004 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.004 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.004 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:17.004 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.004 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.004 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.004 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172777 2024-05-11T23:04:17.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.005 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:04:17.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:04:17.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.005 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.005 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.005 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604068+0000", 2024-05-11T23:04:17.005 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.5199999999999998e-07 2024-05-11T23:04:17.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604075+0000", 2024-05-11T23:04:17.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.109e-06 2024-05-11T23:04:17.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604083+0000", 2024-05-11T23:04:17.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9170000000000006e-06 2024-05-11T23:04:17.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.006 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.006 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.006 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604089+0000", 2024-05-11T23:04:17.006 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.3399999999999997e-06 2024-05-11T23:04:17.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.007 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.007 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662859+0000", 2024-05-11T23:04:17.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058770172000000002 2024-05-11T23:04:17.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.007 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.007 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662868+0000", 2024-05-11T23:04:17.007 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0289999999999994e-06 2024-05-11T23:04:17.007 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.007 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.007 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.007 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.008 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172778 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:17.008 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:04:17.008 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.378127625999994, 2024-05-11T23:04:17.008 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058845435000000001, 2024-05-11T23:04:17.008 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.008 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.008 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:17.008 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.008 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.008 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.008 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172778 2024-05-11T23:04:17.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.008 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.008 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:04:17.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:04:17.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.009 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604096+0000", 2024-05-11T23:04:17.009 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5400000000000002e-07 2024-05-11T23:04:17.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.009 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604099+0000", 2024-05-11T23:04:17.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4650000000000001e-06 2024-05-11T23:04:17.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604105+0000", 2024-05-11T23:04:17.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.314e-06 2024-05-11T23:04:17.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.010 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.010 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.010 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604110+0000", 2024-05-11T23:04:17.010 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6310000000000002e-06 2024-05-11T23:04:17.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.013 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662931+0000", 2024-05-11T23:04:17.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058820960999999998 2024-05-11T23:04:17.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.014 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.014 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662941+0000", 2024-05-11T23:04:17.014 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.061e-05 2024-05-11T23:04:17.014 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.014 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.014 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.014 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172779 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:17.014 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:04:17.015 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.378107466000003, 2024-05-11T23:04:17.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058859163999999999, 2024-05-11T23:04:17.015 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.015 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:17.015 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.015 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.015 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.015 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172779 2024-05-11T23:04:17.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.015 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.015 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.015 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.015 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:04:17.015 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:04:17.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604116+0000", 2024-05-11T23:04:17.016 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9999999999999998e-07 2024-05-11T23:04:17.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.016 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.016 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604119+0000", 2024-05-11T23:04:17.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2529999999999999e-06 2024-05-11T23:04:17.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604125+0000", 2024-05-11T23:04:17.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1670000000000004e-06 2024-05-11T23:04:17.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604130+0000", 2024-05-11T23:04:17.017 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.702e-06 2024-05-11T23:04:17.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.017 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.017 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662967+0000", 2024-05-11T23:04:17.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058837792999999999 2024-05-11T23:04:17.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.018 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.018 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.662975+0000", 2024-05-11T23:04:17.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8490000000000006e-06 2024-05-11T23:04:17.018 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.018 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.018 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.018 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.018 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172780 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:17.018 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:04:17.018 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.378087316000006, 2024-05-11T23:04:17.018 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058872696000000002, 2024-05-11T23:04:17.019 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.019 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.019 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:17.019 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.019 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.019 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.019 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172780 2024-05-11T23:04:17.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.019 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.019 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.019 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:04:17.019 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604136+0000", 2024-05-11T23:04:17.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604137+0000", 2024-05-11T23:04:17.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4400000000000001e-07 2024-05-11T23:04:17.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.020 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.020 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.020 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604139+0000", 2024-05-11T23:04:17.020 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3429999999999998e-06 2024-05-11T23:04:17.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604145+0000", 2024-05-11T23:04:17.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1940000000000003e-06 2024-05-11T23:04:17.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604150+0000", 2024-05-11T23:04:17.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7870000000000001e-06 2024-05-11T23:04:17.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.021 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.021 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663002+0000", 2024-05-11T23:04:17.021 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058851829000000001 2024-05-11T23:04:17.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.022 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.022 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663009+0000", 2024-05-11T23:04:17.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.199e-06 2024-05-11T23:04:17.022 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.022 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.022 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.022 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172781 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:17.022 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:04:17.022 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.378055476, 2024-05-11T23:04:17.022 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058875553999999997, 2024-05-11T23:04:17.022 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.023 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.023 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:17.023 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.023 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.023 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.023 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172781 2024-05-11T23:04:17.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.023 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.023 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:04:17.023 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.023 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.023 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:04:17.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604168+0000", 2024-05-11T23:04:17.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:04:17.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.024 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604171+0000", 2024-05-11T23:04:17.024 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.21e-06 2024-05-11T23:04:17.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.024 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.024 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604176+0000", 2024-05-11T23:04:17.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2970000000000003e-06 2024-05-11T23:04:17.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604179+0000", 2024-05-11T23:04:17.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8459999999999999e-06 2024-05-11T23:04:17.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.025 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.025 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663033+0000", 2024-05-11T23:04:17.025 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058854565999999997 2024-05-11T23:04:17.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.026 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.026 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663044+0000", 2024-05-11T23:04:17.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.0229e-05 2024-05-11T23:04:17.026 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.026 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.026 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.026 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.026 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172782 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:17.026 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:04:17.026 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.378040728000002, 2024-05-11T23:04:17.026 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058894375999999998, 2024-05-11T23:04:17.026 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.026 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.027 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:17.027 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.027 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.027 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.027 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172782 2024-05-11T23:04:17.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.027 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.027 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:04:17.027 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.027 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:04:17.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604183+0000", 2024-05-11T23:04:17.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.11e-07 2024-05-11T23:04:17.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.028 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.028 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604184+0000", 2024-05-11T23:04:17.028 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2979999999999999e-06 2024-05-11T23:04:17.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.028 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604187+0000", 2024-05-11T23:04:17.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.044e-06 2024-05-11T23:04:17.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604190+0000", 2024-05-11T23:04:17.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6460000000000002e-06 2024-05-11T23:04:17.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.029 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.029 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663070+0000", 2024-05-11T23:04:17.029 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058879510000000003 2024-05-11T23:04:17.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.029 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.030 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.030 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663077+0000", 2024-05-11T23:04:17.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6669999999999996e-06 2024-05-11T23:04:17.030 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.030 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.030 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.030 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172783 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:17.030 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:04:17.030 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.378030409000004, 2024-05-11T23:04:17.030 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058913946000000002, 2024-05-11T23:04:17.030 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.030 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.030 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:17.031 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.031 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.031 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.031 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172783 2024-05-11T23:04:17.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.031 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:04:17.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.031 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.031 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:04:17.031 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604193+0000", 2024-05-11T23:04:17.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8000000000000002e-07 2024-05-11T23:04:17.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604195+0000", 2024-05-11T23:04:17.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.19e-06 2024-05-11T23:04:17.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.032 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.032 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604198+0000", 2024-05-11T23:04:17.032 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:04:17.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604200+0000", 2024-05-11T23:04:17.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.4719999999999998e-06 2024-05-11T23:04:17.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663100+0000", 2024-05-11T23:04:17.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058900300000000003 2024-05-11T23:04:17.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.033 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.033 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663107+0000", 2024-05-11T23:04:17.033 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7449999999999998e-06 2024-05-11T23:04:17.034 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.034 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.034 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.034 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.034 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172784 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:17.034 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:04:17.034 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.378020390000003, 2024-05-11T23:04:17.034 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933728999999997, 2024-05-11T23:04:17.034 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.034 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.034 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:17.034 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.034 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.035 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.035 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172784 2024-05-11T23:04:17.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.035 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:04:17.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.035 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:04:17.035 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.035 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.035 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604203+0000", 2024-05-11T23:04:17.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.03e-07 2024-05-11T23:04:17.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604205+0000", 2024-05-11T23:04:17.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2300000000000001e-06 2024-05-11T23:04:17.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.036 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604207+0000", 2024-05-11T23:04:17.036 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8729999999999999e-06 2024-05-11T23:04:17.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.036 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.036 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604210+0000", 2024-05-11T23:04:17.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.858e-06 2024-05-11T23:04:17.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663130+0000", 2024-05-11T23:04:17.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058919314 2024-05-11T23:04:17.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.037 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.037 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.037 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663137+0000", 2024-05-11T23:04:17.037 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2509999999999997e-06 2024-05-11T23:04:17.037 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.037 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.038 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.038 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.038 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172785 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:17.038 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:04:17.038 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.378008120000004, 2024-05-11T23:04:17.038 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058951664000000001, 2024-05-11T23:04:17.038 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.038 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.038 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:17.038 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.038 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.038 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.038 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172785 2024-05-11T23:04:17.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.039 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:04:17.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604215+0000", 2024-05-11T23:04:17.039 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.039 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.039 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604216+0000", 2024-05-11T23:04:17.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:04:17.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604218+0000", 2024-05-11T23:04:17.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7549999999999999e-06 2024-05-11T23:04:17.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604222+0000", 2024-05-11T23:04:17.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6700000000000002e-06 2024-05-11T23:04:17.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.040 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.040 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604227+0000", 2024-05-11T23:04:17.040 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.4399999999999998e-06 2024-05-11T23:04:17.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663160+0000", 2024-05-11T23:04:17.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058933119999999999 2024-05-11T23:04:17.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.041 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.041 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663167+0000", 2024-05-11T23:04:17.041 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1840000000000002e-06 2024-05-11T23:04:17.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.041 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.041 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.042 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172786 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:17.042 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:04:17.042 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.377991066000007, 2024-05-11T23:04:17.042 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058966991000000003, 2024-05-11T23:04:17.042 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.042 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.042 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:17.042 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.042 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.042 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.042 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172786 2024-05-11T23:04:17.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.042 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:04:17.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604232+0000", 2024-05-11T23:04:17.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.043 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.043 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604233+0000", 2024-05-11T23:04:17.043 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6899999999999999e-07 2024-05-11T23:04:17.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604235+0000", 2024-05-11T23:04:17.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8369999999999999e-06 2024-05-11T23:04:17.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604240+0000", 2024-05-11T23:04:17.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0329999999999999e-06 2024-05-11T23:04:17.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.044 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.044 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.044 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604244+0000", 2024-05-11T23:04:17.044 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.9859999999999996e-06 2024-05-11T23:04:17.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663192+0000", 2024-05-11T23:04:17.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058948852000000003 2024-05-11T23:04:17.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.045 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.045 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663199+0000", 2024-05-11T23:04:17.045 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0140000000000001e-06 2024-05-11T23:04:17.045 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.045 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.045 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.045 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.046 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172787 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:17.046 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:04:17.046 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.377974719999997, 2024-05-11T23:04:17.046 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.058982018999999997, 2024-05-11T23:04:17.046 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.046 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.046 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:17.046 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.046 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.046 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.046 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172787 2024-05-11T23:04:17.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.046 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.046 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:04:17.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.604249+0000", 2024-05-11T23:04:17.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.047 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.047 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605916+0000", 2024-05-11T23:04:17.047 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00166752 2024-05-11T23:04:17.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605921+0000", 2024-05-11T23:04:17.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.0669999999999999e-06 2024-05-11T23:04:17.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605934+0000", 2024-05-11T23:04:17.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2271e-05 2024-05-11T23:04:17.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.048 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.048 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.605940+0000", 2024-05-11T23:04:17.048 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6880000000000002e-06 2024-05-11T23:04:17.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663224+0000", 2024-05-11T23:04:17.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057283251 2024-05-11T23:04:17.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.049 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.049 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663231+0000", 2024-05-11T23:04:17.049 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2219999999999996e-06 2024-05-11T23:04:17.049 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.049 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.049 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.050 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172793 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:17.050 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:04:17.050 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.376207872999998, 2024-05-11T23:04:17.050 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057402373999999999, 2024-05-11T23:04:17.050 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.050 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.050 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:17.050 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.050 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.050 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.050 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172793 2024-05-11T23:04:17.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.050 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:04:17.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:04:17.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.051 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.051 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.051 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606016+0000", 2024-05-11T23:04:17.051 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3300000000000001e-07 2024-05-11T23:04:17.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606017+0000", 2024-05-11T23:04:17.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4920000000000001e-06 2024-05-11T23:04:17.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606020+0000", 2024-05-11T23:04:17.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.959e-06 2024-05-11T23:04:17.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.052 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.052 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.052 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606023+0000", 2024-05-11T23:04:17.052 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6680000000000001e-06 2024-05-11T23:04:17.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663411+0000", 2024-05-11T23:04:17.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057388206999999997 2024-05-11T23:04:17.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.053 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.053 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663418+0000", 2024-05-11T23:04:17.053 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.815e-06 2024-05-11T23:04:17.053 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.053 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.053 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.053 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172795 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:17.054 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:04:17.054 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.376166445999999, 2024-05-11T23:04:17.054 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057423015000000001, 2024-05-11T23:04:17.054 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.054 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.054 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:17.054 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.054 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.054 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.054 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172795 2024-05-11T23:04:17.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.054 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.054 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:04:17.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:04:17.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.055 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.055 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606057+0000", 2024-05-11T23:04:17.055 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.9499999999999998e-07 2024-05-11T23:04:17.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.055 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606059+0000", 2024-05-11T23:04:17.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2720000000000001e-06 2024-05-11T23:04:17.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606062+0000", 2024-05-11T23:04:17.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.977e-06 2024-05-11T23:04:17.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.056 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.056 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606064+0000", 2024-05-11T23:04:17.056 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.7410000000000001e-06 2024-05-11T23:04:17.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663473+0000", 2024-05-11T23:04:17.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057408624999999998 2024-05-11T23:04:17.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.057 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.057 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663480+0000", 2024-05-11T23:04:17.057 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1049999999999997e-06 2024-05-11T23:04:17.057 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.057 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.057 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.057 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.057 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172796 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:17.057 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:04:17.057 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.376155882999996, 2024-05-11T23:04:17.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057449235000000001, 2024-05-11T23:04:17.058 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.058 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:17.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.058 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172796 2024-05-11T23:04:17.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.058 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.058 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.058 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:04:17.058 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:04:17.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606068+0000", 2024-05-11T23:04:17.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0900000000000001e-07 2024-05-11T23:04:17.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.059 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.059 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.059 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606069+0000", 2024-05-11T23:04:17.059 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.1200000000000001e-06 2024-05-11T23:04:17.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606072+0000", 2024-05-11T23:04:17.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8700000000000001e-06 2024-05-11T23:04:17.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606074+0000", 2024-05-11T23:04:17.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6230000000000001e-06 2024-05-11T23:04:17.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.060 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.060 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663510+0000", 2024-05-11T23:04:17.060 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.05743529 2024-05-11T23:04:17.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.061 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.061 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663517+0000", 2024-05-11T23:04:17.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1230000000000002e-06 2024-05-11T23:04:17.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.061 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.061 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:172797 3.4 3.cba19fa4 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-05-11T23:04:17.061 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:04:17.061 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 78.376145851000004, 2024-05-11T23:04:17.061 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.057470420000000001, 2024-05-11T23:04:17.061 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.061 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.062 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:17.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.062 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.062 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 172797 2024-05-11T23:04:17.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.062 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:04:17.062 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.062 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.062 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.606078+0000", 2024-05-11T23:04:17.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607915+0000", 2024-05-11T23:04:17.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0018375609999999999 2024-05-11T23:04:17.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607920+0000", 2024-05-11T23:04:17.063 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8899999999999998e-06 2024-05-11T23:04:17.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.063 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.063 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.063 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607930+0000", 2024-05-11T23:04:17.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.4429999999999992e-06 2024-05-11T23:04:17.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.607936+0000", 2024-05-11T23:04:17.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7089999999999996e-06 2024-05-11T23:04:17.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663541+0000", 2024-05-11T23:04:17.064 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.055604537000000002 2024-05-11T23:04:17.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.064 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.064 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.064 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:58.663548+0000", 2024-05-11T23:04:17.065 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2799999999999998e-06 2024-05-11T23:04:17.065 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.065 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.065 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.065 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.065 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.065 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:17.066 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-11T23:04:17.067 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:17.066+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:17.167 INFO:tasks.ceph.osd.5.smithi115.stderr:2024-05-11T23:04:17.167+0000 7f8006387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33179) UID: 0 2024-05-11T23:04:17.177 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:17.177 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:04:17.177 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:04:17.177 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:17.186 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-11T23:04:17.268 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:17.268+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:17.297 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:17.297 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:04:17.297 INFO:teuthology.orchestra.run.smithi005.stdout: "complaint_time": 30, 2024-05-11T23:04:17.297 INFO:teuthology.orchestra.run.smithi005.stdout: "num_blocked_ops": 0 2024-05-11T23:04:17.297 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:17.306 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-11T23:04:17.369 INFO:tasks.ceph.osd.4.smithi115.stderr:2024-05-11T23:04:17.369+0000 7f8370aaa640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33172) UID: 0 2024-05-11T23:04:17.458 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:17.458 INFO:teuthology.orchestra.run.smithi005.stdout: "size": 20, 2024-05-11T23:04:17.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 600, 2024-05-11T23:04:17.458 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [ 2024-05-11T23:04:17.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.458 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:77128 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_get in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-11T23:04:17.458 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:04:17.458 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 122.44107719599999, 2024-05-11T23:04:17.458 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91737855899999998, 2024-05-11T23:04:17.458 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.458 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.459 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:17.459 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.459 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.459 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.459 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 77128 2024-05-11T23:04:17.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.459 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:04:17.459 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.459 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.459 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.459 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016002+0000", 2024-05-11T23:04:17.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016003+0000", 2024-05-11T23:04:17.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.5379999999999999e-06 2024-05-11T23:04:17.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.460 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016011+0000", 2024-05-11T23:04:17.460 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6410000000000002e-06 2024-05-11T23:04:17.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.460 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016026+0000", 2024-05-11T23:04:17.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.4779e-05 2024-05-11T23:04:17.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016032+0000", 2024-05-11T23:04:17.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.066e-06 2024-05-11T23:04:17.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016093+0000", 2024-05-11T23:04:17.461 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0878000000000002e-05 2024-05-11T23:04:17.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.461 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.461 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:04:17.461 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.016100+0000", 2024-05-11T23:04:17.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6799999999999993e-06 2024-05-11T23:04:17.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933302+0000", 2024-05-11T23:04:17.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.91720135300000005 2024-05-11T23:04:17.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.462 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.462 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:15.933380+0000", 2024-05-11T23:04:17.462 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8623999999999998e-05 2024-05-11T23:04:17.462 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.462 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.462 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.463 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98950 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 4161536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:04:17.463 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:04:17.463 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.32499630700001, 2024-05-11T23:04:17.463 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.021419438999999998, 2024-05-11T23:04:17.463 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.463 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.463 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:17.463 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.463 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.463 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.463 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98950 2024-05-11T23:04:17.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.463 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:04:17.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:04:17.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.464 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.464 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132083+0000", 2024-05-11T23:04:17.464 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.34e-07 2024-05-11T23:04:17.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.464 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132085+0000", 2024-05-11T23:04:17.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0049999999999999e-06 2024-05-11T23:04:17.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132093+0000", 2024-05-11T23:04:17.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6660000000000008e-06 2024-05-11T23:04:17.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.465 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.465 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132097+0000", 2024-05-11T23:04:17.465 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2590000000000002e-06 2024-05-11T23:04:17.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135114+0000", 2024-05-11T23:04:17.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.003017573 2024-05-11T23:04:17.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:17.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153416+0000", 2024-05-11T23:04:17.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018301884000000001 2024-05-11T23:04:17.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.466 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.466 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153502+0000", 2024-05-11T23:04:17.466 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5717999999999995e-05 2024-05-11T23:04:17.466 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.467 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.467 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.467 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.467 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98959 3.1e 3:7807da99:::rbd_data.111ed2934718.000000000000006d:head [stat,delete] snapc 24=[24] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:04:17.467 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:04:17.467 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.32450357, 2024-05-11T23:04:17.467 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020525928999999998, 2024-05-11T23:04:17.467 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.467 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.467 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:17.467 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.467 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.467 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.468 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98959 2024-05-11T23:04:17.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.468 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.468 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:04:17.468 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.468 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132575+0000", 2024-05-11T23:04:17.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132576+0000", 2024-05-11T23:04:17.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4199999999999995e-07 2024-05-11T23:04:17.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.469 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.469 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132580+0000", 2024-05-11T23:04:17.469 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2910000000000001e-06 2024-05-11T23:04:17.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.469 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132588+0000", 2024-05-11T23:04:17.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7970000000000001e-06 2024-05-11T23:04:17.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132602+0000", 2024-05-11T23:04:17.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.3743999999999999e-05 2024-05-11T23:04:17.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.470 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.470 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132655+0000", 2024-05-11T23:04:17.470 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2714999999999999e-05 2024-05-11T23:04:17.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.470 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:17.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.132787+0000", 2024-05-11T23:04:17.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013273899999999999 2024-05-11T23:04:17.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.471 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.471 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153101+0000", 2024-05-11T23:04:17.471 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.020313900999999999 2024-05-11T23:04:17.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.471 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.471 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98965 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3694592~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:04:17.471 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:04:17.472 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.321979934, 2024-05-11T23:04:17.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018608983999999999, 2024-05-11T23:04:17.472 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.472 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:17.472 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.472 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.472 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.472 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98965 2024-05-11T23:04:17.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.472 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.472 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.472 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:04:17.472 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135099+0000", 2024-05-11T23:04:17.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135100+0000", 2024-05-11T23:04:17.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.17e-06 2024-05-11T23:04:17.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.473 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.473 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.473 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135105+0000", 2024-05-11T23:04:17.473 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.899e-06 2024-05-11T23:04:17.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135122+0000", 2024-05-11T23:04:17.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.7346999999999998e-05 2024-05-11T23:04:17.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.474 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.474 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135130+0000", 2024-05-11T23:04:17.474 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0639999999999994e-06 2024-05-11T23:04:17.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153533+0000", 2024-05-11T23:04:17.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018402753000000001 2024-05-11T23:04:17.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:17.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153615+0000", 2024-05-11T23:04:17.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.1720999999999999e-05 2024-05-11T23:04:17.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.475 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.475 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.475 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153708+0000", 2024-05-11T23:04:17.475 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.3029999999999995e-05 2024-05-11T23:04:17.475 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.476 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.476 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.476 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.476 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:98966 3.3 3:c74b2f6e:::rbd_data.111e7d9444a9.0000000000000006:22 [read 3760128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-05-11T23:04:17.476 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:04:17.476 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.32190780800001, 2024-05-11T23:04:17.476 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.018716535999999999, 2024-05-11T23:04:17.476 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.476 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.476 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:17.476 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.476 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.476 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.477 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 98966 2024-05-11T23:04:17.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.477 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:04:17.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.477 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135171+0000", 2024-05-11T23:04:17.477 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.477 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135172+0000", 2024-05-11T23:04:17.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.0699999999999997e-07 2024-05-11T23:04:17.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135175+0000", 2024-05-11T23:04:17.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4489999999999999e-06 2024-05-11T23:04:17.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.478 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.478 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135184+0000", 2024-05-11T23:04:17.478 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.1840000000000005e-06 2024-05-11T23:04:17.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152903+0000", 2024-05-11T23:04:17.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017718357000000001 2024-05-11T23:04:17.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153736+0000", 2024-05-11T23:04:17.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00083324799999999997 2024-05-11T23:04:17.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:17.479 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153809+0000", 2024-05-11T23:04:17.479 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2995000000000002e-05 2024-05-11T23:04:17.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.479 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.480 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153888+0000", 2024-05-11T23:04:17.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.8695999999999993e-05 2024-05-11T23:04:17.480 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.480 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.480 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.480 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.480 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "pg_update_log_missing_reply(3.1e epoch 72/53 rep_tid 104 lcod 72'363)", 2024-05-11T23:04:17.480 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:04:17.480 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 111.321844912, 2024-05-11T23:04:17.480 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017871951000000001, 2024-05-11T23:04:17.480 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.480 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "reached pg", 2024-05-11T23:04:17.480 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:04:17.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135234+0000", 2024-05-11T23:04:17.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.481 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.481 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135235+0000", 2024-05-11T23:04:17.481 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8899999999999999e-07 2024-05-11T23:04:17.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.135238+0000", 2024-05-11T23:04:17.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.27e-06 2024-05-11T23:04:17.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152878+0000", 2024-05-11T23:04:17.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.01763986 2024-05-11T23:04:17.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.482 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.482 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152896+0000", 2024-05-11T23:04:17.482 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.8573e-05 2024-05-11T23:04:17.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.152968+0000", 2024-05-11T23:04:17.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.1780999999999996e-05 2024-05-11T23:04:17.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.483 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.483 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:02:26.153106+0000", 2024-05-11T23:04:17.483 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00013777799999999999 2024-05-11T23:04:17.483 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.483 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.483 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.483 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.483 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:239544 3.1a 3:5fe59cad:::rbd_data.111ebabb4604.000000000000016e:head [stat,delete] snapc 2f=[2f] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-05-11T23:04:17.483 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:04:17.483 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 73.802488179999997, 2024-05-11T23:04:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.50514007500000002, 2024-05-11T23:04:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 239544 2024-05-11T23:04:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:04:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654591+0000", 2024-05-11T23:04:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654592+0000", 2024-05-11T23:04:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.404e-06 2024-05-11T23:04:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654598+0000", 2024-05-11T23:04:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7660000000000004e-06 2024-05-11T23:04:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654611+0000", 2024-05-11T23:04:17.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2676e-05 2024-05-11T23:04:17.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654620+0000", 2024-05-11T23:04:17.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2990000000000002e-06 2024-05-11T23:04:17.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.486 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.486 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654698+0000", 2024-05-11T23:04:17.486 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.7739999999999998e-05 2024-05-11T23:04:17.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "waiting for peered", 2024-05-11T23:04:17.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:03.654710+0000", 2024-05-11T23:04:17.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2648e-05 2024-05-11T23:04:17.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.157889+0000", 2024-05-11T23:04:17.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.50317850799999997 2024-05-11T23:04:17.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.487 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:17.487 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.158050+0000", 2024-05-11T23:04:17.487 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016084500000000001 2024-05-11T23:04:17.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.487 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.488 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.488 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:04.159731+0000", 2024-05-11T23:04:17.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.001681189 2024-05-11T23:04:17.488 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.488 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.488 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.488 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262364 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2613248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:17.488 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:04:17.488 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.870615314999995, 2024-05-11T23:04:17.488 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016312508, 2024-05-11T23:04:17.488 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.488 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.488 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:17.489 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.489 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.489 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.489 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262364 2024-05-11T23:04:17.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.489 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:04:17.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.489 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.489 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.489 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:04:17.489 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586464+0000", 2024-05-11T23:04:17.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.13e-07 2024-05-11T23:04:17.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586467+0000", 2024-05-11T23:04:17.490 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.8770000000000002e-06 2024-05-11T23:04:17.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.490 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.490 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586474+0000", 2024-05-11T23:04:17.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.7739999999999999e-06 2024-05-11T23:04:17.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.586483+0000", 2024-05-11T23:04:17.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.0340000000000002e-06 2024-05-11T23:04:17.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602629+0000", 2024-05-11T23:04:17.491 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016145845999999998 2024-05-11T23:04:17.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.491 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:17.491 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602703+0000", 2024-05-11T23:04:17.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.4671000000000004e-05 2024-05-11T23:04:17.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.492 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.492 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.602776+0000", 2024-05-11T23:04:17.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2793000000000006e-05 2024-05-11T23:04:17.492 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.492 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.492 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.492 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262417 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2359296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:17.492 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:04:17.492 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.868869016000005, 2024-05-11T23:04:17.492 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016403456, 2024-05-11T23:04:17.492 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262417 2024-05-11T23:04:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:04:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:04:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588210+0000", 2024-05-11T23:04:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.6199999999999999e-07 2024-05-11T23:04:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588212+0000", 2024-05-11T23:04:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.075e-06 2024-05-11T23:04:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588218+0000", 2024-05-11T23:04:17.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8150000000000002e-06 2024-05-11T23:04:17.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588223+0000", 2024-05-11T23:04:17.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7090000000000001e-06 2024-05-11T23:04:17.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604461+0000", 2024-05-11T23:04:17.495 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016237900999999999 2024-05-11T23:04:17.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.495 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:17.495 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604540+0000", 2024-05-11T23:04:17.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9376999999999996e-05 2024-05-11T23:04:17.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.496 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.496 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604613+0000", 2024-05-11T23:04:17.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3217000000000001e-05 2024-05-11T23:04:17.496 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.496 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.496 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.496 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262418 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2396160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:17.496 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:04:17.496 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.868849107000003, 2024-05-11T23:04:17.496 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016591180000000001, 2024-05-11T23:04:17.496 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.496 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.497 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:17.497 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.497 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.497 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262418 2024-05-11T23:04:17.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.497 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:04:17.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.497 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.497 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.497 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:04:17.497 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588230+0000", 2024-05-11T23:04:17.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6100000000000001e-07 2024-05-11T23:04:17.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588232+0000", 2024-05-11T23:04:17.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0370000000000001e-06 2024-05-11T23:04:17.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.498 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.498 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588238+0000", 2024-05-11T23:04:17.498 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9340000000000003e-06 2024-05-11T23:04:17.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588243+0000", 2024-05-11T23:04:17.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.6179999999999997e-06 2024-05-11T23:04:17.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604642+0000", 2024-05-11T23:04:17.499 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.0163991 2024-05-11T23:04:17.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.499 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.499 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:17.499 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604712+0000", 2024-05-11T23:04:17.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.9935e-05 2024-05-11T23:04:17.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.500 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.500 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604821+0000", 2024-05-11T23:04:17.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000109095 2024-05-11T23:04:17.500 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.500 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.500 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.500 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.500 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262419 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2433024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:17.500 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:04:17.500 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.868829789000003, 2024-05-11T23:04:17.500 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016799692000000001, 2024-05-11T23:04:17.501 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.501 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.501 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:17.501 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.501 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.501 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.501 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262419 2024-05-11T23:04:17.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.501 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.501 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.501 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:04:17.501 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:04:17.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588249+0000", 2024-05-11T23:04:17.502 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:04:17.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.502 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.502 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.502 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588252+0000", 2024-05-11T23:04:17.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.0990000000000001e-06 2024-05-11T23:04:17.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588257+0000", 2024-05-11T23:04:17.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8159999999999999e-06 2024-05-11T23:04:17.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588262+0000", 2024-05-11T23:04:17.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8069999999999999e-06 2024-05-11T23:04:17.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.503 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.503 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604868+0000", 2024-05-11T23:04:17.503 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016605880999999999 2024-05-11T23:04:17.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:17.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.604966+0000", 2024-05-11T23:04:17.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.7597999999999996e-05 2024-05-11T23:04:17.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.504 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.504 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605049+0000", 2024-05-11T23:04:17.504 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3141999999999998e-05 2024-05-11T23:04:17.504 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.504 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.504 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.505 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262420 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2469888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:17.505 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:04:17.505 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.868809678000005, 2024-05-11T23:04:17.505 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016972514000000001, 2024-05-11T23:04:17.505 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.505 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.505 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:17.505 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.505 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.505 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.505 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262420 2024-05-11T23:04:17.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.505 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.505 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.505 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:04:17.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588269+0000", 2024-05-11T23:04:17.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.506 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.506 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588271+0000", 2024-05-11T23:04:17.506 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.6199999999999999e-06 2024-05-11T23:04:17.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588273+0000", 2024-05-11T23:04:17.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3209999999999999e-06 2024-05-11T23:04:17.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588279+0000", 2024-05-11T23:04:17.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2519999999999996e-06 2024-05-11T23:04:17.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.507 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.507 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588285+0000", 2024-05-11T23:04:17.507 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.1440000000000002e-06 2024-05-11T23:04:17.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.507 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605083+0000", 2024-05-11T23:04:17.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016798418999999998 2024-05-11T23:04:17.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:17.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605168+0000", 2024-05-11T23:04:17.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.4851000000000005e-05 2024-05-11T23:04:17.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.508 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.508 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.508 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605242+0000", 2024-05-11T23:04:17.508 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3906999999999996e-05 2024-05-11T23:04:17.508 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.508 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.508 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.509 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262421 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2506752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:17.509 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:04:17.509 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.868787531999999, 2024-05-11T23:04:17.509 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017141427000000001, 2024-05-11T23:04:17.509 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.509 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.509 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:17.509 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.509 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.509 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.509 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262421 2024-05-11T23:04:17.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.509 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:04:17.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588291+0000", 2024-05-11T23:04:17.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.510 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.510 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588292+0000", 2024-05-11T23:04:17.510 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:04:17.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588318+0000", 2024-05-11T23:04:17.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.6376e-05 2024-05-11T23:04:17.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588326+0000", 2024-05-11T23:04:17.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.6699999999999994e-06 2024-05-11T23:04:17.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.511 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.511 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.511 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588332+0000", 2024-05-11T23:04:17.511 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.8540000000000002e-06 2024-05-11T23:04:17.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605274+0000", 2024-05-11T23:04:17.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016942024999999999 2024-05-11T23:04:17.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:17.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605360+0000", 2024-05-11T23:04:17.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.6304999999999994e-05 2024-05-11T23:04:17.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.512 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.512 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.512 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605433+0000", 2024-05-11T23:04:17.512 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.2772000000000001e-05 2024-05-11T23:04:17.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.513 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.513 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.513 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.513 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262422 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2543616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:17.513 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:04:17.513 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.868738308000005, 2024-05-11T23:04:17.513 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017283705, 2024-05-11T23:04:17.513 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.513 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.513 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:17.513 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.513 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.513 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.513 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262422 2024-05-11T23:04:17.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.514 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:04:17.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.514 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:04:17.514 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.514 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.514 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588341+0000", 2024-05-11T23:04:17.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:04:17.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588343+0000", 2024-05-11T23:04:17.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.34e-06 2024-05-11T23:04:17.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.515 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588350+0000", 2024-05-11T23:04:17.515 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1569999999999997e-06 2024-05-11T23:04:17.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.515 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.515 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588354+0000", 2024-05-11T23:04:17.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.865e-06 2024-05-11T23:04:17.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605466+0000", 2024-05-11T23:04:17.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017111498999999999 2024-05-11T23:04:17.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:17.516 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605551+0000", 2024-05-11T23:04:17.516 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.5210999999999994e-05 2024-05-11T23:04:17.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.516 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.516 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.517 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605624+0000", 2024-05-11T23:04:17.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3153999999999998e-05 2024-05-11T23:04:17.517 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.517 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.517 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.517 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262423 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2580480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:17.517 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:04:17.517 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.868718059000003, 2024-05-11T23:04:17.517 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017451194, 2024-05-11T23:04:17.517 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.517 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.517 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:17.517 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.517 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.518 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.518 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262423 2024-05-11T23:04:17.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.518 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:04:17.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.518 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.518 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:04:17.518 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.518 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588361+0000", 2024-05-11T23:04:17.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:04:17.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588363+0000", 2024-05-11T23:04:17.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.855e-06 2024-05-11T23:04:17.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.519 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588369+0000", 2024-05-11T23:04:17.519 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1260000000000003e-06 2024-05-11T23:04:17.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.519 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588375+0000", 2024-05-11T23:04:17.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9630000000000003e-06 2024-05-11T23:04:17.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605656+0000", 2024-05-11T23:04:17.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017280865999999999 2024-05-11T23:04:17.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:17.520 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605737+0000", 2024-05-11T23:04:17.520 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.0773999999999996e-05 2024-05-11T23:04:17.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.520 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.521 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605812+0000", 2024-05-11T23:04:17.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.5203999999999994e-05 2024-05-11T23:04:17.521 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.521 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.521 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.521 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.521 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262424 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 2617344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:17.521 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:04:17.521 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.868696541000006, 2024-05-11T23:04:17.521 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017617544999999998, 2024-05-11T23:04:17.521 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:17.521 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.522 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.522 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.522 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262424 2024-05-11T23:04:17.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.522 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:04:17.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.522 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.522 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.522 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588382+0000", 2024-05-11T23:04:17.522 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588383+0000", 2024-05-11T23:04:17.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.8299999999999997e-07 2024-05-11T23:04:17.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588385+0000", 2024-05-11T23:04:17.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.2869999999999999e-06 2024-05-11T23:04:17.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.523 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.523 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.523 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588391+0000", 2024-05-11T23:04:17.523 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.1689999999999997e-06 2024-05-11T23:04:17.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.588396+0000", 2024-05-11T23:04:17.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.5719999999999996e-06 2024-05-11T23:04:17.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605837+0000", 2024-05-11T23:04:17.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017440812999999999 2024-05-11T23:04:17.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.524 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:17.524 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.605917+0000", 2024-05-11T23:04:17.524 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.9924000000000003e-05 2024-05-11T23:04:17.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.525 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.525 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.606000+0000", 2024-05-11T23:04:17.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3296999999999999e-05 2024-05-11T23:04:17.525 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.525 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.525 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.525 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262896 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 921600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:17.525 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:04:17.525 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.797691904000004, 2024-05-11T23:04:17.525 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016295738000000001, 2024-05-11T23:04:17.525 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.526 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.526 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:17.526 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.526 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.526 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.526 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262896 2024-05-11T23:04:17.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.526 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.526 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.526 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:04:17.526 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:04:17.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659387+0000", 2024-05-11T23:04:17.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.2500000000000001e-07 2024-05-11T23:04:17.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.527 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.527 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.527 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659390+0000", 2024-05-11T23:04:17.527 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1440000000000001e-06 2024-05-11T23:04:17.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659397+0000", 2024-05-11T23:04:17.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.0010000000000004e-06 2024-05-11T23:04:17.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659402+0000", 2024-05-11T23:04:17.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.2179999999999998e-06 2024-05-11T23:04:17.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.528 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.528 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675487+0000", 2024-05-11T23:04:17.528 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016085706000000002 2024-05-11T23:04:17.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:17.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675580+0000", 2024-05-11T23:04:17.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.2884e-05 2024-05-11T23:04:17.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.529 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.529 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675683+0000", 2024-05-11T23:04:17.529 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00010236 2024-05-11T23:04:17.529 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.529 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.529 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.529 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262897 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 958464~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:17.529 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:04:17.529 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.797656563000004, 2024-05-11T23:04:17.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016500932999999999, 2024-05-11T23:04:17.530 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:17.530 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.530 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.530 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.530 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262897 2024-05-11T23:04:17.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.530 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.530 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.530 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.530 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:04:17.530 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659422+0000", 2024-05-11T23:04:17.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659423+0000", 2024-05-11T23:04:17.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.9699999999999996e-07 2024-05-11T23:04:17.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.531 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659425+0000", 2024-05-11T23:04:17.531 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.3199999999999998e-06 2024-05-11T23:04:17.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.531 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.531 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659433+0000", 2024-05-11T23:04:17.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 7.3660000000000003e-06 2024-05-11T23:04:17.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659445+0000", 2024-05-11T23:04:17.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 1.2102e-05 2024-05-11T23:04:17.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675727+0000", 2024-05-11T23:04:17.532 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016282331000000001 2024-05-11T23:04:17.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.532 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.532 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:17.532 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675815+0000", 2024-05-11T23:04:17.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.7484999999999993e-05 2024-05-11T23:04:17.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.533 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.533 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675923+0000", 2024-05-11T23:04:17.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.000108732 2024-05-11T23:04:17.533 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.533 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.533 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.533 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.533 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262898 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:17.533 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:04:17.533 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.797625444999994, 2024-05-11T23:04:17.533 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016685855999999999, 2024-05-11T23:04:17.533 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.534 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.534 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:17.534 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.534 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.534 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.534 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262898 2024-05-11T23:04:17.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.534 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.534 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:04:17.534 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.534 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659453+0000", 2024-05-11T23:04:17.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659454+0000", 2024-05-11T23:04:17.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.2200000000000004e-07 2024-05-11T23:04:17.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.535 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659457+0000", 2024-05-11T23:04:17.535 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.779e-06 2024-05-11T23:04:17.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.535 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659463+0000", 2024-05-11T23:04:17.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.6660000000000002e-06 2024-05-11T23:04:17.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659472+0000", 2024-05-11T23:04:17.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.2139999999999996e-06 2024-05-11T23:04:17.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.675956+0000", 2024-05-11T23:04:17.536 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016484431000000001 2024-05-11T23:04:17.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.536 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.536 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:17.536 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676039+0000", 2024-05-11T23:04:17.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 8.3358999999999999e-05 2024-05-11T23:04:17.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.537 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.537 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676139+0000", 2024-05-11T23:04:17.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 9.9784999999999994e-05 2024-05-11T23:04:17.537 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.537 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.537 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.537 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.537 INFO:teuthology.orchestra.run.smithi005.stdout: "description": "osd_op(client.4382.0:262899 3.9 3:9029f125:::rbd_data.111ec06c817a.000000000000002b:33 [read 1032192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:17.537 INFO:teuthology.orchestra.run.smithi005.stdout: "initiated_at": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:04:17.537 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 64.797599614000006, 2024-05-11T23:04:17.537 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.017030087999999999, 2024-05-11T23:04:17.537 INFO:teuthology.orchestra.run.smithi005.stdout: "continuous": false, 2024-05-11T23:04:17.538 INFO:teuthology.orchestra.run.smithi005.stdout: "type_data": { 2024-05-11T23:04:17.538 INFO:teuthology.orchestra.run.smithi005.stdout: "flag_point": "started", 2024-05-11T23:04:17.538 INFO:teuthology.orchestra.run.smithi005.stdout: "client_info": { 2024-05-11T23:04:17.538 INFO:teuthology.orchestra.run.smithi005.stdout: "client": "client.4382", 2024-05-11T23:04:17.538 INFO:teuthology.orchestra.run.smithi005.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.538 INFO:teuthology.orchestra.run.smithi005.stdout: "tid": 262899 2024-05-11T23:04:17.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.538 INFO:teuthology.orchestra.run.smithi005.stdout: "events": [ 2024-05-11T23:04:17.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.538 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "initiated", 2024-05-11T23:04:17.538 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:04:17.538 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.538 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "header_read", 2024-05-11T23:04:17.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659479+0000", 2024-05-11T23:04:17.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0 2024-05-11T23:04:17.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "throttled", 2024-05-11T23:04:17.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659480+0000", 2024-05-11T23:04:17.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 5.7700000000000004e-07 2024-05-11T23:04:17.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.539 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "all_read", 2024-05-11T23:04:17.539 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659482+0000", 2024-05-11T23:04:17.539 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 2.1390000000000002e-06 2024-05-11T23:04:17.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "dispatched", 2024-05-11T23:04:17.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659489+0000", 2024-05-11T23:04:17.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 6.8279999999999997e-06 2024-05-11T23:04:17.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.659494+0000", 2024-05-11T23:04:17.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 4.9019999999999998e-06 2024-05-11T23:04:17.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.540 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "reached_pg", 2024-05-11T23:04:17.540 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676167+0000", 2024-05-11T23:04:17.540 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.016673621 2024-05-11T23:04:17.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.540 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "started", 2024-05-11T23:04:17.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676329+0000", 2024-05-11T23:04:17.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00016206099999999999 2024-05-11T23:04:17.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-05-11T23:04:17.541 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-05-11T23:04:17.541 INFO:teuthology.orchestra.run.smithi005.stdout: "event": "done", 2024-05-11T23:04:17.541 INFO:teuthology.orchestra.run.smithi005.stdout: "time": "2024-05-11T23:03:12.676509+0000", 2024-05-11T23:04:17.541 INFO:teuthology.orchestra.run.smithi005.stdout: "duration": 0.00017996 2024-05-11T23:04:17.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.541 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-05-11T23:04:17.541 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-05-11T23:04:17.541 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:17.543 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-11T23:04:17.543 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-05-11T23:04:17.543+0000 7f66a732b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32855) UID: 0 2024-05-11T23:04:17.644 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-05-11T23:04:17.644+0000 7f208720e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32840) UID: 0 2024-05-11T23:04:17.659 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:17.660 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:17.660 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:04:17.660 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:17.669 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-11T23:04:17.745 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:17.745+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:17.775 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:17.775 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:17.775 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:04:17.775 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:04:17.775 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:17.784 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-11T23:04:17.846 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:17.846+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:17.891 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:17.891 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:04:17.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:04:17.891 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:04:17.891 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.891 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116446 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8273920~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:17.891 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:04:17.891 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 102.514439309, 2024-05-11T23:04:17.891 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045535299000000001, 2024-05-11T23:04:17.891 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:17.891 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:17.891 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:17.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:17.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:17.892 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.892 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116446 2024-05-11T23:04:17.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.892 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:17.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:17.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:04:17.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.892 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.892 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:17.892 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375546+0000", 2024-05-11T23:04:17.892 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.892 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:17.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375547+0000", 2024-05-11T23:04:17.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4000000000000001e-07 2024-05-11T23:04:17.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:17.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375549+0000", 2024-05-11T23:04:17.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.368e-06 2024-05-11T23:04:17.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.893 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.893 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:17.893 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375556+0000", 2024-05-11T23:04:17.893 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1840000000000004e-06 2024-05-11T23:04:17.893 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375560+0000", 2024-05-11T23:04:17.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.741e-06 2024-05-11T23:04:17.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420896+0000", 2024-05-11T23:04:17.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045335435 2024-05-11T23:04:17.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.894 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.894 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:17.894 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.420987+0000", 2024-05-11T23:04:17.894 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1033999999999996e-05 2024-05-11T23:04:17.894 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.895 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:17.895 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421082+0000", 2024-05-11T23:04:17.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4896999999999996e-05 2024-05-11T23:04:17.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.895 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.895 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.895 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.895 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.895 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116447 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6316032~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:17.895 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:04:17.895 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 102.514417799, 2024-05-11T23:04:17.895 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045745830000000001, 2024-05-11T23:04:17.895 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:17.895 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:17.896 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:17.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:17.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:17.896 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.896 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116447 2024-05-11T23:04:17.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.896 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:17.896 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.896 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:17.896 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:04:17.896 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.896 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:17.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:04:17.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:17.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375568+0000", 2024-05-11T23:04:17.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.58e-07 2024-05-11T23:04:17.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.897 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:17.897 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375570+0000", 2024-05-11T23:04:17.897 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.052e-06 2024-05-11T23:04:17.897 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.897 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:17.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375576+0000", 2024-05-11T23:04:17.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7939999999999999e-06 2024-05-11T23:04:17.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375581+0000", 2024-05-11T23:04:17.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9280000000000001e-06 2024-05-11T23:04:17.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.898 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.898 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421127+0000", 2024-05-11T23:04:17.898 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045545785999999998 2024-05-11T23:04:17.898 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.898 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:17.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421214+0000", 2024-05-11T23:04:17.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.7039000000000005e-05 2024-05-11T23:04:17.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.899 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:17.899 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421314+0000", 2024-05-11T23:04:17.899 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9772999999999995e-05 2024-05-11T23:04:17.899 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.899 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.899 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.899 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.899 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.899 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116448 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8314880~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:17.899 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:04:17.900 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 102.51439872500001, 2024-05-11T23:04:17.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045941753000000002, 2024-05-11T23:04:17.900 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:17.900 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:17.900 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:17.900 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:17.900 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:17.900 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.900 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116448 2024-05-11T23:04:17.900 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.900 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:17.900 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.900 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:17.900 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:04:17.900 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:17.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:04:17.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:17.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375587+0000", 2024-05-11T23:04:17.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.77e-07 2024-05-11T23:04:17.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.901 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.901 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:17.901 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375589+0000", 2024-05-11T23:04:17.901 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.091e-06 2024-05-11T23:04:17.901 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:17.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375596+0000", 2024-05-11T23:04:17.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9179999999999997e-06 2024-05-11T23:04:17.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375601+0000", 2024-05-11T23:04:17.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0030000000000002e-06 2024-05-11T23:04:17.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.902 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.902 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.902 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421363+0000", 2024-05-11T23:04:17.902 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045761352999999998 2024-05-11T23:04:17.902 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:17.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421440+0000", 2024-05-11T23:04:17.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7585999999999999e-05 2024-05-11T23:04:17.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.903 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:17.903 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421529+0000", 2024-05-11T23:04:17.903 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8424999999999994e-05 2024-05-11T23:04:17.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.903 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.903 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.903 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.903 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.903 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116449 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 6438912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:17.903 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:04:17.904 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 102.51437829, 2024-05-11T23:04:17.904 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046082208999999999, 2024-05-11T23:04:17.904 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:17.904 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:17.904 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:17.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:17.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:17.904 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.904 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116449 2024-05-11T23:04:17.904 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.904 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:17.904 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.904 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:17.904 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:04:17.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:17.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.375607+0000", 2024-05-11T23:04:17.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:17.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377225+0000", 2024-05-11T23:04:17.905 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0016177030000000001 2024-05-11T23:04:17.905 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.905 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.905 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:17.905 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377229+0000", 2024-05-11T23:04:17.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8e-06 2024-05-11T23:04:17.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:17.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377238+0000", 2024-05-11T23:04:17.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2229999999999996e-06 2024-05-11T23:04:17.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.906 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377244+0000", 2024-05-11T23:04:17.906 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4060000000000002e-06 2024-05-11T23:04:17.906 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.906 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.906 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421520+0000", 2024-05-11T23:04:17.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044275222000000003 2024-05-11T23:04:17.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:17.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421595+0000", 2024-05-11T23:04:17.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4838000000000004e-05 2024-05-11T23:04:17.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.907 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.907 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:17.907 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.421690+0000", 2024-05-11T23:04:17.907 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5017000000000002e-05 2024-05-11T23:04:17.907 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.907 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.907 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.907 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.908 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116458 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7094272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:17.908 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:04:17.908 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 102.51256407299999, 2024-05-11T23:04:17.908 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045706877999999999, 2024-05-11T23:04:17.908 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:17.908 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:17.908 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:17.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:17.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:17.908 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.908 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116458 2024-05-11T23:04:17.908 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.908 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:17.908 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.908 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:17.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:04:17.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:17.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:04:17.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:17.909 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377422+0000", 2024-05-11T23:04:17.909 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9400000000000005e-07 2024-05-11T23:04:17.909 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.909 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.909 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:17.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377424+0000", 2024-05-11T23:04:17.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1859999999999999e-06 2024-05-11T23:04:17.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:17.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377431+0000", 2024-05-11T23:04:17.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4049999999999997e-06 2024-05-11T23:04:17.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.910 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377436+0000", 2024-05-11T23:04:17.910 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1399999999999999e-06 2024-05-11T23:04:17.910 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.910 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.910 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.422945+0000", 2024-05-11T23:04:17.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045509326000000003 2024-05-11T23:04:17.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:17.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423036+0000", 2024-05-11T23:04:17.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0649999999999994e-05 2024-05-11T23:04:17.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.911 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.911 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:17.911 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423128+0000", 2024-05-11T23:04:17.911 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2577000000000005e-05 2024-05-11T23:04:17.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.911 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.911 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.911 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.912 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116459 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7258112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:17.912 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:04:17.912 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 102.512543341, 2024-05-11T23:04:17.912 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045831945999999998, 2024-05-11T23:04:17.912 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:17.912 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:17.912 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:17.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:17.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:17.912 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.912 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116459 2024-05-11T23:04:17.912 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.912 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:17.912 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.912 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:17.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:04:17.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:17.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377442+0000", 2024-05-11T23:04:17.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:17.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377443+0000", 2024-05-11T23:04:17.913 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4799999999999999e-07 2024-05-11T23:04:17.913 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.913 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.913 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:17.913 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377445+0000", 2024-05-11T23:04:17.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0209999999999998e-06 2024-05-11T23:04:17.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:17.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377451+0000", 2024-05-11T23:04:17.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.071e-06 2024-05-11T23:04:17.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377456+0000", 2024-05-11T23:04:17.914 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8289999999999998e-06 2024-05-11T23:04:17.914 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.914 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.914 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.914 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423155+0000", 2024-05-11T23:04:17.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045699447999999997 2024-05-11T23:04:17.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:17.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423200+0000", 2024-05-11T23:04:17.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5083000000000002e-05 2024-05-11T23:04:17.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.915 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:17.915 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423274+0000", 2024-05-11T23:04:17.915 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4046000000000002e-05 2024-05-11T23:04:17.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.915 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.915 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.915 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.915 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.916 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116460 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7299072~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:17.916 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:04:17.916 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 102.512524279, 2024-05-11T23:04:17.916 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046010987000000003, 2024-05-11T23:04:17.916 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:17.916 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:17.916 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:17.916 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:17.916 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:17.916 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.916 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116460 2024-05-11T23:04:17.916 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.916 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:17.916 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.916 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:17.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:04:17.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:17.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377461+0000", 2024-05-11T23:04:17.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:17.917 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377462+0000", 2024-05-11T23:04:17.917 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3599999999999999e-07 2024-05-11T23:04:17.917 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.917 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.917 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:17.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377464+0000", 2024-05-11T23:04:17.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1600000000000001e-06 2024-05-11T23:04:17.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:17.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377469+0000", 2024-05-11T23:04:17.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5430000000000001e-06 2024-05-11T23:04:17.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.918 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377474+0000", 2024-05-11T23:04:17.918 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8509999999999998e-06 2024-05-11T23:04:17.918 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.918 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.918 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423312+0000", 2024-05-11T23:04:17.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045838080000000003 2024-05-11T23:04:17.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:17.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423401+0000", 2024-05-11T23:04:17.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8170000000000005e-05 2024-05-11T23:04:17.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.919 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.919 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:17.919 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423472+0000", 2024-05-11T23:04:17.919 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1847000000000005e-05 2024-05-11T23:04:17.919 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.919 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.919 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.919 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.920 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116461 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7421952~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:17.920 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:04:17.920 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 102.51250550899999, 2024-05-11T23:04:17.920 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046130749999999998, 2024-05-11T23:04:17.920 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:17.920 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:17.920 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:17.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:17.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:17.920 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.920 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116461 2024-05-11T23:04:17.920 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.920 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:17.920 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.920 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:17.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:04:17.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:17.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:04:17.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:17.921 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377480+0000", 2024-05-11T23:04:17.921 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4700000000000002e-07 2024-05-11T23:04:17.921 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.921 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.921 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:17.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377483+0000", 2024-05-11T23:04:17.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0219999999999999e-06 2024-05-11T23:04:17.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:17.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377489+0000", 2024-05-11T23:04:17.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4420000000000003e-06 2024-05-11T23:04:17.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.922 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377494+0000", 2024-05-11T23:04:17.922 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9239999999999998e-06 2024-05-11T23:04:17.922 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.922 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.922 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423500+0000", 2024-05-11T23:04:17.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046006054999999997 2024-05-11T23:04:17.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:17.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423557+0000", 2024-05-11T23:04:17.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7169000000000001e-05 2024-05-11T23:04:17.923 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.923 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.923 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:17.923 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423611+0000", 2024-05-11T23:04:17.923 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3791e-05 2024-05-11T23:04:17.923 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.923 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.923 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.924 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.924 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116462 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7462912~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:17.924 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:04:17.924 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 102.512485891, 2024-05-11T23:04:17.924 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046254666, 2024-05-11T23:04:17.924 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:17.924 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:17.924 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:17.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:17.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:17.924 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.924 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116462 2024-05-11T23:04:17.924 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.924 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:17.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:17.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:04:17.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:17.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:04:17.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.925 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.925 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:17.925 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377500+0000", 2024-05-11T23:04:17.925 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:04:17.925 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:17.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377502+0000", 2024-05-11T23:04:17.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.154e-06 2024-05-11T23:04:17.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:17.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377508+0000", 2024-05-11T23:04:17.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4500000000000003e-06 2024-05-11T23:04:17.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.926 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.926 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377512+0000", 2024-05-11T23:04:17.926 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7700000000000001e-06 2024-05-11T23:04:17.926 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.926 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423649+0000", 2024-05-11T23:04:17.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046136859000000002 2024-05-11T23:04:17.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:17.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423704+0000", 2024-05-11T23:04:17.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4938000000000002e-05 2024-05-11T23:04:17.927 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.927 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.927 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:17.927 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423754+0000", 2024-05-11T23:04:17.927 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0145999999999997e-05 2024-05-11T23:04:17.927 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.927 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.927 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.928 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.928 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116463 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7585792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:17.928 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:04:17.928 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 102.512467465, 2024-05-11T23:04:17.928 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361763, 2024-05-11T23:04:17.928 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:17.928 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:17.928 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:17.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:17.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:17.928 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.928 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116463 2024-05-11T23:04:17.928 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.928 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:17.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:17.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:04:17.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:17.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377518+0000", 2024-05-11T23:04:17.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.929 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.929 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:17.929 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377519+0000", 2024-05-11T23:04:17.929 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2600000000000002e-07 2024-05-11T23:04:17.929 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:17.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377521+0000", 2024-05-11T23:04:17.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1710000000000001e-06 2024-05-11T23:04:17.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:17.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377527+0000", 2024-05-11T23:04:17.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9649999999999996e-06 2024-05-11T23:04:17.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.930 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.930 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.930 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377532+0000", 2024-05-11T23:04:17.930 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.78e-06 2024-05-11T23:04:17.930 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423779+0000", 2024-05-11T23:04:17.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046247706 2024-05-11T23:04:17.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:17.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423829+0000", 2024-05-11T23:04:17.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9188999999999999e-05 2024-05-11T23:04:17.931 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.931 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.931 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:17.931 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423880+0000", 2024-05-11T23:04:17.931 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1425999999999999e-05 2024-05-11T23:04:17.931 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.931 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.932 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.932 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.932 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116464 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7749632~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:17.932 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:04:17.932 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 102.512448229, 2024-05-11T23:04:17.932 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046491719000000001, 2024-05-11T23:04:17.932 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:17.932 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:17.932 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:17.932 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:17.932 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:17.932 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.932 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116464 2024-05-11T23:04:17.932 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.933 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:17.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:17.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:04:17.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:17.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377537+0000", 2024-05-11T23:04:17.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.933 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.933 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.933 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:17.933 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377538+0000", 2024-05-11T23:04:17.933 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4499999999999998e-07 2024-05-11T23:04:17.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:17.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377540+0000", 2024-05-11T23:04:17.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2340000000000001e-06 2024-05-11T23:04:17.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:17.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377546+0000", 2024-05-11T23:04:17.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8289999999999996e-06 2024-05-11T23:04:17.934 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.934 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.934 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.934 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377551+0000", 2024-05-11T23:04:17.934 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8559999999999998e-06 2024-05-11T23:04:17.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423913+0000", 2024-05-11T23:04:17.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046361885999999998 2024-05-11T23:04:17.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:17.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.423958+0000", 2024-05-11T23:04:17.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5513999999999998e-05 2024-05-11T23:04:17.935 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.935 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.935 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:17.935 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424029+0000", 2024-05-11T23:04:17.935 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1055000000000003e-05 2024-05-11T23:04:17.935 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.936 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.936 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.936 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.936 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.936 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116465 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7626752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:17.936 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:04:17.936 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 102.51242908099999, 2024-05-11T23:04:17.936 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046600057, 2024-05-11T23:04:17.936 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:17.936 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:17.936 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:17.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:17.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:17.936 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.936 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116465 2024-05-11T23:04:17.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.937 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:17.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:17.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:04:17.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:17.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:04:17.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.937 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.937 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.937 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:17.937 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377557+0000", 2024-05-11T23:04:17.937 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3599999999999999e-07 2024-05-11T23:04:17.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:17.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377559+0000", 2024-05-11T23:04:17.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9640000000000002e-06 2024-05-11T23:04:17.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:17.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377572+0000", 2024-05-11T23:04:17.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3188e-05 2024-05-11T23:04:17.938 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.938 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.938 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.938 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377578+0000", 2024-05-11T23:04:17.938 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.942e-06 2024-05-11T23:04:17.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424058+0000", 2024-05-11T23:04:17.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046480021000000003 2024-05-11T23:04:17.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:17.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424103+0000", 2024-05-11T23:04:17.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5021999999999997e-05 2024-05-11T23:04:17.939 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.939 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.939 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:17.939 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424157+0000", 2024-05-11T23:04:17.939 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3483999999999998e-05 2024-05-11T23:04:17.939 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.939 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.940 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.940 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.940 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116466 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7790592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:17.940 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:04:17.940 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 102.51240049899999, 2024-05-11T23:04:17.940 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046743741999999998, 2024-05-11T23:04:17.940 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:17.940 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:17.940 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:17.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:17.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:17.940 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.940 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116466 2024-05-11T23:04:17.940 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.940 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:17.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:17.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:04:17.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:17.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377585+0000", 2024-05-11T23:04:17.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.941 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.941 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:17.941 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377586+0000", 2024-05-11T23:04:17.941 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:04:17.941 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:17.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377588+0000", 2024-05-11T23:04:17.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3039999999999999e-06 2024-05-11T23:04:17.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:17.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377594+0000", 2024-05-11T23:04:17.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9800000000000003e-06 2024-05-11T23:04:17.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.942 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.942 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.942 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377600+0000", 2024-05-11T23:04:17.942 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7699999999999998e-06 2024-05-11T23:04:17.942 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424185+0000", 2024-05-11T23:04:17.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046585891999999997 2024-05-11T23:04:17.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:17.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424235+0000", 2024-05-11T23:04:17.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9356e-05 2024-05-11T23:04:17.943 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.943 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.943 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:17.943 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424329+0000", 2024-05-11T23:04:17.943 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4036999999999994e-05 2024-05-11T23:04:17.943 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.944 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.944 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.944 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.944 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.944 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116467 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 7954432~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:17.944 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:04:17.944 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 102.512380212, 2024-05-11T23:04:17.944 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046930134999999998, 2024-05-11T23:04:17.944 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:17.944 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:17.944 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:17.944 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:17.944 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:17.944 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.944 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116467 2024-05-11T23:04:17.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.945 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:17.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:17.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:04:17.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:17.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.377605+0000", 2024-05-11T23:04:17.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.945 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.945 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.945 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:17.945 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379161+0000", 2024-05-11T23:04:17.945 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0015555009999999999 2024-05-11T23:04:17.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:17.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379166+0000", 2024-05-11T23:04:17.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6689999999999997e-06 2024-05-11T23:04:17.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:17.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379176+0000", 2024-05-11T23:04:17.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0549e-05 2024-05-11T23:04:17.946 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.946 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.946 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.946 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379183+0000", 2024-05-11T23:04:17.946 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1199999999999996e-06 2024-05-11T23:04:17.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424359+0000", 2024-05-11T23:04:17.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045175949999999999 2024-05-11T23:04:17.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:17.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424433+0000", 2024-05-11T23:04:17.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3824000000000003e-05 2024-05-11T23:04:17.947 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.947 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.947 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:17.947 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424536+0000", 2024-05-11T23:04:17.947 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010252200000000001 2024-05-11T23:04:17.947 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.948 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.948 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.948 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.948 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.948 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116469 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8077312~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:17.948 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:04:17.948 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 102.510740173, 2024-05-11T23:04:17.948 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045636687000000002, 2024-05-11T23:04:17.948 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:17.948 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:17.948 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:17.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:17.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:17.948 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.948 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116469 2024-05-11T23:04:17.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.949 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:17.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.949 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:17.949 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:04:17.949 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.949 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.949 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:17.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379245+0000", 2024-05-11T23:04:17.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:17.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379246+0000", 2024-05-11T23:04:17.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.32e-07 2024-05-11T23:04:17.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.950 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.950 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:17.950 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379254+0000", 2024-05-11T23:04:17.950 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.807e-06 2024-05-11T23:04:17.950 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:17.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379261+0000", 2024-05-11T23:04:17.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.985e-06 2024-05-11T23:04:17.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379264+0000", 2024-05-11T23:04:17.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6480000000000001e-06 2024-05-11T23:04:17.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.951 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.951 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424744+0000", 2024-05-11T23:04:17.951 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045479638000000003 2024-05-11T23:04:17.951 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.951 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:17.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424797+0000", 2024-05-11T23:04:17.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2725000000000001e-05 2024-05-11T23:04:17.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.952 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:17.952 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424882+0000", 2024-05-11T23:04:17.952 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5451999999999994e-05 2024-05-11T23:04:17.952 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.952 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.952 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.952 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.952 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.952 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116470 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8118272~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:17.952 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:04:17.953 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 102.51071496100001, 2024-05-11T23:04:17.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045755745, 2024-05-11T23:04:17.953 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:17.953 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:17.953 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:17.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:17.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:17.953 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.953 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116470 2024-05-11T23:04:17.953 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.953 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:17.953 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.953 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:17.953 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:04:17.953 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:17.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:04:17.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:17.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379271+0000", 2024-05-11T23:04:17.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3999999999999997e-07 2024-05-11T23:04:17.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.954 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.954 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:17.954 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379273+0000", 2024-05-11T23:04:17.954 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.508e-06 2024-05-11T23:04:17.954 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:17.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379277+0000", 2024-05-11T23:04:17.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4020000000000004e-06 2024-05-11T23:04:17.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379280+0000", 2024-05-11T23:04:17.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-05-11T23:04:17.955 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.955 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.955 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.955 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424922+0000", 2024-05-11T23:04:17.955 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045642598999999999 2024-05-11T23:04:17.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:17.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.424973+0000", 2024-05-11T23:04:17.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0674999999999998e-05 2024-05-11T23:04:17.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.956 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:17.956 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425026+0000", 2024-05-11T23:04:17.956 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3492000000000002e-05 2024-05-11T23:04:17.956 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.956 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.956 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.956 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.956 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.957 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116471 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8241152~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:17.957 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:04:17.957 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 102.510702178, 2024-05-11T23:04:17.957 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045867374000000002, 2024-05-11T23:04:17.957 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:17.957 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:17.957 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:17.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:17.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:17.957 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.957 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116471 2024-05-11T23:04:17.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.958 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:17.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:17.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:04:17.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:17.958 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379283+0000", 2024-05-11T23:04:17.958 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.958 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.958 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.958 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:17.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379284+0000", 2024-05-11T23:04:17.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.53e-07 2024-05-11T23:04:17.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:17.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379285+0000", 2024-05-11T23:04:17.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.223e-06 2024-05-11T23:04:17.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:17.959 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379288+0000", 2024-05-11T23:04:17.959 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5379999999999998e-06 2024-05-11T23:04:17.959 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.959 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.959 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379291+0000", 2024-05-11T23:04:17.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6699999999999998e-06 2024-05-11T23:04:17.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425055+0000", 2024-05-11T23:04:17.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045763497 2024-05-11T23:04:17.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:17.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425099+0000", 2024-05-11T23:04:17.960 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3899000000000001e-05 2024-05-11T23:04:17.960 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.960 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.960 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:17.960 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425151+0000", 2024-05-11T23:04:17.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2293999999999998e-05 2024-05-11T23:04:17.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.961 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.961 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.961 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.961 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.961 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116472 3.1b 3:da7861af:::rbd_data.111e913a615f.0000000000000002:25 [read 8282112~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:17.961 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:04:17.961 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 102.51069117900001, 2024-05-11T23:04:17.961 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045973778999999999, 2024-05-11T23:04:17.961 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:17.961 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:17.961 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:17.961 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:17.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:17.962 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.962 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116472 2024-05-11T23:04:17.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.962 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:17.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:17.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:04:17.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.962 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.962 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:17.962 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.379294+0000", 2024-05-11T23:04:17.962 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.962 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:17.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381226+0000", 2024-05-11T23:04:17.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001931627 2024-05-11T23:04:17.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:17.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381231+0000", 2024-05-11T23:04:17.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8679999999999998e-06 2024-05-11T23:04:17.963 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.963 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.963 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:17.963 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381242+0000", 2024-05-11T23:04:17.963 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0702e-05 2024-05-11T23:04:17.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381249+0000", 2024-05-11T23:04:17.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9800000000000001e-06 2024-05-11T23:04:17.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425175+0000", 2024-05-11T23:04:17.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043926805999999999 2024-05-11T23:04:17.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.964 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.964 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:17.964 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425215+0000", 2024-05-11T23:04:17.964 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9737999999999998e-05 2024-05-11T23:04:17.964 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.965 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:17.965 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425268+0000", 2024-05-11T23:04:17.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3057999999999999e-05 2024-05-11T23:04:17.965 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.965 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.965 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.965 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.965 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.965 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:116482 3.1b 3:db6e8843:::rbd_object_map.111e67f27da3.0000000000000028:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-05-11T23:04:17.965 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:04:17.965 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 102.508722636, 2024-05-11T23:04:17.965 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046526076, 2024-05-11T23:04:17.965 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:17.965 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:17.965 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:04:17.966 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:17.966 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:17.966 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.966 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 116482 2024-05-11T23:04:17.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.966 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:17.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:17.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:04:17.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.966 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.966 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.966 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:17.966 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381263+0000", 2024-05-11T23:04:17.966 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:17.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381264+0000", 2024-05-11T23:04:17.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5300000000000004e-07 2024-05-11T23:04:17.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:17.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381267+0000", 2024-05-11T23:04:17.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1999999999999999e-06 2024-05-11T23:04:17.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.967 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.967 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:17.967 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381274+0000", 2024-05-11T23:04:17.967 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7109999999999995e-06 2024-05-11T23:04:17.967 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.381285+0000", 2024-05-11T23:04:17.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0856000000000001e-05 2024-05-11T23:04:17.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425292+0000", 2024-05-11T23:04:17.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044006376 2024-05-11T23:04:17.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.968 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.968 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:17.968 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425337+0000", 2024-05-11T23:04:17.968 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4907999999999998e-05 2024-05-11T23:04:17.968 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 7", 2024-05-11T23:04:17.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.425507+0000", 2024-05-11T23:04:17.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017029999999999999 2024-05-11T23:04:17.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:04:17.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427668+0000", 2024-05-11T23:04:17.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0021612430000000002 2024-05-11T23:04:17.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.969 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:04:17.969 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427683+0000", 2024-05-11T23:04:17.969 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.5016e-05 2024-05-11T23:04:17.969 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.969 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:04:17.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427742+0000", 2024-05-11T23:04:17.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8811999999999999e-05 2024-05-11T23:04:17.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.970 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:17.970 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:35.427789+0000", 2024-05-11T23:04:17.970 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7101000000000002e-05 2024-05-11T23:04:17.970 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.970 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.970 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.970 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.970 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.970 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:283279 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-05-11T23:04:17.970 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:53.865673+0000", 2024-05-11T23:04:17.971 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 24.02431262, 2024-05-11T23:04:17.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.93944476799999999, 2024-05-11T23:04:17.971 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:17.971 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:17.971 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:04:17.971 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:17.971 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:17.971 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:17.971 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 283279 2024-05-11T23:04:17.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.971 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:17.971 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.971 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:17.971 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865673+0000", 2024-05-11T23:04:17.971 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.971 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:17.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865673+0000", 2024-05-11T23:04:17.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:17.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:17.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865674+0000", 2024-05-11T23:04:17.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.012e-06 2024-05-11T23:04:17.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.972 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:17.972 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865679+0000", 2024-05-11T23:04:17.972 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7740000000000004e-06 2024-05-11T23:04:17.972 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.972 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:17.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865689+0000", 2024-05-11T23:04:17.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0644e-05 2024-05-11T23:04:17.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:17.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865694+0000", 2024-05-11T23:04:17.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.826e-06 2024-05-11T23:04:17.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.973 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.973 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865757+0000", 2024-05-11T23:04:17.973 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2493e-05 2024-05-11T23:04:17.973 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.973 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:04:17.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:53.865768+0000", 2024-05-11T23:04:17.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0777e-05 2024-05-11T23:04:17.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:17.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.802182+0000", 2024-05-11T23:04:17.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.93641407499999996 2024-05-11T23:04:17.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:17.974 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.802338+0000", 2024-05-11T23:04:17.974 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015636599999999999 2024-05-11T23:04:17.974 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.974 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.974 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 6", 2024-05-11T23:04:17.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.802561+0000", 2024-05-11T23:04:17.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000222585 2024-05-11T23:04:17.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:04:17.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.805050+0000", 2024-05-11T23:04:17.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0024896300000000001 2024-05-11T23:04:17.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:04:17.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.805059+0000", 2024-05-11T23:04:17.975 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6829999999999997e-06 2024-05-11T23:04:17.975 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.975 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.975 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:04:17.975 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.805093+0000", 2024-05-11T23:04:17.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3763000000000002e-05 2024-05-11T23:04:17.976 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:17.976 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:17.976 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:17.976 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:54.805118+0000", 2024-05-11T23:04:17.976 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.514e-05 2024-05-11T23:04:17.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.976 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.976 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:17.976 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:17.976 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:17.978 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is set 2024-05-11T23:04:17.978 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-11T23:04:17.979 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-05-11T23:04:17.977+0000 7f1111744640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32842) UID: 0 2024-05-11T23:04:18.026 INFO:teuthology.orchestra.run.smithi005.stdout:All operations completed A-OK! 2024-05-11T23:04:18.045 DEBUG:teuthology.parallel:result is None 2024-05-11T23:04:18.046 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-05-11T23:04:18.055 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-05-11T23:04:18.073 INFO:tasks.thrashosds:joining thrashosds 2024-05-11T23:04:18.091 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:18.091 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:04:18.091 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:18.099 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-11T23:04:18.206 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:18.206 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:18.206 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:04:18.206 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:04:18.206 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:18.213 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-11T23:04:18.320 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:18.320 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:04:18.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:04:18.320 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:04:18.320 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.320 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267820 3.16 3:6b963329:::rbd_data.111ec06c817a.000000000000000a:33 [read 1036288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:18.320 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:04:18.320 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.376493726000007, 2024-05-11T23:04:18.320 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050141218000000001, 2024-05-11T23:04:18.320 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267820 2024-05-11T23:04:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:04:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.321 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942655+0000", 2024-05-11T23:04:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942754+0000", 2024-05-11T23:04:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8559000000000003e-05 2024-05-11T23:04:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942756+0000", 2024-05-11T23:04:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.604e-06 2024-05-11T23:04:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.322 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942763+0000", 2024-05-11T23:04:18.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7939999999999997e-06 2024-05-11T23:04:18.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942768+0000", 2024-05-11T23:04:18.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4410000000000001e-06 2024-05-11T23:04:18.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.323 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.323 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992601+0000", 2024-05-11T23:04:18.323 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049832754999999999 2024-05-11T23:04:18.323 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.323 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992687+0000", 2024-05-11T23:04:18.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5605000000000005e-05 2024-05-11T23:04:18.324 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.324 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.324 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.324 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992796+0000", 2024-05-11T23:04:18.324 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010946 2024-05-11T23:04:18.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.324 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.324 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.325 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.325 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.325 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267852 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:18.325 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:04:18.325 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.376372521999997, 2024-05-11T23:04:18.325 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050212600000000003, 2024-05-11T23:04:18.325 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.325 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.325 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.325 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.326 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.326 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267852 2024-05-11T23:04:18.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.326 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:04:18.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.326 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.326 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.326 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.942776+0000", 2024-05-11T23:04:18.326 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.326 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943228+0000", 2024-05-11T23:04:18.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00045180600000000002 2024-05-11T23:04:18.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943232+0000", 2024-05-11T23:04:18.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8399999999999997e-06 2024-05-11T23:04:18.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.327 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.327 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.327 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943240+0000", 2024-05-11T23:04:18.327 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8900000000000007e-06 2024-05-11T23:04:18.327 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943246+0000", 2024-05-11T23:04:18.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6239999999999996e-06 2024-05-11T23:04:18.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992828+0000", 2024-05-11T23:04:18.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049581247000000002 2024-05-11T23:04:18.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.328 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.328 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.328 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992905+0000", 2024-05-11T23:04:18.328 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.7656000000000003e-05 2024-05-11T23:04:18.328 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.329 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.329 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.992989+0000", 2024-05-11T23:04:18.329 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3536999999999997e-05 2024-05-11T23:04:18.329 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.329 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.329 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.329 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.329 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.329 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267854 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1134592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:18.329 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:04:18.330 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.375859380999998, 2024-05-11T23:04:18.330 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050077425000000002, 2024-05-11T23:04:18.330 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.330 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.330 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.330 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.330 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.330 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.330 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267854 2024-05-11T23:04:18.330 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.330 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.330 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.330 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.330 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:04:18.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943289+0000", 2024-05-11T23:04:18.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943290+0000", 2024-05-11T23:04:18.331 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2300000000000002e-07 2024-05-11T23:04:18.331 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.331 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.331 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.331 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943291+0000", 2024-05-11T23:04:18.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6190000000000001e-06 2024-05-11T23:04:18.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943296+0000", 2024-05-11T23:04:18.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2889999999999999e-06 2024-05-11T23:04:18.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943299+0000", 2024-05-11T23:04:18.332 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1030000000000002e-06 2024-05-11T23:04:18.332 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.332 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.332 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.332 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993205+0000", 2024-05-11T23:04:18.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049906405000000001 2024-05-11T23:04:18.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993285+0000", 2024-05-11T23:04:18.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9752999999999993e-05 2024-05-11T23:04:18.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.333 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.333 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.333 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993367+0000", 2024-05-11T23:04:18.333 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1933000000000003e-05 2024-05-11T23:04:18.333 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.333 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.333 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.333 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.334 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.334 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267855 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1171456~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:18.334 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:04:18.334 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.375842438999996, 2024-05-11T23:04:18.334 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050250346000000001, 2024-05-11T23:04:18.334 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.334 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.334 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.334 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.334 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267855 2024-05-11T23:04:18.334 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.335 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:04:18.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.943306+0000", 2024-05-11T23:04:18.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.335 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.335 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.335 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.335 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945226+0000", 2024-05-11T23:04:18.335 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001919235 2024-05-11T23:04:18.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945230+0000", 2024-05-11T23:04:18.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9580000000000001e-06 2024-05-11T23:04:18.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945238+0000", 2024-05-11T23:04:18.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1040000000000007e-06 2024-05-11T23:04:18.336 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.336 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.336 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.336 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945244+0000", 2024-05-11T23:04:18.336 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6919999999999996e-06 2024-05-11T23:04:18.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993399+0000", 2024-05-11T23:04:18.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048154651 2024-05-11T23:04:18.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993472+0000", 2024-05-11T23:04:18.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2780000000000005e-05 2024-05-11T23:04:18.337 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.337 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.337 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.337 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.993557+0000", 2024-05-11T23:04:18.337 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4926000000000006e-05 2024-05-11T23:04:18.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.338 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.338 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.338 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.338 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.338 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:267865 3.16 3:68965212:::rbd_data.111ec06c817a.0000000000000000:33 [read 1540096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:18.338 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:04:18.338 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.373690812999996, 2024-05-11T23:04:18.338 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050054409000000001, 2024-05-11T23:04:18.338 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.338 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.338 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.338 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.339 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.339 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 267865 2024-05-11T23:04:18.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.339 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:04:18.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.339 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.339 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.339 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:04:18.339 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.339 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945458+0000", 2024-05-11T23:04:18.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6600000000000002e-07 2024-05-11T23:04:18.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945461+0000", 2024-05-11T23:04:18.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2670000000000001e-06 2024-05-11T23:04:18.340 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.340 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.340 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.340 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945466+0000", 2024-05-11T23:04:18.340 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7200000000000003e-06 2024-05-11T23:04:18.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.945471+0000", 2024-05-11T23:04:18.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9439999999999996e-06 2024-05-11T23:04:18.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995337+0000", 2024-05-11T23:04:18.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049865415000000003 2024-05-11T23:04:18.341 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.341 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.341 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.341 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995413+0000", 2024-05-11T23:04:18.341 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5989999999999996e-05 2024-05-11T23:04:18.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.342 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.342 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.995512+0000", 2024-05-11T23:04:18.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9706999999999999e-05 2024-05-11T23:04:18.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.342 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.342 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.342 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.342 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.342 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268018 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 872448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:18.342 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:04:18.342 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.371714748000002, 2024-05-11T23:04:18.342 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050170927999999997, 2024-05-11T23:04:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268018 2024-05-11T23:04:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:04:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.343 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947434+0000", 2024-05-11T23:04:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947435+0000", 2024-05-11T23:04:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3099999999999997e-07 2024-05-11T23:04:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947437+0000", 2024-05-11T23:04:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6340000000000001e-06 2024-05-11T23:04:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.344 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947443+0000", 2024-05-11T23:04:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7810000000000002e-06 2024-05-11T23:04:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947448+0000", 2024-05-11T23:04:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9679999999999997e-06 2024-05-11T23:04:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997418+0000", 2024-05-11T23:04:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049969587000000003 2024-05-11T23:04:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.345 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997504+0000", 2024-05-11T23:04:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6426000000000002e-05 2024-05-11T23:04:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997605+0000", 2024-05-11T23:04:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000100901 2024-05-11T23:04:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268019 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 909312~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:18.346 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:04:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.371694189999999, 2024-05-11T23:04:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050285074999999999, 2024-05-11T23:04:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268019 2024-05-11T23:04:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.347 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:04:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:04:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947455+0000", 2024-05-11T23:04:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0999999999999999e-07 2024-05-11T23:04:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.348 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947457+0000", 2024-05-11T23:04:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0099999999999998e-06 2024-05-11T23:04:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947462+0000", 2024-05-11T23:04:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2410000000000003e-06 2024-05-11T23:04:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947467+0000", 2024-05-11T23:04:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8339999999999998e-06 2024-05-11T23:04:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.349 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997639+0000", 2024-05-11T23:04:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050171697000000001 2024-05-11T23:04:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997685+0000", 2024-05-11T23:04:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6146000000000002e-05 2024-05-11T23:04:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997740+0000", 2024-05-11T23:04:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4737000000000001e-05 2024-05-11T23:04:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.350 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268020 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:04:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.371675945000007, 2024-05-11T23:04:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050504089000000002, 2024-05-11T23:04:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268020 2024-05-11T23:04:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.351 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:04:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947473+0000", 2024-05-11T23:04:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947591+0000", 2024-05-11T23:04:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011805 2024-05-11T23:04:18.352 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947594+0000", 2024-05-11T23:04:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.1549999999999999e-06 2024-05-11T23:04:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947602+0000", 2024-05-11T23:04:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.174e-06 2024-05-11T23:04:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947608+0000", 2024-05-11T23:04:18.353 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.263e-06 2024-05-11T23:04:18.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997775+0000", 2024-05-11T23:04:18.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050166980999999999 2024-05-11T23:04:18.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997875+0000", 2024-05-11T23:04:18.354 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9844000000000002e-05 2024-05-11T23:04:18.354 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.354 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.354 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.354 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.997977+0000", 2024-05-11T23:04:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101622 2024-05-11T23:04:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268021 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 983040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:04:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.371530484999994, 2024-05-11T23:04:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050498874999999999, 2024-05-11T23:04:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.355 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268021 2024-05-11T23:04:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:04:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947618+0000", 2024-05-11T23:04:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.356 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947695+0000", 2024-05-11T23:04:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6247000000000004e-05 2024-05-11T23:04:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947698+0000", 2024-05-11T23:04:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9199999999999997e-06 2024-05-11T23:04:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947706+0000", 2024-05-11T23:04:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:04:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.357 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947711+0000", 2024-05-11T23:04:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6720000000000001e-06 2024-05-11T23:04:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998017+0000", 2024-05-11T23:04:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050305470999999997 2024-05-11T23:04:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998060+0000", 2024-05-11T23:04:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3374000000000002e-05 2024-05-11T23:04:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.358 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998117+0000", 2024-05-11T23:04:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6971000000000001e-05 2024-05-11T23:04:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268022 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1019904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:04:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.371427448999995, 2024-05-11T23:04:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050731361000000003, 2024-05-11T23:04:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.359 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268022 2024-05-11T23:04:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:04:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947721+0000", 2024-05-11T23:04:18.360 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947722+0000", 2024-05-11T23:04:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8999999999999996e-07 2024-05-11T23:04:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947724+0000", 2024-05-11T23:04:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1639999999999999e-06 2024-05-11T23:04:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947730+0000", 2024-05-11T23:04:18.361 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6099999999999997e-06 2024-05-11T23:04:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947735+0000", 2024-05-11T23:04:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7189999999999998e-06 2024-05-11T23:04:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998140+0000", 2024-05-11T23:04:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050404254000000003 2024-05-11T23:04:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.362 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998239+0000", 2024-05-11T23:04:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9192999999999997e-05 2024-05-11T23:04:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998453+0000", 2024-05-11T23:04:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00021383099999999999 2024-05-11T23:04:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268023 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1069056~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:04:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.371405183999997, 2024-05-11T23:04:18.363 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050874509999999998, 2024-05-11T23:04:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268023 2024-05-11T23:04:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:04:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.364 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.947744+0000", 2024-05-11T23:04:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948934+0000", 2024-05-11T23:04:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0011903339999999999 2024-05-11T23:04:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948938+0000", 2024-05-11T23:04:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1330000000000001e-06 2024-05-11T23:04:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.365 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948949+0000", 2024-05-11T23:04:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0872e-05 2024-05-11T23:04:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.948956+0000", 2024-05-11T23:04:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.131e-06 2024-05-11T23:04:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998494+0000", 2024-05-11T23:04:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049537678000000002 2024-05-11T23:04:18.366 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998551+0000", 2024-05-11T23:04:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7256000000000002e-05 2024-05-11T23:04:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.998618+0000", 2024-05-11T23:04:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7106000000000004e-05 2024-05-11T23:04:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268026 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1179648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:18.367 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:04:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.370128131000001, 2024-05-11T23:04:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050107977999999997, 2024-05-11T23:04:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268026 2024-05-11T23:04:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:04:18.368 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949021+0000", 2024-05-11T23:04:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949029+0000", 2024-05-11T23:04:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2220000000000002e-06 2024-05-11T23:04:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949041+0000", 2024-05-11T23:04:18.369 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.241e-05 2024-05-11T23:04:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949049+0000", 2024-05-11T23:04:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.072e-06 2024-05-11T23:04:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949056+0000", 2024-05-11T23:04:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1380000000000001e-06 2024-05-11T23:04:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999016+0000", 2024-05-11T23:04:18.370 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.049959212000000003 2024-05-11T23:04:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999067+0000", 2024-05-11T23:04:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.1598000000000003e-05 2024-05-11T23:04:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999129+0000", 2024-05-11T23:04:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1326000000000002e-05 2024-05-11T23:04:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.371 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268027 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1216512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:04:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.370083980999993, 2024-05-11T23:04:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050189865, 2024-05-11T23:04:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268027 2024-05-11T23:04:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.372 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:04:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:04:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949065+0000", 2024-05-11T23:04:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7999999999999996e-07 2024-05-11T23:04:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949068+0000", 2024-05-11T23:04:18.373 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.26e-06 2024-05-11T23:04:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949074+0000", 2024-05-11T23:04:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7050000000000002e-06 2024-05-11T23:04:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949080+0000", 2024-05-11T23:04:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3689999999999998e-06 2024-05-11T23:04:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.374 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999152+0000", 2024-05-11T23:04:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050072653000000002 2024-05-11T23:04:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999197+0000", 2024-05-11T23:04:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4972999999999998e-05 2024-05-11T23:04:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999255+0000", 2024-05-11T23:04:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7425e-05 2024-05-11T23:04:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.375 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268028 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1253376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:04:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.370062321000006, 2024-05-11T23:04:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050280017000000003, 2024-05-11T23:04:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268028 2024-05-11T23:04:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.376 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:04:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949086+0000", 2024-05-11T23:04:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.377 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949087+0000", 2024-05-11T23:04:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.03e-07 2024-05-11T23:04:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949089+0000", 2024-05-11T23:04:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8950000000000001e-06 2024-05-11T23:04:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949095+0000", 2024-05-11T23:04:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7540000000000003e-06 2024-05-11T23:04:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.378 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949114+0000", 2024-05-11T23:04:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9738e-05 2024-05-11T23:04:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999278+0000", 2024-05-11T23:04:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050163368999999999 2024-05-11T23:04:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999319+0000", 2024-05-11T23:04:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0979000000000002e-05 2024-05-11T23:04:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.379 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999366+0000", 2024-05-11T23:04:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7879e-05 2024-05-11T23:04:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268029 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1290240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:04:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.370022212999999, 2024-05-11T23:04:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050344904000000003, 2024-05-11T23:04:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.380 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268029 2024-05-11T23:04:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:04:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:04:18.381 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949127+0000", 2024-05-11T23:04:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9500000000000003e-07 2024-05-11T23:04:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949130+0000", 2024-05-11T23:04:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.729e-06 2024-05-11T23:04:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949139+0000", 2024-05-11T23:04:18.382 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9800000000000004e-06 2024-05-11T23:04:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949145+0000", 2024-05-11T23:04:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0510000000000002e-06 2024-05-11T23:04:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999387+0000", 2024-05-11T23:04:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050242167999999997 2024-05-11T23:04:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999427+0000", 2024-05-11T23:04:18.383 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9566e-05 2024-05-11T23:04:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999471+0000", 2024-05-11T23:04:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4914999999999999e-05 2024-05-11T23:04:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268030 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1327104~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:04:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.369996928999996, 2024-05-11T23:04:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050447394, 2024-05-11T23:04:18.384 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268030 2024-05-11T23:04:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:04:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.385 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:04:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949152+0000", 2024-05-11T23:04:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0500000000000004e-07 2024-05-11T23:04:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949155+0000", 2024-05-11T23:04:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3819999999999998e-06 2024-05-11T23:04:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.386 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949162+0000", 2024-05-11T23:04:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9609999999999999e-06 2024-05-11T23:04:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949188+0000", 2024-05-11T23:04:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5877e-05 2024-05-11T23:04:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999494+0000", 2024-05-11T23:04:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050306265000000003 2024-05-11T23:04:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.387 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999534+0000", 2024-05-11T23:04:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0447000000000001e-05 2024-05-11T23:04:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999599+0000", 2024-05-11T23:04:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4956999999999995e-05 2024-05-11T23:04:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268031 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1363968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:04:18.388 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.369951829000001, 2024-05-11T23:04:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050511525000000002, 2024-05-11T23:04:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268031 2024-05-11T23:04:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:04:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.389 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949197+0000", 2024-05-11T23:04:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949198+0000", 2024-05-11T23:04:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5700000000000002e-07 2024-05-11T23:04:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949200+0000", 2024-05-11T23:04:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7599999999999998e-06 2024-05-11T23:04:18.390 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949208+0000", 2024-05-11T23:04:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2840000000000001e-06 2024-05-11T23:04:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949214+0000", 2024-05-11T23:04:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9869999999999996e-06 2024-05-11T23:04:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999619+0000", 2024-05-11T23:04:18.391 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050405278999999997 2024-05-11T23:04:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999660+0000", 2024-05-11T23:04:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1427999999999997e-05 2024-05-11T23:04:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999708+0000", 2024-05-11T23:04:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8229999999999997e-05 2024-05-11T23:04:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.392 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268032 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1400832~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:04:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.369922208999995, 2024-05-11T23:04:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050611863999999999, 2024-05-11T23:04:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268032 2024-05-11T23:04:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.393 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:04:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949227+0000", 2024-05-11T23:04:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949230+0000", 2024-05-11T23:04:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.371e-06 2024-05-11T23:04:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.394 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949233+0000", 2024-05-11T23:04:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.6599999999999999e-06 2024-05-11T23:04:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949241+0000", 2024-05-11T23:04:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8939999999999993e-06 2024-05-11T23:04:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949246+0000", 2024-05-11T23:04:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6590000000000004e-06 2024-05-11T23:04:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.395 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999733+0000", 2024-05-11T23:04:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050487268000000002 2024-05-11T23:04:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999784+0000", 2024-05-11T23:04:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0562000000000001e-05 2024-05-11T23:04:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999838+0000", 2024-05-11T23:04:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4450000000000002e-05 2024-05-11T23:04:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.396 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:268033 3.16 3:6b3f074b:::rbd_data.111ec06c817a.0000000000000008:33 [read 1437696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-05-11T23:04:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:04:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 65.369895081999999, 2024-05-11T23:04:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.050697157999999999, 2024-05-11T23:04:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 268033 2024-05-11T23:04:18.397 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:04:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.949254+0000", 2024-05-11T23:04:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950862+0000", 2024-05-11T23:04:18.398 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.001607975 2024-05-11T23:04:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950866+0000", 2024-05-11T23:04:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7770000000000002e-06 2024-05-11T23:04:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950878+0000", 2024-05-11T23:04:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1421e-05 2024-05-11T23:04:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.399 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.950885+0000", 2024-05-11T23:04:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.6000000000000001e-06 2024-05-11T23:04:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999859+0000", 2024-05-11T23:04:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048973199000000002 2024-05-11T23:04:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999901+0000", 2024-05-11T23:04:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1943000000000001e-05 2024-05-11T23:04:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.400 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:12.999951+0000", 2024-05-11T23:04:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0243e-05 2024-05-11T23:04:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:283084 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e147)", 2024-05-11T23:04:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:03:49.906190+0000", 2024-05-11T23:04:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 28.412959132000001, 2024-05-11T23:04:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.89363789900000001, 2024-05-11T23:04:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.401 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 283084 2024-05-11T23:04:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906190+0000", 2024-05-11T23:04:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906190+0000", 2024-05-11T23:04:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.402 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906191+0000", 2024-05-11T23:04:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6029999999999999e-06 2024-05-11T23:04:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906198+0000", 2024-05-11T23:04:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7170000000000002e-06 2024-05-11T23:04:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906214+0000", 2024-05-11T23:04:18.403 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6290000000000002e-05 2024-05-11T23:04:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906229+0000", 2024-05-11T23:04:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.4735999999999999e-05 2024-05-11T23:04:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906295+0000", 2024-05-11T23:04:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5604000000000004e-05 2024-05-11T23:04:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:04:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:49.906305+0000", 2024-05-11T23:04:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0795000000000001e-05 2024-05-11T23:04:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.404 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:50.799618+0000", 2024-05-11T23:04:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.89331292600000001 2024-05-11T23:04:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:50.799711+0000", 2024-05-11T23:04:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2406999999999998e-05 2024-05-11T23:04:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:03:50.799828+0000", 2024-05-11T23:04:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116821 2024-05-11T23:04:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.405 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.406 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.406 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.406 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:18.407 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-11T23:04:18.513 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:18.513 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:18.513 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:04:18.513 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:18.522 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-11T23:04:18.628 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:18.628 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:18.628 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:04:18.628 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:04:18.628 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:18.637 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-11T23:04:18.744 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:18.744 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:04:18.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:04:18.744 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:04:18.744 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.744 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34402 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 630784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:18.744 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:04:18.744 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.08935075799999, 2024-05-11T23:04:18.744 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023085369000000001, 2024-05-11T23:04:18.744 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.744 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.744 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.745 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.745 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34402 2024-05-11T23:04:18.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.745 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:04:18.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.745 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.745 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.745 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.745 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:04:18.745 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653801+0000", 2024-05-11T23:04:18.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6700000000000003e-07 2024-05-11T23:04:18.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.746 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653813+0000", 2024-05-11T23:04:18.746 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2058999999999999e-05 2024-05-11T23:04:18.746 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.746 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.746 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653821+0000", 2024-05-11T23:04:18.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1540000000000002e-06 2024-05-11T23:04:18.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.653827+0000", 2024-05-11T23:04:18.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0340000000000002e-06 2024-05-11T23:04:18.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.747 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676715+0000", 2024-05-11T23:04:18.747 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022887940999999998 2024-05-11T23:04:18.747 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.747 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.747 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676790+0000", 2024-05-11T23:04:18.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4334999999999999e-05 2024-05-11T23:04:18.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.748 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.748 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.676886+0000", 2024-05-11T23:04:18.748 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6279000000000004e-05 2024-05-11T23:04:18.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.748 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.748 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.748 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.748 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.748 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34641 3.10 3:0a73a884:::rbd_data.111e5e4d8275.000000000000002c:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:18.748 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:04:18.748 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.08596956900001, 2024-05-11T23:04:18.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022946639000000001, 2024-05-11T23:04:18.749 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.749 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.749 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.749 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.749 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34641 2024-05-11T23:04:18.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.749 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.749 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.749 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.749 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:04:18.749 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.749 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:04:18.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657182+0000", 2024-05-11T23:04:18.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.4900000000000001e-07 2024-05-11T23:04:18.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.750 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.750 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.750 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657184+0000", 2024-05-11T23:04:18.750 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8300000000000001e-06 2024-05-11T23:04:18.750 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657190+0000", 2024-05-11T23:04:18.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6910000000000002e-06 2024-05-11T23:04:18.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657195+0000", 2024-05-11T23:04:18.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3410000000000002e-06 2024-05-11T23:04:18.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.751 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.751 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.751 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.679981+0000", 2024-05-11T23:04:18.751 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022786508 2024-05-11T23:04:18.751 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680060+0000", 2024-05-11T23:04:18.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8607000000000001e-05 2024-05-11T23:04:18.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.752 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.752 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680128+0000", 2024-05-11T23:04:18.752 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8312999999999995e-05 2024-05-11T23:04:18.752 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.752 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.752 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.752 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.752 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.752 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34642 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 32768~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:18.752 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:04:18.753 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.085950862, 2024-05-11T23:04:18.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023133918, 2024-05-11T23:04:18.753 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.753 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.753 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.753 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.753 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.753 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.753 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34642 2024-05-11T23:04:18.753 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.753 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.753 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.753 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.753 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:04:18.753 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657200+0000", 2024-05-11T23:04:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657201+0000", 2024-05-11T23:04:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.2100000000000002e-07 2024-05-11T23:04:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657203+0000", 2024-05-11T23:04:18.754 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8339999999999999e-06 2024-05-11T23:04:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657208+0000", 2024-05-11T23:04:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6289999999999998e-06 2024-05-11T23:04:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657213+0000", 2024-05-11T23:04:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5499999999999996e-06 2024-05-11T23:04:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680158+0000", 2024-05-11T23:04:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.022944737999999999 2024-05-11T23:04:18.755 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680242+0000", 2024-05-11T23:04:18.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4322999999999999e-05 2024-05-11T23:04:18.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.756 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.756 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680334+0000", 2024-05-11T23:04:18.756 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2423000000000006e-05 2024-05-11T23:04:18.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.756 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.756 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.756 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.756 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.756 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34643 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 69632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:04:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.085932719, 2024-05-11T23:04:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023305889999999999, 2024-05-11T23:04:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34643 2024-05-11T23:04:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.757 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:04:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:04:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657219+0000", 2024-05-11T23:04:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6800000000000001e-07 2024-05-11T23:04:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.758 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657228+0000", 2024-05-11T23:04:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6410000000000008e-06 2024-05-11T23:04:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657234+0000", 2024-05-11T23:04:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0889999999999996e-06 2024-05-11T23:04:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657239+0000", 2024-05-11T23:04:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8149999999999996e-06 2024-05-11T23:04:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.759 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680374+0000", 2024-05-11T23:04:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023135271999999998 2024-05-11T23:04:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680455+0000", 2024-05-11T23:04:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0992e-05 2024-05-11T23:04:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680525+0000", 2024-05-11T23:04:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9713000000000002e-05 2024-05-11T23:04:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.760 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34644 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 106496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:04:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.08590523199999, 2024-05-11T23:04:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023474332000000001, 2024-05-11T23:04:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34644 2024-05-11T23:04:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.761 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:04:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657246+0000", 2024-05-11T23:04:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657247+0000", 2024-05-11T23:04:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4400000000000001e-07 2024-05-11T23:04:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.762 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657249+0000", 2024-05-11T23:04:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.34e-06 2024-05-11T23:04:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657255+0000", 2024-05-11T23:04:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4969999999999998e-06 2024-05-11T23:04:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657260+0000", 2024-05-11T23:04:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.814e-06 2024-05-11T23:04:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.763 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680555+0000", 2024-05-11T23:04:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023295093999999999 2024-05-11T23:04:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680627+0000", 2024-05-11T23:04:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2071999999999997e-05 2024-05-11T23:04:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680720+0000", 2024-05-11T23:04:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3071000000000004e-05 2024-05-11T23:04:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.764 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34645 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 143360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:04:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.08588470399999, 2024-05-11T23:04:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023642696000000001, 2024-05-11T23:04:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34645 2024-05-11T23:04:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.765 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:04:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:04:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657267+0000", 2024-05-11T23:04:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8299999999999998e-07 2024-05-11T23:04:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.766 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657269+0000", 2024-05-11T23:04:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.0770000000000002e-06 2024-05-11T23:04:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657276+0000", 2024-05-11T23:04:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.421e-06 2024-05-11T23:04:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657280+0000", 2024-05-11T23:04:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5940000000000004e-06 2024-05-11T23:04:18.767 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680759+0000", 2024-05-11T23:04:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023478691999999999 2024-05-11T23:04:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680838+0000", 2024-05-11T23:04:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9493000000000006e-05 2024-05-11T23:04:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680909+0000", 2024-05-11T23:04:18.768 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1036000000000002e-05 2024-05-11T23:04:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34646 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 180224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:04:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.085864929, 2024-05-11T23:04:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023820456, 2024-05-11T23:04:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.769 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34646 2024-05-11T23:04:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:04:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657286+0000", 2024-05-11T23:04:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.770 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657287+0000", 2024-05-11T23:04:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5600000000000001e-07 2024-05-11T23:04:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657289+0000", 2024-05-11T23:04:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.362e-06 2024-05-11T23:04:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657295+0000", 2024-05-11T23:04:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9989999999999997e-06 2024-05-11T23:04:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.771 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657300+0000", 2024-05-11T23:04:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7720000000000002e-06 2024-05-11T23:04:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.680939+0000", 2024-05-11T23:04:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023639262000000001 2024-05-11T23:04:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681012+0000", 2024-05-11T23:04:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2917000000000007e-05 2024-05-11T23:04:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.772 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681107+0000", 2024-05-11T23:04:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4788000000000001e-05 2024-05-11T23:04:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34647 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 217088~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:04:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.08584612300001, 2024-05-11T23:04:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999738999999999, 2024-05-11T23:04:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.773 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34647 2024-05-11T23:04:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:04:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657305+0000", 2024-05-11T23:04:18.774 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657306+0000", 2024-05-11T23:04:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3000000000000002e-07 2024-05-11T23:04:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657307+0000", 2024-05-11T23:04:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.9130000000000001e-06 2024-05-11T23:04:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657313+0000", 2024-05-11T23:04:18.775 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6200000000000004e-06 2024-05-11T23:04:18.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657319+0000", 2024-05-11T23:04:18.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.7250000000000002e-06 2024-05-11T23:04:18.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681145+0000", 2024-05-11T23:04:18.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023826101999999998 2024-05-11T23:04:18.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.776 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.776 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.776 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681231+0000", 2024-05-11T23:04:18.776 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.5699999999999996e-05 2024-05-11T23:04:18.776 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.777 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.777 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681305+0000", 2024-05-11T23:04:18.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.4349000000000003e-05 2024-05-11T23:04:18.777 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.777 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.777 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.777 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.777 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.777 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34648 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 253952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:18.777 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:04:18.777 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.08582720000001, 2024-05-11T23:04:18.777 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024181595, 2024-05-11T23:04:18.777 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.777 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.777 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.778 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.778 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.778 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.778 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34648 2024-05-11T23:04:18.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.778 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.778 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:04:18.778 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.778 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.778 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.778 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:04:18.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657324+0000", 2024-05-11T23:04:18.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5100000000000001e-07 2024-05-11T23:04:18.779 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.779 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.779 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.779 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657327+0000", 2024-05-11T23:04:18.779 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.136e-06 2024-05-11T23:04:18.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657332+0000", 2024-05-11T23:04:18.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8119999999999996e-06 2024-05-11T23:04:18.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657337+0000", 2024-05-11T23:04:18.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8369999999999996e-06 2024-05-11T23:04:18.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.780 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.780 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.780 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681336+0000", 2024-05-11T23:04:18.780 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.023999198999999999 2024-05-11T23:04:18.780 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681410+0000", 2024-05-11T23:04:18.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3421000000000001e-05 2024-05-11T23:04:18.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.781 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.781 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681506+0000", 2024-05-11T23:04:18.781 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.5839000000000002e-05 2024-05-11T23:04:18.781 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.781 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.781 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.781 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.781 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.781 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34649 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 290816~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:18.782 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:04:18.782 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.08580853800001, 2024-05-11T23:04:18.782 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024357444999999998, 2024-05-11T23:04:18.782 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.782 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.782 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.782 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.782 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.782 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.782 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34649 2024-05-11T23:04:18.782 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.782 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.782 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.782 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.782 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:04:18.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:04:18.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657343+0000", 2024-05-11T23:04:18.783 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2399999999999999e-07 2024-05-11T23:04:18.783 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.783 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.783 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.783 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657354+0000", 2024-05-11T23:04:18.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.047e-05 2024-05-11T23:04:18.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657360+0000", 2024-05-11T23:04:18.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.1040000000000003e-06 2024-05-11T23:04:18.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.784 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657365+0000", 2024-05-11T23:04:18.784 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9049999999999996e-06 2024-05-11T23:04:18.784 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.784 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.784 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681543+0000", 2024-05-11T23:04:18.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024178846 2024-05-11T23:04:18.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681625+0000", 2024-05-11T23:04:18.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1731999999999996e-05 2024-05-11T23:04:18.785 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.785 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.785 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.785 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681700+0000", 2024-05-11T23:04:18.785 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5063999999999998e-05 2024-05-11T23:04:18.785 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.785 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.785 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.786 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.786 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34650 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:18.786 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:04:18.786 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.08546337600001, 2024-05-11T23:04:18.786 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024205634, 2024-05-11T23:04:18.786 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.786 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.786 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.786 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.786 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.786 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.786 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34650 2024-05-11T23:04:18.786 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.787 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:04:18.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657688+0000", 2024-05-11T23:04:18.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.787 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.787 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.787 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657689+0000", 2024-05-11T23:04:18.787 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.3600000000000003e-07 2024-05-11T23:04:18.787 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657691+0000", 2024-05-11T23:04:18.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.7350000000000001e-06 2024-05-11T23:04:18.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657699+0000", 2024-05-11T23:04:18.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.106e-06 2024-05-11T23:04:18.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.788 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657705+0000", 2024-05-11T23:04:18.788 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.784e-06 2024-05-11T23:04:18.788 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.788 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.788 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681729+0000", 2024-05-11T23:04:18.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024023973000000001 2024-05-11T23:04:18.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681799+0000", 2024-05-11T23:04:18.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0030000000000005e-05 2024-05-11T23:04:18.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.789 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.789 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.789 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681894+0000", 2024-05-11T23:04:18.789 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4370000000000006e-05 2024-05-11T23:04:18.789 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.789 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.789 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.789 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.790 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.790 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34651 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 364544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:18.790 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:04:18.790 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.085437826, 2024-05-11T23:04:18.790 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024376912000000001, 2024-05-11T23:04:18.790 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.790 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.790 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.790 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.790 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.790 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.790 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34651 2024-05-11T23:04:18.790 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.790 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:04:18.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:04:18.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.791 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657714+0000", 2024-05-11T23:04:18.791 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.96e-07 2024-05-11T23:04:18.791 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.791 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.791 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657720+0000", 2024-05-11T23:04:18.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0730000000000001e-06 2024-05-11T23:04:18.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657727+0000", 2024-05-11T23:04:18.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5560000000000004e-06 2024-05-11T23:04:18.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657732+0000", 2024-05-11T23:04:18.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.0830000000000003e-06 2024-05-11T23:04:18.792 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.792 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.792 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.792 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.681932+0000", 2024-05-11T23:04:18.792 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024200531000000001 2024-05-11T23:04:18.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682011+0000", 2024-05-11T23:04:18.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8924999999999994e-05 2024-05-11T23:04:18.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.793 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.793 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682090+0000", 2024-05-11T23:04:18.793 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9348e-05 2024-05-11T23:04:18.793 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.793 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.793 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.793 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.793 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.793 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34652 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 401408~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:18.793 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:04:18.794 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.08535556800001, 2024-05-11T23:04:18.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024495053999999999, 2024-05-11T23:04:18.794 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.794 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.794 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.794 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.794 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.794 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.794 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34652 2024-05-11T23:04:18.794 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.794 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.794 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.794 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.794 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:04:18.794 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:04:18.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657796+0000", 2024-05-11T23:04:18.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.06e-07 2024-05-11T23:04:18.795 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.795 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.795 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.795 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657799+0000", 2024-05-11T23:04:18.795 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3920000000000001e-06 2024-05-11T23:04:18.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657806+0000", 2024-05-11T23:04:18.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.092e-06 2024-05-11T23:04:18.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657811+0000", 2024-05-11T23:04:18.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4670000000000003e-06 2024-05-11T23:04:18.796 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.796 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.796 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.796 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682122+0000", 2024-05-11T23:04:18.796 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024310333999999999 2024-05-11T23:04:18.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682191+0000", 2024-05-11T23:04:18.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9850000000000004e-05 2024-05-11T23:04:18.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.797 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.797 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682291+0000", 2024-05-11T23:04:18.797 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9413000000000005e-05 2024-05-11T23:04:18.797 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.797 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.797 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.797 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.797 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34653 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 438272~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:04:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.08533312099999, 2024-05-11T23:04:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02467689, 2024-05-11T23:04:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34653 2024-05-11T23:04:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.798 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:04:18.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657818+0000", 2024-05-11T23:04:18.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.799 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.799 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.799 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657819+0000", 2024-05-11T23:04:18.799 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.8500000000000002e-07 2024-05-11T23:04:18.799 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657821+0000", 2024-05-11T23:04:18.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1179999999999999e-06 2024-05-11T23:04:18.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657827+0000", 2024-05-11T23:04:18.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0460000000000002e-06 2024-05-11T23:04:18.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.800 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.800 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.800 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657832+0000", 2024-05-11T23:04:18.800 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7659999999999998e-06 2024-05-11T23:04:18.800 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682330+0000", 2024-05-11T23:04:18.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024498789 2024-05-11T23:04:18.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682411+0000", 2024-05-11T23:04:18.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.0914000000000005e-05 2024-05-11T23:04:18.801 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.801 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.801 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.801 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682495+0000", 2024-05-11T23:04:18.801 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3871999999999999e-05 2024-05-11T23:04:18.801 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.801 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.802 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.802 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.802 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.802 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34654 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 475136~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:18.802 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:04:18.802 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.08531366299999, 2024-05-11T23:04:18.802 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024856438000000002, 2024-05-11T23:04:18.802 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.802 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.802 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.802 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.802 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.802 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.802 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34654 2024-05-11T23:04:18.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.803 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:04:18.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:04:18.803 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.803 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.803 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.803 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.803 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657838+0000", 2024-05-11T23:04:18.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7e-07 2024-05-11T23:04:18.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657840+0000", 2024-05-11T23:04:18.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.001e-06 2024-05-11T23:04:18.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657847+0000", 2024-05-11T23:04:18.804 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5100000000000004e-06 2024-05-11T23:04:18.804 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.804 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.804 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.804 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657851+0000", 2024-05-11T23:04:18.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8840000000000002e-06 2024-05-11T23:04:18.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682528+0000", 2024-05-11T23:04:18.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024676410999999999 2024-05-11T23:04:18.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682601+0000", 2024-05-11T23:04:18.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2803000000000001e-05 2024-05-11T23:04:18.805 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.805 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.805 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.805 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682694+0000", 2024-05-11T23:04:18.805 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.3459000000000001e-05 2024-05-11T23:04:18.806 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.806 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.806 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.806 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.806 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.806 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34655 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 512000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:18.806 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:04:18.806 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.085293603, 2024-05-11T23:04:18.806 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025039428999999998, 2024-05-11T23:04:18.806 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.806 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.806 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.806 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.806 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.806 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.807 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34655 2024-05-11T23:04:18.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.807 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:04:18.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.807 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.807 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:04:18.807 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.807 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.807 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657858+0000", 2024-05-11T23:04:18.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0600000000000001e-07 2024-05-11T23:04:18.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657860+0000", 2024-05-11T23:04:18.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.916e-06 2024-05-11T23:04:18.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.808 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.808 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657866+0000", 2024-05-11T23:04:18.808 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0599999999999996e-06 2024-05-11T23:04:18.808 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.808 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657871+0000", 2024-05-11T23:04:18.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8879999999999996e-06 2024-05-11T23:04:18.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682732+0000", 2024-05-11T23:04:18.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024860641999999999 2024-05-11T23:04:18.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.809 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.809 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682811+0000", 2024-05-11T23:04:18.809 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8993999999999996e-05 2024-05-11T23:04:18.809 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.809 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.810 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.810 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682897+0000", 2024-05-11T23:04:18.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6522999999999998e-05 2024-05-11T23:04:18.810 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.810 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.810 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.810 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.810 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.810 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34656 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 548864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:18.810 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:04:18.810 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.085274333, 2024-05-11T23:04:18.810 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025211291, 2024-05-11T23:04:18.810 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.810 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.811 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.811 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.811 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34656 2024-05-11T23:04:18.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.811 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.811 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:04:18.811 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.811 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.811 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.811 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:04:18.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657877+0000", 2024-05-11T23:04:18.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.6399999999999998e-07 2024-05-11T23:04:18.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.812 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657880+0000", 2024-05-11T23:04:18.812 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2010000000000002e-06 2024-05-11T23:04:18.812 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.812 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.812 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657885+0000", 2024-05-11T23:04:18.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.8109999999999999e-06 2024-05-11T23:04:18.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.657890+0000", 2024-05-11T23:04:18.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7979999999999996e-06 2024-05-11T23:04:18.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.813 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682925+0000", 2024-05-11T23:04:18.813 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025034935000000001 2024-05-11T23:04:18.813 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.813 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.813 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.682997+0000", 2024-05-11T23:04:18.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1489999999999995e-05 2024-05-11T23:04:18.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.814 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.814 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683088+0000", 2024-05-11T23:04:18.814 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1692000000000002e-05 2024-05-11T23:04:18.814 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.814 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.814 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.814 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.814 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.814 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34657 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 585728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:18.814 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:04:18.814 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.084728267, 2024-05-11T23:04:18.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024878747, 2024-05-11T23:04:18.815 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.815 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.815 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.815 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.815 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.815 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.815 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34657 2024-05-11T23:04:18.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.815 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.815 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.815 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.815 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:04:18.815 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.815 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658423+0000", 2024-05-11T23:04:18.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658424+0000", 2024-05-11T23:04:18.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9699999999999995e-07 2024-05-11T23:04:18.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.816 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.816 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.816 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658427+0000", 2024-05-11T23:04:18.816 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.2969999999999999e-06 2024-05-11T23:04:18.816 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658438+0000", 2024-05-11T23:04:18.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0458999999999999e-05 2024-05-11T23:04:18.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658444+0000", 2024-05-11T23:04:18.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7120000000000003e-06 2024-05-11T23:04:18.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.817 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.817 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.817 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683126+0000", 2024-05-11T23:04:18.817 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.02468184 2024-05-11T23:04:18.817 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683205+0000", 2024-05-11T23:04:18.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.8843000000000004e-05 2024-05-11T23:04:18.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.818 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.818 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683302+0000", 2024-05-11T23:04:18.818 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.6898999999999995e-05 2024-05-11T23:04:18.818 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.818 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.818 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.818 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.818 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.818 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:34658 3.10 3:0bd3c855:::rbd_data.111e5e4d8275.000000000000002d:11 [read 622592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-11T23:04:18.819 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:04:18.819 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 146.08469648299999, 2024-05-11T23:04:18.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.025043365000000001, 2024-05-11T23:04:18.819 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.819 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.819 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:18.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.819 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.819 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 34658 2024-05-11T23:04:18.819 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.819 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.819 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.819 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.819 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:04:18.819 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:04:18.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658455+0000", 2024-05-11T23:04:18.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.9900000000000001e-07 2024-05-11T23:04:18.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.820 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.820 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.820 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658458+0000", 2024-05-11T23:04:18.820 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.1830000000000001e-06 2024-05-11T23:04:18.820 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658465+0000", 2024-05-11T23:04:18.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0740000000000004e-06 2024-05-11T23:04:18.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.658470+0000", 2024-05-11T23:04:18.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.2259999999999996e-06 2024-05-11T23:04:18.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.821 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.821 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.821 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683331+0000", 2024-05-11T23:04:18.821 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024861606000000001 2024-05-11T23:04:18.821 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683406+0000", 2024-05-11T23:04:18.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5006000000000007e-05 2024-05-11T23:04:18.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.822 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.822 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:52.683498+0000", 2024-05-11T23:04:18.822 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1770999999999999e-05 2024-05-11T23:04:18.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.822 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.822 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.822 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.822 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.822 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:64399 3.1e 3:7f1683c3:::rbd_data.111ecf60147b.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 14054115~51366] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-11T23:04:18.823 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:04:18.823 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 136.64745101899999, 2024-05-11T23:04:18.823 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.024115527000000001, 2024-05-11T23:04:18.823 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:18.823 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:18.823 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:04:18.823 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:18.823 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:18.823 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:18.823 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 64399 2024-05-11T23:04:18.823 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.823 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:18.823 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.823 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:18.823 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:04:18.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:18.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095700+0000", 2024-05-11T23:04:18.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:18.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:18.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.095702+0000", 2024-05-11T23:04:18.824 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3230000000000001e-06 2024-05-11T23:04:18.824 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.824 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.824 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:18.824 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103650+0000", 2024-05-11T23:04:18.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0079488000000000007 2024-05-11T23:04:18.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:18.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103669+0000", 2024-05-11T23:04:18.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.8753000000000002e-05 2024-05-11T23:04:18.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:18.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103677+0000", 2024-05-11T23:04:18.825 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1280000000000008e-06 2024-05-11T23:04:18.825 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.825 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.825 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:18.825 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103748+0000", 2024-05-11T23:04:18.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.0432000000000006e-05 2024-05-11T23:04:18.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:18.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.103865+0000", 2024-05-11T23:04:18.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116838 2024-05-11T23:04:18.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 2", 2024-05-11T23:04:18.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.104436+0000", 2024-05-11T23:04:18.826 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00057132999999999999 2024-05-11T23:04:18.826 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.826 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.826 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:04:18.826 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.113383+0000", 2024-05-11T23:04:18.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0089468729999999993 2024-05-11T23:04:18.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:04:18.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119733+0000", 2024-05-11T23:04:18.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0063497759999999997 2024-05-11T23:04:18.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:04:18.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119772+0000", 2024-05-11T23:04:18.827 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9443000000000002e-05 2024-05-11T23:04:18.827 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:18.827 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:18.827 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:18.827 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:02.119816+0000", 2024-05-11T23:04:18.828 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3831000000000001e-05 2024-05-11T23:04:18.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.828 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.828 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:18.828 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:18.828 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:18.829 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-11T23:04:18.924 INFO:tasks.thrashosds.thrasher:reviving osd 2024-05-11T23:04:18.924 INFO:tasks.thrashosds.thrasher:Reviving osd 3 2024-05-11T23:04:18.924 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-11T23:04:18.924 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-11T23:04:18.927 INFO:tasks.ceph.osd.3:Started 2024-05-11T23:04:18.928 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:04:18.934 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:18.935 INFO:teuthology.orchestra.run.smithi115.stdout: "num_ops": 0 2024-05-11T23:04:18.935 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:18.943 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-11T23:04:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:no valid command found; 10 closest matches: 2024-05-11T23:04:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:0 2024-05-11T23:04:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:1 2024-05-11T23:04:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2 2024-05-11T23:04:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:abort 2024-05-11T23:04:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:assert 2024-05-11T23:04:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:config diff 2024-05-11T23:04:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:config diff get 2024-05-11T23:04:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:config get 2024-05-11T23:04:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:config help [] 2024-05-11T23:04:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:config set ... 2024-05-11T23:04:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:admin_socket: invalid command 2024-05-11T23:04:19.046 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-11T23:04:19.047 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-05-11T23:04:19.048 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:19.049 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [], 2024-05-11T23:04:19.049 INFO:teuthology.orchestra.run.smithi115.stdout: "complaint_time": 30, 2024-05-11T23:04:19.049 INFO:teuthology.orchestra.run.smithi115.stdout: "num_blocked_ops": 0 2024-05-11T23:04:19.049 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:19.057 DEBUG:teuthology.orchestra.run.smithi115:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-11T23:04:19.164 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-05-11T23:04:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: "size": 20, 2024-05-11T23:04:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 600, 2024-05-11T23:04:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: "ops": [ 2024-05-11T23:04:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-11T23:04:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:04:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 181.999891687, 2024-05-11T23:04:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.001990545, 2024-05-11T23:04:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:19.164 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:04:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4215", 2024-05-11T23:04:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/849530673", 2024-05-11T23:04:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 1 2024-05-11T23:04:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:04:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:19.165 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163252+0000", 2024-05-11T23:04:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163253+0000", 2024-05-11T23:04:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2279999999999999e-06 2024-05-11T23:04:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163261+0000", 2024-05-11T23:04:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5100000000000001e-06 2024-05-11T23:04:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.166 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163277+0000", 2024-05-11T23:04:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.6314999999999999e-05 2024-05-11T23:04:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163290+0000", 2024-05-11T23:04:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3399e-05 2024-05-11T23:04:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163353+0000", 2024-05-11T23:04:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2906999999999999e-05 2024-05-11T23:04:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:04:19.167 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:17.163366+0000", 2024-05-11T23:04:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2329e-05 2024-05-11T23:04:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163674+0000", 2024-05-11T23:04:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0003086450000001 2024-05-11T23:04:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.163839+0000", 2024-05-11T23:04:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000165308 2024-05-11T23:04:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 0", 2024-05-11T23:04:19.168 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164036+0000", 2024-05-11T23:04:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019651599999999999 2024-05-11T23:04:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:04:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.164962+0000", 2024-05-11T23:04:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000925982 2024-05-11T23:04:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:04:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165126+0000", 2024-05-11T23:04:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00016370999999999999 2024-05-11T23:04:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:04:19.169 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165188+0000", 2024-05-11T23:04:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2335999999999994e-05 2024-05-11T23:04:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:01:18.165242+0000", 2024-05-11T23:04:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4360000000000001e-05 2024-05-11T23:04:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78429 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 729088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:04:19.170 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.198622345, 2024-05-11T23:04:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044563497000000001, 2024-05-11T23:04:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78429 2024-05-11T23:04:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:04:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.171 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964521+0000", 2024-05-11T23:04:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964872+0000", 2024-05-11T23:04:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00035070300000000001 2024-05-11T23:04:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964876+0000", 2024-05-11T23:04:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5009999999999998e-06 2024-05-11T23:04:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:19.172 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964902+0000", 2024-05-11T23:04:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5715999999999999e-05 2024-05-11T23:04:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964910+0000", 2024-05-11T23:04:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.3930000000000002e-06 2024-05-11T23:04:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008898+0000", 2024-05-11T23:04:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043988772000000002 2024-05-11T23:04:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.008981+0000", 2024-05-11T23:04:19.173 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3140999999999996e-05 2024-05-11T23:04:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009085+0000", 2024-05-11T23:04:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010327099999999999 2024-05-11T23:04:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78431 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 761856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:04:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.198162037, 2024-05-11T23:04:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044542033000000002, 2024-05-11T23:04:19.174 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78431 2024-05-11T23:04:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:04:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.175 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.964982+0000", 2024-05-11T23:04:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965038+0000", 2024-05-11T23:04:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6993000000000001e-05 2024-05-11T23:04:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965043+0000", 2024-05-11T23:04:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7040000000000002e-06 2024-05-11T23:04:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.176 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965053+0000", 2024-05-11T23:04:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8539999999999992e-06 2024-05-11T23:04:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965061+0000", 2024-05-11T23:04:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9810000000000003e-06 2024-05-11T23:04:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009350+0000", 2024-05-11T23:04:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044288739000000001 2024-05-11T23:04:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.177 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009429+0000", 2024-05-11T23:04:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.9354e-05 2024-05-11T23:04:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009524+0000", 2024-05-11T23:04:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4407999999999995e-05 2024-05-11T23:04:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78432 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:19.178 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:04:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.198071189, 2024-05-11T23:04:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044635421000000002, 2024-05-11T23:04:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78432 2024-05-11T23:04:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:04:19.179 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965072+0000", 2024-05-11T23:04:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965343+0000", 2024-05-11T23:04:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00027056500000000001 2024-05-11T23:04:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965347+0000", 2024-05-11T23:04:19.180 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3540000000000002e-06 2024-05-11T23:04:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965356+0000", 2024-05-11T23:04:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.2140000000000002e-06 2024-05-11T23:04:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965363+0000", 2024-05-11T23:04:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.4150000000000004e-06 2024-05-11T23:04:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009551+0000", 2024-05-11T23:04:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044187917 2024-05-11T23:04:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.181 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009626+0000", 2024-05-11T23:04:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5024000000000005e-05 2024-05-11T23:04:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009708+0000", 2024-05-11T23:04:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.1932000000000001e-05 2024-05-11T23:04:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.182 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78433 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:04:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.197770302, 2024-05-11T23:04:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044564864000000003, 2024-05-11T23:04:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78433 2024-05-11T23:04:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:04:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.183 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.965373+0000", 2024-05-11T23:04:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966225+0000", 2024-05-11T23:04:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085220700000000001 2024-05-11T23:04:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966230+0000", 2024-05-11T23:04:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.0690000000000003e-06 2024-05-11T23:04:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.184 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966239+0000", 2024-05-11T23:04:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0270000000000001e-06 2024-05-11T23:04:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.966245+0000", 2024-05-11T23:04:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.5520000000000001e-06 2024-05-11T23:04:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009743+0000", 2024-05-11T23:04:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043498311999999997 2024-05-11T23:04:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.185 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009848+0000", 2024-05-11T23:04:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000104885 2024-05-11T23:04:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.009938+0000", 2024-05-11T23:04:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9812e-05 2024-05-11T23:04:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78580 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 999424~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:04:19.186 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.194657619, 2024-05-11T23:04:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044493076999999999, 2024-05-11T23:04:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78580 2024-05-11T23:04:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:04:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.187 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:04:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968486+0000", 2024-05-11T23:04:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4099999999999999e-07 2024-05-11T23:04:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968489+0000", 2024-05-11T23:04:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.52e-06 2024-05-11T23:04:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.188 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:19.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968496+0000", 2024-05-11T23:04:19.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.9419999999999998e-06 2024-05-11T23:04:19.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:19.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968501+0000", 2024-05-11T23:04:19.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.8720000000000001e-06 2024-05-11T23:04:19.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.189 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.189 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012690+0000", 2024-05-11T23:04:19.189 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044189655000000001 2024-05-11T23:04:19.189 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.189 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:19.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012848+0000", 2024-05-11T23:04:19.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00015758100000000001 2024-05-11T23:04:19.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.190 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:19.190 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.012979+0000", 2024-05-11T23:04:19.190 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000131066 2024-05-11T23:04:19.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.190 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.190 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.190 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.190 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.190 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78581 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1015808~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:19.190 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:04:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.19463661499999, 2024-05-11T23:04:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044616243, 2024-05-11T23:04:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78581 2024-05-11T23:04:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:04:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.191 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:19.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.968507+0000", 2024-05-11T23:04:19.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:19.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970227+0000", 2024-05-11T23:04:19.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0017200049999999999 2024-05-11T23:04:19.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.192 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:19.192 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970232+0000", 2024-05-11T23:04:19.192 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6600000000000003e-06 2024-05-11T23:04:19.192 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.192 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:19.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970242+0000", 2024-05-11T23:04:19.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.0192e-05 2024-05-11T23:04:19.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:19.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970249+0000", 2024-05-11T23:04:19.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.888e-06 2024-05-11T23:04:19.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013008+0000", 2024-05-11T23:04:19.193 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.042759495000000002 2024-05-11T23:04:19.193 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.193 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.193 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:19.193 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013063+0000", 2024-05-11T23:04:19.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.5071999999999997e-05 2024-05-11T23:04:19.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.194 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:19.194 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.013123+0000", 2024-05-11T23:04:19.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9930999999999999e-05 2024-05-11T23:04:19.194 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.194 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.194 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.194 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.194 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.194 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78589 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:19.194 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:04:19.194 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.192717371, 2024-05-11T23:04:19.194 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044587697000000003, 2024-05-11T23:04:19.194 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:19.195 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:19.195 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:19.195 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:19.195 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:19.195 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:19.195 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78589 2024-05-11T23:04:19.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.195 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:19.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:19.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:04:19.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.195 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.195 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.195 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:19.195 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970426+0000", 2024-05-11T23:04:19.195 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:19.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970427+0000", 2024-05-11T23:04:19.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.7899999999999999e-07 2024-05-11T23:04:19.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:19.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970429+0000", 2024-05-11T23:04:19.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3800000000000001e-06 2024-05-11T23:04:19.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.196 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.196 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:19.196 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970435+0000", 2024-05-11T23:04:19.196 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.2530000000000001e-06 2024-05-11T23:04:19.196 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:19.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970441+0000", 2024-05-11T23:04:19.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.3879999999999999e-06 2024-05-11T23:04:19.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014822+0000", 2024-05-11T23:04:19.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04438168 2024-05-11T23:04:19.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.197 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.197 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:19.197 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.014911+0000", 2024-05-11T23:04:19.197 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.8882999999999996e-05 2024-05-11T23:04:19.197 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.198 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:19.198 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015014+0000", 2024-05-11T23:04:19.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00010263399999999999 2024-05-11T23:04:19.198 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.198 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.198 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.198 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.198 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.198 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78590 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:19.198 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:04:19.198 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.192696572, 2024-05-11T23:04:19.198 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044880120000000003, 2024-05-11T23:04:19.198 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:19.198 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:19.199 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:19.199 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:19.199 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:19.199 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:19.199 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78590 2024-05-11T23:04:19.199 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.199 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:19.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.201 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:19.201 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:04:19.201 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.201 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.201 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:19.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970447+0000", 2024-05-11T23:04:19.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:19.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970918+0000", 2024-05-11T23:04:19.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000470834 2024-05-11T23:04:19.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.202 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.202 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:19.202 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970925+0000", 2024-05-11T23:04:19.202 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.1690000000000003e-06 2024-05-11T23:04:19.202 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:19.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970934+0000", 2024-05-11T23:04:19.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.9099999999999994e-06 2024-05-11T23:04:19.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:19.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970941+0000", 2024-05-11T23:04:19.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7410000000000003e-06 2024-05-11T23:04:19.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.203 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.203 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.203 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015049+0000", 2024-05-11T23:04:19.203 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044108392000000003 2024-05-11T23:04:19.203 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:19.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015133+0000", 2024-05-11T23:04:19.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3568999999999999e-05 2024-05-11T23:04:19.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.204 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:19.204 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015327+0000", 2024-05-11T23:04:19.204 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00019450500000000001 2024-05-11T23:04:19.204 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.204 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.204 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.204 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.204 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.204 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78591 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 925696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:19.205 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:04:19.205 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.192193156, 2024-05-11T23:04:19.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044618497, 2024-05-11T23:04:19.205 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:19.205 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:19.205 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:19.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:19.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:19.205 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:19.205 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78591 2024-05-11T23:04:19.205 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.205 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:19.205 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.205 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:19.205 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:04:19.205 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:19.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970950+0000", 2024-05-11T23:04:19.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:19.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970951+0000", 2024-05-11T23:04:19.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.9000000000000002e-07 2024-05-11T23:04:19.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.206 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.206 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:19.206 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970953+0000", 2024-05-11T23:04:19.206 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.745e-06 2024-05-11T23:04:19.206 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:19.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970956+0000", 2024-05-11T23:04:19.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7189999999999999e-06 2024-05-11T23:04:19.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:19.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970959+0000", 2024-05-11T23:04:19.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9409999999999999e-06 2024-05-11T23:04:19.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.207 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.207 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015367+0000", 2024-05-11T23:04:19.207 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044407621000000001 2024-05-11T23:04:19.207 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.207 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:19.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015453+0000", 2024-05-11T23:04:19.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6685000000000001e-05 2024-05-11T23:04:19.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.208 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:19.208 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015569+0000", 2024-05-11T23:04:19.208 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000115396 2024-05-11T23:04:19.208 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.208 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.208 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.208 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.208 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.208 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78592 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 942080~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:19.208 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:04:19.209 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.19218034799999, 2024-05-11T23:04:19.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.044831546999999999, 2024-05-11T23:04:19.209 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:19.209 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:19.209 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:19.209 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:19.209 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:19.209 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:19.209 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78592 2024-05-11T23:04:19.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.209 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:19.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.209 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:19.209 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:04:19.209 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.209 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.209 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:19.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.970963+0000", 2024-05-11T23:04:19.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:19.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972226+0000", 2024-05-11T23:04:19.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00126246 2024-05-11T23:04:19.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.210 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:19.210 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972230+0000", 2024-05-11T23:04:19.210 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.1860000000000002e-06 2024-05-11T23:04:19.210 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.210 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:19.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972237+0000", 2024-05-11T23:04:19.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.2200000000000003e-06 2024-05-11T23:04:19.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:19.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972241+0000", 2024-05-11T23:04:19.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.284e-06 2024-05-11T23:04:19.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015611+0000", 2024-05-11T23:04:19.211 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.043369461999999998 2024-05-11T23:04:19.211 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.211 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.211 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:19.211 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015695+0000", 2024-05-11T23:04:19.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.4458999999999999e-05 2024-05-11T23:04:19.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.212 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:19.212 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.015795+0000", 2024-05-11T23:04:19.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.9475999999999994e-05 2024-05-11T23:04:19.212 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.212 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.212 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.212 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.212 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.212 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78593 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 958464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:19.212 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:04:19.212 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.190893429, 2024-05-11T23:04:19.212 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046045214000000001, 2024-05-11T23:04:19.212 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:19.213 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:19.213 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:19.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:19.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:19.213 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:19.213 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78593 2024-05-11T23:04:19.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.213 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:19.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:19.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:04:19.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.213 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.213 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.213 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:19.213 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:04:19.213 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:19.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972250+0000", 2024-05-11T23:04:19.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.7599999999999998e-07 2024-05-11T23:04:19.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:19.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972252+0000", 2024-05-11T23:04:19.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.7349999999999999e-06 2024-05-11T23:04:19.214 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.214 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.214 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:19.214 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972256+0000", 2024-05-11T23:04:19.214 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5659999999999998e-06 2024-05-11T23:04:19.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:19.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972259+0000", 2024-05-11T23:04:19.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8040000000000002e-06 2024-05-11T23:04:19.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018088+0000", 2024-05-11T23:04:19.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.045829316000000002 2024-05-11T23:04:19.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.215 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:19.215 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018179+0000", 2024-05-11T23:04:19.215 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.1299999999999997e-05 2024-05-11T23:04:19.215 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.215 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.216 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:19.216 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018295+0000", 2024-05-11T23:04:19.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000116117 2024-05-11T23:04:19.216 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.216 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.216 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.216 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.216 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.216 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78594 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:19.216 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:04:19.216 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.190881069, 2024-05-11T23:04:19.216 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046245204999999998, 2024-05-11T23:04:19.216 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:19.216 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:19.216 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:19.217 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:19.217 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:19.217 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:19.217 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78594 2024-05-11T23:04:19.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.217 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:19.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:19.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:04:19.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.217 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.217 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.217 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:19.217 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972262+0000", 2024-05-11T23:04:19.217 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:19.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972263+0000", 2024-05-11T23:04:19.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:04:19.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:19.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972264+0000", 2024-05-11T23:04:19.218 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.206e-06 2024-05-11T23:04:19.218 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.218 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.218 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:19.218 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972267+0000", 2024-05-11T23:04:19.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.0690000000000002e-06 2024-05-11T23:04:19.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:19.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972270+0000", 2024-05-11T23:04:19.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5610000000000001e-06 2024-05-11T23:04:19.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018330+0000", 2024-05-11T23:04:19.219 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.04606085 2024-05-11T23:04:19.219 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.219 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.219 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:19.219 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018414+0000", 2024-05-11T23:04:19.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.3125000000000001e-05 2024-05-11T23:04:19.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.220 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:19.220 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018508+0000", 2024-05-11T23:04:19.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.4153000000000005e-05 2024-05-11T23:04:19.220 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.220 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.220 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.220 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.220 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.220 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78595 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:19.220 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:04:19.220 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.190868204, 2024-05-11T23:04:19.220 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046396698, 2024-05-11T23:04:19.220 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:19.221 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:19.221 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:19.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:19.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:19.221 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:19.221 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78595 2024-05-11T23:04:19.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.221 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:19.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:19.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:04:19.221 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.221 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.221 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.221 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:19.221 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972275+0000", 2024-05-11T23:04:19.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:19.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972276+0000", 2024-05-11T23:04:19.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5600000000000002e-07 2024-05-11T23:04:19.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:19.222 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972277+0000", 2024-05-11T23:04:19.222 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.576e-06 2024-05-11T23:04:19.222 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.222 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.222 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:19.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972280+0000", 2024-05-11T23:04:19.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.242e-06 2024-05-11T23:04:19.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:19.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972283+0000", 2024-05-11T23:04:19.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4770000000000002e-06 2024-05-11T23:04:19.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018538+0000", 2024-05-11T23:04:19.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046255255000000002 2024-05-11T23:04:19.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.223 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.223 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:19.223 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018605+0000", 2024-05-11T23:04:19.223 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.6847000000000006e-05 2024-05-11T23:04:19.223 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018672+0000", 2024-05-11T23:04:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.7045000000000006e-05 2024-05-11T23:04:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78596 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:04:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.19085735500001, 2024-05-11T23:04:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046586806000000001, 2024-05-11T23:04:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:19.224 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78596 2024-05-11T23:04:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:04:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:19.225 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:04:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972286+0000", 2024-05-11T23:04:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.3099999999999999e-07 2024-05-11T23:04:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972288+0000", 2024-05-11T23:04:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2550000000000001e-06 2024-05-11T23:04:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:19.226 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972291+0000", 2024-05-11T23:04:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9490000000000001e-06 2024-05-11T23:04:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972293+0000", 2024-05-11T23:04:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5069999999999999e-06 2024-05-11T23:04:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018707+0000", 2024-05-11T23:04:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046413604999999997 2024-05-11T23:04:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018782+0000", 2024-05-11T23:04:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5556000000000006e-05 2024-05-11T23:04:19.227 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018873+0000", 2024-05-11T23:04:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.0703000000000002e-05 2024-05-11T23:04:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78597 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1024000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:04:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.190847582, 2024-05-11T23:04:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046825787000000001, 2024-05-11T23:04:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:19.228 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78597 2024-05-11T23:04:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:04:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:19.229 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:04:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972296+0000", 2024-05-11T23:04:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.11e-07 2024-05-11T23:04:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972297+0000", 2024-05-11T23:04:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.3069999999999999e-06 2024-05-11T23:04:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:19.230 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972300+0000", 2024-05-11T23:04:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.9019999999999999e-06 2024-05-11T23:04:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972303+0000", 2024-05-11T23:04:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5509999999999998e-06 2024-05-11T23:04:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018911+0000", 2024-05-11T23:04:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046608194999999998 2024-05-11T23:04:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:19.231 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.018986+0000", 2024-05-11T23:04:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.5360999999999999e-05 2024-05-11T23:04:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019122+0000", 2024-05-11T23:04:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00013526 2024-05-11T23:04:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:78598 3.f 3:f2824726:::rbd_data.111e7090a382.0000000000000001:1a [read 1040384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e62)", 2024-05-11T23:04:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:04:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 123.19083768199999, 2024-05-11T23:04:19.232 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046998572000000002, 2024-05-11T23:04:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "started", 2024-05-11T23:04:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 78598 2024-05-11T23:04:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:04:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.233 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:04:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972306+0000", 2024-05-11T23:04:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.2000000000000001e-07 2024-05-11T23:04:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972307+0000", 2024-05-11T23:04:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.1799999999999999e-06 2024-05-11T23:04:19.234 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972310+0000", 2024-05-11T23:04:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.943e-06 2024-05-11T23:04:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:15.972313+0000", 2024-05-11T23:04:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4760000000000001e-06 2024-05-11T23:04:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.235 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019149+0000", 2024-05-11T23:04:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.046836056000000001 2024-05-11T23:04:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019203+0000", 2024-05-11T23:04:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.4160000000000003e-05 2024-05-11T23:04:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:16.019304+0000", 2024-05-11T23:04:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000101537 2024-05-11T23:04:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.236 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:105234 3.15 3:adf885e5:::rbd_object_map.111e913a615f.0000000000000025:head [writefull 0~39] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-05-11T23:04:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:04:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 109.155521972, 2024-05-11T23:04:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77797599100000003, 2024-05-11T23:04:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:04:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 105234 2024-05-11T23:04:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.237 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:04:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007622+0000", 2024-05-11T23:04:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007623+0000", 2024-05-11T23:04:19.238 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.2240000000000001e-06 2024-05-11T23:04:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007627+0000", 2024-05-11T23:04:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.5000000000000001e-06 2024-05-11T23:04:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007636+0000", 2024-05-11T23:04:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.2409999999999995e-06 2024-05-11T23:04:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007641+0000", 2024-05-11T23:04:19.239 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6810000000000003e-06 2024-05-11T23:04:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007676+0000", 2024-05-11T23:04:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.5061999999999997e-05 2024-05-11T23:04:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for peered", 2024-05-11T23:04:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.007682+0000", 2024-05-11T23:04:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.0399999999999998e-06 2024-05-11T23:04:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783880+0000", 2024-05-11T23:04:19.240 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.77619752900000005 2024-05-11T23:04:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.783995+0000", 2024-05-11T23:04:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00011550800000000001 2024-05-11T23:04:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 3", 2024-05-11T23:04:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784082+0000", 2024-05-11T23:04:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 8.6347999999999994e-05 2024-05-11T23:04:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:04:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.784682+0000", 2024-05-11T23:04:19.241 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.000600162 2024-05-11T23:04:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:04:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785539+0000", 2024-05-11T23:04:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00085699099999999998 2024-05-11T23:04:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:04:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785572+0000", 2024-05-11T23:04:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3182999999999997e-05 2024-05-11T23:04:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:30.785598+0000", 2024-05-11T23:04:19.242 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.5522000000000001e-05 2024-05-11T23:04:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113761 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [call rbd.sparse_copyup in=999496b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:04:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:04:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.63283505299999, 2024-05-11T23:04:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.048572964000000003, 2024-05-11T23:04:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:04:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:19.243 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:19.244 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113761 2024-05-11T23:04:19.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.244 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:19.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:19.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:04:19.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:19.244 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530308+0000", 2024-05-11T23:04:19.244 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.244 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.244 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.244 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:19.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.530309+0000", 2024-05-11T23:04:19.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.41e-07 2024-05-11T23:04:19.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:19.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531242+0000", 2024-05-11T23:04:19.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00093316899999999997 2024-05-11T23:04:19.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:19.245 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531249+0000", 2024-05-11T23:04:19.245 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 6.8140000000000003e-06 2024-05-11T23:04:19.245 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.245 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.245 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:19.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531253+0000", 2024-05-11T23:04:19.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.4290000000000003e-06 2024-05-11T23:04:19.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570325+0000", 2024-05-11T23:04:19.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.039071379000000003 2024-05-11T23:04:19.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:19.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570382+0000", 2024-05-11T23:04:19.246 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.6988999999999999e-05 2024-05-11T23:04:19.246 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.246 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.246 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:04:19.246 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.570555+0000", 2024-05-11T23:04:19.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00017379799999999999 2024-05-11T23:04:19.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:04:19.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.575953+0000", 2024-05-11T23:04:19.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.0053975539999999997 2024-05-11T23:04:19.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:04:19.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578778+0000", 2024-05-11T23:04:19.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.002825616 2024-05-11T23:04:19.247 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.247 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.247 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:04:19.247 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578837+0000", 2024-05-11T23:04:19.247 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 5.9005000000000002e-05 2024-05-11T23:04:19.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.248 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:19.248 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578881+0000", 2024-05-11T23:04:19.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.3970000000000001e-05 2024-05-11T23:04:19.248 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.248 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.248 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.248 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.248 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.248 INFO:teuthology.orchestra.run.smithi115.stdout: "description": "osd_op(client.4382.0:113762 3.e 3:70b30400:::rbd_data.111ed8ed3219.0000000000000075:head [delete] snapc 27=[27] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-11T23:04:19.248 INFO:teuthology.orchestra.run.smithi115.stdout: "initiated_at": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:04:19.248 INFO:teuthology.orchestra.run.smithi115.stdout: "age": 104.63188110199999, 2024-05-11T23:04:19.248 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.047916260000000002, 2024-05-11T23:04:19.248 INFO:teuthology.orchestra.run.smithi115.stdout: "continuous": false, 2024-05-11T23:04:19.249 INFO:teuthology.orchestra.run.smithi115.stdout: "type_data": { 2024-05-11T23:04:19.249 INFO:teuthology.orchestra.run.smithi115.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-11T23:04:19.249 INFO:teuthology.orchestra.run.smithi115.stdout: "client_info": { 2024-05-11T23:04:19.249 INFO:teuthology.orchestra.run.smithi115.stdout: "client": "client.4382", 2024-05-11T23:04:19.249 INFO:teuthology.orchestra.run.smithi115.stdout: "client_addr": "172.21.15.5:0/331665891", 2024-05-11T23:04:19.249 INFO:teuthology.orchestra.run.smithi115.stdout: "tid": 113762 2024-05-11T23:04:19.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.249 INFO:teuthology.orchestra.run.smithi115.stdout: "events": [ 2024-05-11T23:04:19.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "initiated", 2024-05-11T23:04:19.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:04:19.249 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.249 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.249 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.249 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "header_read", 2024-05-11T23:04:19.249 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531262+0000", 2024-05-11T23:04:19.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0 2024-05-11T23:04:19.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "throttled", 2024-05-11T23:04:19.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531263+0000", 2024-05-11T23:04:19.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 7.85e-07 2024-05-11T23:04:19.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "all_read", 2024-05-11T23:04:19.250 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531265+0000", 2024-05-11T23:04:19.250 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 1.827e-06 2024-05-11T23:04:19.250 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.250 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.250 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "dispatched", 2024-05-11T23:04:19.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531268+0000", 2024-05-11T23:04:19.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 3.3780000000000001e-06 2024-05-11T23:04:19.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "queued_for_pg", 2024-05-11T23:04:19.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.531271+0000", 2024-05-11T23:04:19.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8509999999999999e-06 2024-05-11T23:04:19.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "reached_pg", 2024-05-11T23:04:19.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572298+0000", 2024-05-11T23:04:19.251 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.041026990999999999 2024-05-11T23:04:19.251 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.251 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.251 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "started", 2024-05-11T23:04:19.251 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572345+0000", 2024-05-11T23:04:19.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 4.6878000000000001e-05 2024-05-11T23:04:19.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "waiting for subops from 4", 2024-05-11T23:04:19.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.572443+0000", 2024-05-11T23:04:19.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 9.8184999999999996e-05 2024-05-11T23:04:19.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "op_commit", 2024-05-11T23:04:19.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.578585+0000", 2024-05-11T23:04:19.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.006141919 2024-05-11T23:04:19.252 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.252 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.252 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "sub_op_commit_rec", 2024-05-11T23:04:19.252 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579126+0000", 2024-05-11T23:04:19.252 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 0.00054063799999999997 2024-05-11T23:04:19.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "commit_sent", 2024-05-11T23:04:19.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579154+0000", 2024-05-11T23:04:19.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.8042999999999999e-05 2024-05-11T23:04:19.253 INFO:teuthology.orchestra.run.smithi115.stdout: }, 2024-05-11T23:04:19.253 INFO:teuthology.orchestra.run.smithi115.stdout: { 2024-05-11T23:04:19.253 INFO:teuthology.orchestra.run.smithi115.stdout: "event": "done", 2024-05-11T23:04:19.253 INFO:teuthology.orchestra.run.smithi115.stdout: "time": "2024-05-11T23:02:34.579179+0000", 2024-05-11T23:04:19.253 INFO:teuthology.orchestra.run.smithi115.stdout: "duration": 2.4765000000000001e-05 2024-05-11T23:04:19.253 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.253 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.253 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.253 INFO:teuthology.orchestra.run.smithi115.stdout: } 2024-05-11T23:04:19.253 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-05-11T23:04:19.254 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-05-11T23:04:19.535 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:04:19.534+0000 7fb123344640 -1 Falling back to public interface 2024-05-11T23:04:19.980 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-11T23:04:20.941 INFO:teuthology.orchestra.run.smithi005.stderr:noscrub is unset 2024-05-11T23:04:20.959 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-11T23:04:21.946 INFO:teuthology.orchestra.run.smithi005.stderr:nodeep-scrub is unset 2024-05-11T23:04:23.095 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:04:23.095+0000 7fb123344640 -1 osd.3 129 log_to_monitors true 2024-05-11T23:04:23.996 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-05-11T23:04:23.996+0000 7fb1170fa640 -1 osd.3 178 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T23:04:24.047 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-11T23:04:24.166 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:24.166 INFO:teuthology.orchestra.run.smithi005.stdout: "ops": [], 2024-05-11T23:04:24.166 INFO:teuthology.orchestra.run.smithi005.stdout: "num_ops": 0 2024-05-11T23:04:24.166 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:24.175 INFO:tasks.thrashosds.thrasher:inning osd 2024-05-11T23:04:24.175 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-05-11T23:04:24.176 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-05-11T23:04:24.484 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-05-11T23:04:24.484+0000 7fd16247e640 -1 mon.a@0(leader).osd e178 definitely_dead 0 2024-05-11T23:04:24.953 INFO:teuthology.orchestra.run.smithi005.stderr:marked in osd.0. 2024-05-11T23:04:24.981 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-05-11T23:04:24.982 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-05-11T23:04:25.100 INFO:teuthology.orchestra.run.smithi005.stdout:mon.a: {} 2024-05-11T23:04:25.101 INFO:teuthology.orchestra.run.smithi005.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-11T23:04:25.111 INFO:teuthology.orchestra.run.smithi005.stdout:mon.b: {} 2024-05-11T23:04:25.111 INFO:teuthology.orchestra.run.smithi005.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-11T23:04:25.119 INFO:teuthology.orchestra.run.smithi005.stdout:mon.c: {} 2024-05-11T23:04:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-11T23:04:25.130 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-05-11T23:04:25.792 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.0 to 1 (10000) 2024-05-11T23:04:25.822 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-05-11T23:04:26.798 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-05-11T23:04:26.823 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-05-11T23:04:27.795 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.1 to 1 (10000) 2024-05-11T23:04:27.818 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-05-11T23:04:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-05-11T23:04:28.817 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-05-11T23:04:29.798 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.2 to 1 (10000) 2024-05-11T23:04:29.820 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-05-11T23:04:30.985 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-05-11T23:04:31.002 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-05-11T23:04:31.985 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.4 to 1 (10000) 2024-05-11T23:04:32.003 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-05-11T23:04:32.990 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-05-11T23:04:33.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-05-11T23:04:33.999 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.5 to 1 (10000) 2024-05-11T23:04:34.017 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-05-11T23:04:35.000 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-05-11T23:04:35.019 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-05-11T23:04:35.797 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.6 to 1 (10000) 2024-05-11T23:04:35.812 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-05-11T23:04:37.011 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-05-11T23:04:37.030 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-05-11T23:04:38.018 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.7 to 1 (10000) 2024-05-11T23:04:38.035 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-05-11T23:04:39.028 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-05-11T23:04:39.045 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-05-11T23:04:40.032 INFO:teuthology.orchestra.run.smithi005.stderr:reweighted osd.3 to 1 (10000) 2024-05-11T23:04:40.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-05-11T23:04:41.043 INFO:teuthology.orchestra.run.smithi005.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-05-11T23:04:41.061 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-05-11T23:04:41.061 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-05-11T23:04:41.061 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-05-11T23:04:41.061 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-05-11T23:04:41.061 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-11T23:04:41.061 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T23:04:41.371 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:04:41.371 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":195,"fsid":"34b20022-f11b-4169-a5f6-6386270a2f8f","created":"2024-05-11T23:01:05.977693+0000","modified":"2024-05-11T23:04:41.037937+0000","last_up_change":"2024-05-11T23:04:24.952411+0000","last_in_change":"2024-05-11T23:04:24.952411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T23:01:15.187614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-11T23:01:17.689285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-05-11T23:01:25.743347+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":53,"snap_epoch":175,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57ab1c70-9478-4a26-bb41-d753b4f3eb79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6801","nonce":3452725643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6803","nonce":3452725643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6808","nonce":3452725643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6805","nonce":3452725643}]},"public_addr":"172.21.15.5:6801/3452725643","cluster_addr":"172.21.15.5:6803/3452725643","heartbeat_back_addr":"172.21.15.5:6808/3452725643","heartbeat_front_addr":"172.21.15.5:6805/3452725643","state":["exists","up"]},{"osd":1,"uuid":"05ece110-4a99-46f0-b061-5980f7ef2a55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6825","nonce":4028727138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6827","nonce":4028727138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6831","nonce":4028727138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6829","nonce":4028727138}]},"public_addr":"172.21.15.5:6825/4028727138","cluster_addr":"172.21.15.5:6827/4028727138","heartbeat_back_addr":"172.21.15.5:6831/4028727138","heartbeat_front_addr":"172.21.15.5:6829/4028727138","state":["exists","up"]},{"osd":2,"uuid":"4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6809","nonce":2059959509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6811","nonce":2059959509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6815","nonce":2059959509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6813","nonce":2059959509}]},"public_addr":"172.21.15.5:6809/2059959509","cluster_addr":"172.21.15.5:6811/2059959509","heartbeat_back_addr":"172.21.15.5:6815/2059959509","heartbeat_front_addr":"172.21.15.5:6813/2059959509","state":["exists","up"]},{"osd":3,"uuid":"5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":129,"up_from":179,"up_thru":180,"down_at":130,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2390324318},{"type":"v1","addr":"172.21.15.5:6817","nonce":2390324318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2390324318},{"type":"v1","addr":"172.21.15.5:6819","nonce":2390324318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2390324318},{"type":"v1","addr":"172.21.15.5:6823","nonce":2390324318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2390324318},{"type":"v1","addr":"172.21.15.5:6821","nonce":2390324318}]},"public_addr":"172.21.15.5:6817/2390324318","cluster_addr":"172.21.15.5:6819/2390324318","heartbeat_back_addr":"172.21.15.5:6823/2390324318","heartbeat_front_addr":"172.21.15.5:6821/2390324318","state":["exists","up"]},{"osd":4,"uuid":"4e87e4f0-288e-4ee6-919e-9337925a5d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6801","nonce":495703077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6803","nonce":495703077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6807","nonce":495703077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6805","nonce":495703077}]},"public_addr":"172.21.15.115:6801/495703077","cluster_addr":"172.21.15.115:6803/495703077","heartbeat_back_addr":"172.21.15.115:6807/495703077","heartbeat_front_addr":"172.21.15.115:6805/495703077","state":["exists","up"]},{"osd":5,"uuid":"e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6811","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6813","nonce":902573489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6817","nonce":902573489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6823","nonce":902573489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6819","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6821","nonce":902573489}]},"public_addr":"172.21.15.115:6813/902573489","cluster_addr":"172.21.15.115:6817/902573489","heartbeat_back_addr":"172.21.15.115:6823/902573489","heartbeat_front_addr":"172.21.15.115:6821/902573489","state":["exists","up"]},{"osd":6,"uuid":"edb686ec-e1a2-40d6-975a-e32e0e7760ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6825","nonce":2749332272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6827","nonce":2749332272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6831","nonce":2749332272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6829","nonce":2749332272}]},"public_addr":"172.21.15.115:6825/2749332272","cluster_addr":"172.21.15.115:6827/2749332272","heartbeat_back_addr":"172.21.15.115:6831/2749332272","heartbeat_front_addr":"172.21.15.115:6829/2749332272","state":["exists","up"]},{"osd":7,"uuid":"7aa14dbb-c6cb-4540-b160-3fd346f6a046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6809","nonce":1043826337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6812","nonce":1043826337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6820","nonce":1043826337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6816","nonce":1043826337}]},"public_addr":"172.21.15.115:6809/1043826337","cluster_addr":"172.21.15.115:6812/1043826337","heartbeat_back_addr":"172.21.15.115:6820/1043826337","heartbeat_front_addr":"172.21.15.115:6816/1043826337","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.113232+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.174679+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.131118+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.207668+0000","dead_epoch":130},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.124542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.200832+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.169560+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.182465+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T23:04:41.384 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-11T23:04:41.385 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T23:04:41.685 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:04:41.685 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":195,"fsid":"34b20022-f11b-4169-a5f6-6386270a2f8f","created":"2024-05-11T23:01:05.977693+0000","modified":"2024-05-11T23:04:41.037937+0000","last_up_change":"2024-05-11T23:04:24.952411+0000","last_in_change":"2024-05-11T23:04:24.952411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T23:01:15.187614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-11T23:01:17.689285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-05-11T23:01:25.743347+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":53,"snap_epoch":175,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57ab1c70-9478-4a26-bb41-d753b4f3eb79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6801","nonce":3452725643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6803","nonce":3452725643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6808","nonce":3452725643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6805","nonce":3452725643}]},"public_addr":"172.21.15.5:6801/3452725643","cluster_addr":"172.21.15.5:6803/3452725643","heartbeat_back_addr":"172.21.15.5:6808/3452725643","heartbeat_front_addr":"172.21.15.5:6805/3452725643","state":["exists","up"]},{"osd":1,"uuid":"05ece110-4a99-46f0-b061-5980f7ef2a55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6825","nonce":4028727138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6827","nonce":4028727138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6831","nonce":4028727138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6829","nonce":4028727138}]},"public_addr":"172.21.15.5:6825/4028727138","cluster_addr":"172.21.15.5:6827/4028727138","heartbeat_back_addr":"172.21.15.5:6831/4028727138","heartbeat_front_addr":"172.21.15.5:6829/4028727138","state":["exists","up"]},{"osd":2,"uuid":"4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6809","nonce":2059959509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6811","nonce":2059959509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6815","nonce":2059959509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6813","nonce":2059959509}]},"public_addr":"172.21.15.5:6809/2059959509","cluster_addr":"172.21.15.5:6811/2059959509","heartbeat_back_addr":"172.21.15.5:6815/2059959509","heartbeat_front_addr":"172.21.15.5:6813/2059959509","state":["exists","up"]},{"osd":3,"uuid":"5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":129,"up_from":179,"up_thru":180,"down_at":130,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2390324318},{"type":"v1","addr":"172.21.15.5:6817","nonce":2390324318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2390324318},{"type":"v1","addr":"172.21.15.5:6819","nonce":2390324318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2390324318},{"type":"v1","addr":"172.21.15.5:6823","nonce":2390324318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2390324318},{"type":"v1","addr":"172.21.15.5:6821","nonce":2390324318}]},"public_addr":"172.21.15.5:6817/2390324318","cluster_addr":"172.21.15.5:6819/2390324318","heartbeat_back_addr":"172.21.15.5:6823/2390324318","heartbeat_front_addr":"172.21.15.5:6821/2390324318","state":["exists","up"]},{"osd":4,"uuid":"4e87e4f0-288e-4ee6-919e-9337925a5d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6801","nonce":495703077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6803","nonce":495703077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6807","nonce":495703077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6805","nonce":495703077}]},"public_addr":"172.21.15.115:6801/495703077","cluster_addr":"172.21.15.115:6803/495703077","heartbeat_back_addr":"172.21.15.115:6807/495703077","heartbeat_front_addr":"172.21.15.115:6805/495703077","state":["exists","up"]},{"osd":5,"uuid":"e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6811","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6813","nonce":902573489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6817","nonce":902573489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6823","nonce":902573489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6819","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6821","nonce":902573489}]},"public_addr":"172.21.15.115:6813/902573489","cluster_addr":"172.21.15.115:6817/902573489","heartbeat_back_addr":"172.21.15.115:6823/902573489","heartbeat_front_addr":"172.21.15.115:6821/902573489","state":["exists","up"]},{"osd":6,"uuid":"edb686ec-e1a2-40d6-975a-e32e0e7760ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6825","nonce":2749332272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6827","nonce":2749332272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6831","nonce":2749332272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6829","nonce":2749332272}]},"public_addr":"172.21.15.115:6825/2749332272","cluster_addr":"172.21.15.115:6827/2749332272","heartbeat_back_addr":"172.21.15.115:6831/2749332272","heartbeat_front_addr":"172.21.15.115:6829/2749332272","state":["exists","up"]},{"osd":7,"uuid":"7aa14dbb-c6cb-4540-b160-3fd346f6a046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6809","nonce":1043826337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6812","nonce":1043826337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6820","nonce":1043826337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6816","nonce":1043826337}]},"public_addr":"172.21.15.115:6809/1043826337","cluster_addr":"172.21.15.115:6812/1043826337","heartbeat_back_addr":"172.21.15.115:6820/1043826337","heartbeat_front_addr":"172.21.15.115:6816/1043826337","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.113232+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.174679+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.131118+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.207668+0000","dead_epoch":130},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.124542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.200832+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.169560+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.182465+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T23:04:41.698 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-11T23:04:41.698 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-11T23:04:41.699 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-11T23:04:41.699 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-11T23:04:41.699 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-11T23:04:41.700 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-11T23:04:41.700 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-11T23:04:41.700 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-11T23:04:41.959 INFO:teuthology.orchestra.run.smithi005.stdout:768799145989 2024-05-11T23:04:41.960 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-11T23:04:41.961 INFO:teuthology.orchestra.run.smithi005.stdout:60129542188 2024-05-11T23:04:41.961 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-11T23:04:41.962 INFO:teuthology.orchestra.run.smithi005.stdout:55834574893 2024-05-11T23:04:41.962 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-11T23:04:41.963 INFO:teuthology.orchestra.run.smithi005.stdout:64424509483 2024-05-11T23:04:41.964 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-11T23:04:41.965 INFO:teuthology.orchestra.run.smithi005.stdout:64424509483 2024-05-11T23:04:41.965 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-11T23:04:41.966 INFO:teuthology.orchestra.run.smithi005.stdout:55834574892 2024-05-11T23:04:41.966 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-11T23:04:41.967 INFO:teuthology.orchestra.run.smithi005.stdout:64424509484 2024-05-11T23:04:41.967 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-11T23:04:41.968 INFO:teuthology.orchestra.run.smithi005.stdout:64424509484 2024-05-11T23:04:41.968 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-11T23:04:42.635 INFO:teuthology.orchestra.run.smithi005.stdout:55834574891 2024-05-11T23:04:42.637 INFO:teuthology.orchestra.run.smithi005.stdout:64424509482 2024-05-11T23:04:42.646 INFO:teuthology.orchestra.run.smithi005.stdout:768799145988 2024-05-11T23:04:42.649 INFO:teuthology.orchestra.run.smithi005.stdout:64424509483 2024-05-11T23:04:42.656 INFO:teuthology.orchestra.run.smithi005.stdout:64424509483 2024-05-11T23:04:42.657 INFO:teuthology.orchestra.run.smithi005.stdout:55834574892 2024-05-11T23:04:42.660 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509483 got 64424509482 for osd.7 2024-05-11T23:04:42.661 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574892 got 55834574891 for osd.2 2024-05-11T23:04:42.667 INFO:tasks.ceph.ceph_manager.ceph:need seq 768799145989 got 768799145988 for osd.3 2024-05-11T23:04:42.668 INFO:teuthology.orchestra.run.smithi005.stdout:60129542186 2024-05-11T23:04:42.672 INFO:teuthology.orchestra.run.smithi005.stdout:64424509482 2024-05-11T23:04:42.673 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509484 got 64424509483 for osd.5 2024-05-11T23:04:42.676 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509484 got 64424509483 for osd.4 2024-05-11T23:04:42.677 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574893 got 55834574892 for osd.0 2024-05-11T23:04:42.688 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542188 got 60129542186 for osd.1 2024-05-11T23:04:42.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509483 got 64424509482 for osd.6 2024-05-11T23:04:43.662 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-11T23:04:43.662 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-11T23:04:43.670 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-11T23:04:43.674 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-11T23:04:43.676 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-11T23:04:43.678 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-11T23:04:43.689 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-11T23:04:43.690 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-11T23:04:44.281 INFO:teuthology.orchestra.run.smithi005.stdout:55834574892 2024-05-11T23:04:44.296 INFO:teuthology.orchestra.run.smithi005.stdout:64424509483 2024-05-11T23:04:44.302 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574892 got 55834574892 for osd.2 2024-05-11T23:04:44.302 DEBUG:teuthology.parallel:result is None 2024-05-11T23:04:44.316 INFO:teuthology.orchestra.run.smithi005.stdout:64424509484 2024-05-11T23:04:44.317 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509483 got 64424509483 for osd.7 2024-05-11T23:04:44.317 DEBUG:teuthology.parallel:result is None 2024-05-11T23:04:44.321 INFO:teuthology.orchestra.run.smithi005.stdout:64424509484 2024-05-11T23:04:44.335 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509483 got 64424509484 for osd.6 2024-05-11T23:04:44.335 DEBUG:teuthology.parallel:result is None 2024-05-11T23:04:44.339 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509484 got 64424509484 for osd.4 2024-05-11T23:04:44.339 DEBUG:teuthology.parallel:result is None 2024-05-11T23:04:44.340 INFO:teuthology.orchestra.run.smithi005.stdout:64424509484 2024-05-11T23:04:44.350 INFO:teuthology.orchestra.run.smithi005.stdout:768799145989 2024-05-11T23:04:44.354 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509484 got 64424509484 for osd.5 2024-05-11T23:04:44.354 DEBUG:teuthology.parallel:result is None 2024-05-11T23:04:44.357 INFO:teuthology.orchestra.run.smithi005.stdout:55834574893 2024-05-11T23:04:44.360 INFO:teuthology.orchestra.run.smithi005.stdout:60129542188 2024-05-11T23:04:44.364 INFO:tasks.ceph.ceph_manager.ceph:need seq 768799145989 got 768799145989 for osd.3 2024-05-11T23:04:44.364 DEBUG:teuthology.parallel:result is None 2024-05-11T23:04:44.372 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574893 got 55834574893 for osd.0 2024-05-11T23:04:44.372 DEBUG:teuthology.parallel:result is None 2024-05-11T23:04:44.374 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542188 got 60129542188 for osd.1 2024-05-11T23:04:44.374 DEBUG:teuthology.parallel:result is None 2024-05-11T23:04:44.374 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-05-11T23:04:44.374 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T23:04:44.712 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:04:44.712 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-11T23:04:44.724 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-05-11T23:04:43.173884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":1467,"num_object_clones":0,"num_object_copies":2934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1467,"num_whiteouts":1461,"num_read":339547,"num_read_kb":1023467,"num_write":28662,"num_write_kb":501200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8735,"num_bytes_recovered":383662413,"num_keys_recovered":951,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26684,"ondisk_log_size":26684,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":453660,"kb_used_data":18420,"kb_used_omap":60,"kb_used_meta":435075,"kb_avail":754521060,"statfs":{"total":773094113280,"available":772629565440,"internally_reserved":0,"allocated":18862080,"data_stored":10293335,"data_compressed":66655,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":62250,"internal_metadata":445517014},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.147643"},"pg_stats":[{"pgid":"3.1f","version":"175'540","reported_seq":5571,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044941+0000","last_change":"2024-05-11T23:04:28.806612+0000","last_active":"2024-05-11T23:04:41.044941+0000","last_peered":"2024-05-11T23:04:41.044941+0000","last_clean":"2024-05-11T23:04:41.044941+0000","last_became_active":"2024-05-11T23:04:28.804255+0000","last_became_peered":"2024-05-11T23:04:28.804255+0000","last_unstale":"2024-05-11T23:04:41.044941+0000","last_undegraded":"2024-05-11T23:04:41.044941+0000","last_fullsized":"2024-05-11T23:04:41.044941+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:57:34.663961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098957899999999998,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":5393,"num_read_kb":14723,"num_write":488,"num_write_kb":18081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":877932,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"180'934","reported_seq":6541,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.042173+0000","last_change":"2024-05-11T23:04:25.935404+0000","last_active":"2024-05-11T23:04:41.042173+0000","last_peered":"2024-05-11T23:04:41.042173+0000","last_clean":"2024-05-11T23:04:41.042173+0000","last_became_active":"2024-05-11T23:04:25.808413+0000","last_became_peered":"2024-05-11T23:04:25.808413+0000","last_unstale":"2024-05-11T23:04:41.042173+0000","last_undegraded":"2024-05-11T23:04:41.042173+0000","last_fullsized":"2024-05-11T23:04:41.042173+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":934,"log_dups_size":0,"ondisk_log_size":934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:09:17.133440+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.084167985000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":6916,"num_read_kb":49448,"num_write":971,"num_write_kb":22687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":53290749,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"167'508","reported_seq":4528,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045582+0000","last_change":"2024-05-11T23:04:17.950967+0000","last_active":"2024-05-11T23:04:41.045582+0000","last_peered":"2024-05-11T23:04:41.045582+0000","last_clean":"2024-05-11T23:04:41.045582+0000","last_became_active":"2024-05-11T23:03:48.889806+0000","last_became_peered":"2024-05-11T23:03:48.889806+0000","last_unstale":"2024-05-11T23:04:41.045582+0000","last_undegraded":"2024-05-11T23:04:41.045582+0000","last_fullsized":"2024-05-11T23:04:41.045582+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:46:37.757197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060389999999999999,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":3180,"num_read_kb":16428,"num_write":406,"num_write_kb":9629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":7067368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"175'784","reported_seq":19158,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045367+0000","last_change":"2024-05-11T23:04:17.951789+0000","last_active":"2024-05-11T23:04:41.045367+0000","last_peered":"2024-05-11T23:04:41.045367+0000","last_clean":"2024-05-11T23:04:41.045367+0000","last_became_active":"2024-05-11T23:03:50.798632+0000","last_became_peered":"2024-05-11T23:03:50.798632+0000","last_unstale":"2024-05-11T23:04:41.045367+0000","last_undegraded":"2024-05-11T23:04:41.045367+0000","last_fullsized":"2024-05-11T23:04:41.045367+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:13:59.992713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248566,"stat_sum":{"num_bytes":0,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":63,"num_read":18429,"num_read_kb":39778,"num_write":674,"num_write_kb":10510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":9396782,"num_keys_recovered":182,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"174'594","reported_seq":15111,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045102+0000","last_change":"2024-05-11T23:04:29.807594+0000","last_active":"2024-05-11T23:04:41.045102+0000","last_peered":"2024-05-11T23:04:41.045102+0000","last_clean":"2024-05-11T23:04:41.045102+0000","last_became_active":"2024-05-11T23:04:29.804143+0000","last_became_peered":"2024-05-11T23:04:29.804143+0000","last_unstale":"2024-05-11T23:04:41.045102+0000","last_undegraded":"2024-05-11T23:04:41.045102+0000","last_fullsized":"2024-05-11T23:04:41.045102+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:25:20.570143+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001388731,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":13979,"num_read_kb":68609,"num_write":598,"num_write_kb":35793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":532,"num_bytes_recovered":20898869,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"173'1664","reported_seq":21043,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.896600+0000","last_change":"2024-05-11T23:04:34.010543+0000","last_active":"2024-05-11T23:04:41.896600+0000","last_peered":"2024-05-11T23:04:41.896600+0000","last_clean":"2024-05-11T23:04:41.896600+0000","last_became_active":"2024-05-11T23:04:34.007358+0000","last_became_peered":"2024-05-11T23:04:34.007358+0000","last_unstale":"2024-05-11T23:04:41.896600+0000","last_undegraded":"2024-05-11T23:04:41.896600+0000","last_fullsized":"2024-05-11T23:04:41.896600+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1664,"log_dups_size":0,"ondisk_log_size":1664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:31:26.529561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097887899999999999,"stat_sum":{"num_bytes":19,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":25336,"num_read_kb":48131,"num_write":2723,"num_write_kb":23444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":6434587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"180'1052","reported_seq":15385,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.042982+0000","last_change":"2024-05-11T23:04:25.959047+0000","last_active":"2024-05-11T23:04:41.042982+0000","last_peered":"2024-05-11T23:04:41.042982+0000","last_clean":"2024-05-11T23:04:41.042982+0000","last_became_active":"2024-05-11T23:04:25.807274+0000","last_became_peered":"2024-05-11T23:04:25.807274+0000","last_unstale":"2024-05-11T23:04:41.042982+0000","last_undegraded":"2024-05-11T23:04:41.042982+0000","last_fullsized":"2024-05-11T23:04:41.042982+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1052,"log_dups_size":0,"ondisk_log_size":1052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:17:04.610363+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.061503582000000001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":16282,"num_read_kb":43182,"num_write":1369,"num_write_kb":14705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":13589947,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"170'967","reported_seq":3980,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.041570+0000","last_change":"2024-05-11T23:04:26.853599+0000","last_active":"2024-05-11T23:04:41.041570+0000","last_peered":"2024-05-11T23:04:41.041570+0000","last_clean":"2024-05-11T23:04:41.041570+0000","last_became_active":"2024-05-11T23:04:26.807825+0000","last_became_peered":"2024-05-11T23:04:26.807825+0000","last_unstale":"2024-05-11T23:04:41.041570+0000","last_undegraded":"2024-05-11T23:04:41.041570+0000","last_fullsized":"2024-05-11T23:04:41.041570+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:26:49.247146+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001621796,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":5754,"num_read_kb":9891,"num_write":1117,"num_write_kb":16486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":7487386,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"172'667","reported_seq":3869,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045052+0000","last_change":"2024-05-11T23:04:27.810906+0000","last_active":"2024-05-11T23:04:41.045052+0000","last_peered":"2024-05-11T23:04:41.045052+0000","last_clean":"2024-05-11T23:04:41.045052+0000","last_became_active":"2024-05-11T23:04:27.809321+0000","last_became_peered":"2024-05-11T23:04:27.809321+0000","last_unstale":"2024-05-11T23:04:41.045052+0000","last_undegraded":"2024-05-11T23:04:41.045052+0000","last_fullsized":"2024-05-11T23:04:41.045052+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:32:33.820874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013760389999999999,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":3972,"num_read_kb":15246,"num_write":808,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"172'559","reported_seq":15688,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045292+0000","last_change":"2024-05-11T23:04:17.950903+0000","last_active":"2024-05-11T23:04:41.045292+0000","last_peered":"2024-05-11T23:04:41.045292+0000","last_clean":"2024-05-11T23:04:41.045292+0000","last_became_active":"2024-05-11T23:01:27.463623+0000","last_became_peered":"2024-05-11T23:01:27.463623+0000","last_unstale":"2024-05-11T23:04:41.045292+0000","last_undegraded":"2024-05-11T23:04:41.045292+0000","last_fullsized":"2024-05-11T23:04:41.045292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:56:10.783772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054116699999999995,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":15017,"num_read_kb":54784,"num_write":481,"num_write_kb":18602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"180'622","reported_seq":16411,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044641+0000","last_change":"2024-05-11T23:04:26.015543+0000","last_active":"2024-05-11T23:04:41.044641+0000","last_peered":"2024-05-11T23:04:41.044641+0000","last_clean":"2024-05-11T23:04:41.044641+0000","last_became_active":"2024-05-11T23:04:25.807935+0000","last_became_peered":"2024-05-11T23:04:25.807935+0000","last_unstale":"2024-05-11T23:04:41.044641+0000","last_undegraded":"2024-05-11T23:04:41.044641+0000","last_fullsized":"2024-05-11T23:04:41.044641+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:50:30.593291+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.072194399000000006,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":14264,"num_read_kb":25298,"num_write":546,"num_write_kb":13154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":12363304,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"172'661","reported_seq":19582,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.896664+0000","last_change":"2024-05-11T23:04:34.010412+0000","last_active":"2024-05-11T23:04:41.896664+0000","last_peered":"2024-05-11T23:04:41.896664+0000","last_clean":"2024-05-11T23:04:41.896664+0000","last_became_active":"2024-05-11T23:04:34.007861+0000","last_became_peered":"2024-05-11T23:04:34.007861+0000","last_unstale":"2024-05-11T23:04:41.896664+0000","last_undegraded":"2024-05-11T23:04:41.896664+0000","last_fullsized":"2024-05-11T23:04:41.896664+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:13:07.675246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001389319,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":19402,"num_read_kb":42118,"num_write":569,"num_write_kb":6424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":4215359,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118756+0000","last_change":"2024-05-11T23:02:39.044686+0000","last_active":"2024-05-11T23:04:42.118756+0000","last_peered":"2024-05-11T23:04:42.118756+0000","last_clean":"2024-05-11T23:04:42.118756+0000","last_became_active":"2024-05-11T23:02:39.044216+0000","last_became_peered":"2024-05-11T23:02:39.044216+0000","last_unstale":"2024-05-11T23:04:42.118756+0000","last_undegraded":"2024-05-11T23:04:42.118756+0000","last_fullsized":"2024-05-11T23:04:42.118756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:31:03.826529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"175'1049","reported_seq":7444,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044838+0000","last_change":"2024-05-11T23:04:28.808149+0000","last_active":"2024-05-11T23:04:41.044838+0000","last_peered":"2024-05-11T23:04:41.044838+0000","last_clean":"2024-05-11T23:04:41.044838+0000","last_became_active":"2024-05-11T23:04:28.805428+0000","last_became_peered":"2024-05-11T23:04:28.805428+0000","last_unstale":"2024-05-11T23:04:41.044838+0000","last_undegraded":"2024-05-11T23:04:41.044838+0000","last_fullsized":"2024-05-11T23:04:41.044838+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:36:30.546501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002036403,"stat_sum":{"num_bytes":0,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":60,"num_read":9703,"num_read_kb":23103,"num_write":1155,"num_write_kb":12619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":2180842,"num_keys_recovered":21,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":335,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.043163+0000","last_change":"2024-05-11T23:04:29.806455+0000","last_active":"2024-05-11T23:04:41.043163+0000","last_peered":"2024-05-11T23:04:41.043163+0000","last_clean":"2024-05-11T23:04:41.043163+0000","last_became_active":"2024-05-11T23:04:29.805946+0000","last_became_peered":"2024-05-11T23:04:29.805946+0000","last_unstale":"2024-05-11T23:04:41.043163+0000","last_undegraded":"2024-05-11T23:04:41.043163+0000","last_fullsized":"2024-05-11T23:04:41.043163+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:09:54.127025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"180'926","reported_seq":6106,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-05-11T23:04:38.090033+0000","last_change":"2024-05-11T23:04:25.923075+0000","last_active":"2024-05-11T23:04:38.090033+0000","last_peered":"2024-05-11T23:04:38.090033+0000","last_clean":"2024-05-11T23:04:38.090033+0000","last_became_active":"2024-05-11T23:04:25.806037+0000","last_became_peered":"2024-05-11T23:04:25.806037+0000","last_unstale":"2024-05-11T23:04:38.090033+0000","last_undegraded":"2024-05-11T23:04:38.090033+0000","last_fullsized":"2024-05-11T23:04:38.090033+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:34:55.146392+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.064543953000000001,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":7310,"num_read_kb":19251,"num_write":1088,"num_write_kb":7185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":4415701,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":374,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044868+0000","last_change":"2024-05-11T23:04:25.806665+0000","last_active":"2024-05-11T23:04:41.044868+0000","last_peered":"2024-05-11T23:04:41.044868+0000","last_clean":"2024-05-11T23:04:41.044868+0000","last_became_active":"2024-05-11T23:04:25.806292+0000","last_became_peered":"2024-05-11T23:04:25.806292+0000","last_unstale":"2024-05-11T23:04:41.044868+0000","last_undegraded":"2024-05-11T23:04:41.044868+0000","last_fullsized":"2024-05-11T23:04:41.044868+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:12:45.144479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"175'564","reported_seq":14906,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.043149+0000","last_change":"2024-05-11T23:04:29.809156+0000","last_active":"2024-05-11T23:04:41.043149+0000","last_peered":"2024-05-11T23:04:41.043149+0000","last_clean":"2024-05-11T23:04:41.043149+0000","last_became_active":"2024-05-11T23:04:29.807343+0000","last_became_peered":"2024-05-11T23:04:29.807343+0000","last_unstale":"2024-05-11T23:04:41.043149+0000","last_undegraded":"2024-05-11T23:04:41.043149+0000","last_fullsized":"2024-05-11T23:04:41.043149+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:35:56.026223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095379299999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":14156,"num_read_kb":53683,"num_write":532,"num_write_kb":33731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":339,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.043292+0000","last_change":"2024-05-11T23:04:29.805774+0000","last_active":"2024-05-11T23:04:41.043292+0000","last_peered":"2024-05-11T23:04:41.043292+0000","last_clean":"2024-05-11T23:04:41.043292+0000","last_became_active":"2024-05-11T23:04:29.805447+0000","last_became_peered":"2024-05-11T23:04:29.805447+0000","last_unstale":"2024-05-11T23:04:41.043292+0000","last_undegraded":"2024-05-11T23:04:41.043292+0000","last_fullsized":"2024-05-11T23:04:41.043292+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:56:59.210181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"180'758","reported_seq":6310,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045646+0000","last_change":"2024-05-11T23:04:25.999769+0000","last_active":"2024-05-11T23:04:41.045646+0000","last_peered":"2024-05-11T23:04:41.045646+0000","last_clean":"2024-05-11T23:04:41.045646+0000","last_became_active":"2024-05-11T23:04:25.806236+0000","last_became_peered":"2024-05-11T23:04:25.806236+0000","last_unstale":"2024-05-11T23:04:41.045646+0000","last_undegraded":"2024-05-11T23:04:41.045646+0000","last_fullsized":"2024-05-11T23:04:41.045646+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:19:42.475478+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.081084693999999999,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":6715,"num_read_kb":25684,"num_write":652,"num_write_kb":13121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":13712581,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"175'1022","reported_seq":11245,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.896816+0000","last_change":"2024-05-11T23:04:34.011637+0000","last_active":"2024-05-11T23:04:41.896816+0000","last_peered":"2024-05-11T23:04:41.896816+0000","last_clean":"2024-05-11T23:04:41.896816+0000","last_became_active":"2024-05-11T23:04:34.008456+0000","last_became_peered":"2024-05-11T23:04:34.008456+0000","last_unstale":"2024-05-11T23:04:41.896816+0000","last_undegraded":"2024-05-11T23:04:41.896816+0000","last_fullsized":"2024-05-11T23:04:41.896816+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1022,"log_dups_size":0,"ondisk_log_size":1022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:56:16.226740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017329069999999999,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":56,"num_read":10698,"num_read_kb":33938,"num_write":973,"num_write_kb":17254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":898,"num_bytes_recovered":16208786,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":366,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045946+0000","last_change":"2024-05-11T23:01:19.221118+0000","last_active":"2024-05-11T23:04:41.045946+0000","last_peered":"2024-05-11T23:04:41.045946+0000","last_clean":"2024-05-11T23:04:41.045946+0000","last_became_active":"2024-05-11T23:01:19.220855+0000","last_became_peered":"2024-05-11T23:01:19.220855+0000","last_unstale":"2024-05-11T23:04:41.045946+0000","last_undegraded":"2024-05-11T23:04:41.045946+0000","last_fullsized":"2024-05-11T23:04:41.045946+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:01:56.076963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"173'1148","reported_seq":6343,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.042746+0000","last_change":"2024-05-11T23:04:17.940887+0000","last_active":"2024-05-11T23:04:41.042746+0000","last_peered":"2024-05-11T23:04:41.042746+0000","last_clean":"2024-05-11T23:04:41.042746+0000","last_became_active":"2024-05-11T23:01:27.459593+0000","last_became_peered":"2024-05-11T23:01:27.459593+0000","last_unstale":"2024-05-11T23:04:41.042746+0000","last_undegraded":"2024-05-11T23:04:41.042746+0000","last_fullsized":"2024-05-11T23:04:41.042746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1148,"log_dups_size":0,"ondisk_log_size":1148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:45.922442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017756580000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9995,"num_read_kb":18445,"num_write":1457,"num_write_kb":4408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":384,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.041751+0000","last_change":"2024-05-11T23:04:25.805119+0000","last_active":"2024-05-11T23:04:41.041751+0000","last_peered":"2024-05-11T23:04:41.041751+0000","last_clean":"2024-05-11T23:04:41.041751+0000","last_became_active":"2024-05-11T23:04:25.804674+0000","last_became_peered":"2024-05-11T23:04:25.804674+0000","last_unstale":"2024-05-11T23:04:41.041751+0000","last_undegraded":"2024-05-11T23:04:41.041751+0000","last_fullsized":"2024-05-11T23:04:41.041751+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:37:37.191290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"175'656","reported_seq":15312,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044755+0000","last_change":"2024-05-11T23:04:26.807059+0000","last_active":"2024-05-11T23:04:41.044755+0000","last_peered":"2024-05-11T23:04:41.044755+0000","last_clean":"2024-05-11T23:04:41.044755+0000","last_became_active":"2024-05-11T23:04:26.804248+0000","last_became_peered":"2024-05-11T23:04:26.804248+0000","last_unstale":"2024-05-11T23:04:41.044755+0000","last_undegraded":"2024-05-11T23:04:41.044755+0000","last_fullsized":"2024-05-11T23:04:41.044755+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:32:14.359230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014815119999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":15309,"num_read_kb":24676,"num_write":622,"num_write_kb":6352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":299,"num_bytes_recovered":6345105,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":364,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044805+0000","last_change":"2024-05-11T23:01:19.219755+0000","last_active":"2024-05-11T23:04:41.044805+0000","last_peered":"2024-05-11T23:04:41.044805+0000","last_clean":"2024-05-11T23:04:41.044805+0000","last_became_active":"2024-05-11T23:01:19.219471+0000","last_became_peered":"2024-05-11T23:01:19.219471+0000","last_unstale":"2024-05-11T23:04:41.044805+0000","last_undegraded":"2024-05-11T23:04:41.044805+0000","last_fullsized":"2024-05-11T23:04:41.044805+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:39:05.284213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"180'690","reported_seq":4329,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-05-11T23:04:38.022395+0000","last_change":"2024-05-11T23:04:25.884271+0000","last_active":"2024-05-11T23:04:38.022395+0000","last_peered":"2024-05-11T23:04:38.022395+0000","last_clean":"2024-05-11T23:04:38.022395+0000","last_became_active":"2024-05-11T23:04:25.805994+0000","last_became_peered":"2024-05-11T23:04:25.805994+0000","last_unstale":"2024-05-11T23:04:38.022395+0000","last_undegraded":"2024-05-11T23:04:38.022395+0000","last_fullsized":"2024-05-11T23:04:38.022395+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:43:03.144032+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.055474953,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":3497,"num_read_kb":16044,"num_write":640,"num_write_kb":10409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":10719266,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":366,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045169+0000","last_change":"2024-05-11T23:01:19.219128+0000","last_active":"2024-05-11T23:04:41.045169+0000","last_peered":"2024-05-11T23:04:41.045169+0000","last_clean":"2024-05-11T23:04:41.045169+0000","last_became_active":"2024-05-11T23:01:19.218846+0000","last_became_peered":"2024-05-11T23:01:19.218846+0000","last_unstale":"2024-05-11T23:04:41.045169+0000","last_undegraded":"2024-05-11T23:04:41.045169+0000","last_fullsized":"2024-05-11T23:04:41.045169+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:41:09.111000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":542,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044642+0000","last_change":"2024-05-11T23:04:28.806566+0000","last_active":"2024-05-11T23:04:41.044642+0000","last_peered":"2024-05-11T23:04:41.044642+0000","last_clean":"2024-05-11T23:04:41.044642+0000","last_became_active":"2024-05-11T23:04:28.806124+0000","last_became_peered":"2024-05-11T23:04:28.806124+0000","last_unstale":"2024-05-11T23:04:41.044642+0000","last_undegraded":"2024-05-11T23:04:41.044642+0000","last_fullsized":"2024-05-11T23:04:41.044642+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_clean_scrub_stamp":"2024-05-11T23:01:16.147564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:03:40.590362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"180'966","reported_seq":5552,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.041792+0000","last_change":"2024-05-11T23:04:25.885127+0000","last_active":"2024-05-11T23:04:41.041792+0000","last_peered":"2024-05-11T23:04:41.041792+0000","last_clean":"2024-05-11T23:04:41.041792+0000","last_became_active":"2024-05-11T23:04:25.802390+0000","last_became_peered":"2024-05-11T23:04:25.802390+0000","last_unstale":"2024-05-11T23:04:41.041792+0000","last_undegraded":"2024-05-11T23:04:41.041792+0000","last_fullsized":"2024-05-11T23:04:41.041792+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":966,"log_dups_size":0,"ondisk_log_size":966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:45:25.229272+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.057229434000000003,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":6339,"num_read_kb":20167,"num_write":1151,"num_write_kb":10790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":4220183,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"172'676","reported_seq":9167,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.896968+0000","last_change":"2024-05-11T23:04:34.008013+0000","last_active":"2024-05-11T23:04:41.896968+0000","last_peered":"2024-05-11T23:04:41.896968+0000","last_clean":"2024-05-11T23:04:41.896968+0000","last_became_active":"2024-05-11T23:04:34.005569+0000","last_became_peered":"2024-05-11T23:04:34.005569+0000","last_unstale":"2024-05-11T23:04:41.896968+0000","last_undegraded":"2024-05-11T23:04:41.896968+0000","last_fullsized":"2024-05-11T23:04:41.896968+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:27:04.828149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064185299999999995,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":9043,"num_read_kb":47546,"num_write":541,"num_write_kb":29805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":37320801,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"175'785","reported_seq":6658,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118728+0000","last_change":"2024-05-11T23:04:17.987769+0000","last_active":"2024-05-11T23:04:42.118728+0000","last_peered":"2024-05-11T23:04:42.118728+0000","last_clean":"2024-05-11T23:04:42.118728+0000","last_became_active":"2024-05-11T23:03:51.797214+0000","last_became_peered":"2024-05-11T23:03:51.797214+0000","last_unstale":"2024-05-11T23:04:42.118728+0000","last_undegraded":"2024-05-11T23:04:42.118728+0000","last_fullsized":"2024-05-11T23:04:42.118728+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:04:04.928406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001270001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":44,"num_read":5814,"num_read_kb":22550,"num_write":685,"num_write_kb":9625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":538,"num_bytes_recovered":16997094,"num_keys_recovered":140,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"173'657","reported_seq":10543,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-05-11T23:04:39.159213+0000","last_change":"2024-05-11T23:04:27.905632+0000","last_active":"2024-05-11T23:04:39.159213+0000","last_peered":"2024-05-11T23:04:39.159213+0000","last_clean":"2024-05-11T23:04:39.159213+0000","last_became_active":"2024-05-11T23:04:27.803860+0000","last_became_peered":"2024-05-11T23:04:27.803860+0000","last_unstale":"2024-05-11T23:04:39.159213+0000","last_undegraded":"2024-05-11T23:04:39.159213+0000","last_fullsized":"2024-05-11T23:04:39.159213+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T09:48:49.893537+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00088000000000000003,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":9829,"num_read_kb":52232,"num_write":599,"num_write_kb":42645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":47434023,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"180'694","reported_seq":5214,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045475+0000","last_change":"2024-05-11T23:04:26.020729+0000","last_active":"2024-05-11T23:04:41.045475+0000","last_peered":"2024-05-11T23:04:41.045475+0000","last_clean":"2024-05-11T23:04:41.045475+0000","last_became_active":"2024-05-11T23:04:25.806852+0000","last_became_peered":"2024-05-11T23:04:25.806852+0000","last_unstale":"2024-05-11T23:04:41.045475+0000","last_undegraded":"2024-05-11T23:04:41.045475+0000","last_fullsized":"2024-05-11T23:04:41.045475+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:12:25.591743+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.060182605,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":6324,"num_read_kb":25218,"num_write":659,"num_write_kb":11709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":10615766,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"174'897","reported_seq":21500,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044816+0000","last_change":"2024-05-11T23:04:18.014080+0000","last_active":"2024-05-11T23:04:41.044816+0000","last_peered":"2024-05-11T23:04:41.044816+0000","last_clean":"2024-05-11T23:04:41.044816+0000","last_became_active":"2024-05-11T23:01:27.465570+0000","last_became_peered":"2024-05-11T23:01:27.465570+0000","last_unstale":"2024-05-11T23:04:41.044816+0000","last_undegraded":"2024-05-11T23:04:41.044816+0000","last_fullsized":"2024-05-11T23:04:41.044816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:14:28.607937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003620968,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":22252,"num_read_kb":40862,"num_write":875,"num_write_kb":12354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"175'1308","reported_seq":14448,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044911+0000","last_change":"2024-05-11T23:04:18.014428+0000","last_active":"2024-05-11T23:04:41.044911+0000","last_peered":"2024-05-11T23:04:41.044911+0000","last_clean":"2024-05-11T23:04:41.044911+0000","last_became_active":"2024-05-11T23:03:52.804360+0000","last_became_peered":"2024-05-11T23:03:52.804360+0000","last_unstale":"2024-05-11T23:04:41.044911+0000","last_undegraded":"2024-05-11T23:04:41.044911+0000","last_fullsized":"2024-05-11T23:04:41.044911+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:09:37.091464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039667269999999998,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":17126,"num_read_kb":45672,"num_write":1555,"num_write_kb":23766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":18760061,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"168'1114","reported_seq":9524,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045480+0000","last_change":"2024-05-11T23:04:18.014829+0000","last_active":"2024-05-11T23:04:41.045480+0000","last_peered":"2024-05-11T23:04:41.045480+0000","last_clean":"2024-05-11T23:04:41.045480+0000","last_became_active":"2024-05-11T23:03:49.895043+0000","last_became_peered":"2024-05-11T23:03:49.895043+0000","last_unstale":"2024-05-11T23:04:41.045480+0000","last_undegraded":"2024-05-11T23:04:41.045480+0000","last_fullsized":"2024-05-11T23:04:41.045480+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:55:15.798774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040936139999999998,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":11168,"num_read_kb":37882,"num_write":1390,"num_write_kb":15155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":362,"num_bytes_recovered":15895238,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"174'678","reported_seq":7038,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118720+0000","last_change":"2024-05-11T23:04:28.807759+0000","last_active":"2024-05-11T23:04:42.118720+0000","last_peered":"2024-05-11T23:04:42.118720+0000","last_clean":"2024-05-11T23:04:42.118720+0000","last_became_active":"2024-05-11T23:04:28.806050+0000","last_became_peered":"2024-05-11T23:04:28.806050+0000","last_unstale":"2024-05-11T23:04:42.118720+0000","last_undegraded":"2024-05-11T23:04:42.118720+0000","last_fullsized":"2024-05-11T23:04:42.118720+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:58:42.310196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023348000000000002,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":7071,"num_read_kb":24639,"num_write":633,"num_write_kb":13630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":19437463,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"173'853","reported_seq":6346,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044618+0000","last_change":"2024-05-11T23:04:18.011274+0000","last_active":"2024-05-11T23:04:41.044618+0000","last_peered":"2024-05-11T23:04:41.044618+0000","last_clean":"2024-05-11T23:04:41.044618+0000","last_became_active":"2024-05-11T23:03:53.806713+0000","last_became_peered":"2024-05-11T23:03:53.806713+0000","last_unstale":"2024-05-11T23:04:41.044618+0000","last_undegraded":"2024-05-11T23:04:41.044618+0000","last_fullsized":"2024-05-11T23:04:41.044618+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":853,"log_dups_size":0,"ondisk_log_size":853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:25:53.472036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011357120000000001,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":7833,"num_read_kb":21923,"num_write":862,"num_write_kb":7573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":450,"num_bytes_recovered":18021482,"num_keys_recovered":78,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"175'975","reported_seq":5304,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044876+0000","last_change":"2024-05-11T23:04:27.810986+0000","last_active":"2024-05-11T23:04:41.044876+0000","last_peered":"2024-05-11T23:04:41.044876+0000","last_clean":"2024-05-11T23:04:41.044876+0000","last_became_active":"2024-05-11T23:04:27.809030+0000","last_became_peered":"2024-05-11T23:04:27.809030+0000","last_unstale":"2024-05-11T23:04:41.044876+0000","last_undegraded":"2024-05-11T23:04:41.044876+0000","last_fullsized":"2024-05-11T23:04:41.044876+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:20:47.733184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028179780000000001,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":6275,"num_read_kb":24022,"num_write":1130,"num_write_kb":13297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"173'599","reported_seq":5605,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044706+0000","last_change":"2024-05-11T23:04:18.013459+0000","last_active":"2024-05-11T23:04:41.044706+0000","last_peered":"2024-05-11T23:04:41.044706+0000","last_clean":"2024-05-11T23:04:41.044706+0000","last_became_active":"2024-05-11T23:03:52.818374+0000","last_became_peered":"2024-05-11T23:03:52.818374+0000","last_unstale":"2024-05-11T23:04:41.044706+0000","last_undegraded":"2024-05-11T23:04:41.044706+0000","last_fullsized":"2024-05-11T23:04:41.044706+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:21:58.570294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031987780000000002,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":5050,"num_read_kb":18194,"num_write":554,"num_write_kb":7419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4575002,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1465,"num_object_clones":0,"num_object_copies":2930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1465,"num_whiteouts":1461,"num_read":339431,"num_read_kb":1023367,"num_write":28503,"num_write_kb":498784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8731,"num_bytes_recovered":382481677,"num_keys_recovered":951,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16167,"internal_metadata":0},"log_size":26567,"ondisk_log_size":26567,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100800,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69608,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302232,"statfs":{"total":96636764160,"available":96565485568,"internally_reserved":0,"allocated":2387968,"data_stored":1719992,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509484,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59840,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312000,"statfs":{"total":96636764160,"available":96575488000,"internally_reserved":0,"allocated":2347008,"data_stored":1141250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509484,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65412,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306428,"statfs":{"total":96636764160,"available":96569782272,"internally_reserved":0,"allocated":2351104,"data_stored":1141269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33472,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338368,"statfs":{"total":96636764160,"available":96602488832,"internally_reserved":0,"allocated":2347008,"data_stored":1141250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":179,"seq":768799145989,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36028,"kb_used_data":2280,"kb_used_omap":49,"kb_used_meta":33678,"kb_avail":94335812,"statfs":{"total":96636764160,"available":96599871488,"internally_reserved":0,"allocated":2334720,"data_stored":1135437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51091,"internal_metadata":34486381},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73732,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298108,"statfs":{"total":96636764160,"available":96561262592,"internally_reserved":0,"allocated":2351104,"data_stored":1141269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37632,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334208,"statfs":{"total":96636764160,"available":96598228992,"internally_reserved":0,"allocated":2347008,"data_stored":1141250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77936,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293904,"statfs":{"total":96636764160,"available":96556957696,"internally_reserved":0,"allocated":2396160,"data_stored":1731618,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16167,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T23:04:44.726 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T23:04:45.031 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:04:45.031 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-11T23:04:45.043 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-05-11T23:04:43.173884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":1467,"num_object_clones":0,"num_object_copies":2934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1467,"num_whiteouts":1461,"num_read":339547,"num_read_kb":1023467,"num_write":28662,"num_write_kb":501200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8735,"num_bytes_recovered":383662413,"num_keys_recovered":951,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26684,"ondisk_log_size":26684,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":453660,"kb_used_data":18420,"kb_used_omap":60,"kb_used_meta":435075,"kb_avail":754521060,"statfs":{"total":773094113280,"available":772629565440,"internally_reserved":0,"allocated":18862080,"data_stored":10293335,"data_compressed":66655,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":62250,"internal_metadata":445517014},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.147643"},"pg_stats":[{"pgid":"3.1f","version":"175'540","reported_seq":5571,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044941+0000","last_change":"2024-05-11T23:04:28.806612+0000","last_active":"2024-05-11T23:04:41.044941+0000","last_peered":"2024-05-11T23:04:41.044941+0000","last_clean":"2024-05-11T23:04:41.044941+0000","last_became_active":"2024-05-11T23:04:28.804255+0000","last_became_peered":"2024-05-11T23:04:28.804255+0000","last_unstale":"2024-05-11T23:04:41.044941+0000","last_undegraded":"2024-05-11T23:04:41.044941+0000","last_fullsized":"2024-05-11T23:04:41.044941+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:57:34.663961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098957899999999998,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":5393,"num_read_kb":14723,"num_write":488,"num_write_kb":18081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":877932,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"180'934","reported_seq":6541,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.042173+0000","last_change":"2024-05-11T23:04:25.935404+0000","last_active":"2024-05-11T23:04:41.042173+0000","last_peered":"2024-05-11T23:04:41.042173+0000","last_clean":"2024-05-11T23:04:41.042173+0000","last_became_active":"2024-05-11T23:04:25.808413+0000","last_became_peered":"2024-05-11T23:04:25.808413+0000","last_unstale":"2024-05-11T23:04:41.042173+0000","last_undegraded":"2024-05-11T23:04:41.042173+0000","last_fullsized":"2024-05-11T23:04:41.042173+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":934,"log_dups_size":0,"ondisk_log_size":934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:09:17.133440+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.084167985000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":6916,"num_read_kb":49448,"num_write":971,"num_write_kb":22687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":53290749,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"167'508","reported_seq":4528,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045582+0000","last_change":"2024-05-11T23:04:17.950967+0000","last_active":"2024-05-11T23:04:41.045582+0000","last_peered":"2024-05-11T23:04:41.045582+0000","last_clean":"2024-05-11T23:04:41.045582+0000","last_became_active":"2024-05-11T23:03:48.889806+0000","last_became_peered":"2024-05-11T23:03:48.889806+0000","last_unstale":"2024-05-11T23:04:41.045582+0000","last_undegraded":"2024-05-11T23:04:41.045582+0000","last_fullsized":"2024-05-11T23:04:41.045582+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:46:37.757197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060389999999999999,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":3180,"num_read_kb":16428,"num_write":406,"num_write_kb":9629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":7067368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"175'784","reported_seq":19158,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045367+0000","last_change":"2024-05-11T23:04:17.951789+0000","last_active":"2024-05-11T23:04:41.045367+0000","last_peered":"2024-05-11T23:04:41.045367+0000","last_clean":"2024-05-11T23:04:41.045367+0000","last_became_active":"2024-05-11T23:03:50.798632+0000","last_became_peered":"2024-05-11T23:03:50.798632+0000","last_unstale":"2024-05-11T23:04:41.045367+0000","last_undegraded":"2024-05-11T23:04:41.045367+0000","last_fullsized":"2024-05-11T23:04:41.045367+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:13:59.992713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248566,"stat_sum":{"num_bytes":0,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":63,"num_read":18429,"num_read_kb":39778,"num_write":674,"num_write_kb":10510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":9396782,"num_keys_recovered":182,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"174'594","reported_seq":15111,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045102+0000","last_change":"2024-05-11T23:04:29.807594+0000","last_active":"2024-05-11T23:04:41.045102+0000","last_peered":"2024-05-11T23:04:41.045102+0000","last_clean":"2024-05-11T23:04:41.045102+0000","last_became_active":"2024-05-11T23:04:29.804143+0000","last_became_peered":"2024-05-11T23:04:29.804143+0000","last_unstale":"2024-05-11T23:04:41.045102+0000","last_undegraded":"2024-05-11T23:04:41.045102+0000","last_fullsized":"2024-05-11T23:04:41.045102+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:25:20.570143+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001388731,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":13979,"num_read_kb":68609,"num_write":598,"num_write_kb":35793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":532,"num_bytes_recovered":20898869,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"173'1664","reported_seq":21043,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.896600+0000","last_change":"2024-05-11T23:04:34.010543+0000","last_active":"2024-05-11T23:04:41.896600+0000","last_peered":"2024-05-11T23:04:41.896600+0000","last_clean":"2024-05-11T23:04:41.896600+0000","last_became_active":"2024-05-11T23:04:34.007358+0000","last_became_peered":"2024-05-11T23:04:34.007358+0000","last_unstale":"2024-05-11T23:04:41.896600+0000","last_undegraded":"2024-05-11T23:04:41.896600+0000","last_fullsized":"2024-05-11T23:04:41.896600+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1664,"log_dups_size":0,"ondisk_log_size":1664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:31:26.529561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097887899999999999,"stat_sum":{"num_bytes":19,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":25336,"num_read_kb":48131,"num_write":2723,"num_write_kb":23444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":6434587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"180'1052","reported_seq":15385,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.042982+0000","last_change":"2024-05-11T23:04:25.959047+0000","last_active":"2024-05-11T23:04:41.042982+0000","last_peered":"2024-05-11T23:04:41.042982+0000","last_clean":"2024-05-11T23:04:41.042982+0000","last_became_active":"2024-05-11T23:04:25.807274+0000","last_became_peered":"2024-05-11T23:04:25.807274+0000","last_unstale":"2024-05-11T23:04:41.042982+0000","last_undegraded":"2024-05-11T23:04:41.042982+0000","last_fullsized":"2024-05-11T23:04:41.042982+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1052,"log_dups_size":0,"ondisk_log_size":1052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:17:04.610363+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.061503582000000001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":16282,"num_read_kb":43182,"num_write":1369,"num_write_kb":14705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":13589947,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"170'967","reported_seq":3980,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.041570+0000","last_change":"2024-05-11T23:04:26.853599+0000","last_active":"2024-05-11T23:04:41.041570+0000","last_peered":"2024-05-11T23:04:41.041570+0000","last_clean":"2024-05-11T23:04:41.041570+0000","last_became_active":"2024-05-11T23:04:26.807825+0000","last_became_peered":"2024-05-11T23:04:26.807825+0000","last_unstale":"2024-05-11T23:04:41.041570+0000","last_undegraded":"2024-05-11T23:04:41.041570+0000","last_fullsized":"2024-05-11T23:04:41.041570+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:26:49.247146+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001621796,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":5754,"num_read_kb":9891,"num_write":1117,"num_write_kb":16486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":7487386,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"172'667","reported_seq":3869,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045052+0000","last_change":"2024-05-11T23:04:27.810906+0000","last_active":"2024-05-11T23:04:41.045052+0000","last_peered":"2024-05-11T23:04:41.045052+0000","last_clean":"2024-05-11T23:04:41.045052+0000","last_became_active":"2024-05-11T23:04:27.809321+0000","last_became_peered":"2024-05-11T23:04:27.809321+0000","last_unstale":"2024-05-11T23:04:41.045052+0000","last_undegraded":"2024-05-11T23:04:41.045052+0000","last_fullsized":"2024-05-11T23:04:41.045052+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:32:33.820874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013760389999999999,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":3972,"num_read_kb":15246,"num_write":808,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"172'559","reported_seq":15688,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045292+0000","last_change":"2024-05-11T23:04:17.950903+0000","last_active":"2024-05-11T23:04:41.045292+0000","last_peered":"2024-05-11T23:04:41.045292+0000","last_clean":"2024-05-11T23:04:41.045292+0000","last_became_active":"2024-05-11T23:01:27.463623+0000","last_became_peered":"2024-05-11T23:01:27.463623+0000","last_unstale":"2024-05-11T23:04:41.045292+0000","last_undegraded":"2024-05-11T23:04:41.045292+0000","last_fullsized":"2024-05-11T23:04:41.045292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:56:10.783772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054116699999999995,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":15017,"num_read_kb":54784,"num_write":481,"num_write_kb":18602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"180'622","reported_seq":16411,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044641+0000","last_change":"2024-05-11T23:04:26.015543+0000","last_active":"2024-05-11T23:04:41.044641+0000","last_peered":"2024-05-11T23:04:41.044641+0000","last_clean":"2024-05-11T23:04:41.044641+0000","last_became_active":"2024-05-11T23:04:25.807935+0000","last_became_peered":"2024-05-11T23:04:25.807935+0000","last_unstale":"2024-05-11T23:04:41.044641+0000","last_undegraded":"2024-05-11T23:04:41.044641+0000","last_fullsized":"2024-05-11T23:04:41.044641+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:50:30.593291+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.072194399000000006,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":14264,"num_read_kb":25298,"num_write":546,"num_write_kb":13154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":12363304,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"172'661","reported_seq":19582,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.896664+0000","last_change":"2024-05-11T23:04:34.010412+0000","last_active":"2024-05-11T23:04:41.896664+0000","last_peered":"2024-05-11T23:04:41.896664+0000","last_clean":"2024-05-11T23:04:41.896664+0000","last_became_active":"2024-05-11T23:04:34.007861+0000","last_became_peered":"2024-05-11T23:04:34.007861+0000","last_unstale":"2024-05-11T23:04:41.896664+0000","last_undegraded":"2024-05-11T23:04:41.896664+0000","last_fullsized":"2024-05-11T23:04:41.896664+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:13:07.675246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001389319,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":19402,"num_read_kb":42118,"num_write":569,"num_write_kb":6424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":4215359,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118756+0000","last_change":"2024-05-11T23:02:39.044686+0000","last_active":"2024-05-11T23:04:42.118756+0000","last_peered":"2024-05-11T23:04:42.118756+0000","last_clean":"2024-05-11T23:04:42.118756+0000","last_became_active":"2024-05-11T23:02:39.044216+0000","last_became_peered":"2024-05-11T23:02:39.044216+0000","last_unstale":"2024-05-11T23:04:42.118756+0000","last_undegraded":"2024-05-11T23:04:42.118756+0000","last_fullsized":"2024-05-11T23:04:42.118756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:31:03.826529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"175'1049","reported_seq":7444,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044838+0000","last_change":"2024-05-11T23:04:28.808149+0000","last_active":"2024-05-11T23:04:41.044838+0000","last_peered":"2024-05-11T23:04:41.044838+0000","last_clean":"2024-05-11T23:04:41.044838+0000","last_became_active":"2024-05-11T23:04:28.805428+0000","last_became_peered":"2024-05-11T23:04:28.805428+0000","last_unstale":"2024-05-11T23:04:41.044838+0000","last_undegraded":"2024-05-11T23:04:41.044838+0000","last_fullsized":"2024-05-11T23:04:41.044838+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:36:30.546501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002036403,"stat_sum":{"num_bytes":0,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":60,"num_read":9703,"num_read_kb":23103,"num_write":1155,"num_write_kb":12619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":2180842,"num_keys_recovered":21,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":335,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.043163+0000","last_change":"2024-05-11T23:04:29.806455+0000","last_active":"2024-05-11T23:04:41.043163+0000","last_peered":"2024-05-11T23:04:41.043163+0000","last_clean":"2024-05-11T23:04:41.043163+0000","last_became_active":"2024-05-11T23:04:29.805946+0000","last_became_peered":"2024-05-11T23:04:29.805946+0000","last_unstale":"2024-05-11T23:04:41.043163+0000","last_undegraded":"2024-05-11T23:04:41.043163+0000","last_fullsized":"2024-05-11T23:04:41.043163+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:09:54.127025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"180'926","reported_seq":6106,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-05-11T23:04:38.090033+0000","last_change":"2024-05-11T23:04:25.923075+0000","last_active":"2024-05-11T23:04:38.090033+0000","last_peered":"2024-05-11T23:04:38.090033+0000","last_clean":"2024-05-11T23:04:38.090033+0000","last_became_active":"2024-05-11T23:04:25.806037+0000","last_became_peered":"2024-05-11T23:04:25.806037+0000","last_unstale":"2024-05-11T23:04:38.090033+0000","last_undegraded":"2024-05-11T23:04:38.090033+0000","last_fullsized":"2024-05-11T23:04:38.090033+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:34:55.146392+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.064543953000000001,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":7310,"num_read_kb":19251,"num_write":1088,"num_write_kb":7185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":4415701,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":374,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044868+0000","last_change":"2024-05-11T23:04:25.806665+0000","last_active":"2024-05-11T23:04:41.044868+0000","last_peered":"2024-05-11T23:04:41.044868+0000","last_clean":"2024-05-11T23:04:41.044868+0000","last_became_active":"2024-05-11T23:04:25.806292+0000","last_became_peered":"2024-05-11T23:04:25.806292+0000","last_unstale":"2024-05-11T23:04:41.044868+0000","last_undegraded":"2024-05-11T23:04:41.044868+0000","last_fullsized":"2024-05-11T23:04:41.044868+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:12:45.144479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"175'564","reported_seq":14906,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.043149+0000","last_change":"2024-05-11T23:04:29.809156+0000","last_active":"2024-05-11T23:04:41.043149+0000","last_peered":"2024-05-11T23:04:41.043149+0000","last_clean":"2024-05-11T23:04:41.043149+0000","last_became_active":"2024-05-11T23:04:29.807343+0000","last_became_peered":"2024-05-11T23:04:29.807343+0000","last_unstale":"2024-05-11T23:04:41.043149+0000","last_undegraded":"2024-05-11T23:04:41.043149+0000","last_fullsized":"2024-05-11T23:04:41.043149+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:35:56.026223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095379299999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":14156,"num_read_kb":53683,"num_write":532,"num_write_kb":33731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":339,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.043292+0000","last_change":"2024-05-11T23:04:29.805774+0000","last_active":"2024-05-11T23:04:41.043292+0000","last_peered":"2024-05-11T23:04:41.043292+0000","last_clean":"2024-05-11T23:04:41.043292+0000","last_became_active":"2024-05-11T23:04:29.805447+0000","last_became_peered":"2024-05-11T23:04:29.805447+0000","last_unstale":"2024-05-11T23:04:41.043292+0000","last_undegraded":"2024-05-11T23:04:41.043292+0000","last_fullsized":"2024-05-11T23:04:41.043292+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:56:59.210181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"180'758","reported_seq":6310,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045646+0000","last_change":"2024-05-11T23:04:25.999769+0000","last_active":"2024-05-11T23:04:41.045646+0000","last_peered":"2024-05-11T23:04:41.045646+0000","last_clean":"2024-05-11T23:04:41.045646+0000","last_became_active":"2024-05-11T23:04:25.806236+0000","last_became_peered":"2024-05-11T23:04:25.806236+0000","last_unstale":"2024-05-11T23:04:41.045646+0000","last_undegraded":"2024-05-11T23:04:41.045646+0000","last_fullsized":"2024-05-11T23:04:41.045646+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:19:42.475478+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.081084693999999999,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":6715,"num_read_kb":25684,"num_write":652,"num_write_kb":13121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":13712581,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"175'1022","reported_seq":11245,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.896816+0000","last_change":"2024-05-11T23:04:34.011637+0000","last_active":"2024-05-11T23:04:41.896816+0000","last_peered":"2024-05-11T23:04:41.896816+0000","last_clean":"2024-05-11T23:04:41.896816+0000","last_became_active":"2024-05-11T23:04:34.008456+0000","last_became_peered":"2024-05-11T23:04:34.008456+0000","last_unstale":"2024-05-11T23:04:41.896816+0000","last_undegraded":"2024-05-11T23:04:41.896816+0000","last_fullsized":"2024-05-11T23:04:41.896816+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1022,"log_dups_size":0,"ondisk_log_size":1022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:56:16.226740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017329069999999999,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":56,"num_read":10698,"num_read_kb":33938,"num_write":973,"num_write_kb":17254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":898,"num_bytes_recovered":16208786,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":366,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045946+0000","last_change":"2024-05-11T23:01:19.221118+0000","last_active":"2024-05-11T23:04:41.045946+0000","last_peered":"2024-05-11T23:04:41.045946+0000","last_clean":"2024-05-11T23:04:41.045946+0000","last_became_active":"2024-05-11T23:01:19.220855+0000","last_became_peered":"2024-05-11T23:01:19.220855+0000","last_unstale":"2024-05-11T23:04:41.045946+0000","last_undegraded":"2024-05-11T23:04:41.045946+0000","last_fullsized":"2024-05-11T23:04:41.045946+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:01:56.076963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"173'1148","reported_seq":6343,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.042746+0000","last_change":"2024-05-11T23:04:17.940887+0000","last_active":"2024-05-11T23:04:41.042746+0000","last_peered":"2024-05-11T23:04:41.042746+0000","last_clean":"2024-05-11T23:04:41.042746+0000","last_became_active":"2024-05-11T23:01:27.459593+0000","last_became_peered":"2024-05-11T23:01:27.459593+0000","last_unstale":"2024-05-11T23:04:41.042746+0000","last_undegraded":"2024-05-11T23:04:41.042746+0000","last_fullsized":"2024-05-11T23:04:41.042746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1148,"log_dups_size":0,"ondisk_log_size":1148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:45.922442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017756580000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9995,"num_read_kb":18445,"num_write":1457,"num_write_kb":4408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":384,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.041751+0000","last_change":"2024-05-11T23:04:25.805119+0000","last_active":"2024-05-11T23:04:41.041751+0000","last_peered":"2024-05-11T23:04:41.041751+0000","last_clean":"2024-05-11T23:04:41.041751+0000","last_became_active":"2024-05-11T23:04:25.804674+0000","last_became_peered":"2024-05-11T23:04:25.804674+0000","last_unstale":"2024-05-11T23:04:41.041751+0000","last_undegraded":"2024-05-11T23:04:41.041751+0000","last_fullsized":"2024-05-11T23:04:41.041751+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:37:37.191290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"175'656","reported_seq":15312,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044755+0000","last_change":"2024-05-11T23:04:26.807059+0000","last_active":"2024-05-11T23:04:41.044755+0000","last_peered":"2024-05-11T23:04:41.044755+0000","last_clean":"2024-05-11T23:04:41.044755+0000","last_became_active":"2024-05-11T23:04:26.804248+0000","last_became_peered":"2024-05-11T23:04:26.804248+0000","last_unstale":"2024-05-11T23:04:41.044755+0000","last_undegraded":"2024-05-11T23:04:41.044755+0000","last_fullsized":"2024-05-11T23:04:41.044755+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:32:14.359230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014815119999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":15309,"num_read_kb":24676,"num_write":622,"num_write_kb":6352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":299,"num_bytes_recovered":6345105,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":364,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044805+0000","last_change":"2024-05-11T23:01:19.219755+0000","last_active":"2024-05-11T23:04:41.044805+0000","last_peered":"2024-05-11T23:04:41.044805+0000","last_clean":"2024-05-11T23:04:41.044805+0000","last_became_active":"2024-05-11T23:01:19.219471+0000","last_became_peered":"2024-05-11T23:01:19.219471+0000","last_unstale":"2024-05-11T23:04:41.044805+0000","last_undegraded":"2024-05-11T23:04:41.044805+0000","last_fullsized":"2024-05-11T23:04:41.044805+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:39:05.284213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"180'690","reported_seq":4329,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-05-11T23:04:38.022395+0000","last_change":"2024-05-11T23:04:25.884271+0000","last_active":"2024-05-11T23:04:38.022395+0000","last_peered":"2024-05-11T23:04:38.022395+0000","last_clean":"2024-05-11T23:04:38.022395+0000","last_became_active":"2024-05-11T23:04:25.805994+0000","last_became_peered":"2024-05-11T23:04:25.805994+0000","last_unstale":"2024-05-11T23:04:38.022395+0000","last_undegraded":"2024-05-11T23:04:38.022395+0000","last_fullsized":"2024-05-11T23:04:38.022395+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:43:03.144032+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.055474953,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":3497,"num_read_kb":16044,"num_write":640,"num_write_kb":10409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":10719266,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":366,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045169+0000","last_change":"2024-05-11T23:01:19.219128+0000","last_active":"2024-05-11T23:04:41.045169+0000","last_peered":"2024-05-11T23:04:41.045169+0000","last_clean":"2024-05-11T23:04:41.045169+0000","last_became_active":"2024-05-11T23:01:19.218846+0000","last_became_peered":"2024-05-11T23:01:19.218846+0000","last_unstale":"2024-05-11T23:04:41.045169+0000","last_undegraded":"2024-05-11T23:04:41.045169+0000","last_fullsized":"2024-05-11T23:04:41.045169+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:41:09.111000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":542,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044642+0000","last_change":"2024-05-11T23:04:28.806566+0000","last_active":"2024-05-11T23:04:41.044642+0000","last_peered":"2024-05-11T23:04:41.044642+0000","last_clean":"2024-05-11T23:04:41.044642+0000","last_became_active":"2024-05-11T23:04:28.806124+0000","last_became_peered":"2024-05-11T23:04:28.806124+0000","last_unstale":"2024-05-11T23:04:41.044642+0000","last_undegraded":"2024-05-11T23:04:41.044642+0000","last_fullsized":"2024-05-11T23:04:41.044642+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_clean_scrub_stamp":"2024-05-11T23:01:16.147564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:03:40.590362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"180'966","reported_seq":5552,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.041792+0000","last_change":"2024-05-11T23:04:25.885127+0000","last_active":"2024-05-11T23:04:41.041792+0000","last_peered":"2024-05-11T23:04:41.041792+0000","last_clean":"2024-05-11T23:04:41.041792+0000","last_became_active":"2024-05-11T23:04:25.802390+0000","last_became_peered":"2024-05-11T23:04:25.802390+0000","last_unstale":"2024-05-11T23:04:41.041792+0000","last_undegraded":"2024-05-11T23:04:41.041792+0000","last_fullsized":"2024-05-11T23:04:41.041792+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":966,"log_dups_size":0,"ondisk_log_size":966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:45:25.229272+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.057229434000000003,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":6339,"num_read_kb":20167,"num_write":1151,"num_write_kb":10790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":4220183,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"172'676","reported_seq":9167,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.896968+0000","last_change":"2024-05-11T23:04:34.008013+0000","last_active":"2024-05-11T23:04:41.896968+0000","last_peered":"2024-05-11T23:04:41.896968+0000","last_clean":"2024-05-11T23:04:41.896968+0000","last_became_active":"2024-05-11T23:04:34.005569+0000","last_became_peered":"2024-05-11T23:04:34.005569+0000","last_unstale":"2024-05-11T23:04:41.896968+0000","last_undegraded":"2024-05-11T23:04:41.896968+0000","last_fullsized":"2024-05-11T23:04:41.896968+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:27:04.828149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064185299999999995,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":9043,"num_read_kb":47546,"num_write":541,"num_write_kb":29805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":37320801,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"175'785","reported_seq":6658,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118728+0000","last_change":"2024-05-11T23:04:17.987769+0000","last_active":"2024-05-11T23:04:42.118728+0000","last_peered":"2024-05-11T23:04:42.118728+0000","last_clean":"2024-05-11T23:04:42.118728+0000","last_became_active":"2024-05-11T23:03:51.797214+0000","last_became_peered":"2024-05-11T23:03:51.797214+0000","last_unstale":"2024-05-11T23:04:42.118728+0000","last_undegraded":"2024-05-11T23:04:42.118728+0000","last_fullsized":"2024-05-11T23:04:42.118728+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:04:04.928406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001270001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":44,"num_read":5814,"num_read_kb":22550,"num_write":685,"num_write_kb":9625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":538,"num_bytes_recovered":16997094,"num_keys_recovered":140,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"173'657","reported_seq":10543,"reported_epoch":193,"state":"active+clean","last_fresh":"2024-05-11T23:04:39.159213+0000","last_change":"2024-05-11T23:04:27.905632+0000","last_active":"2024-05-11T23:04:39.159213+0000","last_peered":"2024-05-11T23:04:39.159213+0000","last_clean":"2024-05-11T23:04:39.159213+0000","last_became_active":"2024-05-11T23:04:27.803860+0000","last_became_peered":"2024-05-11T23:04:27.803860+0000","last_unstale":"2024-05-11T23:04:39.159213+0000","last_undegraded":"2024-05-11T23:04:39.159213+0000","last_fullsized":"2024-05-11T23:04:39.159213+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T09:48:49.893537+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00088000000000000003,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":9829,"num_read_kb":52232,"num_write":599,"num_write_kb":42645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":401,"num_bytes_recovered":47434023,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"180'694","reported_seq":5214,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045475+0000","last_change":"2024-05-11T23:04:26.020729+0000","last_active":"2024-05-11T23:04:41.045475+0000","last_peered":"2024-05-11T23:04:41.045475+0000","last_clean":"2024-05-11T23:04:41.045475+0000","last_became_active":"2024-05-11T23:04:25.806852+0000","last_became_peered":"2024-05-11T23:04:25.806852+0000","last_unstale":"2024-05-11T23:04:41.045475+0000","last_undegraded":"2024-05-11T23:04:41.045475+0000","last_fullsized":"2024-05-11T23:04:41.045475+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:12:25.591743+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.060182605,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":6324,"num_read_kb":25218,"num_write":659,"num_write_kb":11709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":10615766,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"174'897","reported_seq":21500,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044816+0000","last_change":"2024-05-11T23:04:18.014080+0000","last_active":"2024-05-11T23:04:41.044816+0000","last_peered":"2024-05-11T23:04:41.044816+0000","last_clean":"2024-05-11T23:04:41.044816+0000","last_became_active":"2024-05-11T23:01:27.465570+0000","last_became_peered":"2024-05-11T23:01:27.465570+0000","last_unstale":"2024-05-11T23:04:41.044816+0000","last_undegraded":"2024-05-11T23:04:41.044816+0000","last_fullsized":"2024-05-11T23:04:41.044816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:14:28.607937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003620968,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":22252,"num_read_kb":40862,"num_write":875,"num_write_kb":12354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"175'1308","reported_seq":14448,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044911+0000","last_change":"2024-05-11T23:04:18.014428+0000","last_active":"2024-05-11T23:04:41.044911+0000","last_peered":"2024-05-11T23:04:41.044911+0000","last_clean":"2024-05-11T23:04:41.044911+0000","last_became_active":"2024-05-11T23:03:52.804360+0000","last_became_peered":"2024-05-11T23:03:52.804360+0000","last_unstale":"2024-05-11T23:04:41.044911+0000","last_undegraded":"2024-05-11T23:04:41.044911+0000","last_fullsized":"2024-05-11T23:04:41.044911+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:09:37.091464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039667269999999998,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":17126,"num_read_kb":45672,"num_write":1555,"num_write_kb":23766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":18760061,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"168'1114","reported_seq":9524,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045480+0000","last_change":"2024-05-11T23:04:18.014829+0000","last_active":"2024-05-11T23:04:41.045480+0000","last_peered":"2024-05-11T23:04:41.045480+0000","last_clean":"2024-05-11T23:04:41.045480+0000","last_became_active":"2024-05-11T23:03:49.895043+0000","last_became_peered":"2024-05-11T23:03:49.895043+0000","last_unstale":"2024-05-11T23:04:41.045480+0000","last_undegraded":"2024-05-11T23:04:41.045480+0000","last_fullsized":"2024-05-11T23:04:41.045480+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:55:15.798774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040936139999999998,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":11168,"num_read_kb":37882,"num_write":1390,"num_write_kb":15155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":362,"num_bytes_recovered":15895238,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"174'678","reported_seq":7038,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118720+0000","last_change":"2024-05-11T23:04:28.807759+0000","last_active":"2024-05-11T23:04:42.118720+0000","last_peered":"2024-05-11T23:04:42.118720+0000","last_clean":"2024-05-11T23:04:42.118720+0000","last_became_active":"2024-05-11T23:04:28.806050+0000","last_became_peered":"2024-05-11T23:04:28.806050+0000","last_unstale":"2024-05-11T23:04:42.118720+0000","last_undegraded":"2024-05-11T23:04:42.118720+0000","last_fullsized":"2024-05-11T23:04:42.118720+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:58:42.310196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023348000000000002,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":7071,"num_read_kb":24639,"num_write":633,"num_write_kb":13630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":19437463,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"173'853","reported_seq":6346,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044618+0000","last_change":"2024-05-11T23:04:18.011274+0000","last_active":"2024-05-11T23:04:41.044618+0000","last_peered":"2024-05-11T23:04:41.044618+0000","last_clean":"2024-05-11T23:04:41.044618+0000","last_became_active":"2024-05-11T23:03:53.806713+0000","last_became_peered":"2024-05-11T23:03:53.806713+0000","last_unstale":"2024-05-11T23:04:41.044618+0000","last_undegraded":"2024-05-11T23:04:41.044618+0000","last_fullsized":"2024-05-11T23:04:41.044618+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":853,"log_dups_size":0,"ondisk_log_size":853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:25:53.472036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011357120000000001,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":7833,"num_read_kb":21923,"num_write":862,"num_write_kb":7573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":450,"num_bytes_recovered":18021482,"num_keys_recovered":78,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"175'975","reported_seq":5304,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044876+0000","last_change":"2024-05-11T23:04:27.810986+0000","last_active":"2024-05-11T23:04:41.044876+0000","last_peered":"2024-05-11T23:04:41.044876+0000","last_clean":"2024-05-11T23:04:41.044876+0000","last_became_active":"2024-05-11T23:04:27.809030+0000","last_became_peered":"2024-05-11T23:04:27.809030+0000","last_unstale":"2024-05-11T23:04:41.044876+0000","last_undegraded":"2024-05-11T23:04:41.044876+0000","last_fullsized":"2024-05-11T23:04:41.044876+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:20:47.733184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028179780000000001,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":6275,"num_read_kb":24022,"num_write":1130,"num_write_kb":13297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"173'599","reported_seq":5605,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044706+0000","last_change":"2024-05-11T23:04:18.013459+0000","last_active":"2024-05-11T23:04:41.044706+0000","last_peered":"2024-05-11T23:04:41.044706+0000","last_clean":"2024-05-11T23:04:41.044706+0000","last_became_active":"2024-05-11T23:03:52.818374+0000","last_became_peered":"2024-05-11T23:03:52.818374+0000","last_unstale":"2024-05-11T23:04:41.044706+0000","last_undegraded":"2024-05-11T23:04:41.044706+0000","last_fullsized":"2024-05-11T23:04:41.044706+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:21:58.570294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031987780000000002,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":5050,"num_read_kb":18194,"num_write":554,"num_write_kb":7419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4575002,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1465,"num_object_clones":0,"num_object_copies":2930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1465,"num_whiteouts":1461,"num_read":339431,"num_read_kb":1023367,"num_write":28503,"num_write_kb":498784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8731,"num_bytes_recovered":382481677,"num_keys_recovered":951,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16167,"internal_metadata":0},"log_size":26567,"ondisk_log_size":26567,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100800,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509483,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69608,"kb_used_data":2332,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302232,"statfs":{"total":96636764160,"available":96565485568,"internally_reserved":0,"allocated":2387968,"data_stored":1719992,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509484,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59840,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312000,"statfs":{"total":96636764160,"available":96575488000,"internally_reserved":0,"allocated":2347008,"data_stored":1141250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509484,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65412,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306428,"statfs":{"total":96636764160,"available":96569782272,"internally_reserved":0,"allocated":2351104,"data_stored":1141269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33472,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338368,"statfs":{"total":96636764160,"available":96602488832,"internally_reserved":0,"allocated":2347008,"data_stored":1141250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":179,"seq":768799145989,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36028,"kb_used_data":2280,"kb_used_omap":49,"kb_used_meta":33678,"kb_avail":94335812,"statfs":{"total":96636764160,"available":96599871488,"internally_reserved":0,"allocated":2334720,"data_stored":1135437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51091,"internal_metadata":34486381},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73732,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298108,"statfs":{"total":96636764160,"available":96561262592,"internally_reserved":0,"allocated":2351104,"data_stored":1141269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37632,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334208,"statfs":{"total":96636764160,"available":96598228992,"internally_reserved":0,"allocated":2347008,"data_stored":1141250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77936,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293904,"statfs":{"total":96636764160,"available":96556957696,"internally_reserved":0,"allocated":2396160,"data_stored":1731618,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16167,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T23:04:45.045 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-05-11T23:04:45.045 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-05-11T23:04:45.363 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-11T23:04:45.372 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-11T23:04:45.373 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T23:04:45.674 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:04:45.675 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-11T23:04:45.686 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-05-11T23:04:45.174196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":1467,"num_object_clones":0,"num_object_copies":2934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1467,"num_whiteouts":1461,"num_read":339547,"num_read_kb":1023467,"num_write":28662,"num_write_kb":501200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8674,"num_bytes_recovered":383662413,"num_keys_recovered":951,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26684,"ondisk_log_size":26684,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":453748,"kb_used_data":18504,"kb_used_omap":60,"kb_used_meta":435075,"kb_avail":754520972,"statfs":{"total":773094113280,"available":772629475328,"internally_reserved":0,"allocated":18948096,"data_stored":10334026,"data_compressed":66655,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":62250,"internal_metadata":445517014},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000923"},"pg_stats":[{"pgid":"3.1f","version":"175'540","reported_seq":5571,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044941+0000","last_change":"2024-05-11T23:04:28.806612+0000","last_active":"2024-05-11T23:04:41.044941+0000","last_peered":"2024-05-11T23:04:41.044941+0000","last_clean":"2024-05-11T23:04:41.044941+0000","last_became_active":"2024-05-11T23:04:28.804255+0000","last_became_peered":"2024-05-11T23:04:28.804255+0000","last_unstale":"2024-05-11T23:04:41.044941+0000","last_undegraded":"2024-05-11T23:04:41.044941+0000","last_fullsized":"2024-05-11T23:04:41.044941+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:57:34.663961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098957899999999998,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":5393,"num_read_kb":14723,"num_write":488,"num_write_kb":18081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":877932,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"180'934","reported_seq":6543,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.057168+0000","last_change":"2024-05-11T23:04:25.935404+0000","last_active":"2024-05-11T23:04:42.057168+0000","last_peered":"2024-05-11T23:04:42.057168+0000","last_clean":"2024-05-11T23:04:42.057168+0000","last_became_active":"2024-05-11T23:04:25.808413+0000","last_became_peered":"2024-05-11T23:04:25.808413+0000","last_unstale":"2024-05-11T23:04:42.057168+0000","last_undegraded":"2024-05-11T23:04:42.057168+0000","last_fullsized":"2024-05-11T23:04:42.057168+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":934,"log_dups_size":0,"ondisk_log_size":934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:09:17.133440+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.084167985000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":6916,"num_read_kb":49448,"num_write":971,"num_write_kb":22687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":53290749,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"167'508","reported_seq":4530,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058246+0000","last_change":"2024-05-11T23:04:17.950967+0000","last_active":"2024-05-11T23:04:42.058246+0000","last_peered":"2024-05-11T23:04:42.058246+0000","last_clean":"2024-05-11T23:04:42.058246+0000","last_became_active":"2024-05-11T23:03:48.889806+0000","last_became_peered":"2024-05-11T23:03:48.889806+0000","last_unstale":"2024-05-11T23:04:42.058246+0000","last_undegraded":"2024-05-11T23:04:42.058246+0000","last_fullsized":"2024-05-11T23:04:42.058246+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:46:37.757197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060389999999999999,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":3180,"num_read_kb":16428,"num_write":406,"num_write_kb":9629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":7067368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"175'784","reported_seq":19160,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058324+0000","last_change":"2024-05-11T23:04:17.951789+0000","last_active":"2024-05-11T23:04:42.058324+0000","last_peered":"2024-05-11T23:04:42.058324+0000","last_clean":"2024-05-11T23:04:42.058324+0000","last_became_active":"2024-05-11T23:03:50.798632+0000","last_became_peered":"2024-05-11T23:03:50.798632+0000","last_unstale":"2024-05-11T23:04:42.058324+0000","last_undegraded":"2024-05-11T23:04:42.058324+0000","last_fullsized":"2024-05-11T23:04:42.058324+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:13:59.992713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248566,"stat_sum":{"num_bytes":0,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":63,"num_read":18429,"num_read_kb":39778,"num_write":674,"num_write_kb":10510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":9396782,"num_keys_recovered":182,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"174'594","reported_seq":15111,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045102+0000","last_change":"2024-05-11T23:04:29.807594+0000","last_active":"2024-05-11T23:04:41.045102+0000","last_peered":"2024-05-11T23:04:41.045102+0000","last_clean":"2024-05-11T23:04:41.045102+0000","last_became_active":"2024-05-11T23:04:29.804143+0000","last_became_peered":"2024-05-11T23:04:29.804143+0000","last_unstale":"2024-05-11T23:04:41.045102+0000","last_undegraded":"2024-05-11T23:04:41.045102+0000","last_fullsized":"2024-05-11T23:04:41.045102+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:25:20.570143+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001388731,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":13979,"num_read_kb":68609,"num_write":598,"num_write_kb":35793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":532,"num_bytes_recovered":20898869,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"173'1664","reported_seq":21045,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118917+0000","last_change":"2024-05-11T23:04:34.010543+0000","last_active":"2024-05-11T23:04:42.118917+0000","last_peered":"2024-05-11T23:04:42.118917+0000","last_clean":"2024-05-11T23:04:42.118917+0000","last_became_active":"2024-05-11T23:04:34.007358+0000","last_became_peered":"2024-05-11T23:04:34.007358+0000","last_unstale":"2024-05-11T23:04:42.118917+0000","last_undegraded":"2024-05-11T23:04:42.118917+0000","last_fullsized":"2024-05-11T23:04:42.118917+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1664,"log_dups_size":0,"ondisk_log_size":1664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:31:26.529561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097887899999999999,"stat_sum":{"num_bytes":19,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":25336,"num_read_kb":48131,"num_write":2723,"num_write_kb":23444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":6434587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"180'1052","reported_seq":15385,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.042982+0000","last_change":"2024-05-11T23:04:25.959047+0000","last_active":"2024-05-11T23:04:41.042982+0000","last_peered":"2024-05-11T23:04:41.042982+0000","last_clean":"2024-05-11T23:04:41.042982+0000","last_became_active":"2024-05-11T23:04:25.807274+0000","last_became_peered":"2024-05-11T23:04:25.807274+0000","last_unstale":"2024-05-11T23:04:41.042982+0000","last_undegraded":"2024-05-11T23:04:41.042982+0000","last_fullsized":"2024-05-11T23:04:41.042982+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1052,"log_dups_size":0,"ondisk_log_size":1052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:17:04.610363+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.061503582000000001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":16282,"num_read_kb":43182,"num_write":1369,"num_write_kb":14705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":13589947,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"170'967","reported_seq":3982,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.056362+0000","last_change":"2024-05-11T23:04:26.853599+0000","last_active":"2024-05-11T23:04:42.056362+0000","last_peered":"2024-05-11T23:04:42.056362+0000","last_clean":"2024-05-11T23:04:42.056362+0000","last_became_active":"2024-05-11T23:04:26.807825+0000","last_became_peered":"2024-05-11T23:04:26.807825+0000","last_unstale":"2024-05-11T23:04:42.056362+0000","last_undegraded":"2024-05-11T23:04:42.056362+0000","last_fullsized":"2024-05-11T23:04:42.056362+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:26:49.247146+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001621796,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":5754,"num_read_kb":9891,"num_write":1117,"num_write_kb":16486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":7487386,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"172'667","reported_seq":3869,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045052+0000","last_change":"2024-05-11T23:04:27.810906+0000","last_active":"2024-05-11T23:04:41.045052+0000","last_peered":"2024-05-11T23:04:41.045052+0000","last_clean":"2024-05-11T23:04:41.045052+0000","last_became_active":"2024-05-11T23:04:27.809321+0000","last_became_peered":"2024-05-11T23:04:27.809321+0000","last_unstale":"2024-05-11T23:04:41.045052+0000","last_undegraded":"2024-05-11T23:04:41.045052+0000","last_fullsized":"2024-05-11T23:04:41.045052+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:32:33.820874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013760389999999999,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":3972,"num_read_kb":15246,"num_write":808,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"172'559","reported_seq":15690,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058382+0000","last_change":"2024-05-11T23:04:17.950903+0000","last_active":"2024-05-11T23:04:42.058382+0000","last_peered":"2024-05-11T23:04:42.058382+0000","last_clean":"2024-05-11T23:04:42.058382+0000","last_became_active":"2024-05-11T23:01:27.463623+0000","last_became_peered":"2024-05-11T23:01:27.463623+0000","last_unstale":"2024-05-11T23:04:42.058382+0000","last_undegraded":"2024-05-11T23:04:42.058382+0000","last_fullsized":"2024-05-11T23:04:42.058382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:56:10.783772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054116699999999995,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":15017,"num_read_kb":54784,"num_write":481,"num_write_kb":18602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"180'622","reported_seq":16413,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058371+0000","last_change":"2024-05-11T23:04:26.015543+0000","last_active":"2024-05-11T23:04:42.058371+0000","last_peered":"2024-05-11T23:04:42.058371+0000","last_clean":"2024-05-11T23:04:42.058371+0000","last_became_active":"2024-05-11T23:04:25.807935+0000","last_became_peered":"2024-05-11T23:04:25.807935+0000","last_unstale":"2024-05-11T23:04:42.058371+0000","last_undegraded":"2024-05-11T23:04:42.058371+0000","last_fullsized":"2024-05-11T23:04:42.058371+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:50:30.593291+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.072194399000000006,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":14264,"num_read_kb":25298,"num_write":546,"num_write_kb":13154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":12363304,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"172'661","reported_seq":19584,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.119099+0000","last_change":"2024-05-11T23:04:34.010412+0000","last_active":"2024-05-11T23:04:42.119099+0000","last_peered":"2024-05-11T23:04:42.119099+0000","last_clean":"2024-05-11T23:04:42.119099+0000","last_became_active":"2024-05-11T23:04:34.007861+0000","last_became_peered":"2024-05-11T23:04:34.007861+0000","last_unstale":"2024-05-11T23:04:42.119099+0000","last_undegraded":"2024-05-11T23:04:42.119099+0000","last_fullsized":"2024-05-11T23:04:42.119099+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:13:07.675246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001389319,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":19402,"num_read_kb":42118,"num_write":569,"num_write_kb":6424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":4215359,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118756+0000","last_change":"2024-05-11T23:02:39.044686+0000","last_active":"2024-05-11T23:04:42.118756+0000","last_peered":"2024-05-11T23:04:42.118756+0000","last_clean":"2024-05-11T23:04:42.118756+0000","last_became_active":"2024-05-11T23:02:39.044216+0000","last_became_peered":"2024-05-11T23:02:39.044216+0000","last_unstale":"2024-05-11T23:04:42.118756+0000","last_undegraded":"2024-05-11T23:04:42.118756+0000","last_fullsized":"2024-05-11T23:04:42.118756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:31:03.826529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"175'1049","reported_seq":7444,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044838+0000","last_change":"2024-05-11T23:04:28.808149+0000","last_active":"2024-05-11T23:04:41.044838+0000","last_peered":"2024-05-11T23:04:41.044838+0000","last_clean":"2024-05-11T23:04:41.044838+0000","last_became_active":"2024-05-11T23:04:28.805428+0000","last_became_peered":"2024-05-11T23:04:28.805428+0000","last_unstale":"2024-05-11T23:04:41.044838+0000","last_undegraded":"2024-05-11T23:04:41.044838+0000","last_fullsized":"2024-05-11T23:04:41.044838+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:36:30.546501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002036403,"stat_sum":{"num_bytes":0,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":60,"num_read":9703,"num_read_kb":23103,"num_write":1155,"num_write_kb":12619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":2180842,"num_keys_recovered":21,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":335,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.043163+0000","last_change":"2024-05-11T23:04:29.806455+0000","last_active":"2024-05-11T23:04:41.043163+0000","last_peered":"2024-05-11T23:04:41.043163+0000","last_clean":"2024-05-11T23:04:41.043163+0000","last_became_active":"2024-05-11T23:04:29.805946+0000","last_became_peered":"2024-05-11T23:04:29.805946+0000","last_unstale":"2024-05-11T23:04:41.043163+0000","last_undegraded":"2024-05-11T23:04:41.043163+0000","last_fullsized":"2024-05-11T23:04:41.043163+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:09:54.127025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"180'926","reported_seq":6079,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.063271+0000","last_change":"2024-05-11T23:04:42.063271+0000","last_active":"2024-05-11T23:04:42.063271+0000","last_peered":"2024-05-11T23:04:42.063271+0000","last_clean":"2024-05-11T23:04:42.063271+0000","last_became_active":"2024-05-11T23:04:42.060396+0000","last_became_peered":"2024-05-11T23:04:42.060396+0000","last_unstale":"2024-05-11T23:04:42.063271+0000","last_undegraded":"2024-05-11T23:04:42.063271+0000","last_fullsized":"2024-05-11T23:04:42.063271+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:02:26.606646+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.055140898000000001,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":7310,"num_read_kb":19251,"num_write":1088,"num_write_kb":7185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":4415701,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":376,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058504+0000","last_change":"2024-05-11T23:04:25.806665+0000","last_active":"2024-05-11T23:04:42.058504+0000","last_peered":"2024-05-11T23:04:42.058504+0000","last_clean":"2024-05-11T23:04:42.058504+0000","last_became_active":"2024-05-11T23:04:25.806292+0000","last_became_peered":"2024-05-11T23:04:25.806292+0000","last_unstale":"2024-05-11T23:04:42.058504+0000","last_undegraded":"2024-05-11T23:04:42.058504+0000","last_fullsized":"2024-05-11T23:04:42.058504+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:12:45.144479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"175'564","reported_seq":14906,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.043149+0000","last_change":"2024-05-11T23:04:29.809156+0000","last_active":"2024-05-11T23:04:41.043149+0000","last_peered":"2024-05-11T23:04:41.043149+0000","last_clean":"2024-05-11T23:04:41.043149+0000","last_became_active":"2024-05-11T23:04:29.807343+0000","last_became_peered":"2024-05-11T23:04:29.807343+0000","last_unstale":"2024-05-11T23:04:41.043149+0000","last_undegraded":"2024-05-11T23:04:41.043149+0000","last_fullsized":"2024-05-11T23:04:41.043149+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:35:56.026223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095379299999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":14156,"num_read_kb":53683,"num_write":532,"num_write_kb":33731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":339,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.043292+0000","last_change":"2024-05-11T23:04:29.805774+0000","last_active":"2024-05-11T23:04:41.043292+0000","last_peered":"2024-05-11T23:04:41.043292+0000","last_clean":"2024-05-11T23:04:41.043292+0000","last_became_active":"2024-05-11T23:04:29.805447+0000","last_became_peered":"2024-05-11T23:04:29.805447+0000","last_unstale":"2024-05-11T23:04:41.043292+0000","last_undegraded":"2024-05-11T23:04:41.043292+0000","last_fullsized":"2024-05-11T23:04:41.043292+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:56:59.210181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"180'758","reported_seq":6312,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058458+0000","last_change":"2024-05-11T23:04:25.999769+0000","last_active":"2024-05-11T23:04:42.058458+0000","last_peered":"2024-05-11T23:04:42.058458+0000","last_clean":"2024-05-11T23:04:42.058458+0000","last_became_active":"2024-05-11T23:04:25.806236+0000","last_became_peered":"2024-05-11T23:04:25.806236+0000","last_unstale":"2024-05-11T23:04:42.058458+0000","last_undegraded":"2024-05-11T23:04:42.058458+0000","last_fullsized":"2024-05-11T23:04:42.058458+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:19:42.475478+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.081084693999999999,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":6715,"num_read_kb":25684,"num_write":652,"num_write_kb":13121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":13712581,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"175'1022","reported_seq":11247,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.119002+0000","last_change":"2024-05-11T23:04:34.011637+0000","last_active":"2024-05-11T23:04:42.119002+0000","last_peered":"2024-05-11T23:04:42.119002+0000","last_clean":"2024-05-11T23:04:42.119002+0000","last_became_active":"2024-05-11T23:04:34.008456+0000","last_became_peered":"2024-05-11T23:04:34.008456+0000","last_unstale":"2024-05-11T23:04:42.119002+0000","last_undegraded":"2024-05-11T23:04:42.119002+0000","last_fullsized":"2024-05-11T23:04:42.119002+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1022,"log_dups_size":0,"ondisk_log_size":1022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:56:16.226740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017329069999999999,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":56,"num_read":10698,"num_read_kb":33938,"num_write":973,"num_write_kb":17254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":898,"num_bytes_recovered":16208786,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":368,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058873+0000","last_change":"2024-05-11T23:01:19.221118+0000","last_active":"2024-05-11T23:04:42.058873+0000","last_peered":"2024-05-11T23:04:42.058873+0000","last_clean":"2024-05-11T23:04:42.058873+0000","last_became_active":"2024-05-11T23:01:19.220855+0000","last_became_peered":"2024-05-11T23:01:19.220855+0000","last_unstale":"2024-05-11T23:04:42.058873+0000","last_undegraded":"2024-05-11T23:04:42.058873+0000","last_fullsized":"2024-05-11T23:04:42.058873+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:01:56.076963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"173'1148","reported_seq":6343,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.042746+0000","last_change":"2024-05-11T23:04:17.940887+0000","last_active":"2024-05-11T23:04:41.042746+0000","last_peered":"2024-05-11T23:04:41.042746+0000","last_clean":"2024-05-11T23:04:41.042746+0000","last_became_active":"2024-05-11T23:01:27.459593+0000","last_became_peered":"2024-05-11T23:01:27.459593+0000","last_unstale":"2024-05-11T23:04:41.042746+0000","last_undegraded":"2024-05-11T23:04:41.042746+0000","last_fullsized":"2024-05-11T23:04:41.042746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1148,"log_dups_size":0,"ondisk_log_size":1148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:45.922442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017756580000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9995,"num_read_kb":18445,"num_write":1457,"num_write_kb":4408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":384,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.041751+0000","last_change":"2024-05-11T23:04:25.805119+0000","last_active":"2024-05-11T23:04:41.041751+0000","last_peered":"2024-05-11T23:04:41.041751+0000","last_clean":"2024-05-11T23:04:41.041751+0000","last_became_active":"2024-05-11T23:04:25.804674+0000","last_became_peered":"2024-05-11T23:04:25.804674+0000","last_unstale":"2024-05-11T23:04:41.041751+0000","last_undegraded":"2024-05-11T23:04:41.041751+0000","last_fullsized":"2024-05-11T23:04:41.041751+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:37:37.191290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"175'656","reported_seq":15312,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044755+0000","last_change":"2024-05-11T23:04:26.807059+0000","last_active":"2024-05-11T23:04:41.044755+0000","last_peered":"2024-05-11T23:04:41.044755+0000","last_clean":"2024-05-11T23:04:41.044755+0000","last_became_active":"2024-05-11T23:04:26.804248+0000","last_became_peered":"2024-05-11T23:04:26.804248+0000","last_unstale":"2024-05-11T23:04:41.044755+0000","last_undegraded":"2024-05-11T23:04:41.044755+0000","last_fullsized":"2024-05-11T23:04:41.044755+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:32:14.359230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014815119999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":15309,"num_read_kb":24676,"num_write":622,"num_write_kb":6352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":299,"num_bytes_recovered":6345105,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":366,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058188+0000","last_change":"2024-05-11T23:01:19.219755+0000","last_active":"2024-05-11T23:04:42.058188+0000","last_peered":"2024-05-11T23:04:42.058188+0000","last_clean":"2024-05-11T23:04:42.058188+0000","last_became_active":"2024-05-11T23:01:19.219471+0000","last_became_peered":"2024-05-11T23:01:19.219471+0000","last_unstale":"2024-05-11T23:04:42.058188+0000","last_undegraded":"2024-05-11T23:04:42.058188+0000","last_fullsized":"2024-05-11T23:04:42.058188+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:39:05.284213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"180'690","reported_seq":4302,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.064001+0000","last_change":"2024-05-11T23:04:42.064001+0000","last_active":"2024-05-11T23:04:42.064001+0000","last_peered":"2024-05-11T23:04:42.064001+0000","last_clean":"2024-05-11T23:04:42.064001+0000","last_became_active":"2024-05-11T23:04:42.060595+0000","last_became_peered":"2024-05-11T23:04:42.060595+0000","last_unstale":"2024-05-11T23:04:42.064001+0000","last_undegraded":"2024-05-11T23:04:42.064001+0000","last_fullsized":"2024-05-11T23:04:42.064001+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:38:46.487523+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.040069711000000001,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":3497,"num_read_kb":16044,"num_write":640,"num_write_kb":10409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":10719266,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":368,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058198+0000","last_change":"2024-05-11T23:01:19.219128+0000","last_active":"2024-05-11T23:04:42.058198+0000","last_peered":"2024-05-11T23:04:42.058198+0000","last_clean":"2024-05-11T23:04:42.058198+0000","last_became_active":"2024-05-11T23:01:19.218846+0000","last_became_peered":"2024-05-11T23:01:19.218846+0000","last_unstale":"2024-05-11T23:04:42.058198+0000","last_undegraded":"2024-05-11T23:04:42.058198+0000","last_fullsized":"2024-05-11T23:04:42.058198+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:41:09.111000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":544,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058200+0000","last_change":"2024-05-11T23:04:28.806566+0000","last_active":"2024-05-11T23:04:42.058200+0000","last_peered":"2024-05-11T23:04:42.058200+0000","last_clean":"2024-05-11T23:04:42.058200+0000","last_became_active":"2024-05-11T23:04:28.806124+0000","last_became_peered":"2024-05-11T23:04:28.806124+0000","last_unstale":"2024-05-11T23:04:42.058200+0000","last_undegraded":"2024-05-11T23:04:42.058200+0000","last_fullsized":"2024-05-11T23:04:42.058200+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_clean_scrub_stamp":"2024-05-11T23:01:16.147564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:03:40.590362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"180'966","reported_seq":5554,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.056462+0000","last_change":"2024-05-11T23:04:25.885127+0000","last_active":"2024-05-11T23:04:42.056462+0000","last_peered":"2024-05-11T23:04:42.056462+0000","last_clean":"2024-05-11T23:04:42.056462+0000","last_became_active":"2024-05-11T23:04:25.802390+0000","last_became_peered":"2024-05-11T23:04:25.802390+0000","last_unstale":"2024-05-11T23:04:42.056462+0000","last_undegraded":"2024-05-11T23:04:42.056462+0000","last_fullsized":"2024-05-11T23:04:42.056462+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":966,"log_dups_size":0,"ondisk_log_size":966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:45:25.229272+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.057229434000000003,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":6339,"num_read_kb":20167,"num_write":1151,"num_write_kb":10790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":4220183,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"172'676","reported_seq":9169,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118976+0000","last_change":"2024-05-11T23:04:34.008013+0000","last_active":"2024-05-11T23:04:42.118976+0000","last_peered":"2024-05-11T23:04:42.118976+0000","last_clean":"2024-05-11T23:04:42.118976+0000","last_became_active":"2024-05-11T23:04:34.005569+0000","last_became_peered":"2024-05-11T23:04:34.005569+0000","last_unstale":"2024-05-11T23:04:42.118976+0000","last_undegraded":"2024-05-11T23:04:42.118976+0000","last_fullsized":"2024-05-11T23:04:42.118976+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:27:04.828149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064185299999999995,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":9043,"num_read_kb":47546,"num_write":541,"num_write_kb":29805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":37320801,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"175'785","reported_seq":6658,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118728+0000","last_change":"2024-05-11T23:04:17.987769+0000","last_active":"2024-05-11T23:04:42.118728+0000","last_peered":"2024-05-11T23:04:42.118728+0000","last_clean":"2024-05-11T23:04:42.118728+0000","last_became_active":"2024-05-11T23:03:51.797214+0000","last_became_peered":"2024-05-11T23:03:51.797214+0000","last_unstale":"2024-05-11T23:04:42.118728+0000","last_undegraded":"2024-05-11T23:04:42.118728+0000","last_fullsized":"2024-05-11T23:04:42.118728+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:04:04.928406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001270001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":44,"num_read":5814,"num_read_kb":22550,"num_write":685,"num_write_kb":9625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":538,"num_bytes_recovered":16997094,"num_keys_recovered":140,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"173'657","reported_seq":9945,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.076988+0000","last_change":"2024-05-11T23:04:42.076988+0000","last_active":"2024-05-11T23:04:42.076988+0000","last_peered":"2024-05-11T23:04:42.076988+0000","last_clean":"2024-05-11T23:04:42.076988+0000","last_became_active":"2024-05-11T23:04:42.073145+0000","last_became_peered":"2024-05-11T23:04:42.073145+0000","last_unstale":"2024-05-11T23:04:42.076988+0000","last_undegraded":"2024-05-11T23:04:42.076988+0000","last_fullsized":"2024-05-11T23:04:42.076988+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:50:27.744009+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00088000000000000003,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":9829,"num_read_kb":52232,"num_write":599,"num_write_kb":42645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":47434023,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"180'694","reported_seq":5216,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058529+0000","last_change":"2024-05-11T23:04:26.020729+0000","last_active":"2024-05-11T23:04:42.058529+0000","last_peered":"2024-05-11T23:04:42.058529+0000","last_clean":"2024-05-11T23:04:42.058529+0000","last_became_active":"2024-05-11T23:04:25.806852+0000","last_became_peered":"2024-05-11T23:04:25.806852+0000","last_unstale":"2024-05-11T23:04:42.058529+0000","last_undegraded":"2024-05-11T23:04:42.058529+0000","last_fullsized":"2024-05-11T23:04:42.058529+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:12:25.591743+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.060182605,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":6324,"num_read_kb":25218,"num_write":659,"num_write_kb":11709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":10615766,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"174'897","reported_seq":21502,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058455+0000","last_change":"2024-05-11T23:04:18.014080+0000","last_active":"2024-05-11T23:04:42.058455+0000","last_peered":"2024-05-11T23:04:42.058455+0000","last_clean":"2024-05-11T23:04:42.058455+0000","last_became_active":"2024-05-11T23:01:27.465570+0000","last_became_peered":"2024-05-11T23:01:27.465570+0000","last_unstale":"2024-05-11T23:04:42.058455+0000","last_undegraded":"2024-05-11T23:04:42.058455+0000","last_fullsized":"2024-05-11T23:04:42.058455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:14:28.607937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003620968,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":22252,"num_read_kb":40862,"num_write":875,"num_write_kb":12354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"175'1308","reported_seq":14450,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058370+0000","last_change":"2024-05-11T23:04:18.014428+0000","last_active":"2024-05-11T23:04:42.058370+0000","last_peered":"2024-05-11T23:04:42.058370+0000","last_clean":"2024-05-11T23:04:42.058370+0000","last_became_active":"2024-05-11T23:03:52.804360+0000","last_became_peered":"2024-05-11T23:03:52.804360+0000","last_unstale":"2024-05-11T23:04:42.058370+0000","last_undegraded":"2024-05-11T23:04:42.058370+0000","last_fullsized":"2024-05-11T23:04:42.058370+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:09:37.091464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039667269999999998,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":17126,"num_read_kb":45672,"num_write":1555,"num_write_kb":23766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":18760061,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"168'1114","reported_seq":9526,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058558+0000","last_change":"2024-05-11T23:04:18.014829+0000","last_active":"2024-05-11T23:04:42.058558+0000","last_peered":"2024-05-11T23:04:42.058558+0000","last_clean":"2024-05-11T23:04:42.058558+0000","last_became_active":"2024-05-11T23:03:49.895043+0000","last_became_peered":"2024-05-11T23:03:49.895043+0000","last_unstale":"2024-05-11T23:04:42.058558+0000","last_undegraded":"2024-05-11T23:04:42.058558+0000","last_fullsized":"2024-05-11T23:04:42.058558+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:55:15.798774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040936139999999998,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":11168,"num_read_kb":37882,"num_write":1390,"num_write_kb":15155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":362,"num_bytes_recovered":15895238,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"174'678","reported_seq":7038,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118720+0000","last_change":"2024-05-11T23:04:28.807759+0000","last_active":"2024-05-11T23:04:42.118720+0000","last_peered":"2024-05-11T23:04:42.118720+0000","last_clean":"2024-05-11T23:04:42.118720+0000","last_became_active":"2024-05-11T23:04:28.806050+0000","last_became_peered":"2024-05-11T23:04:28.806050+0000","last_unstale":"2024-05-11T23:04:42.118720+0000","last_undegraded":"2024-05-11T23:04:42.118720+0000","last_fullsized":"2024-05-11T23:04:42.118720+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:58:42.310196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023348000000000002,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":7071,"num_read_kb":24639,"num_write":633,"num_write_kb":13630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":19437463,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"173'853","reported_seq":6348,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058241+0000","last_change":"2024-05-11T23:04:18.011274+0000","last_active":"2024-05-11T23:04:42.058241+0000","last_peered":"2024-05-11T23:04:42.058241+0000","last_clean":"2024-05-11T23:04:42.058241+0000","last_became_active":"2024-05-11T23:03:53.806713+0000","last_became_peered":"2024-05-11T23:03:53.806713+0000","last_unstale":"2024-05-11T23:04:42.058241+0000","last_undegraded":"2024-05-11T23:04:42.058241+0000","last_fullsized":"2024-05-11T23:04:42.058241+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":853,"log_dups_size":0,"ondisk_log_size":853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:25:53.472036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011357120000000001,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":7833,"num_read_kb":21923,"num_write":862,"num_write_kb":7573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":450,"num_bytes_recovered":18021482,"num_keys_recovered":78,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"175'975","reported_seq":5304,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044876+0000","last_change":"2024-05-11T23:04:27.810986+0000","last_active":"2024-05-11T23:04:41.044876+0000","last_peered":"2024-05-11T23:04:41.044876+0000","last_clean":"2024-05-11T23:04:41.044876+0000","last_became_active":"2024-05-11T23:04:27.809030+0000","last_became_peered":"2024-05-11T23:04:27.809030+0000","last_unstale":"2024-05-11T23:04:41.044876+0000","last_undegraded":"2024-05-11T23:04:41.044876+0000","last_fullsized":"2024-05-11T23:04:41.044876+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:20:47.733184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028179780000000001,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":6275,"num_read_kb":24022,"num_write":1130,"num_write_kb":13297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"173'599","reported_seq":5607,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058311+0000","last_change":"2024-05-11T23:04:18.013459+0000","last_active":"2024-05-11T23:04:42.058311+0000","last_peered":"2024-05-11T23:04:42.058311+0000","last_clean":"2024-05-11T23:04:42.058311+0000","last_became_active":"2024-05-11T23:03:52.818374+0000","last_became_peered":"2024-05-11T23:03:52.818374+0000","last_unstale":"2024-05-11T23:04:42.058311+0000","last_undegraded":"2024-05-11T23:04:42.058311+0000","last_fullsized":"2024-05-11T23:04:42.058311+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:21:58.570294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031987780000000002,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":5050,"num_read_kb":18194,"num_write":554,"num_write_kb":7419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4575002,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1465,"num_object_clones":0,"num_object_copies":2930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1465,"num_whiteouts":1461,"num_read":339431,"num_read_kb":1023367,"num_write":28503,"num_write_kb":498784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8670,"num_bytes_recovered":382481677,"num_keys_recovered":951,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16167,"internal_metadata":0},"log_size":26567,"ondisk_log_size":26567,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100800,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509484,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69644,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302196,"statfs":{"total":96636764160,"available":96565448704,"internally_reserved":0,"allocated":2424832,"data_stored":1737431,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59856,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311984,"statfs":{"total":96636764160,"available":96575471616,"internally_reserved":0,"allocated":2359296,"data_stored":1147063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509485,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65428,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306412,"statfs":{"total":96636764160,"available":96569765888,"internally_reserved":0,"allocated":2363392,"data_stored":1147082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33472,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338368,"statfs":{"total":96636764160,"available":96602488832,"internally_reserved":0,"allocated":2347008,"data_stored":1141250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":179,"seq":768799145990,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36048,"kb_used_data":2304,"kb_used_omap":49,"kb_used_meta":33678,"kb_avail":94335792,"statfs":{"total":96636764160,"available":96599851008,"internally_reserved":0,"allocated":2359296,"data_stored":1147063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51091,"internal_metadata":34486381},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73732,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298108,"statfs":{"total":96636764160,"available":96561262592,"internally_reserved":0,"allocated":2351104,"data_stored":1141269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37632,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334208,"statfs":{"total":96636764160,"available":96598228992,"internally_reserved":0,"allocated":2347008,"data_stored":1141250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77936,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293904,"statfs":{"total":96636764160,"available":96556957696,"internally_reserved":0,"allocated":2396160,"data_stored":1731618,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16167,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T23:04:45.688 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T23:04:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:04:45.990 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-11T23:04:46.002 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-05-11T23:04:45.174196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":1467,"num_object_clones":0,"num_object_copies":2934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1467,"num_whiteouts":1461,"num_read":339547,"num_read_kb":1023467,"num_write":28662,"num_write_kb":501200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8674,"num_bytes_recovered":383662413,"num_keys_recovered":951,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26684,"ondisk_log_size":26684,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":453748,"kb_used_data":18504,"kb_used_omap":60,"kb_used_meta":435075,"kb_avail":754520972,"statfs":{"total":773094113280,"available":772629475328,"internally_reserved":0,"allocated":18948096,"data_stored":10334026,"data_compressed":66655,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":62250,"internal_metadata":445517014},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000923"},"pg_stats":[{"pgid":"3.1f","version":"175'540","reported_seq":5571,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044941+0000","last_change":"2024-05-11T23:04:28.806612+0000","last_active":"2024-05-11T23:04:41.044941+0000","last_peered":"2024-05-11T23:04:41.044941+0000","last_clean":"2024-05-11T23:04:41.044941+0000","last_became_active":"2024-05-11T23:04:28.804255+0000","last_became_peered":"2024-05-11T23:04:28.804255+0000","last_unstale":"2024-05-11T23:04:41.044941+0000","last_undegraded":"2024-05-11T23:04:41.044941+0000","last_fullsized":"2024-05-11T23:04:41.044941+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:57:34.663961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098957899999999998,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":5393,"num_read_kb":14723,"num_write":488,"num_write_kb":18081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":877932,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"180'934","reported_seq":6543,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.057168+0000","last_change":"2024-05-11T23:04:25.935404+0000","last_active":"2024-05-11T23:04:42.057168+0000","last_peered":"2024-05-11T23:04:42.057168+0000","last_clean":"2024-05-11T23:04:42.057168+0000","last_became_active":"2024-05-11T23:04:25.808413+0000","last_became_peered":"2024-05-11T23:04:25.808413+0000","last_unstale":"2024-05-11T23:04:42.057168+0000","last_undegraded":"2024-05-11T23:04:42.057168+0000","last_fullsized":"2024-05-11T23:04:42.057168+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":934,"log_dups_size":0,"ondisk_log_size":934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:09:17.133440+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.084167985000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":6916,"num_read_kb":49448,"num_write":971,"num_write_kb":22687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":53290749,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"167'508","reported_seq":4530,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058246+0000","last_change":"2024-05-11T23:04:17.950967+0000","last_active":"2024-05-11T23:04:42.058246+0000","last_peered":"2024-05-11T23:04:42.058246+0000","last_clean":"2024-05-11T23:04:42.058246+0000","last_became_active":"2024-05-11T23:03:48.889806+0000","last_became_peered":"2024-05-11T23:03:48.889806+0000","last_unstale":"2024-05-11T23:04:42.058246+0000","last_undegraded":"2024-05-11T23:04:42.058246+0000","last_fullsized":"2024-05-11T23:04:42.058246+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:46:37.757197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060389999999999999,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":3180,"num_read_kb":16428,"num_write":406,"num_write_kb":9629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":7067368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"175'784","reported_seq":19160,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058324+0000","last_change":"2024-05-11T23:04:17.951789+0000","last_active":"2024-05-11T23:04:42.058324+0000","last_peered":"2024-05-11T23:04:42.058324+0000","last_clean":"2024-05-11T23:04:42.058324+0000","last_became_active":"2024-05-11T23:03:50.798632+0000","last_became_peered":"2024-05-11T23:03:50.798632+0000","last_unstale":"2024-05-11T23:04:42.058324+0000","last_undegraded":"2024-05-11T23:04:42.058324+0000","last_fullsized":"2024-05-11T23:04:42.058324+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:13:59.992713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248566,"stat_sum":{"num_bytes":0,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":63,"num_read":18429,"num_read_kb":39778,"num_write":674,"num_write_kb":10510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":9396782,"num_keys_recovered":182,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"174'594","reported_seq":15111,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045102+0000","last_change":"2024-05-11T23:04:29.807594+0000","last_active":"2024-05-11T23:04:41.045102+0000","last_peered":"2024-05-11T23:04:41.045102+0000","last_clean":"2024-05-11T23:04:41.045102+0000","last_became_active":"2024-05-11T23:04:29.804143+0000","last_became_peered":"2024-05-11T23:04:29.804143+0000","last_unstale":"2024-05-11T23:04:41.045102+0000","last_undegraded":"2024-05-11T23:04:41.045102+0000","last_fullsized":"2024-05-11T23:04:41.045102+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:25:20.570143+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001388731,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":13979,"num_read_kb":68609,"num_write":598,"num_write_kb":35793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":532,"num_bytes_recovered":20898869,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"173'1664","reported_seq":21045,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118917+0000","last_change":"2024-05-11T23:04:34.010543+0000","last_active":"2024-05-11T23:04:42.118917+0000","last_peered":"2024-05-11T23:04:42.118917+0000","last_clean":"2024-05-11T23:04:42.118917+0000","last_became_active":"2024-05-11T23:04:34.007358+0000","last_became_peered":"2024-05-11T23:04:34.007358+0000","last_unstale":"2024-05-11T23:04:42.118917+0000","last_undegraded":"2024-05-11T23:04:42.118917+0000","last_fullsized":"2024-05-11T23:04:42.118917+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1664,"log_dups_size":0,"ondisk_log_size":1664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:31:26.529561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097887899999999999,"stat_sum":{"num_bytes":19,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":25336,"num_read_kb":48131,"num_write":2723,"num_write_kb":23444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":6434587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"180'1052","reported_seq":15385,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.042982+0000","last_change":"2024-05-11T23:04:25.959047+0000","last_active":"2024-05-11T23:04:41.042982+0000","last_peered":"2024-05-11T23:04:41.042982+0000","last_clean":"2024-05-11T23:04:41.042982+0000","last_became_active":"2024-05-11T23:04:25.807274+0000","last_became_peered":"2024-05-11T23:04:25.807274+0000","last_unstale":"2024-05-11T23:04:41.042982+0000","last_undegraded":"2024-05-11T23:04:41.042982+0000","last_fullsized":"2024-05-11T23:04:41.042982+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1052,"log_dups_size":0,"ondisk_log_size":1052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:17:04.610363+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.061503582000000001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":16282,"num_read_kb":43182,"num_write":1369,"num_write_kb":14705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":13589947,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"170'967","reported_seq":3982,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.056362+0000","last_change":"2024-05-11T23:04:26.853599+0000","last_active":"2024-05-11T23:04:42.056362+0000","last_peered":"2024-05-11T23:04:42.056362+0000","last_clean":"2024-05-11T23:04:42.056362+0000","last_became_active":"2024-05-11T23:04:26.807825+0000","last_became_peered":"2024-05-11T23:04:26.807825+0000","last_unstale":"2024-05-11T23:04:42.056362+0000","last_undegraded":"2024-05-11T23:04:42.056362+0000","last_fullsized":"2024-05-11T23:04:42.056362+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:26:49.247146+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001621796,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":5754,"num_read_kb":9891,"num_write":1117,"num_write_kb":16486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":7487386,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"172'667","reported_seq":3869,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045052+0000","last_change":"2024-05-11T23:04:27.810906+0000","last_active":"2024-05-11T23:04:41.045052+0000","last_peered":"2024-05-11T23:04:41.045052+0000","last_clean":"2024-05-11T23:04:41.045052+0000","last_became_active":"2024-05-11T23:04:27.809321+0000","last_became_peered":"2024-05-11T23:04:27.809321+0000","last_unstale":"2024-05-11T23:04:41.045052+0000","last_undegraded":"2024-05-11T23:04:41.045052+0000","last_fullsized":"2024-05-11T23:04:41.045052+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:32:33.820874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013760389999999999,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":3972,"num_read_kb":15246,"num_write":808,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"172'559","reported_seq":15690,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058382+0000","last_change":"2024-05-11T23:04:17.950903+0000","last_active":"2024-05-11T23:04:42.058382+0000","last_peered":"2024-05-11T23:04:42.058382+0000","last_clean":"2024-05-11T23:04:42.058382+0000","last_became_active":"2024-05-11T23:01:27.463623+0000","last_became_peered":"2024-05-11T23:01:27.463623+0000","last_unstale":"2024-05-11T23:04:42.058382+0000","last_undegraded":"2024-05-11T23:04:42.058382+0000","last_fullsized":"2024-05-11T23:04:42.058382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:56:10.783772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054116699999999995,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":15017,"num_read_kb":54784,"num_write":481,"num_write_kb":18602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"180'622","reported_seq":16413,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058371+0000","last_change":"2024-05-11T23:04:26.015543+0000","last_active":"2024-05-11T23:04:42.058371+0000","last_peered":"2024-05-11T23:04:42.058371+0000","last_clean":"2024-05-11T23:04:42.058371+0000","last_became_active":"2024-05-11T23:04:25.807935+0000","last_became_peered":"2024-05-11T23:04:25.807935+0000","last_unstale":"2024-05-11T23:04:42.058371+0000","last_undegraded":"2024-05-11T23:04:42.058371+0000","last_fullsized":"2024-05-11T23:04:42.058371+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:50:30.593291+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.072194399000000006,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":14264,"num_read_kb":25298,"num_write":546,"num_write_kb":13154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":12363304,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"172'661","reported_seq":19584,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.119099+0000","last_change":"2024-05-11T23:04:34.010412+0000","last_active":"2024-05-11T23:04:42.119099+0000","last_peered":"2024-05-11T23:04:42.119099+0000","last_clean":"2024-05-11T23:04:42.119099+0000","last_became_active":"2024-05-11T23:04:34.007861+0000","last_became_peered":"2024-05-11T23:04:34.007861+0000","last_unstale":"2024-05-11T23:04:42.119099+0000","last_undegraded":"2024-05-11T23:04:42.119099+0000","last_fullsized":"2024-05-11T23:04:42.119099+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:13:07.675246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001389319,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":19402,"num_read_kb":42118,"num_write":569,"num_write_kb":6424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":4215359,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118756+0000","last_change":"2024-05-11T23:02:39.044686+0000","last_active":"2024-05-11T23:04:42.118756+0000","last_peered":"2024-05-11T23:04:42.118756+0000","last_clean":"2024-05-11T23:04:42.118756+0000","last_became_active":"2024-05-11T23:02:39.044216+0000","last_became_peered":"2024-05-11T23:02:39.044216+0000","last_unstale":"2024-05-11T23:04:42.118756+0000","last_undegraded":"2024-05-11T23:04:42.118756+0000","last_fullsized":"2024-05-11T23:04:42.118756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:31:03.826529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"175'1049","reported_seq":7444,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044838+0000","last_change":"2024-05-11T23:04:28.808149+0000","last_active":"2024-05-11T23:04:41.044838+0000","last_peered":"2024-05-11T23:04:41.044838+0000","last_clean":"2024-05-11T23:04:41.044838+0000","last_became_active":"2024-05-11T23:04:28.805428+0000","last_became_peered":"2024-05-11T23:04:28.805428+0000","last_unstale":"2024-05-11T23:04:41.044838+0000","last_undegraded":"2024-05-11T23:04:41.044838+0000","last_fullsized":"2024-05-11T23:04:41.044838+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:36:30.546501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002036403,"stat_sum":{"num_bytes":0,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":60,"num_read":9703,"num_read_kb":23103,"num_write":1155,"num_write_kb":12619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":2180842,"num_keys_recovered":21,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":335,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.043163+0000","last_change":"2024-05-11T23:04:29.806455+0000","last_active":"2024-05-11T23:04:41.043163+0000","last_peered":"2024-05-11T23:04:41.043163+0000","last_clean":"2024-05-11T23:04:41.043163+0000","last_became_active":"2024-05-11T23:04:29.805946+0000","last_became_peered":"2024-05-11T23:04:29.805946+0000","last_unstale":"2024-05-11T23:04:41.043163+0000","last_undegraded":"2024-05-11T23:04:41.043163+0000","last_fullsized":"2024-05-11T23:04:41.043163+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:09:54.127025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"180'926","reported_seq":6079,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.063271+0000","last_change":"2024-05-11T23:04:42.063271+0000","last_active":"2024-05-11T23:04:42.063271+0000","last_peered":"2024-05-11T23:04:42.063271+0000","last_clean":"2024-05-11T23:04:42.063271+0000","last_became_active":"2024-05-11T23:04:42.060396+0000","last_became_peered":"2024-05-11T23:04:42.060396+0000","last_unstale":"2024-05-11T23:04:42.063271+0000","last_undegraded":"2024-05-11T23:04:42.063271+0000","last_fullsized":"2024-05-11T23:04:42.063271+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:02:26.606646+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.055140898000000001,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":7310,"num_read_kb":19251,"num_write":1088,"num_write_kb":7185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":4415701,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":376,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058504+0000","last_change":"2024-05-11T23:04:25.806665+0000","last_active":"2024-05-11T23:04:42.058504+0000","last_peered":"2024-05-11T23:04:42.058504+0000","last_clean":"2024-05-11T23:04:42.058504+0000","last_became_active":"2024-05-11T23:04:25.806292+0000","last_became_peered":"2024-05-11T23:04:25.806292+0000","last_unstale":"2024-05-11T23:04:42.058504+0000","last_undegraded":"2024-05-11T23:04:42.058504+0000","last_fullsized":"2024-05-11T23:04:42.058504+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:12:45.144479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"175'564","reported_seq":14906,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.043149+0000","last_change":"2024-05-11T23:04:29.809156+0000","last_active":"2024-05-11T23:04:41.043149+0000","last_peered":"2024-05-11T23:04:41.043149+0000","last_clean":"2024-05-11T23:04:41.043149+0000","last_became_active":"2024-05-11T23:04:29.807343+0000","last_became_peered":"2024-05-11T23:04:29.807343+0000","last_unstale":"2024-05-11T23:04:41.043149+0000","last_undegraded":"2024-05-11T23:04:41.043149+0000","last_fullsized":"2024-05-11T23:04:41.043149+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:35:56.026223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095379299999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":14156,"num_read_kb":53683,"num_write":532,"num_write_kb":33731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":339,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.043292+0000","last_change":"2024-05-11T23:04:29.805774+0000","last_active":"2024-05-11T23:04:41.043292+0000","last_peered":"2024-05-11T23:04:41.043292+0000","last_clean":"2024-05-11T23:04:41.043292+0000","last_became_active":"2024-05-11T23:04:29.805447+0000","last_became_peered":"2024-05-11T23:04:29.805447+0000","last_unstale":"2024-05-11T23:04:41.043292+0000","last_undegraded":"2024-05-11T23:04:41.043292+0000","last_fullsized":"2024-05-11T23:04:41.043292+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:56:59.210181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"180'758","reported_seq":6312,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058458+0000","last_change":"2024-05-11T23:04:25.999769+0000","last_active":"2024-05-11T23:04:42.058458+0000","last_peered":"2024-05-11T23:04:42.058458+0000","last_clean":"2024-05-11T23:04:42.058458+0000","last_became_active":"2024-05-11T23:04:25.806236+0000","last_became_peered":"2024-05-11T23:04:25.806236+0000","last_unstale":"2024-05-11T23:04:42.058458+0000","last_undegraded":"2024-05-11T23:04:42.058458+0000","last_fullsized":"2024-05-11T23:04:42.058458+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:19:42.475478+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.081084693999999999,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":6715,"num_read_kb":25684,"num_write":652,"num_write_kb":13121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":13712581,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"175'1022","reported_seq":11247,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.119002+0000","last_change":"2024-05-11T23:04:34.011637+0000","last_active":"2024-05-11T23:04:42.119002+0000","last_peered":"2024-05-11T23:04:42.119002+0000","last_clean":"2024-05-11T23:04:42.119002+0000","last_became_active":"2024-05-11T23:04:34.008456+0000","last_became_peered":"2024-05-11T23:04:34.008456+0000","last_unstale":"2024-05-11T23:04:42.119002+0000","last_undegraded":"2024-05-11T23:04:42.119002+0000","last_fullsized":"2024-05-11T23:04:42.119002+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1022,"log_dups_size":0,"ondisk_log_size":1022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:56:16.226740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017329069999999999,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":56,"num_read":10698,"num_read_kb":33938,"num_write":973,"num_write_kb":17254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":898,"num_bytes_recovered":16208786,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":368,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058873+0000","last_change":"2024-05-11T23:01:19.221118+0000","last_active":"2024-05-11T23:04:42.058873+0000","last_peered":"2024-05-11T23:04:42.058873+0000","last_clean":"2024-05-11T23:04:42.058873+0000","last_became_active":"2024-05-11T23:01:19.220855+0000","last_became_peered":"2024-05-11T23:01:19.220855+0000","last_unstale":"2024-05-11T23:04:42.058873+0000","last_undegraded":"2024-05-11T23:04:42.058873+0000","last_fullsized":"2024-05-11T23:04:42.058873+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:01:56.076963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"173'1148","reported_seq":6343,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.042746+0000","last_change":"2024-05-11T23:04:17.940887+0000","last_active":"2024-05-11T23:04:41.042746+0000","last_peered":"2024-05-11T23:04:41.042746+0000","last_clean":"2024-05-11T23:04:41.042746+0000","last_became_active":"2024-05-11T23:01:27.459593+0000","last_became_peered":"2024-05-11T23:01:27.459593+0000","last_unstale":"2024-05-11T23:04:41.042746+0000","last_undegraded":"2024-05-11T23:04:41.042746+0000","last_fullsized":"2024-05-11T23:04:41.042746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1148,"log_dups_size":0,"ondisk_log_size":1148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:45.922442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017756580000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9995,"num_read_kb":18445,"num_write":1457,"num_write_kb":4408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":384,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.041751+0000","last_change":"2024-05-11T23:04:25.805119+0000","last_active":"2024-05-11T23:04:41.041751+0000","last_peered":"2024-05-11T23:04:41.041751+0000","last_clean":"2024-05-11T23:04:41.041751+0000","last_became_active":"2024-05-11T23:04:25.804674+0000","last_became_peered":"2024-05-11T23:04:25.804674+0000","last_unstale":"2024-05-11T23:04:41.041751+0000","last_undegraded":"2024-05-11T23:04:41.041751+0000","last_fullsized":"2024-05-11T23:04:41.041751+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:37:37.191290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"175'656","reported_seq":15312,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044755+0000","last_change":"2024-05-11T23:04:26.807059+0000","last_active":"2024-05-11T23:04:41.044755+0000","last_peered":"2024-05-11T23:04:41.044755+0000","last_clean":"2024-05-11T23:04:41.044755+0000","last_became_active":"2024-05-11T23:04:26.804248+0000","last_became_peered":"2024-05-11T23:04:26.804248+0000","last_unstale":"2024-05-11T23:04:41.044755+0000","last_undegraded":"2024-05-11T23:04:41.044755+0000","last_fullsized":"2024-05-11T23:04:41.044755+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:32:14.359230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014815119999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":15309,"num_read_kb":24676,"num_write":622,"num_write_kb":6352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":299,"num_bytes_recovered":6345105,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":366,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058188+0000","last_change":"2024-05-11T23:01:19.219755+0000","last_active":"2024-05-11T23:04:42.058188+0000","last_peered":"2024-05-11T23:04:42.058188+0000","last_clean":"2024-05-11T23:04:42.058188+0000","last_became_active":"2024-05-11T23:01:19.219471+0000","last_became_peered":"2024-05-11T23:01:19.219471+0000","last_unstale":"2024-05-11T23:04:42.058188+0000","last_undegraded":"2024-05-11T23:04:42.058188+0000","last_fullsized":"2024-05-11T23:04:42.058188+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:39:05.284213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"180'690","reported_seq":4302,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.064001+0000","last_change":"2024-05-11T23:04:42.064001+0000","last_active":"2024-05-11T23:04:42.064001+0000","last_peered":"2024-05-11T23:04:42.064001+0000","last_clean":"2024-05-11T23:04:42.064001+0000","last_became_active":"2024-05-11T23:04:42.060595+0000","last_became_peered":"2024-05-11T23:04:42.060595+0000","last_unstale":"2024-05-11T23:04:42.064001+0000","last_undegraded":"2024-05-11T23:04:42.064001+0000","last_fullsized":"2024-05-11T23:04:42.064001+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:38:46.487523+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.040069711000000001,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":3497,"num_read_kb":16044,"num_write":640,"num_write_kb":10409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":10719266,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":368,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058198+0000","last_change":"2024-05-11T23:01:19.219128+0000","last_active":"2024-05-11T23:04:42.058198+0000","last_peered":"2024-05-11T23:04:42.058198+0000","last_clean":"2024-05-11T23:04:42.058198+0000","last_became_active":"2024-05-11T23:01:19.218846+0000","last_became_peered":"2024-05-11T23:01:19.218846+0000","last_unstale":"2024-05-11T23:04:42.058198+0000","last_undegraded":"2024-05-11T23:04:42.058198+0000","last_fullsized":"2024-05-11T23:04:42.058198+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:41:09.111000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":544,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058200+0000","last_change":"2024-05-11T23:04:28.806566+0000","last_active":"2024-05-11T23:04:42.058200+0000","last_peered":"2024-05-11T23:04:42.058200+0000","last_clean":"2024-05-11T23:04:42.058200+0000","last_became_active":"2024-05-11T23:04:28.806124+0000","last_became_peered":"2024-05-11T23:04:28.806124+0000","last_unstale":"2024-05-11T23:04:42.058200+0000","last_undegraded":"2024-05-11T23:04:42.058200+0000","last_fullsized":"2024-05-11T23:04:42.058200+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_clean_scrub_stamp":"2024-05-11T23:01:16.147564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:03:40.590362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"180'966","reported_seq":5554,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.056462+0000","last_change":"2024-05-11T23:04:25.885127+0000","last_active":"2024-05-11T23:04:42.056462+0000","last_peered":"2024-05-11T23:04:42.056462+0000","last_clean":"2024-05-11T23:04:42.056462+0000","last_became_active":"2024-05-11T23:04:25.802390+0000","last_became_peered":"2024-05-11T23:04:25.802390+0000","last_unstale":"2024-05-11T23:04:42.056462+0000","last_undegraded":"2024-05-11T23:04:42.056462+0000","last_fullsized":"2024-05-11T23:04:42.056462+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":966,"log_dups_size":0,"ondisk_log_size":966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:45:25.229272+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.057229434000000003,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":6339,"num_read_kb":20167,"num_write":1151,"num_write_kb":10790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":4220183,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"172'676","reported_seq":9169,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118976+0000","last_change":"2024-05-11T23:04:34.008013+0000","last_active":"2024-05-11T23:04:42.118976+0000","last_peered":"2024-05-11T23:04:42.118976+0000","last_clean":"2024-05-11T23:04:42.118976+0000","last_became_active":"2024-05-11T23:04:34.005569+0000","last_became_peered":"2024-05-11T23:04:34.005569+0000","last_unstale":"2024-05-11T23:04:42.118976+0000","last_undegraded":"2024-05-11T23:04:42.118976+0000","last_fullsized":"2024-05-11T23:04:42.118976+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:27:04.828149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064185299999999995,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":9043,"num_read_kb":47546,"num_write":541,"num_write_kb":29805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":37320801,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"175'785","reported_seq":6658,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118728+0000","last_change":"2024-05-11T23:04:17.987769+0000","last_active":"2024-05-11T23:04:42.118728+0000","last_peered":"2024-05-11T23:04:42.118728+0000","last_clean":"2024-05-11T23:04:42.118728+0000","last_became_active":"2024-05-11T23:03:51.797214+0000","last_became_peered":"2024-05-11T23:03:51.797214+0000","last_unstale":"2024-05-11T23:04:42.118728+0000","last_undegraded":"2024-05-11T23:04:42.118728+0000","last_fullsized":"2024-05-11T23:04:42.118728+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:04:04.928406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001270001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":44,"num_read":5814,"num_read_kb":22550,"num_write":685,"num_write_kb":9625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":538,"num_bytes_recovered":16997094,"num_keys_recovered":140,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"173'657","reported_seq":9945,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.076988+0000","last_change":"2024-05-11T23:04:42.076988+0000","last_active":"2024-05-11T23:04:42.076988+0000","last_peered":"2024-05-11T23:04:42.076988+0000","last_clean":"2024-05-11T23:04:42.076988+0000","last_became_active":"2024-05-11T23:04:42.073145+0000","last_became_peered":"2024-05-11T23:04:42.073145+0000","last_unstale":"2024-05-11T23:04:42.076988+0000","last_undegraded":"2024-05-11T23:04:42.076988+0000","last_fullsized":"2024-05-11T23:04:42.076988+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:50:27.744009+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00088000000000000003,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":9829,"num_read_kb":52232,"num_write":599,"num_write_kb":42645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":47434023,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"180'694","reported_seq":5216,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058529+0000","last_change":"2024-05-11T23:04:26.020729+0000","last_active":"2024-05-11T23:04:42.058529+0000","last_peered":"2024-05-11T23:04:42.058529+0000","last_clean":"2024-05-11T23:04:42.058529+0000","last_became_active":"2024-05-11T23:04:25.806852+0000","last_became_peered":"2024-05-11T23:04:25.806852+0000","last_unstale":"2024-05-11T23:04:42.058529+0000","last_undegraded":"2024-05-11T23:04:42.058529+0000","last_fullsized":"2024-05-11T23:04:42.058529+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:12:25.591743+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.060182605,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":6324,"num_read_kb":25218,"num_write":659,"num_write_kb":11709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":10615766,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"174'897","reported_seq":21502,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058455+0000","last_change":"2024-05-11T23:04:18.014080+0000","last_active":"2024-05-11T23:04:42.058455+0000","last_peered":"2024-05-11T23:04:42.058455+0000","last_clean":"2024-05-11T23:04:42.058455+0000","last_became_active":"2024-05-11T23:01:27.465570+0000","last_became_peered":"2024-05-11T23:01:27.465570+0000","last_unstale":"2024-05-11T23:04:42.058455+0000","last_undegraded":"2024-05-11T23:04:42.058455+0000","last_fullsized":"2024-05-11T23:04:42.058455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:14:28.607937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003620968,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":22252,"num_read_kb":40862,"num_write":875,"num_write_kb":12354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"175'1308","reported_seq":14450,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058370+0000","last_change":"2024-05-11T23:04:18.014428+0000","last_active":"2024-05-11T23:04:42.058370+0000","last_peered":"2024-05-11T23:04:42.058370+0000","last_clean":"2024-05-11T23:04:42.058370+0000","last_became_active":"2024-05-11T23:03:52.804360+0000","last_became_peered":"2024-05-11T23:03:52.804360+0000","last_unstale":"2024-05-11T23:04:42.058370+0000","last_undegraded":"2024-05-11T23:04:42.058370+0000","last_fullsized":"2024-05-11T23:04:42.058370+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:09:37.091464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039667269999999998,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":17126,"num_read_kb":45672,"num_write":1555,"num_write_kb":23766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":18760061,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"168'1114","reported_seq":9526,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058558+0000","last_change":"2024-05-11T23:04:18.014829+0000","last_active":"2024-05-11T23:04:42.058558+0000","last_peered":"2024-05-11T23:04:42.058558+0000","last_clean":"2024-05-11T23:04:42.058558+0000","last_became_active":"2024-05-11T23:03:49.895043+0000","last_became_peered":"2024-05-11T23:03:49.895043+0000","last_unstale":"2024-05-11T23:04:42.058558+0000","last_undegraded":"2024-05-11T23:04:42.058558+0000","last_fullsized":"2024-05-11T23:04:42.058558+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:55:15.798774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040936139999999998,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":11168,"num_read_kb":37882,"num_write":1390,"num_write_kb":15155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":362,"num_bytes_recovered":15895238,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"174'678","reported_seq":7038,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118720+0000","last_change":"2024-05-11T23:04:28.807759+0000","last_active":"2024-05-11T23:04:42.118720+0000","last_peered":"2024-05-11T23:04:42.118720+0000","last_clean":"2024-05-11T23:04:42.118720+0000","last_became_active":"2024-05-11T23:04:28.806050+0000","last_became_peered":"2024-05-11T23:04:28.806050+0000","last_unstale":"2024-05-11T23:04:42.118720+0000","last_undegraded":"2024-05-11T23:04:42.118720+0000","last_fullsized":"2024-05-11T23:04:42.118720+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:58:42.310196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023348000000000002,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":7071,"num_read_kb":24639,"num_write":633,"num_write_kb":13630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":19437463,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"173'853","reported_seq":6348,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058241+0000","last_change":"2024-05-11T23:04:18.011274+0000","last_active":"2024-05-11T23:04:42.058241+0000","last_peered":"2024-05-11T23:04:42.058241+0000","last_clean":"2024-05-11T23:04:42.058241+0000","last_became_active":"2024-05-11T23:03:53.806713+0000","last_became_peered":"2024-05-11T23:03:53.806713+0000","last_unstale":"2024-05-11T23:04:42.058241+0000","last_undegraded":"2024-05-11T23:04:42.058241+0000","last_fullsized":"2024-05-11T23:04:42.058241+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":853,"log_dups_size":0,"ondisk_log_size":853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:25:53.472036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011357120000000001,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":7833,"num_read_kb":21923,"num_write":862,"num_write_kb":7573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":450,"num_bytes_recovered":18021482,"num_keys_recovered":78,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"175'975","reported_seq":5304,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044876+0000","last_change":"2024-05-11T23:04:27.810986+0000","last_active":"2024-05-11T23:04:41.044876+0000","last_peered":"2024-05-11T23:04:41.044876+0000","last_clean":"2024-05-11T23:04:41.044876+0000","last_became_active":"2024-05-11T23:04:27.809030+0000","last_became_peered":"2024-05-11T23:04:27.809030+0000","last_unstale":"2024-05-11T23:04:41.044876+0000","last_undegraded":"2024-05-11T23:04:41.044876+0000","last_fullsized":"2024-05-11T23:04:41.044876+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:20:47.733184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028179780000000001,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":6275,"num_read_kb":24022,"num_write":1130,"num_write_kb":13297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"173'599","reported_seq":5607,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058311+0000","last_change":"2024-05-11T23:04:18.013459+0000","last_active":"2024-05-11T23:04:42.058311+0000","last_peered":"2024-05-11T23:04:42.058311+0000","last_clean":"2024-05-11T23:04:42.058311+0000","last_became_active":"2024-05-11T23:03:52.818374+0000","last_became_peered":"2024-05-11T23:03:52.818374+0000","last_unstale":"2024-05-11T23:04:42.058311+0000","last_undegraded":"2024-05-11T23:04:42.058311+0000","last_fullsized":"2024-05-11T23:04:42.058311+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:21:58.570294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031987780000000002,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":5050,"num_read_kb":18194,"num_write":554,"num_write_kb":7419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4575002,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1465,"num_object_clones":0,"num_object_copies":2930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1465,"num_whiteouts":1461,"num_read":339431,"num_read_kb":1023367,"num_write":28503,"num_write_kb":498784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8670,"num_bytes_recovered":382481677,"num_keys_recovered":951,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16167,"internal_metadata":0},"log_size":26567,"ondisk_log_size":26567,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100800,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509484,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69644,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302196,"statfs":{"total":96636764160,"available":96565448704,"internally_reserved":0,"allocated":2424832,"data_stored":1737431,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59856,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311984,"statfs":{"total":96636764160,"available":96575471616,"internally_reserved":0,"allocated":2359296,"data_stored":1147063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509485,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65428,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306412,"statfs":{"total":96636764160,"available":96569765888,"internally_reserved":0,"allocated":2363392,"data_stored":1147082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33472,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338368,"statfs":{"total":96636764160,"available":96602488832,"internally_reserved":0,"allocated":2347008,"data_stored":1141250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":179,"seq":768799145990,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36048,"kb_used_data":2304,"kb_used_omap":49,"kb_used_meta":33678,"kb_avail":94335792,"statfs":{"total":96636764160,"available":96599851008,"internally_reserved":0,"allocated":2359296,"data_stored":1147063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51091,"internal_metadata":34486381},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73732,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298108,"statfs":{"total":96636764160,"available":96561262592,"internally_reserved":0,"allocated":2351104,"data_stored":1141269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37632,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334208,"statfs":{"total":96636764160,"available":96598228992,"internally_reserved":0,"allocated":2347008,"data_stored":1141250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77936,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293904,"statfs":{"total":96636764160,"available":96556957696,"internally_reserved":0,"allocated":2396160,"data_stored":1731618,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16167,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T23:04:46.004 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-11T23:04:46.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T23:04:46.307 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:04:46.308 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-11T23:04:46.319 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-05-11T23:04:45.174196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":1467,"num_object_clones":0,"num_object_copies":2934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1467,"num_whiteouts":1461,"num_read":339547,"num_read_kb":1023467,"num_write":28662,"num_write_kb":501200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8674,"num_bytes_recovered":383662413,"num_keys_recovered":951,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26684,"ondisk_log_size":26684,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":453748,"kb_used_data":18504,"kb_used_omap":60,"kb_used_meta":435075,"kb_avail":754520972,"statfs":{"total":773094113280,"available":772629475328,"internally_reserved":0,"allocated":18948096,"data_stored":10334026,"data_compressed":66655,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":62250,"internal_metadata":445517014},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000923"},"pg_stats":[{"pgid":"3.1f","version":"175'540","reported_seq":5571,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044941+0000","last_change":"2024-05-11T23:04:28.806612+0000","last_active":"2024-05-11T23:04:41.044941+0000","last_peered":"2024-05-11T23:04:41.044941+0000","last_clean":"2024-05-11T23:04:41.044941+0000","last_became_active":"2024-05-11T23:04:28.804255+0000","last_became_peered":"2024-05-11T23:04:28.804255+0000","last_unstale":"2024-05-11T23:04:41.044941+0000","last_undegraded":"2024-05-11T23:04:41.044941+0000","last_fullsized":"2024-05-11T23:04:41.044941+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:57:34.663961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098957899999999998,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":5393,"num_read_kb":14723,"num_write":488,"num_write_kb":18081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":877932,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"180'934","reported_seq":6543,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.057168+0000","last_change":"2024-05-11T23:04:25.935404+0000","last_active":"2024-05-11T23:04:42.057168+0000","last_peered":"2024-05-11T23:04:42.057168+0000","last_clean":"2024-05-11T23:04:42.057168+0000","last_became_active":"2024-05-11T23:04:25.808413+0000","last_became_peered":"2024-05-11T23:04:25.808413+0000","last_unstale":"2024-05-11T23:04:42.057168+0000","last_undegraded":"2024-05-11T23:04:42.057168+0000","last_fullsized":"2024-05-11T23:04:42.057168+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":934,"log_dups_size":0,"ondisk_log_size":934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:09:17.133440+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.084167985000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":6916,"num_read_kb":49448,"num_write":971,"num_write_kb":22687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":53290749,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"167'508","reported_seq":4530,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058246+0000","last_change":"2024-05-11T23:04:17.950967+0000","last_active":"2024-05-11T23:04:42.058246+0000","last_peered":"2024-05-11T23:04:42.058246+0000","last_clean":"2024-05-11T23:04:42.058246+0000","last_became_active":"2024-05-11T23:03:48.889806+0000","last_became_peered":"2024-05-11T23:03:48.889806+0000","last_unstale":"2024-05-11T23:04:42.058246+0000","last_undegraded":"2024-05-11T23:04:42.058246+0000","last_fullsized":"2024-05-11T23:04:42.058246+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:46:37.757197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060389999999999999,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":3180,"num_read_kb":16428,"num_write":406,"num_write_kb":9629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":7067368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"175'784","reported_seq":19160,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058324+0000","last_change":"2024-05-11T23:04:17.951789+0000","last_active":"2024-05-11T23:04:42.058324+0000","last_peered":"2024-05-11T23:04:42.058324+0000","last_clean":"2024-05-11T23:04:42.058324+0000","last_became_active":"2024-05-11T23:03:50.798632+0000","last_became_peered":"2024-05-11T23:03:50.798632+0000","last_unstale":"2024-05-11T23:04:42.058324+0000","last_undegraded":"2024-05-11T23:04:42.058324+0000","last_fullsized":"2024-05-11T23:04:42.058324+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:13:59.992713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248566,"stat_sum":{"num_bytes":0,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":63,"num_read":18429,"num_read_kb":39778,"num_write":674,"num_write_kb":10510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":9396782,"num_keys_recovered":182,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"174'594","reported_seq":15111,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045102+0000","last_change":"2024-05-11T23:04:29.807594+0000","last_active":"2024-05-11T23:04:41.045102+0000","last_peered":"2024-05-11T23:04:41.045102+0000","last_clean":"2024-05-11T23:04:41.045102+0000","last_became_active":"2024-05-11T23:04:29.804143+0000","last_became_peered":"2024-05-11T23:04:29.804143+0000","last_unstale":"2024-05-11T23:04:41.045102+0000","last_undegraded":"2024-05-11T23:04:41.045102+0000","last_fullsized":"2024-05-11T23:04:41.045102+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:25:20.570143+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001388731,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":13979,"num_read_kb":68609,"num_write":598,"num_write_kb":35793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":532,"num_bytes_recovered":20898869,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"173'1664","reported_seq":21045,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118917+0000","last_change":"2024-05-11T23:04:34.010543+0000","last_active":"2024-05-11T23:04:42.118917+0000","last_peered":"2024-05-11T23:04:42.118917+0000","last_clean":"2024-05-11T23:04:42.118917+0000","last_became_active":"2024-05-11T23:04:34.007358+0000","last_became_peered":"2024-05-11T23:04:34.007358+0000","last_unstale":"2024-05-11T23:04:42.118917+0000","last_undegraded":"2024-05-11T23:04:42.118917+0000","last_fullsized":"2024-05-11T23:04:42.118917+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1664,"log_dups_size":0,"ondisk_log_size":1664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:31:26.529561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097887899999999999,"stat_sum":{"num_bytes":19,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":25336,"num_read_kb":48131,"num_write":2723,"num_write_kb":23444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":6434587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"180'1052","reported_seq":15385,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.042982+0000","last_change":"2024-05-11T23:04:25.959047+0000","last_active":"2024-05-11T23:04:41.042982+0000","last_peered":"2024-05-11T23:04:41.042982+0000","last_clean":"2024-05-11T23:04:41.042982+0000","last_became_active":"2024-05-11T23:04:25.807274+0000","last_became_peered":"2024-05-11T23:04:25.807274+0000","last_unstale":"2024-05-11T23:04:41.042982+0000","last_undegraded":"2024-05-11T23:04:41.042982+0000","last_fullsized":"2024-05-11T23:04:41.042982+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1052,"log_dups_size":0,"ondisk_log_size":1052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:17:04.610363+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.061503582000000001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":16282,"num_read_kb":43182,"num_write":1369,"num_write_kb":14705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":13589947,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"170'967","reported_seq":3982,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.056362+0000","last_change":"2024-05-11T23:04:26.853599+0000","last_active":"2024-05-11T23:04:42.056362+0000","last_peered":"2024-05-11T23:04:42.056362+0000","last_clean":"2024-05-11T23:04:42.056362+0000","last_became_active":"2024-05-11T23:04:26.807825+0000","last_became_peered":"2024-05-11T23:04:26.807825+0000","last_unstale":"2024-05-11T23:04:42.056362+0000","last_undegraded":"2024-05-11T23:04:42.056362+0000","last_fullsized":"2024-05-11T23:04:42.056362+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:26:49.247146+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001621796,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":5754,"num_read_kb":9891,"num_write":1117,"num_write_kb":16486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":7487386,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"172'667","reported_seq":3869,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.045052+0000","last_change":"2024-05-11T23:04:27.810906+0000","last_active":"2024-05-11T23:04:41.045052+0000","last_peered":"2024-05-11T23:04:41.045052+0000","last_clean":"2024-05-11T23:04:41.045052+0000","last_became_active":"2024-05-11T23:04:27.809321+0000","last_became_peered":"2024-05-11T23:04:27.809321+0000","last_unstale":"2024-05-11T23:04:41.045052+0000","last_undegraded":"2024-05-11T23:04:41.045052+0000","last_fullsized":"2024-05-11T23:04:41.045052+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:32:33.820874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013760389999999999,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":3972,"num_read_kb":15246,"num_write":808,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"172'559","reported_seq":15690,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058382+0000","last_change":"2024-05-11T23:04:17.950903+0000","last_active":"2024-05-11T23:04:42.058382+0000","last_peered":"2024-05-11T23:04:42.058382+0000","last_clean":"2024-05-11T23:04:42.058382+0000","last_became_active":"2024-05-11T23:01:27.463623+0000","last_became_peered":"2024-05-11T23:01:27.463623+0000","last_unstale":"2024-05-11T23:04:42.058382+0000","last_undegraded":"2024-05-11T23:04:42.058382+0000","last_fullsized":"2024-05-11T23:04:42.058382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:56:10.783772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054116699999999995,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":15017,"num_read_kb":54784,"num_write":481,"num_write_kb":18602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"180'622","reported_seq":16413,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058371+0000","last_change":"2024-05-11T23:04:26.015543+0000","last_active":"2024-05-11T23:04:42.058371+0000","last_peered":"2024-05-11T23:04:42.058371+0000","last_clean":"2024-05-11T23:04:42.058371+0000","last_became_active":"2024-05-11T23:04:25.807935+0000","last_became_peered":"2024-05-11T23:04:25.807935+0000","last_unstale":"2024-05-11T23:04:42.058371+0000","last_undegraded":"2024-05-11T23:04:42.058371+0000","last_fullsized":"2024-05-11T23:04:42.058371+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:50:30.593291+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.072194399000000006,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":14264,"num_read_kb":25298,"num_write":546,"num_write_kb":13154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":12363304,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"172'661","reported_seq":19584,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.119099+0000","last_change":"2024-05-11T23:04:34.010412+0000","last_active":"2024-05-11T23:04:42.119099+0000","last_peered":"2024-05-11T23:04:42.119099+0000","last_clean":"2024-05-11T23:04:42.119099+0000","last_became_active":"2024-05-11T23:04:34.007861+0000","last_became_peered":"2024-05-11T23:04:34.007861+0000","last_unstale":"2024-05-11T23:04:42.119099+0000","last_undegraded":"2024-05-11T23:04:42.119099+0000","last_fullsized":"2024-05-11T23:04:42.119099+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:13:07.675246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001389319,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":19402,"num_read_kb":42118,"num_write":569,"num_write_kb":6424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":4215359,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118756+0000","last_change":"2024-05-11T23:02:39.044686+0000","last_active":"2024-05-11T23:04:42.118756+0000","last_peered":"2024-05-11T23:04:42.118756+0000","last_clean":"2024-05-11T23:04:42.118756+0000","last_became_active":"2024-05-11T23:02:39.044216+0000","last_became_peered":"2024-05-11T23:02:39.044216+0000","last_unstale":"2024-05-11T23:04:42.118756+0000","last_undegraded":"2024-05-11T23:04:42.118756+0000","last_fullsized":"2024-05-11T23:04:42.118756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:31:03.826529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"175'1049","reported_seq":7444,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044838+0000","last_change":"2024-05-11T23:04:28.808149+0000","last_active":"2024-05-11T23:04:41.044838+0000","last_peered":"2024-05-11T23:04:41.044838+0000","last_clean":"2024-05-11T23:04:41.044838+0000","last_became_active":"2024-05-11T23:04:28.805428+0000","last_became_peered":"2024-05-11T23:04:28.805428+0000","last_unstale":"2024-05-11T23:04:41.044838+0000","last_undegraded":"2024-05-11T23:04:41.044838+0000","last_fullsized":"2024-05-11T23:04:41.044838+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:36:30.546501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002036403,"stat_sum":{"num_bytes":0,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":60,"num_read":9703,"num_read_kb":23103,"num_write":1155,"num_write_kb":12619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":2180842,"num_keys_recovered":21,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":335,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.043163+0000","last_change":"2024-05-11T23:04:29.806455+0000","last_active":"2024-05-11T23:04:41.043163+0000","last_peered":"2024-05-11T23:04:41.043163+0000","last_clean":"2024-05-11T23:04:41.043163+0000","last_became_active":"2024-05-11T23:04:29.805946+0000","last_became_peered":"2024-05-11T23:04:29.805946+0000","last_unstale":"2024-05-11T23:04:41.043163+0000","last_undegraded":"2024-05-11T23:04:41.043163+0000","last_fullsized":"2024-05-11T23:04:41.043163+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:09:54.127025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"180'926","reported_seq":6079,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.063271+0000","last_change":"2024-05-11T23:04:42.063271+0000","last_active":"2024-05-11T23:04:42.063271+0000","last_peered":"2024-05-11T23:04:42.063271+0000","last_clean":"2024-05-11T23:04:42.063271+0000","last_became_active":"2024-05-11T23:04:42.060396+0000","last_became_peered":"2024-05-11T23:04:42.060396+0000","last_unstale":"2024-05-11T23:04:42.063271+0000","last_undegraded":"2024-05-11T23:04:42.063271+0000","last_fullsized":"2024-05-11T23:04:42.063271+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:02:26.606646+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.055140898000000001,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":7310,"num_read_kb":19251,"num_write":1088,"num_write_kb":7185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":4415701,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":376,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058504+0000","last_change":"2024-05-11T23:04:25.806665+0000","last_active":"2024-05-11T23:04:42.058504+0000","last_peered":"2024-05-11T23:04:42.058504+0000","last_clean":"2024-05-11T23:04:42.058504+0000","last_became_active":"2024-05-11T23:04:25.806292+0000","last_became_peered":"2024-05-11T23:04:25.806292+0000","last_unstale":"2024-05-11T23:04:42.058504+0000","last_undegraded":"2024-05-11T23:04:42.058504+0000","last_fullsized":"2024-05-11T23:04:42.058504+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:12:45.144479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"175'564","reported_seq":14906,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.043149+0000","last_change":"2024-05-11T23:04:29.809156+0000","last_active":"2024-05-11T23:04:41.043149+0000","last_peered":"2024-05-11T23:04:41.043149+0000","last_clean":"2024-05-11T23:04:41.043149+0000","last_became_active":"2024-05-11T23:04:29.807343+0000","last_became_peered":"2024-05-11T23:04:29.807343+0000","last_unstale":"2024-05-11T23:04:41.043149+0000","last_undegraded":"2024-05-11T23:04:41.043149+0000","last_fullsized":"2024-05-11T23:04:41.043149+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:35:56.026223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095379299999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":14156,"num_read_kb":53683,"num_write":532,"num_write_kb":33731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":339,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.043292+0000","last_change":"2024-05-11T23:04:29.805774+0000","last_active":"2024-05-11T23:04:41.043292+0000","last_peered":"2024-05-11T23:04:41.043292+0000","last_clean":"2024-05-11T23:04:41.043292+0000","last_became_active":"2024-05-11T23:04:29.805447+0000","last_became_peered":"2024-05-11T23:04:29.805447+0000","last_unstale":"2024-05-11T23:04:41.043292+0000","last_undegraded":"2024-05-11T23:04:41.043292+0000","last_fullsized":"2024-05-11T23:04:41.043292+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:56:59.210181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"180'758","reported_seq":6312,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058458+0000","last_change":"2024-05-11T23:04:25.999769+0000","last_active":"2024-05-11T23:04:42.058458+0000","last_peered":"2024-05-11T23:04:42.058458+0000","last_clean":"2024-05-11T23:04:42.058458+0000","last_became_active":"2024-05-11T23:04:25.806236+0000","last_became_peered":"2024-05-11T23:04:25.806236+0000","last_unstale":"2024-05-11T23:04:42.058458+0000","last_undegraded":"2024-05-11T23:04:42.058458+0000","last_fullsized":"2024-05-11T23:04:42.058458+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:19:42.475478+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.081084693999999999,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":6715,"num_read_kb":25684,"num_write":652,"num_write_kb":13121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":13712581,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"175'1022","reported_seq":11247,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.119002+0000","last_change":"2024-05-11T23:04:34.011637+0000","last_active":"2024-05-11T23:04:42.119002+0000","last_peered":"2024-05-11T23:04:42.119002+0000","last_clean":"2024-05-11T23:04:42.119002+0000","last_became_active":"2024-05-11T23:04:34.008456+0000","last_became_peered":"2024-05-11T23:04:34.008456+0000","last_unstale":"2024-05-11T23:04:42.119002+0000","last_undegraded":"2024-05-11T23:04:42.119002+0000","last_fullsized":"2024-05-11T23:04:42.119002+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1022,"log_dups_size":0,"ondisk_log_size":1022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:56:16.226740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017329069999999999,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":56,"num_read":10698,"num_read_kb":33938,"num_write":973,"num_write_kb":17254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":898,"num_bytes_recovered":16208786,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":368,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058873+0000","last_change":"2024-05-11T23:01:19.221118+0000","last_active":"2024-05-11T23:04:42.058873+0000","last_peered":"2024-05-11T23:04:42.058873+0000","last_clean":"2024-05-11T23:04:42.058873+0000","last_became_active":"2024-05-11T23:01:19.220855+0000","last_became_peered":"2024-05-11T23:01:19.220855+0000","last_unstale":"2024-05-11T23:04:42.058873+0000","last_undegraded":"2024-05-11T23:04:42.058873+0000","last_fullsized":"2024-05-11T23:04:42.058873+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:01:56.076963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"173'1148","reported_seq":6343,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.042746+0000","last_change":"2024-05-11T23:04:17.940887+0000","last_active":"2024-05-11T23:04:41.042746+0000","last_peered":"2024-05-11T23:04:41.042746+0000","last_clean":"2024-05-11T23:04:41.042746+0000","last_became_active":"2024-05-11T23:01:27.459593+0000","last_became_peered":"2024-05-11T23:01:27.459593+0000","last_unstale":"2024-05-11T23:04:41.042746+0000","last_undegraded":"2024-05-11T23:04:41.042746+0000","last_fullsized":"2024-05-11T23:04:41.042746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1148,"log_dups_size":0,"ondisk_log_size":1148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:45.922442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017756580000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9995,"num_read_kb":18445,"num_write":1457,"num_write_kb":4408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":384,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.041751+0000","last_change":"2024-05-11T23:04:25.805119+0000","last_active":"2024-05-11T23:04:41.041751+0000","last_peered":"2024-05-11T23:04:41.041751+0000","last_clean":"2024-05-11T23:04:41.041751+0000","last_became_active":"2024-05-11T23:04:25.804674+0000","last_became_peered":"2024-05-11T23:04:25.804674+0000","last_unstale":"2024-05-11T23:04:41.041751+0000","last_undegraded":"2024-05-11T23:04:41.041751+0000","last_fullsized":"2024-05-11T23:04:41.041751+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:37:37.191290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"175'656","reported_seq":15312,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044755+0000","last_change":"2024-05-11T23:04:26.807059+0000","last_active":"2024-05-11T23:04:41.044755+0000","last_peered":"2024-05-11T23:04:41.044755+0000","last_clean":"2024-05-11T23:04:41.044755+0000","last_became_active":"2024-05-11T23:04:26.804248+0000","last_became_peered":"2024-05-11T23:04:26.804248+0000","last_unstale":"2024-05-11T23:04:41.044755+0000","last_undegraded":"2024-05-11T23:04:41.044755+0000","last_fullsized":"2024-05-11T23:04:41.044755+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:32:14.359230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014815119999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":15309,"num_read_kb":24676,"num_write":622,"num_write_kb":6352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":299,"num_bytes_recovered":6345105,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":366,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058188+0000","last_change":"2024-05-11T23:01:19.219755+0000","last_active":"2024-05-11T23:04:42.058188+0000","last_peered":"2024-05-11T23:04:42.058188+0000","last_clean":"2024-05-11T23:04:42.058188+0000","last_became_active":"2024-05-11T23:01:19.219471+0000","last_became_peered":"2024-05-11T23:01:19.219471+0000","last_unstale":"2024-05-11T23:04:42.058188+0000","last_undegraded":"2024-05-11T23:04:42.058188+0000","last_fullsized":"2024-05-11T23:04:42.058188+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:39:05.284213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"180'690","reported_seq":4302,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.064001+0000","last_change":"2024-05-11T23:04:42.064001+0000","last_active":"2024-05-11T23:04:42.064001+0000","last_peered":"2024-05-11T23:04:42.064001+0000","last_clean":"2024-05-11T23:04:42.064001+0000","last_became_active":"2024-05-11T23:04:42.060595+0000","last_became_peered":"2024-05-11T23:04:42.060595+0000","last_unstale":"2024-05-11T23:04:42.064001+0000","last_undegraded":"2024-05-11T23:04:42.064001+0000","last_fullsized":"2024-05-11T23:04:42.064001+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:38:46.487523+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.040069711000000001,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":3497,"num_read_kb":16044,"num_write":640,"num_write_kb":10409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":10719266,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":368,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058198+0000","last_change":"2024-05-11T23:01:19.219128+0000","last_active":"2024-05-11T23:04:42.058198+0000","last_peered":"2024-05-11T23:04:42.058198+0000","last_clean":"2024-05-11T23:04:42.058198+0000","last_became_active":"2024-05-11T23:01:19.218846+0000","last_became_peered":"2024-05-11T23:01:19.218846+0000","last_unstale":"2024-05-11T23:04:42.058198+0000","last_undegraded":"2024-05-11T23:04:42.058198+0000","last_fullsized":"2024-05-11T23:04:42.058198+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:41:09.111000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":544,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058200+0000","last_change":"2024-05-11T23:04:28.806566+0000","last_active":"2024-05-11T23:04:42.058200+0000","last_peered":"2024-05-11T23:04:42.058200+0000","last_clean":"2024-05-11T23:04:42.058200+0000","last_became_active":"2024-05-11T23:04:28.806124+0000","last_became_peered":"2024-05-11T23:04:28.806124+0000","last_unstale":"2024-05-11T23:04:42.058200+0000","last_undegraded":"2024-05-11T23:04:42.058200+0000","last_fullsized":"2024-05-11T23:04:42.058200+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_clean_scrub_stamp":"2024-05-11T23:01:16.147564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:03:40.590362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"180'966","reported_seq":5554,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.056462+0000","last_change":"2024-05-11T23:04:25.885127+0000","last_active":"2024-05-11T23:04:42.056462+0000","last_peered":"2024-05-11T23:04:42.056462+0000","last_clean":"2024-05-11T23:04:42.056462+0000","last_became_active":"2024-05-11T23:04:25.802390+0000","last_became_peered":"2024-05-11T23:04:25.802390+0000","last_unstale":"2024-05-11T23:04:42.056462+0000","last_undegraded":"2024-05-11T23:04:42.056462+0000","last_fullsized":"2024-05-11T23:04:42.056462+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":966,"log_dups_size":0,"ondisk_log_size":966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:45:25.229272+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.057229434000000003,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":6339,"num_read_kb":20167,"num_write":1151,"num_write_kb":10790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":4220183,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"172'676","reported_seq":9169,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118976+0000","last_change":"2024-05-11T23:04:34.008013+0000","last_active":"2024-05-11T23:04:42.118976+0000","last_peered":"2024-05-11T23:04:42.118976+0000","last_clean":"2024-05-11T23:04:42.118976+0000","last_became_active":"2024-05-11T23:04:34.005569+0000","last_became_peered":"2024-05-11T23:04:34.005569+0000","last_unstale":"2024-05-11T23:04:42.118976+0000","last_undegraded":"2024-05-11T23:04:42.118976+0000","last_fullsized":"2024-05-11T23:04:42.118976+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:27:04.828149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064185299999999995,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":9043,"num_read_kb":47546,"num_write":541,"num_write_kb":29805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":37320801,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"175'785","reported_seq":6658,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118728+0000","last_change":"2024-05-11T23:04:17.987769+0000","last_active":"2024-05-11T23:04:42.118728+0000","last_peered":"2024-05-11T23:04:42.118728+0000","last_clean":"2024-05-11T23:04:42.118728+0000","last_became_active":"2024-05-11T23:03:51.797214+0000","last_became_peered":"2024-05-11T23:03:51.797214+0000","last_unstale":"2024-05-11T23:04:42.118728+0000","last_undegraded":"2024-05-11T23:04:42.118728+0000","last_fullsized":"2024-05-11T23:04:42.118728+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:04:04.928406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001270001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":44,"num_read":5814,"num_read_kb":22550,"num_write":685,"num_write_kb":9625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":538,"num_bytes_recovered":16997094,"num_keys_recovered":140,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"173'657","reported_seq":9945,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.076988+0000","last_change":"2024-05-11T23:04:42.076988+0000","last_active":"2024-05-11T23:04:42.076988+0000","last_peered":"2024-05-11T23:04:42.076988+0000","last_clean":"2024-05-11T23:04:42.076988+0000","last_became_active":"2024-05-11T23:04:42.073145+0000","last_became_peered":"2024-05-11T23:04:42.073145+0000","last_unstale":"2024-05-11T23:04:42.076988+0000","last_undegraded":"2024-05-11T23:04:42.076988+0000","last_fullsized":"2024-05-11T23:04:42.076988+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:50:27.744009+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00088000000000000003,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":9829,"num_read_kb":52232,"num_write":599,"num_write_kb":42645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":47434023,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"180'694","reported_seq":5216,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058529+0000","last_change":"2024-05-11T23:04:26.020729+0000","last_active":"2024-05-11T23:04:42.058529+0000","last_peered":"2024-05-11T23:04:42.058529+0000","last_clean":"2024-05-11T23:04:42.058529+0000","last_became_active":"2024-05-11T23:04:25.806852+0000","last_became_peered":"2024-05-11T23:04:25.806852+0000","last_unstale":"2024-05-11T23:04:42.058529+0000","last_undegraded":"2024-05-11T23:04:42.058529+0000","last_fullsized":"2024-05-11T23:04:42.058529+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:12:25.591743+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.060182605,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":6324,"num_read_kb":25218,"num_write":659,"num_write_kb":11709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":10615766,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"174'897","reported_seq":21502,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058455+0000","last_change":"2024-05-11T23:04:18.014080+0000","last_active":"2024-05-11T23:04:42.058455+0000","last_peered":"2024-05-11T23:04:42.058455+0000","last_clean":"2024-05-11T23:04:42.058455+0000","last_became_active":"2024-05-11T23:01:27.465570+0000","last_became_peered":"2024-05-11T23:01:27.465570+0000","last_unstale":"2024-05-11T23:04:42.058455+0000","last_undegraded":"2024-05-11T23:04:42.058455+0000","last_fullsized":"2024-05-11T23:04:42.058455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:14:28.607937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003620968,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":22252,"num_read_kb":40862,"num_write":875,"num_write_kb":12354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"175'1308","reported_seq":14450,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058370+0000","last_change":"2024-05-11T23:04:18.014428+0000","last_active":"2024-05-11T23:04:42.058370+0000","last_peered":"2024-05-11T23:04:42.058370+0000","last_clean":"2024-05-11T23:04:42.058370+0000","last_became_active":"2024-05-11T23:03:52.804360+0000","last_became_peered":"2024-05-11T23:03:52.804360+0000","last_unstale":"2024-05-11T23:04:42.058370+0000","last_undegraded":"2024-05-11T23:04:42.058370+0000","last_fullsized":"2024-05-11T23:04:42.058370+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:09:37.091464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039667269999999998,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":17126,"num_read_kb":45672,"num_write":1555,"num_write_kb":23766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":18760061,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"168'1114","reported_seq":9526,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058558+0000","last_change":"2024-05-11T23:04:18.014829+0000","last_active":"2024-05-11T23:04:42.058558+0000","last_peered":"2024-05-11T23:04:42.058558+0000","last_clean":"2024-05-11T23:04:42.058558+0000","last_became_active":"2024-05-11T23:03:49.895043+0000","last_became_peered":"2024-05-11T23:03:49.895043+0000","last_unstale":"2024-05-11T23:04:42.058558+0000","last_undegraded":"2024-05-11T23:04:42.058558+0000","last_fullsized":"2024-05-11T23:04:42.058558+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:55:15.798774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040936139999999998,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":11168,"num_read_kb":37882,"num_write":1390,"num_write_kb":15155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":362,"num_bytes_recovered":15895238,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"174'678","reported_seq":7038,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118720+0000","last_change":"2024-05-11T23:04:28.807759+0000","last_active":"2024-05-11T23:04:42.118720+0000","last_peered":"2024-05-11T23:04:42.118720+0000","last_clean":"2024-05-11T23:04:42.118720+0000","last_became_active":"2024-05-11T23:04:28.806050+0000","last_became_peered":"2024-05-11T23:04:28.806050+0000","last_unstale":"2024-05-11T23:04:42.118720+0000","last_undegraded":"2024-05-11T23:04:42.118720+0000","last_fullsized":"2024-05-11T23:04:42.118720+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:58:42.310196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023348000000000002,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":7071,"num_read_kb":24639,"num_write":633,"num_write_kb":13630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":19437463,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"173'853","reported_seq":6348,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058241+0000","last_change":"2024-05-11T23:04:18.011274+0000","last_active":"2024-05-11T23:04:42.058241+0000","last_peered":"2024-05-11T23:04:42.058241+0000","last_clean":"2024-05-11T23:04:42.058241+0000","last_became_active":"2024-05-11T23:03:53.806713+0000","last_became_peered":"2024-05-11T23:03:53.806713+0000","last_unstale":"2024-05-11T23:04:42.058241+0000","last_undegraded":"2024-05-11T23:04:42.058241+0000","last_fullsized":"2024-05-11T23:04:42.058241+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":853,"log_dups_size":0,"ondisk_log_size":853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:25:53.472036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011357120000000001,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":7833,"num_read_kb":21923,"num_write":862,"num_write_kb":7573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":450,"num_bytes_recovered":18021482,"num_keys_recovered":78,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"175'975","reported_seq":5304,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-05-11T23:04:41.044876+0000","last_change":"2024-05-11T23:04:27.810986+0000","last_active":"2024-05-11T23:04:41.044876+0000","last_peered":"2024-05-11T23:04:41.044876+0000","last_clean":"2024-05-11T23:04:41.044876+0000","last_became_active":"2024-05-11T23:04:27.809030+0000","last_became_peered":"2024-05-11T23:04:27.809030+0000","last_unstale":"2024-05-11T23:04:41.044876+0000","last_undegraded":"2024-05-11T23:04:41.044876+0000","last_fullsized":"2024-05-11T23:04:41.044876+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:20:47.733184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028179780000000001,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":6275,"num_read_kb":24022,"num_write":1130,"num_write_kb":13297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"173'599","reported_seq":5607,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058311+0000","last_change":"2024-05-11T23:04:18.013459+0000","last_active":"2024-05-11T23:04:42.058311+0000","last_peered":"2024-05-11T23:04:42.058311+0000","last_clean":"2024-05-11T23:04:42.058311+0000","last_became_active":"2024-05-11T23:03:52.818374+0000","last_became_peered":"2024-05-11T23:03:52.818374+0000","last_unstale":"2024-05-11T23:04:42.058311+0000","last_undegraded":"2024-05-11T23:04:42.058311+0000","last_fullsized":"2024-05-11T23:04:42.058311+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:21:58.570294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031987780000000002,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":5050,"num_read_kb":18194,"num_write":554,"num_write_kb":7419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4575002,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1465,"num_object_clones":0,"num_object_copies":2930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1465,"num_whiteouts":1461,"num_read":339431,"num_read_kb":1023367,"num_write":28503,"num_write_kb":498784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8670,"num_bytes_recovered":382481677,"num_keys_recovered":951,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16167,"internal_metadata":0},"log_size":26567,"ondisk_log_size":26567,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100800,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509484,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69644,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302196,"statfs":{"total":96636764160,"available":96565448704,"internally_reserved":0,"allocated":2424832,"data_stored":1737431,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59856,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311984,"statfs":{"total":96636764160,"available":96575471616,"internally_reserved":0,"allocated":2359296,"data_stored":1147063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509485,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65428,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306412,"statfs":{"total":96636764160,"available":96569765888,"internally_reserved":0,"allocated":2363392,"data_stored":1147082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33472,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338368,"statfs":{"total":96636764160,"available":96602488832,"internally_reserved":0,"allocated":2347008,"data_stored":1141250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":179,"seq":768799145990,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36048,"kb_used_data":2304,"kb_used_omap":49,"kb_used_meta":33678,"kb_avail":94335792,"statfs":{"total":96636764160,"available":96599851008,"internally_reserved":0,"allocated":2359296,"data_stored":1147063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51091,"internal_metadata":34486381},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73732,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298108,"statfs":{"total":96636764160,"available":96561262592,"internally_reserved":0,"allocated":2351104,"data_stored":1141269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37632,"kb_used_data":2292,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334208,"statfs":{"total":96636764160,"available":96598228992,"internally_reserved":0,"allocated":2347008,"data_stored":1141250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77936,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293904,"statfs":{"total":96636764160,"available":96556957696,"internally_reserved":0,"allocated":2396160,"data_stored":1731618,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16167,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T23:04:46.321 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T23:04:46.621 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:04:46.622 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":196,"fsid":"34b20022-f11b-4169-a5f6-6386270a2f8f","created":"2024-05-11T23:01:05.977693+0000","modified":"2024-05-11T23:04:42.039348+0000","last_up_change":"2024-05-11T23:04:24.952411+0000","last_in_change":"2024-05-11T23:04:24.952411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T23:01:15.187614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-11T23:01:17.689285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-05-11T23:01:25.743347+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":53,"snap_epoch":175,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57ab1c70-9478-4a26-bb41-d753b4f3eb79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6801","nonce":3452725643}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6803","nonce":3452725643}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6808","nonce":3452725643}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3452725643},{"type":"v1","addr":"172.21.15.5:6805","nonce":3452725643}]},"public_addr":"172.21.15.5:6801/3452725643","cluster_addr":"172.21.15.5:6803/3452725643","heartbeat_back_addr":"172.21.15.5:6808/3452725643","heartbeat_front_addr":"172.21.15.5:6805/3452725643","state":["exists","up"]},{"osd":1,"uuid":"05ece110-4a99-46f0-b061-5980f7ef2a55","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6825","nonce":4028727138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6827","nonce":4028727138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6831","nonce":4028727138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":4028727138},{"type":"v1","addr":"172.21.15.5:6829","nonce":4028727138}]},"public_addr":"172.21.15.5:6825/4028727138","cluster_addr":"172.21.15.5:6827/4028727138","heartbeat_back_addr":"172.21.15.5:6831/4028727138","heartbeat_front_addr":"172.21.15.5:6829/4028727138","state":["exists","up"]},{"osd":2,"uuid":"4a662fd6-6ac2-4afb-a4e5-2d87cafb8ea5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6809","nonce":2059959509}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6811","nonce":2059959509}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6815","nonce":2059959509}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2059959509},{"type":"v1","addr":"172.21.15.5:6813","nonce":2059959509}]},"public_addr":"172.21.15.5:6809/2059959509","cluster_addr":"172.21.15.5:6811/2059959509","heartbeat_back_addr":"172.21.15.5:6815/2059959509","heartbeat_front_addr":"172.21.15.5:6813/2059959509","state":["exists","up"]},{"osd":3,"uuid":"5fc8f4fd-4e8b-4758-8184-9f53a93d4f2a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":129,"up_from":179,"up_thru":195,"down_at":130,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2390324318},{"type":"v1","addr":"172.21.15.5:6817","nonce":2390324318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2390324318},{"type":"v1","addr":"172.21.15.5:6819","nonce":2390324318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2390324318},{"type":"v1","addr":"172.21.15.5:6823","nonce":2390324318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2390324318},{"type":"v1","addr":"172.21.15.5:6821","nonce":2390324318}]},"public_addr":"172.21.15.5:6817/2390324318","cluster_addr":"172.21.15.5:6819/2390324318","heartbeat_back_addr":"172.21.15.5:6823/2390324318","heartbeat_front_addr":"172.21.15.5:6821/2390324318","state":["exists","up"]},{"osd":4,"uuid":"4e87e4f0-288e-4ee6-919e-9337925a5d5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":187,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6801","nonce":495703077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6803","nonce":495703077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6807","nonce":495703077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":495703077},{"type":"v1","addr":"172.21.15.115:6805","nonce":495703077}]},"public_addr":"172.21.15.115:6801/495703077","cluster_addr":"172.21.15.115:6803/495703077","heartbeat_back_addr":"172.21.15.115:6807/495703077","heartbeat_front_addr":"172.21.15.115:6805/495703077","state":["exists","up"]},{"osd":5,"uuid":"e149a6f6-a7e5-47bb-a40e-7aa9eb292cd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6811","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6813","nonce":902573489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6815","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6817","nonce":902573489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6823","nonce":902573489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6819","nonce":902573489},{"type":"v1","addr":"172.21.15.115:6821","nonce":902573489}]},"public_addr":"172.21.15.115:6813/902573489","cluster_addr":"172.21.15.115:6817/902573489","heartbeat_back_addr":"172.21.15.115:6823/902573489","heartbeat_front_addr":"172.21.15.115:6821/902573489","state":["exists","up"]},{"osd":6,"uuid":"edb686ec-e1a2-40d6-975a-e32e0e7760ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6825","nonce":2749332272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6827","nonce":2749332272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6830","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6831","nonce":2749332272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6828","nonce":2749332272},{"type":"v1","addr":"172.21.15.115:6829","nonce":2749332272}]},"public_addr":"172.21.15.115:6825/2749332272","cluster_addr":"172.21.15.115:6827/2749332272","heartbeat_back_addr":"172.21.15.115:6831/2749332272","heartbeat_front_addr":"172.21.15.115:6829/2749332272","state":["exists","up"]},{"osd":7,"uuid":"7aa14dbb-c6cb-4540-b160-3fd346f6a046","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6809","nonce":1043826337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6812","nonce":1043826337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6820","nonce":1043826337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1043826337},{"type":"v1","addr":"172.21.15.115:6816","nonce":1043826337}]},"public_addr":"172.21.15.115:6809/1043826337","cluster_addr":"172.21.15.115:6812/1043826337","heartbeat_back_addr":"172.21.15.115:6820/1043826337","heartbeat_front_addr":"172.21.15.115:6816/1043826337","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.113232+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.174679+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.131118+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.207668+0000","dead_epoch":130},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.124542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.200832+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.169560+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T23:01:14.182465+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T23:04:47.636 INFO:tasks.ceph:Scrubbing osd.0 2024-05-11T23:04:47.636 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-11T23:04:47.780 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:47.780 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T23:04:47.780 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:47.791 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-11T23:04:48.093 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 0 to deep-scrub 2024-05-11T23:04:48.105 INFO:tasks.ceph:Scrubbing osd.1 2024-05-11T23:04:48.106 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-11T23:04:48.235 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:48.235 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T23:04:48.235 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:48.245 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-11T23:04:48.547 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 1 to deep-scrub 2024-05-11T23:04:48.560 INFO:tasks.ceph:Scrubbing osd.2 2024-05-11T23:04:48.560 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-11T23:04:48.693 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:48.694 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T23:04:48.694 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:48.704 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-11T23:04:49.005 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 2 to deep-scrub 2024-05-11T23:04:49.018 INFO:tasks.ceph:Scrubbing osd.3 2024-05-11T23:04:49.019 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-11T23:04:49.158 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:49.159 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-05-11T23:04:49.159 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:49.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-11T23:04:49.467 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 3 to deep-scrub 2024-05-11T23:04:49.480 INFO:tasks.ceph:Scrubbing osd.4 2024-05-11T23:04:49.481 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-11T23:04:49.615 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:49.615 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T23:04:49.615 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:49.627 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-11T23:04:49.931 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 4 to deep-scrub 2024-05-11T23:04:49.943 INFO:tasks.ceph:Scrubbing osd.5 2024-05-11T23:04:49.943 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-11T23:04:50.074 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:50.074 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T23:04:50.074 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:50.085 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-11T23:04:50.401 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 5 to deep-scrub 2024-05-11T23:04:50.414 INFO:tasks.ceph:Scrubbing osd.6 2024-05-11T23:04:50.414 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-11T23:04:50.543 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:50.543 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T23:04:50.544 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:50.554 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-11T23:04:50.862 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 6 to deep-scrub 2024-05-11T23:04:50.875 INFO:tasks.ceph:Scrubbing osd.7 2024-05-11T23:04:50.875 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-11T23:04:51.005 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-05-11T23:04:51.006 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-11T23:04:51.006 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-05-11T23:04:51.017 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-11T23:04:51.329 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 7 to deep-scrub 2024-05-11T23:04:51.342 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T23:04:51.643 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:04:51.644 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-11T23:04:51.657 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-05-11T23:04:51.175485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":1467,"num_object_clones":0,"num_object_copies":2934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1467,"num_whiteouts":1461,"num_read":339547,"num_read_kb":1023467,"num_write":28662,"num_write_kb":501200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8674,"num_bytes_recovered":383662413,"num_keys_recovered":951,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26770,"ondisk_log_size":26770,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":453796,"kb_used_data":18552,"kb_used_omap":60,"kb_used_meta":435075,"kb_avail":754520924,"statfs":{"total":773094113280,"available":772629426176,"internally_reserved":0,"allocated":18997248,"data_stored":10357278,"data_compressed":66655,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":62250,"internal_metadata":445517014},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001942"},"pg_stats":[{"pgid":"3.1f","version":"175'540","reported_seq":5574,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.059827+0000","last_change":"2024-05-11T23:04:28.806612+0000","last_active":"2024-05-11T23:04:42.059827+0000","last_peered":"2024-05-11T23:04:42.059827+0000","last_clean":"2024-05-11T23:04:42.059827+0000","last_became_active":"2024-05-11T23:04:28.804255+0000","last_became_peered":"2024-05-11T23:04:28.804255+0000","last_unstale":"2024-05-11T23:04:42.059827+0000","last_undegraded":"2024-05-11T23:04:42.059827+0000","last_fullsized":"2024-05-11T23:04:42.059827+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098957899999999998,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":5393,"num_read_kb":14723,"num_write":488,"num_write_kb":18081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":877932,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"180'934","reported_seq":6543,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.057168+0000","last_change":"2024-05-11T23:04:25.935404+0000","last_active":"2024-05-11T23:04:42.057168+0000","last_peered":"2024-05-11T23:04:42.057168+0000","last_clean":"2024-05-11T23:04:42.057168+0000","last_became_active":"2024-05-11T23:04:25.808413+0000","last_became_peered":"2024-05-11T23:04:25.808413+0000","last_unstale":"2024-05-11T23:04:42.057168+0000","last_undegraded":"2024-05-11T23:04:42.057168+0000","last_fullsized":"2024-05-11T23:04:42.057168+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":934,"log_dups_size":0,"ondisk_log_size":934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:09:17.133440+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.084167985000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":6916,"num_read_kb":49448,"num_write":971,"num_write_kb":22687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":53290749,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"167'508","reported_seq":4530,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058246+0000","last_change":"2024-05-11T23:04:17.950967+0000","last_active":"2024-05-11T23:04:42.058246+0000","last_peered":"2024-05-11T23:04:42.058246+0000","last_clean":"2024-05-11T23:04:42.058246+0000","last_became_active":"2024-05-11T23:03:48.889806+0000","last_became_peered":"2024-05-11T23:03:48.889806+0000","last_unstale":"2024-05-11T23:04:42.058246+0000","last_undegraded":"2024-05-11T23:04:42.058246+0000","last_fullsized":"2024-05-11T23:04:42.058246+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:46:37.757197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060389999999999999,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":3180,"num_read_kb":16428,"num_write":406,"num_write_kb":9629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":7067368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"175'784","reported_seq":19160,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058324+0000","last_change":"2024-05-11T23:04:17.951789+0000","last_active":"2024-05-11T23:04:42.058324+0000","last_peered":"2024-05-11T23:04:42.058324+0000","last_clean":"2024-05-11T23:04:42.058324+0000","last_became_active":"2024-05-11T23:03:50.798632+0000","last_became_peered":"2024-05-11T23:03:50.798632+0000","last_unstale":"2024-05-11T23:04:42.058324+0000","last_undegraded":"2024-05-11T23:04:42.058324+0000","last_fullsized":"2024-05-11T23:04:42.058324+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:13:59.992713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001248566,"stat_sum":{"num_bytes":0,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":63,"num_read":18429,"num_read_kb":39778,"num_write":674,"num_write_kb":10510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":9396782,"num_keys_recovered":182,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"174'594","reported_seq":15114,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.060584+0000","last_change":"2024-05-11T23:04:29.807594+0000","last_active":"2024-05-11T23:04:42.060584+0000","last_peered":"2024-05-11T23:04:42.060584+0000","last_clean":"2024-05-11T23:04:42.060584+0000","last_became_active":"2024-05-11T23:04:29.804143+0000","last_became_peered":"2024-05-11T23:04:29.804143+0000","last_unstale":"2024-05-11T23:04:42.060584+0000","last_undegraded":"2024-05-11T23:04:42.060584+0000","last_fullsized":"2024-05-11T23:04:42.060584+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001388731,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":13979,"num_read_kb":68609,"num_write":598,"num_write_kb":35793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":532,"num_bytes_recovered":20898869,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"173'1664","reported_seq":21045,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118917+0000","last_change":"2024-05-11T23:04:34.010543+0000","last_active":"2024-05-11T23:04:42.118917+0000","last_peered":"2024-05-11T23:04:42.118917+0000","last_clean":"2024-05-11T23:04:42.118917+0000","last_became_active":"2024-05-11T23:04:34.007358+0000","last_became_peered":"2024-05-11T23:04:34.007358+0000","last_unstale":"2024-05-11T23:04:42.118917+0000","last_undegraded":"2024-05-11T23:04:42.118917+0000","last_fullsized":"2024-05-11T23:04:42.118917+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1664,"log_dups_size":0,"ondisk_log_size":1664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:31:26.529561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097887899999999999,"stat_sum":{"num_bytes":19,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":25336,"num_read_kb":48131,"num_write":2723,"num_write_kb":23444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":6434587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"180'1052","reported_seq":15387,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.059508+0000","last_change":"2024-05-11T23:04:25.959047+0000","last_active":"2024-05-11T23:04:42.059508+0000","last_peered":"2024-05-11T23:04:42.059508+0000","last_clean":"2024-05-11T23:04:42.059508+0000","last_became_active":"2024-05-11T23:04:25.807274+0000","last_became_peered":"2024-05-11T23:04:25.807274+0000","last_unstale":"2024-05-11T23:04:42.059508+0000","last_undegraded":"2024-05-11T23:04:42.059508+0000","last_fullsized":"2024-05-11T23:04:42.059508+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1052,"log_dups_size":0,"ondisk_log_size":1052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:17:04.610363+0000","scrub_duration":0,"objects_trimmed":30,"snaptrim_duration":0.061503582000000001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":16282,"num_read_kb":43182,"num_write":1369,"num_write_kb":14705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":13589947,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"170'967","reported_seq":3982,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.056362+0000","last_change":"2024-05-11T23:04:26.853599+0000","last_active":"2024-05-11T23:04:42.056362+0000","last_peered":"2024-05-11T23:04:42.056362+0000","last_clean":"2024-05-11T23:04:42.056362+0000","last_became_active":"2024-05-11T23:04:26.807825+0000","last_became_peered":"2024-05-11T23:04:26.807825+0000","last_unstale":"2024-05-11T23:04:42.056362+0000","last_undegraded":"2024-05-11T23:04:42.056362+0000","last_fullsized":"2024-05-11T23:04:42.056362+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:26:49.247146+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001621796,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":5754,"num_read_kb":9891,"num_write":1117,"num_write_kb":16486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":7487386,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"196'709","reported_seq":3923,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:49.625606+0000","last_change":"2024-05-11T23:04:49.625707+0000","last_active":"2024-05-11T23:04:49.625606+0000","last_peered":"2024-05-11T23:04:49.625606+0000","last_clean":"2024-05-11T23:04:49.625606+0000","last_became_active":"2024-05-11T23:04:27.809321+0000","last_became_peered":"2024-05-11T23:04:27.809321+0000","last_unstale":"2024-05-11T23:04:49.625606+0000","last_undegraded":"2024-05-11T23:04:49.625606+0000","last_fullsized":"2024-05-11T23:04:49.625606+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'709","last_scrub_stamp":"2024-05-11T23:04:49.625504+0000","last_deep_scrub":"196'709","last_deep_scrub_stamp":"2024-05-11T23:04:49.625504+0000","last_clean_scrub_stamp":"2024-05-11T23:04:49.625504+0000","objects_scrubbed":42,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:53:33.656821+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0.0013760389999999999,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":3972,"num_read_kb":15246,"num_write":808,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"172'559","reported_seq":15690,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058382+0000","last_change":"2024-05-11T23:04:17.950903+0000","last_active":"2024-05-11T23:04:42.058382+0000","last_peered":"2024-05-11T23:04:42.058382+0000","last_clean":"2024-05-11T23:04:42.058382+0000","last_became_active":"2024-05-11T23:01:27.463623+0000","last_became_peered":"2024-05-11T23:01:27.463623+0000","last_unstale":"2024-05-11T23:04:42.058382+0000","last_undegraded":"2024-05-11T23:04:42.058382+0000","last_fullsized":"2024-05-11T23:04:42.058382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:56:10.783772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054116699999999995,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":15017,"num_read_kb":54784,"num_write":481,"num_write_kb":18602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"180'622","reported_seq":16413,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058371+0000","last_change":"2024-05-11T23:04:26.015543+0000","last_active":"2024-05-11T23:04:42.058371+0000","last_peered":"2024-05-11T23:04:42.058371+0000","last_clean":"2024-05-11T23:04:42.058371+0000","last_became_active":"2024-05-11T23:04:25.807935+0000","last_became_peered":"2024-05-11T23:04:25.807935+0000","last_unstale":"2024-05-11T23:04:42.058371+0000","last_undegraded":"2024-05-11T23:04:42.058371+0000","last_fullsized":"2024-05-11T23:04:42.058371+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:50:30.593291+0000","scrub_duration":0,"objects_trimmed":36,"snaptrim_duration":0.072194399000000006,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":14264,"num_read_kb":25298,"num_write":546,"num_write_kb":13154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":12363304,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"172'661","reported_seq":19584,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.119099+0000","last_change":"2024-05-11T23:04:34.010412+0000","last_active":"2024-05-11T23:04:42.119099+0000","last_peered":"2024-05-11T23:04:42.119099+0000","last_clean":"2024-05-11T23:04:42.119099+0000","last_became_active":"2024-05-11T23:04:34.007861+0000","last_became_peered":"2024-05-11T23:04:34.007861+0000","last_unstale":"2024-05-11T23:04:42.119099+0000","last_undegraded":"2024-05-11T23:04:42.119099+0000","last_fullsized":"2024-05-11T23:04:42.119099+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:13:07.675246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001389319,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":19402,"num_read_kb":42118,"num_write":569,"num_write_kb":6424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":4215359,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118756+0000","last_change":"2024-05-11T23:02:39.044686+0000","last_active":"2024-05-11T23:04:42.118756+0000","last_peered":"2024-05-11T23:04:42.118756+0000","last_clean":"2024-05-11T23:04:42.118756+0000","last_became_active":"2024-05-11T23:02:39.044216+0000","last_became_peered":"2024-05-11T23:02:39.044216+0000","last_unstale":"2024-05-11T23:04:42.118756+0000","last_undegraded":"2024-05-11T23:04:42.118756+0000","last_fullsized":"2024-05-11T23:04:42.118756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:31:03.826529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"175'1049","reported_seq":7447,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.059255+0000","last_change":"2024-05-11T23:04:28.808149+0000","last_active":"2024-05-11T23:04:42.059255+0000","last_peered":"2024-05-11T23:04:42.059255+0000","last_clean":"2024-05-11T23:04:42.059255+0000","last_became_active":"2024-05-11T23:04:28.805428+0000","last_became_peered":"2024-05-11T23:04:28.805428+0000","last_unstale":"2024-05-11T23:04:42.059255+0000","last_undegraded":"2024-05-11T23:04:42.059255+0000","last_fullsized":"2024-05-11T23:04:42.059255+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002036403,"stat_sum":{"num_bytes":0,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":60,"num_read":9703,"num_read_kb":23103,"num_write":1155,"num_write_kb":12619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":2180842,"num_keys_recovered":21,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":337,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.059701+0000","last_change":"2024-05-11T23:04:29.806455+0000","last_active":"2024-05-11T23:04:42.059701+0000","last_peered":"2024-05-11T23:04:42.059701+0000","last_clean":"2024-05-11T23:04:42.059701+0000","last_became_active":"2024-05-11T23:04:29.805946+0000","last_became_peered":"2024-05-11T23:04:29.805946+0000","last_unstale":"2024-05-11T23:04:42.059701+0000","last_undegraded":"2024-05-11T23:04:42.059701+0000","last_fullsized":"2024-05-11T23:04:42.059701+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:09:54.127025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"180'926","reported_seq":6079,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.063271+0000","last_change":"2024-05-11T23:04:42.063271+0000","last_active":"2024-05-11T23:04:42.063271+0000","last_peered":"2024-05-11T23:04:42.063271+0000","last_clean":"2024-05-11T23:04:42.063271+0000","last_became_active":"2024-05-11T23:04:42.060396+0000","last_became_peered":"2024-05-11T23:04:42.060396+0000","last_unstale":"2024-05-11T23:04:42.063271+0000","last_undegraded":"2024-05-11T23:04:42.063271+0000","last_fullsized":"2024-05-11T23:04:42.063271+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:02:26.606646+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.055140898000000001,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":7310,"num_read_kb":19251,"num_write":1088,"num_write_kb":7185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":4415701,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":376,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058504+0000","last_change":"2024-05-11T23:04:25.806665+0000","last_active":"2024-05-11T23:04:42.058504+0000","last_peered":"2024-05-11T23:04:42.058504+0000","last_clean":"2024-05-11T23:04:42.058504+0000","last_became_active":"2024-05-11T23:04:25.806292+0000","last_became_peered":"2024-05-11T23:04:25.806292+0000","last_unstale":"2024-05-11T23:04:42.058504+0000","last_undegraded":"2024-05-11T23:04:42.058504+0000","last_fullsized":"2024-05-11T23:04:42.058504+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:12:45.144479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"175'564","reported_seq":14908,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.059307+0000","last_change":"2024-05-11T23:04:29.809156+0000","last_active":"2024-05-11T23:04:42.059307+0000","last_peered":"2024-05-11T23:04:42.059307+0000","last_clean":"2024-05-11T23:04:42.059307+0000","last_became_active":"2024-05-11T23:04:29.807343+0000","last_became_peered":"2024-05-11T23:04:29.807343+0000","last_unstale":"2024-05-11T23:04:42.059307+0000","last_undegraded":"2024-05-11T23:04:42.059307+0000","last_fullsized":"2024-05-11T23:04:42.059307+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:35:56.026223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095379299999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":14156,"num_read_kb":53683,"num_write":532,"num_write_kb":33731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":341,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.059407+0000","last_change":"2024-05-11T23:04:29.805774+0000","last_active":"2024-05-11T23:04:42.059407+0000","last_peered":"2024-05-11T23:04:42.059407+0000","last_clean":"2024-05-11T23:04:42.059407+0000","last_became_active":"2024-05-11T23:04:29.805447+0000","last_became_peered":"2024-05-11T23:04:29.805447+0000","last_unstale":"2024-05-11T23:04:42.059407+0000","last_undegraded":"2024-05-11T23:04:42.059407+0000","last_fullsized":"2024-05-11T23:04:42.059407+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:56:59.210181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"180'758","reported_seq":6312,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058458+0000","last_change":"2024-05-11T23:04:25.999769+0000","last_active":"2024-05-11T23:04:42.058458+0000","last_peered":"2024-05-11T23:04:42.058458+0000","last_clean":"2024-05-11T23:04:42.058458+0000","last_became_active":"2024-05-11T23:04:25.806236+0000","last_became_peered":"2024-05-11T23:04:25.806236+0000","last_unstale":"2024-05-11T23:04:42.058458+0000","last_undegraded":"2024-05-11T23:04:42.058458+0000","last_fullsized":"2024-05-11T23:04:42.058458+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:19:42.475478+0000","scrub_duration":0,"objects_trimmed":38,"snaptrim_duration":0.081084693999999999,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":6715,"num_read_kb":25684,"num_write":652,"num_write_kb":13121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":13712581,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"175'1022","reported_seq":11247,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.119002+0000","last_change":"2024-05-11T23:04:34.011637+0000","last_active":"2024-05-11T23:04:42.119002+0000","last_peered":"2024-05-11T23:04:42.119002+0000","last_clean":"2024-05-11T23:04:42.119002+0000","last_became_active":"2024-05-11T23:04:34.008456+0000","last_became_peered":"2024-05-11T23:04:34.008456+0000","last_unstale":"2024-05-11T23:04:42.119002+0000","last_undegraded":"2024-05-11T23:04:42.119002+0000","last_fullsized":"2024-05-11T23:04:42.119002+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1022,"log_dups_size":0,"ondisk_log_size":1022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:56:16.226740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017329069999999999,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":56,"num_read":10698,"num_read_kb":33938,"num_write":973,"num_write_kb":17254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":898,"num_bytes_recovered":16208786,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":368,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058873+0000","last_change":"2024-05-11T23:01:19.221118+0000","last_active":"2024-05-11T23:04:42.058873+0000","last_peered":"2024-05-11T23:04:42.058873+0000","last_clean":"2024-05-11T23:04:42.058873+0000","last_became_active":"2024-05-11T23:01:19.220855+0000","last_became_peered":"2024-05-11T23:01:19.220855+0000","last_unstale":"2024-05-11T23:04:42.058873+0000","last_undegraded":"2024-05-11T23:04:42.058873+0000","last_fullsized":"2024-05-11T23:04:42.058873+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:01:56.076963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"173'1148","reported_seq":6345,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.059014+0000","last_change":"2024-05-11T23:04:17.940887+0000","last_active":"2024-05-11T23:04:42.059014+0000","last_peered":"2024-05-11T23:04:42.059014+0000","last_clean":"2024-05-11T23:04:42.059014+0000","last_became_active":"2024-05-11T23:01:27.459593+0000","last_became_peered":"2024-05-11T23:01:27.459593+0000","last_unstale":"2024-05-11T23:04:42.059014+0000","last_undegraded":"2024-05-11T23:04:42.059014+0000","last_fullsized":"2024-05-11T23:04:42.059014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1148,"log_dups_size":0,"ondisk_log_size":1148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:20:45.922442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017756580000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9995,"num_read_kb":18445,"num_write":1457,"num_write_kb":4408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":395,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:49.642963+0000","last_change":"2024-05-11T23:04:49.642984+0000","last_active":"2024-05-11T23:04:49.642963+0000","last_peered":"2024-05-11T23:04:49.642963+0000","last_clean":"2024-05-11T23:04:49.642963+0000","last_became_active":"2024-05-11T23:04:25.804674+0000","last_became_peered":"2024-05-11T23:04:25.804674+0000","last_unstale":"2024-05-11T23:04:49.642963+0000","last_undegraded":"2024-05-11T23:04:49.642963+0000","last_fullsized":"2024-05-11T23:04:49.642963+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:04:49.642885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:04:49.642885+0000","last_clean_scrub_stamp":"2024-05-11T23:04:49.642885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:58:15.177305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"196'700","reported_seq":15368,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:48.609109+0000","last_change":"2024-05-11T23:04:48.609182+0000","last_active":"2024-05-11T23:04:48.609109+0000","last_peered":"2024-05-11T23:04:48.609109+0000","last_clean":"2024-05-11T23:04:48.609109+0000","last_became_active":"2024-05-11T23:04:26.804248+0000","last_became_peered":"2024-05-11T23:04:26.804248+0000","last_unstale":"2024-05-11T23:04:48.609109+0000","last_undegraded":"2024-05-11T23:04:48.609109+0000","last_fullsized":"2024-05-11T23:04:48.609109+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'700","last_scrub_stamp":"2024-05-11T23:04:48.608991+0000","last_deep_scrub":"196'700","last_deep_scrub_stamp":"2024-05-11T23:04:48.608991+0000","last_clean_scrub_stamp":"2024-05-11T23:04:48.608991+0000","objects_scrubbed":44,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T09:28:40.682780+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0.0014815119999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":15309,"num_read_kb":24676,"num_write":622,"num_write_kb":6352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":299,"num_bytes_recovered":6345105,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":366,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058188+0000","last_change":"2024-05-11T23:01:19.219755+0000","last_active":"2024-05-11T23:04:42.058188+0000","last_peered":"2024-05-11T23:04:42.058188+0000","last_clean":"2024-05-11T23:04:42.058188+0000","last_became_active":"2024-05-11T23:01:19.219471+0000","last_became_peered":"2024-05-11T23:01:19.219471+0000","last_unstale":"2024-05-11T23:04:42.058188+0000","last_undegraded":"2024-05-11T23:04:42.058188+0000","last_fullsized":"2024-05-11T23:04:42.058188+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:39:05.284213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"180'690","reported_seq":4302,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.064001+0000","last_change":"2024-05-11T23:04:42.064001+0000","last_active":"2024-05-11T23:04:42.064001+0000","last_peered":"2024-05-11T23:04:42.064001+0000","last_clean":"2024-05-11T23:04:42.064001+0000","last_became_active":"2024-05-11T23:04:42.060595+0000","last_became_peered":"2024-05-11T23:04:42.060595+0000","last_unstale":"2024-05-11T23:04:42.064001+0000","last_undegraded":"2024-05-11T23:04:42.064001+0000","last_fullsized":"2024-05-11T23:04:42.064001+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:38:46.487523+0000","scrub_duration":0,"objects_trimmed":28,"snaptrim_duration":0.040069711000000001,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":3497,"num_read_kb":16044,"num_write":640,"num_write_kb":10409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":10719266,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":368,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058198+0000","last_change":"2024-05-11T23:01:19.219128+0000","last_active":"2024-05-11T23:04:42.058198+0000","last_peered":"2024-05-11T23:04:42.058198+0000","last_clean":"2024-05-11T23:04:42.058198+0000","last_became_active":"2024-05-11T23:01:19.218846+0000","last_became_peered":"2024-05-11T23:01:19.218846+0000","last_unstale":"2024-05-11T23:04:42.058198+0000","last_undegraded":"2024-05-11T23:04:42.058198+0000","last_fullsized":"2024-05-11T23:04:42.058198+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:18.153704+0000","last_clean_scrub_stamp":"2024-05-11T23:01:18.153704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:41:09.111000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":544,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058200+0000","last_change":"2024-05-11T23:04:28.806566+0000","last_active":"2024-05-11T23:04:42.058200+0000","last_peered":"2024-05-11T23:04:42.058200+0000","last_clean":"2024-05-11T23:04:42.058200+0000","last_became_active":"2024-05-11T23:04:28.806124+0000","last_became_peered":"2024-05-11T23:04:28.806124+0000","last_unstale":"2024-05-11T23:04:42.058200+0000","last_undegraded":"2024-05-11T23:04:42.058200+0000","last_fullsized":"2024-05-11T23:04:42.058200+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:16.147564+0000","last_clean_scrub_stamp":"2024-05-11T23:01:16.147564+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:03:40.590362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"180'966","reported_seq":5554,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.056462+0000","last_change":"2024-05-11T23:04:25.885127+0000","last_active":"2024-05-11T23:04:42.056462+0000","last_peered":"2024-05-11T23:04:42.056462+0000","last_clean":"2024-05-11T23:04:42.056462+0000","last_became_active":"2024-05-11T23:04:25.802390+0000","last_became_peered":"2024-05-11T23:04:25.802390+0000","last_unstale":"2024-05-11T23:04:42.056462+0000","last_undegraded":"2024-05-11T23:04:42.056462+0000","last_fullsized":"2024-05-11T23:04:42.056462+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":966,"log_dups_size":0,"ondisk_log_size":966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:45:25.229272+0000","scrub_duration":0,"objects_trimmed":24,"snaptrim_duration":0.057229434000000003,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":6339,"num_read_kb":20167,"num_write":1151,"num_write_kb":10790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":4220183,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"172'676","reported_seq":9169,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118976+0000","last_change":"2024-05-11T23:04:34.008013+0000","last_active":"2024-05-11T23:04:42.118976+0000","last_peered":"2024-05-11T23:04:42.118976+0000","last_clean":"2024-05-11T23:04:42.118976+0000","last_became_active":"2024-05-11T23:04:34.005569+0000","last_became_peered":"2024-05-11T23:04:34.005569+0000","last_unstale":"2024-05-11T23:04:42.118976+0000","last_undegraded":"2024-05-11T23:04:42.118976+0000","last_fullsized":"2024-05-11T23:04:42.118976+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:27:04.828149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064185299999999995,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":9043,"num_read_kb":47546,"num_write":541,"num_write_kb":29805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":37320801,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"175'785","reported_seq":6658,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118728+0000","last_change":"2024-05-11T23:04:17.987769+0000","last_active":"2024-05-11T23:04:42.118728+0000","last_peered":"2024-05-11T23:04:42.118728+0000","last_clean":"2024-05-11T23:04:42.118728+0000","last_became_active":"2024-05-11T23:03:51.797214+0000","last_became_peered":"2024-05-11T23:03:51.797214+0000","last_unstale":"2024-05-11T23:04:42.118728+0000","last_undegraded":"2024-05-11T23:04:42.118728+0000","last_fullsized":"2024-05-11T23:04:42.118728+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:04:04.928406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001270001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":44,"num_read":5814,"num_read_kb":22550,"num_write":685,"num_write_kb":9625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":538,"num_bytes_recovered":16997094,"num_keys_recovered":140,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"173'657","reported_seq":9945,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.076988+0000","last_change":"2024-05-11T23:04:42.076988+0000","last_active":"2024-05-11T23:04:42.076988+0000","last_peered":"2024-05-11T23:04:42.076988+0000","last_clean":"2024-05-11T23:04:42.076988+0000","last_became_active":"2024-05-11T23:04:42.073145+0000","last_became_peered":"2024-05-11T23:04:42.073145+0000","last_unstale":"2024-05-11T23:04:42.076988+0000","last_undegraded":"2024-05-11T23:04:42.076988+0000","last_fullsized":"2024-05-11T23:04:42.076988+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:50:27.744009+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00088000000000000003,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":9829,"num_read_kb":52232,"num_write":599,"num_write_kb":42645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":47434023,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"180'694","reported_seq":5216,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058529+0000","last_change":"2024-05-11T23:04:26.020729+0000","last_active":"2024-05-11T23:04:42.058529+0000","last_peered":"2024-05-11T23:04:42.058529+0000","last_clean":"2024-05-11T23:04:42.058529+0000","last_became_active":"2024-05-11T23:04:25.806852+0000","last_became_peered":"2024-05-11T23:04:25.806852+0000","last_unstale":"2024-05-11T23:04:42.058529+0000","last_undegraded":"2024-05-11T23:04:42.058529+0000","last_fullsized":"2024-05-11T23:04:42.058529+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:12:25.591743+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.060182605,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":6324,"num_read_kb":25218,"num_write":659,"num_write_kb":11709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":10615766,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"174'897","reported_seq":21502,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058455+0000","last_change":"2024-05-11T23:04:18.014080+0000","last_active":"2024-05-11T23:04:42.058455+0000","last_peered":"2024-05-11T23:04:42.058455+0000","last_clean":"2024-05-11T23:04:42.058455+0000","last_became_active":"2024-05-11T23:01:27.465570+0000","last_became_peered":"2024-05-11T23:01:27.465570+0000","last_unstale":"2024-05-11T23:04:42.058455+0000","last_undegraded":"2024-05-11T23:04:42.058455+0000","last_fullsized":"2024-05-11T23:04:42.058455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":897,"log_dups_size":0,"ondisk_log_size":897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:14:28.607937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003620968,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":22252,"num_read_kb":40862,"num_write":875,"num_write_kb":12354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"175'1308","reported_seq":14450,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058370+0000","last_change":"2024-05-11T23:04:18.014428+0000","last_active":"2024-05-11T23:04:42.058370+0000","last_peered":"2024-05-11T23:04:42.058370+0000","last_clean":"2024-05-11T23:04:42.058370+0000","last_became_active":"2024-05-11T23:03:52.804360+0000","last_became_peered":"2024-05-11T23:03:52.804360+0000","last_unstale":"2024-05-11T23:04:42.058370+0000","last_undegraded":"2024-05-11T23:04:42.058370+0000","last_fullsized":"2024-05-11T23:04:42.058370+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:09:37.091464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039667269999999998,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":17126,"num_read_kb":45672,"num_write":1555,"num_write_kb":23766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":18760061,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"168'1114","reported_seq":9526,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058558+0000","last_change":"2024-05-11T23:04:18.014829+0000","last_active":"2024-05-11T23:04:42.058558+0000","last_peered":"2024-05-11T23:04:42.058558+0000","last_clean":"2024-05-11T23:04:42.058558+0000","last_became_active":"2024-05-11T23:03:49.895043+0000","last_became_peered":"2024-05-11T23:03:49.895043+0000","last_unstale":"2024-05-11T23:04:42.058558+0000","last_undegraded":"2024-05-11T23:04:42.058558+0000","last_fullsized":"2024-05-11T23:04:42.058558+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:55:15.798774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040936139999999998,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":11168,"num_read_kb":37882,"num_write":1390,"num_write_kb":15155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":362,"num_bytes_recovered":15895238,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"174'678","reported_seq":7038,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.118720+0000","last_change":"2024-05-11T23:04:28.807759+0000","last_active":"2024-05-11T23:04:42.118720+0000","last_peered":"2024-05-11T23:04:42.118720+0000","last_clean":"2024-05-11T23:04:42.118720+0000","last_became_active":"2024-05-11T23:04:28.806050+0000","last_became_peered":"2024-05-11T23:04:28.806050+0000","last_unstale":"2024-05-11T23:04:42.118720+0000","last_undegraded":"2024-05-11T23:04:42.118720+0000","last_fullsized":"2024-05-11T23:04:42.118720+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:58:42.310196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023348000000000002,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":7071,"num_read_kb":24639,"num_write":633,"num_write_kb":13630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":19437463,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"173'853","reported_seq":6348,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058241+0000","last_change":"2024-05-11T23:04:18.011274+0000","last_active":"2024-05-11T23:04:42.058241+0000","last_peered":"2024-05-11T23:04:42.058241+0000","last_clean":"2024-05-11T23:04:42.058241+0000","last_became_active":"2024-05-11T23:03:53.806713+0000","last_became_peered":"2024-05-11T23:03:53.806713+0000","last_unstale":"2024-05-11T23:04:42.058241+0000","last_undegraded":"2024-05-11T23:04:42.058241+0000","last_fullsized":"2024-05-11T23:04:42.058241+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":853,"log_dups_size":0,"ondisk_log_size":853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:25:53.472036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011357120000000001,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":7833,"num_read_kb":21923,"num_write":862,"num_write_kb":7573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":450,"num_bytes_recovered":18021482,"num_keys_recovered":78,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"175'975","reported_seq":5307,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.059377+0000","last_change":"2024-05-11T23:04:27.810986+0000","last_active":"2024-05-11T23:04:42.059377+0000","last_peered":"2024-05-11T23:04:42.059377+0000","last_clean":"2024-05-11T23:04:42.059377+0000","last_became_active":"2024-05-11T23:04:27.809030+0000","last_became_peered":"2024-05-11T23:04:27.809030+0000","last_unstale":"2024-05-11T23:04:42.059377+0000","last_undegraded":"2024-05-11T23:04:42.059377+0000","last_fullsized":"2024-05-11T23:04:42.059377+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028179780000000001,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":6275,"num_read_kb":24022,"num_write":1130,"num_write_kb":13297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"173'599","reported_seq":5607,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:42.058311+0000","last_change":"2024-05-11T23:04:18.013459+0000","last_active":"2024-05-11T23:04:42.058311+0000","last_peered":"2024-05-11T23:04:42.058311+0000","last_clean":"2024-05-11T23:04:42.058311+0000","last_became_active":"2024-05-11T23:03:52.818374+0000","last_became_peered":"2024-05-11T23:03:52.818374+0000","last_unstale":"2024-05-11T23:04:42.058311+0000","last_undegraded":"2024-05-11T23:04:42.058311+0000","last_fullsized":"2024-05-11T23:04:42.058311+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:01:26.442571+0000","last_clean_scrub_stamp":"2024-05-11T23:01:26.442571+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:21:58.570294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031987780000000002,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":5050,"num_read_kb":18194,"num_write":554,"num_write_kb":7419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4575002,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1465,"num_object_clones":0,"num_object_copies":2930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1465,"num_whiteouts":1461,"num_read":339431,"num_read_kb":1023367,"num_write":28503,"num_write_kb":498784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8670,"num_bytes_recovered":382481677,"num_keys_recovered":951,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16167,"internal_metadata":0},"log_size":26653,"ondisk_log_size":26653,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100800,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509485,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69644,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302196,"statfs":{"total":96636764160,"available":96565448704,"internally_reserved":0,"allocated":2424832,"data_stored":1737431,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509486,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59856,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311984,"statfs":{"total":96636764160,"available":96575471616,"internally_reserved":0,"allocated":2359296,"data_stored":1147063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509486,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65428,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306412,"statfs":{"total":96636764160,"available":96569765888,"internally_reserved":0,"allocated":2363392,"data_stored":1147082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509485,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33488,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338352,"statfs":{"total":96636764160,"available":96602472448,"internally_reserved":0,"allocated":2359296,"data_stored":1147063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":179,"seq":768799145991,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36048,"kb_used_data":2304,"kb_used_omap":49,"kb_used_meta":33678,"kb_avail":94335792,"statfs":{"total":96636764160,"available":96599851008,"internally_reserved":0,"allocated":2359296,"data_stored":1147063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51091,"internal_metadata":34486381},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73748,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298092,"statfs":{"total":96636764160,"available":96561246208,"internally_reserved":0,"allocated":2363392,"data_stored":1147082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37640,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334200,"statfs":{"total":96636764160,"available":96598220800,"internally_reserved":0,"allocated":2359296,"data_stored":1147063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77944,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293896,"statfs":{"total":96636764160,"available":96556949504,"internally_reserved":0,"allocated":2408448,"data_stored":1737431,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16167,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T23:04:51.663 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.663 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.663 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.663 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.663 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.663 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.663 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.663 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.664 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.664 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.664 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.664 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-05-11T23:01:18.153704+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=18, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.664 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.664 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-05-11T23:01:18.153704+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=18, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.664 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.664 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-05-11T23:01:18.153704+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=18, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.664 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.665 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-05-11T23:01:18.153704+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=18, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.665 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.665 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.665 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-05-11T23:01:18.153704+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=18, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.665 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.665 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-05-11T23:01:18.153704+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=18, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.665 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.665 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-05-11T23:01:18.153704+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=18, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.665 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-11T23:01:16.147564+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=16, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.666 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.666 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.666 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.666 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.666 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.666 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.666 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.666 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.666 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.667 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.667 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.667 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-05-11T23:01:26.442571+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=1, tm_sec=26, tm_wday=5, tm_yday=132, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=11, tm_hour=23, tm_min=4, tm_sec=46, tm_wday=5, tm_yday=132, tm_isdst=0) 2024-05-11T23:04:51.667 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-11T23:05:11.668 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T23:05:11.982 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-05-11T23:05:11.983 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-05-11T23:05:11.994 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-05-11T23:05:11.180371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":1467,"num_object_clones":0,"num_object_copies":2934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1467,"num_whiteouts":1461,"num_read":339547,"num_read_kb":1023467,"num_write":28662,"num_write_kb":501200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8674,"num_bytes_recovered":383662413,"num_keys_recovered":951,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28151,"ondisk_log_size":28151,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":484280,"kb_used_data":18552,"kb_used_omap":60,"kb_used_meta":465603,"kb_avail":754490440,"statfs":{"total":773094113280,"available":772598210560,"internally_reserved":0,"allocated":18997248,"data_stored":10357278,"data_compressed":66655,"data_compressed_allocated":90112,"data_compressed_original":1163264,"omap_allocated":62250,"internal_metadata":476777686},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002950"},"pg_stats":[{"pgid":"3.1f","version":"196'586","reported_seq":5629,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:51.679248+0000","last_change":"2024-05-11T23:04:51.679311+0000","last_active":"2024-05-11T23:04:51.679248+0000","last_peered":"2024-05-11T23:04:51.679248+0000","last_clean":"2024-05-11T23:04:51.679248+0000","last_became_active":"2024-05-11T23:04:28.804255+0000","last_became_peered":"2024-05-11T23:04:28.804255+0000","last_unstale":"2024-05-11T23:04:51.679248+0000","last_undegraded":"2024-05-11T23:04:51.679248+0000","last_fullsized":"2024-05-11T23:04:51.679248+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'586","last_scrub_stamp":"2024-05-11T23:04:51.679156+0000","last_deep_scrub":"196'586","last_deep_scrub_stamp":"2024-05-11T23:04:51.679156+0000","last_clean_scrub_stamp":"2024-05-11T23:04:51.679156+0000","objects_scrubbed":46,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:08.305367+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0.00098957899999999998,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":5393,"num_read_kb":14723,"num_write":488,"num_write_kb":18081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":877932,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"196'985","reported_seq":6605,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:51.088863+0000","last_change":"2024-05-11T23:04:51.088887+0000","last_active":"2024-05-11T23:04:51.088863+0000","last_peered":"2024-05-11T23:04:51.088863+0000","last_clean":"2024-05-11T23:04:51.088863+0000","last_became_active":"2024-05-11T23:04:25.808413+0000","last_became_peered":"2024-05-11T23:04:25.808413+0000","last_unstale":"2024-05-11T23:04:51.088863+0000","last_undegraded":"2024-05-11T23:04:51.088863+0000","last_fullsized":"2024-05-11T23:04:51.088863+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'985","last_scrub_stamp":"2024-05-11T23:04:51.088796+0000","last_deep_scrub":"196'985","last_deep_scrub_stamp":"2024-05-11T23:04:51.088796+0000","last_clean_scrub_stamp":"2024-05-11T23:04:51.088796+0000","objects_scrubbed":51,"log_size":985,"log_dups_size":0,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T09:32:32.250500+0000","scrub_duration":31,"objects_trimmed":36,"snaptrim_duration":0.084167985000000001,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":6916,"num_read_kb":49448,"num_write":971,"num_write_kb":22687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":53290749,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"196'558","reported_seq":4591,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:54.400824+0000","last_change":"2024-05-11T23:04:54.400847+0000","last_active":"2024-05-11T23:04:54.400824+0000","last_peered":"2024-05-11T23:04:54.400824+0000","last_clean":"2024-05-11T23:04:54.400824+0000","last_became_active":"2024-05-11T23:03:48.889806+0000","last_became_peered":"2024-05-11T23:03:48.889806+0000","last_unstale":"2024-05-11T23:04:54.400824+0000","last_undegraded":"2024-05-11T23:04:54.400824+0000","last_fullsized":"2024-05-11T23:04:54.400824+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'558","last_scrub_stamp":"2024-05-11T23:04:54.400750+0000","last_deep_scrub":"196'558","last_deep_scrub_stamp":"2024-05-11T23:04:54.400750+0000","last_clean_scrub_stamp":"2024-05-11T23:04:54.400750+0000","objects_scrubbed":50,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:55:53.850946+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0.00060389999999999999,"stat_sum":{"num_bytes":0,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":50,"num_read":3180,"num_read_kb":16428,"num_write":406,"num_write_kb":9629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":7067368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"196'848","reported_seq":19235,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:55.386461+0000","last_change":"2024-05-11T23:04:55.386497+0000","last_active":"2024-05-11T23:04:55.386461+0000","last_peered":"2024-05-11T23:04:55.386461+0000","last_clean":"2024-05-11T23:04:55.386461+0000","last_became_active":"2024-05-11T23:03:50.798632+0000","last_became_peered":"2024-05-11T23:03:50.798632+0000","last_unstale":"2024-05-11T23:04:55.386461+0000","last_undegraded":"2024-05-11T23:04:55.386461+0000","last_fullsized":"2024-05-11T23:04:55.386461+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'848","last_scrub_stamp":"2024-05-11T23:04:55.386385+0000","last_deep_scrub":"196'848","last_deep_scrub_stamp":"2024-05-11T23:04:55.386385+0000","last_clean_scrub_stamp":"2024-05-11T23:04:55.386385+0000","objects_scrubbed":64,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:29:17.841536+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0.001248566,"stat_sum":{"num_bytes":0,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":63,"num_read":18429,"num_read_kb":39778,"num_write":674,"num_write_kb":10510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":9396782,"num_keys_recovered":182,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"196'639","reported_seq":15168,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:53.702006+0000","last_change":"2024-05-11T23:04:53.702061+0000","last_active":"2024-05-11T23:04:53.702006+0000","last_peered":"2024-05-11T23:04:53.702006+0000","last_clean":"2024-05-11T23:04:53.702006+0000","last_became_active":"2024-05-11T23:04:29.804143+0000","last_became_peered":"2024-05-11T23:04:29.804143+0000","last_unstale":"2024-05-11T23:04:53.702006+0000","last_undegraded":"2024-05-11T23:04:53.702006+0000","last_fullsized":"2024-05-11T23:04:53.702006+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'639","last_scrub_stamp":"2024-05-11T23:04:53.701919+0000","last_deep_scrub":"196'639","last_deep_scrub_stamp":"2024-05-11T23:04:53.701919+0000","last_clean_scrub_stamp":"2024-05-11T23:04:53.701919+0000","objects_scrubbed":45,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:43:34.812181+0000","scrub_duration":28,"objects_trimmed":2,"snaptrim_duration":0.001388731,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":13979,"num_read_kb":68609,"num_write":598,"num_write_kb":35793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":532,"num_bytes_recovered":20898869,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"196'1705","reported_seq":21096,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:52.800412+0000","last_change":"2024-05-11T23:04:52.800453+0000","last_active":"2024-05-11T23:04:52.800412+0000","last_peered":"2024-05-11T23:04:52.800412+0000","last_clean":"2024-05-11T23:04:52.800412+0000","last_became_active":"2024-05-11T23:04:34.007358+0000","last_became_peered":"2024-05-11T23:04:34.007358+0000","last_unstale":"2024-05-11T23:04:52.800412+0000","last_undegraded":"2024-05-11T23:04:52.800412+0000","last_fullsized":"2024-05-11T23:04:52.800412+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'1705","last_scrub_stamp":"2024-05-11T23:04:52.800332+0000","last_deep_scrub":"196'1705","last_deep_scrub_stamp":"2024-05-11T23:04:52.800332+0000","last_clean_scrub_stamp":"2024-05-11T23:04:52.800332+0000","objects_scrubbed":41,"log_size":1705,"log_dups_size":0,"ondisk_log_size":1705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:50:06.369448+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.00097887899999999999,"stat_sum":{"num_bytes":19,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":40,"num_read":25336,"num_read_kb":48131,"num_write":2723,"num_write_kb":23444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":6434587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"196'1097","reported_seq":15442,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:50.171848+0000","last_change":"2024-05-11T23:04:50.171963+0000","last_active":"2024-05-11T23:04:50.171848+0000","last_peered":"2024-05-11T23:04:50.171848+0000","last_clean":"2024-05-11T23:04:50.171848+0000","last_became_active":"2024-05-11T23:04:25.807274+0000","last_became_peered":"2024-05-11T23:04:25.807274+0000","last_unstale":"2024-05-11T23:04:50.171848+0000","last_undegraded":"2024-05-11T23:04:50.171848+0000","last_fullsized":"2024-05-11T23:04:50.171848+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'1097","last_scrub_stamp":"2024-05-11T23:04:50.171727+0000","last_deep_scrub":"196'1097","last_deep_scrub_stamp":"2024-05-11T23:04:50.171727+0000","last_clean_scrub_stamp":"2024-05-11T23:04:50.171727+0000","objects_scrubbed":45,"log_size":1097,"log_dups_size":0,"ondisk_log_size":1097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:38:40.135814+0000","scrub_duration":34,"objects_trimmed":30,"snaptrim_duration":0.061503582000000001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":16282,"num_read_kb":43182,"num_write":1369,"num_write_kb":14705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":13589947,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"196'1007","reported_seq":4032,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:52.065297+0000","last_change":"2024-05-11T23:04:52.065337+0000","last_active":"2024-05-11T23:04:52.065297+0000","last_peered":"2024-05-11T23:04:52.065297+0000","last_clean":"2024-05-11T23:04:52.065297+0000","last_became_active":"2024-05-11T23:04:26.807825+0000","last_became_peered":"2024-05-11T23:04:26.807825+0000","last_unstale":"2024-05-11T23:04:52.065297+0000","last_undegraded":"2024-05-11T23:04:52.065297+0000","last_fullsized":"2024-05-11T23:04:52.065297+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'1007","last_scrub_stamp":"2024-05-11T23:04:52.065200+0000","last_deep_scrub":"196'1007","last_deep_scrub_stamp":"2024-05-11T23:04:52.065200+0000","last_clean_scrub_stamp":"2024-05-11T23:04:52.065200+0000","objects_scrubbed":40,"log_size":1007,"log_dups_size":0,"ondisk_log_size":1007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T09:04:05.514902+0000","scrub_duration":29,"objects_trimmed":2,"snaptrim_duration":0.001621796,"stat_sum":{"num_bytes":0,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":40,"num_read":5754,"num_read_kb":9891,"num_write":1117,"num_write_kb":16486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":7487386,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"196'709","reported_seq":3923,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:49.625606+0000","last_change":"2024-05-11T23:04:49.625707+0000","last_active":"2024-05-11T23:04:49.625606+0000","last_peered":"2024-05-11T23:04:49.625606+0000","last_clean":"2024-05-11T23:04:49.625606+0000","last_became_active":"2024-05-11T23:04:27.809321+0000","last_became_peered":"2024-05-11T23:04:27.809321+0000","last_unstale":"2024-05-11T23:04:49.625606+0000","last_undegraded":"2024-05-11T23:04:49.625606+0000","last_fullsized":"2024-05-11T23:04:49.625606+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'709","last_scrub_stamp":"2024-05-11T23:04:49.625504+0000","last_deep_scrub":"196'709","last_deep_scrub_stamp":"2024-05-11T23:04:49.625504+0000","last_clean_scrub_stamp":"2024-05-11T23:04:49.625504+0000","objects_scrubbed":42,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:53:33.656821+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0.0013760389999999999,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":3972,"num_read_kb":15246,"num_write":808,"num_write_kb":6422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"196'607","reported_seq":15748,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:53.396652+0000","last_change":"2024-05-11T23:04:53.396704+0000","last_active":"2024-05-11T23:04:53.396652+0000","last_peered":"2024-05-11T23:04:53.396652+0000","last_clean":"2024-05-11T23:04:53.396652+0000","last_became_active":"2024-05-11T23:01:27.463623+0000","last_became_peered":"2024-05-11T23:01:27.463623+0000","last_unstale":"2024-05-11T23:04:53.396652+0000","last_undegraded":"2024-05-11T23:04:53.396652+0000","last_fullsized":"2024-05-11T23:04:53.396652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'607","last_scrub_stamp":"2024-05-11T23:04:53.396578+0000","last_deep_scrub":"196'607","last_deep_scrub_stamp":"2024-05-11T23:04:53.396578+0000","last_clean_scrub_stamp":"2024-05-11T23:04:53.396578+0000","objects_scrubbed":48,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:24:29.585597+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.00054116699999999995,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":15017,"num_read_kb":54784,"num_write":481,"num_write_kb":18602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"196'678","reported_seq":16480,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:59.024371+0000","last_change":"2024-05-11T23:04:59.024406+0000","last_active":"2024-05-11T23:04:59.024371+0000","last_peered":"2024-05-11T23:04:59.024371+0000","last_clean":"2024-05-11T23:04:59.024371+0000","last_became_active":"2024-05-11T23:04:25.807935+0000","last_became_peered":"2024-05-11T23:04:25.807935+0000","last_unstale":"2024-05-11T23:04:59.024371+0000","last_undegraded":"2024-05-11T23:04:59.024371+0000","last_fullsized":"2024-05-11T23:04:59.024371+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'678","last_scrub_stamp":"2024-05-11T23:04:59.024291+0000","last_deep_scrub":"196'678","last_deep_scrub_stamp":"2024-05-11T23:04:59.024291+0000","last_clean_scrub_stamp":"2024-05-11T23:04:59.024291+0000","objects_scrubbed":56,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:24:14.569380+0000","scrub_duration":42,"objects_trimmed":36,"snaptrim_duration":0.072194399000000006,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":14264,"num_read_kb":25298,"num_write":546,"num_write_kb":13154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":133,"num_bytes_recovered":12363304,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"196'699","reported_seq":19632,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:51.836562+0000","last_change":"2024-05-11T23:04:51.836608+0000","last_active":"2024-05-11T23:04:51.836562+0000","last_peered":"2024-05-11T23:04:51.836562+0000","last_clean":"2024-05-11T23:04:51.836562+0000","last_became_active":"2024-05-11T23:04:34.007861+0000","last_became_peered":"2024-05-11T23:04:34.007861+0000","last_unstale":"2024-05-11T23:04:51.836562+0000","last_undegraded":"2024-05-11T23:04:51.836562+0000","last_fullsized":"2024-05-11T23:04:51.836562+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'699","last_scrub_stamp":"2024-05-11T23:04:51.836477+0000","last_deep_scrub":"196'699","last_deep_scrub_stamp":"2024-05-11T23:04:51.836477+0000","last_clean_scrub_stamp":"2024-05-11T23:04:51.836477+0000","objects_scrubbed":38,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:40:15.986796+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.001389319,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":19402,"num_read_kb":42118,"num_write":569,"num_write_kb":6424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":4215359,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":243,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:50.976084+0000","last_change":"2024-05-11T23:04:50.976084+0000","last_active":"2024-05-11T23:04:50.976084+0000","last_peered":"2024-05-11T23:04:50.976084+0000","last_clean":"2024-05-11T23:04:50.976084+0000","last_became_active":"2024-05-11T23:02:39.044216+0000","last_became_peered":"2024-05-11T23:02:39.044216+0000","last_unstale":"2024-05-11T23:04:50.976084+0000","last_undegraded":"2024-05-11T23:04:50.976084+0000","last_fullsized":"2024-05-11T23:04:50.976084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:04:50.975972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:04:50.975972+0000","last_clean_scrub_stamp":"2024-05-11T23:04:50.975972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:53:50.988739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"196'1109","reported_seq":7517,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:52.739584+0000","last_change":"2024-05-11T23:04:52.739621+0000","last_active":"2024-05-11T23:04:52.739584+0000","last_peered":"2024-05-11T23:04:52.739584+0000","last_clean":"2024-05-11T23:04:52.739584+0000","last_became_active":"2024-05-11T23:04:28.805428+0000","last_became_peered":"2024-05-11T23:04:28.805428+0000","last_unstale":"2024-05-11T23:04:52.739584+0000","last_undegraded":"2024-05-11T23:04:52.739584+0000","last_fullsized":"2024-05-11T23:04:52.739584+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'1109","last_scrub_stamp":"2024-05-11T23:04:52.739500+0000","last_deep_scrub":"196'1109","last_deep_scrub_stamp":"2024-05-11T23:04:52.739500+0000","last_clean_scrub_stamp":"2024-05-11T23:04:52.739500+0000","objects_scrubbed":60,"log_size":1109,"log_dups_size":0,"ondisk_log_size":1109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:28:14.635226+0000","scrub_duration":52,"objects_trimmed":0,"snaptrim_duration":0.002036403,"stat_sum":{"num_bytes":0,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":60,"num_read":9703,"num_read_kb":23103,"num_write":1155,"num_write_kb":12619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":2180842,"num_keys_recovered":21,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":346,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:52.150285+0000","last_change":"2024-05-11T23:04:52.150308+0000","last_active":"2024-05-11T23:04:52.150285+0000","last_peered":"2024-05-11T23:04:52.150285+0000","last_clean":"2024-05-11T23:04:52.150285+0000","last_became_active":"2024-05-11T23:04:29.805946+0000","last_became_peered":"2024-05-11T23:04:29.805946+0000","last_unstale":"2024-05-11T23:04:52.150285+0000","last_undegraded":"2024-05-11T23:04:52.150285+0000","last_fullsized":"2024-05-11T23:04:52.150285+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:04:52.150204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:04:52.150204+0000","last_clean_scrub_stamp":"2024-05-11T23:04:52.150204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:28:35.232464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"196'973","reported_seq":6136,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:53.110638+0000","last_change":"2024-05-11T23:04:53.110696+0000","last_active":"2024-05-11T23:04:53.110638+0000","last_peered":"2024-05-11T23:04:53.110638+0000","last_clean":"2024-05-11T23:04:53.110638+0000","last_became_active":"2024-05-11T23:04:42.060396+0000","last_became_peered":"2024-05-11T23:04:42.060396+0000","last_unstale":"2024-05-11T23:04:53.110638+0000","last_undegraded":"2024-05-11T23:04:53.110638+0000","last_fullsized":"2024-05-11T23:04:53.110638+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'973","last_scrub_stamp":"2024-05-11T23:04:53.110558+0000","last_deep_scrub":"196'973","last_deep_scrub_stamp":"2024-05-11T23:04:53.110558+0000","last_clean_scrub_stamp":"2024-05-11T23:04:53.110558+0000","objects_scrubbed":47,"log_size":973,"log_dups_size":0,"ondisk_log_size":973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:02:56.118587+0000","scrub_duration":37,"objects_trimmed":24,"snaptrim_duration":0.055140898000000001,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":7310,"num_read_kb":19251,"num_write":1088,"num_write_kb":7185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":4415701,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":385,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:57.979089+0000","last_change":"2024-05-11T23:04:57.979115+0000","last_active":"2024-05-11T23:04:57.979089+0000","last_peered":"2024-05-11T23:04:57.979089+0000","last_clean":"2024-05-11T23:04:57.979089+0000","last_became_active":"2024-05-11T23:04:25.806292+0000","last_became_peered":"2024-05-11T23:04:25.806292+0000","last_unstale":"2024-05-11T23:04:57.979089+0000","last_undegraded":"2024-05-11T23:04:57.979089+0000","last_fullsized":"2024-05-11T23:04:57.979089+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:04:57.978998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:04:57.978998+0000","last_clean_scrub_stamp":"2024-05-11T23:04:57.978998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:17:50.981337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"196'605","reported_seq":14959,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:53.220996+0000","last_change":"2024-05-11T23:04:53.221054+0000","last_active":"2024-05-11T23:04:53.220996+0000","last_peered":"2024-05-11T23:04:53.220996+0000","last_clean":"2024-05-11T23:04:53.220996+0000","last_became_active":"2024-05-11T23:04:29.807343+0000","last_became_peered":"2024-05-11T23:04:29.807343+0000","last_unstale":"2024-05-11T23:04:53.220996+0000","last_undegraded":"2024-05-11T23:04:53.220996+0000","last_fullsized":"2024-05-11T23:04:53.220996+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'605","last_scrub_stamp":"2024-05-11T23:04:53.220916+0000","last_deep_scrub":"196'605","last_deep_scrub_stamp":"2024-05-11T23:04:53.220916+0000","last_clean_scrub_stamp":"2024-05-11T23:04:53.220916+0000","objects_scrubbed":41,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:50:26.295929+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0.00095379299999999998,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":14156,"num_read_kb":53683,"num_write":532,"num_write_kb":33731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":350,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:51.154220+0000","last_change":"2024-05-11T23:04:51.154241+0000","last_active":"2024-05-11T23:04:51.154220+0000","last_peered":"2024-05-11T23:04:51.154220+0000","last_clean":"2024-05-11T23:04:51.154220+0000","last_became_active":"2024-05-11T23:04:29.805447+0000","last_became_peered":"2024-05-11T23:04:29.805447+0000","last_unstale":"2024-05-11T23:04:51.154220+0000","last_undegraded":"2024-05-11T23:04:51.154220+0000","last_fullsized":"2024-05-11T23:04:51.154220+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:04:51.154140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:04:51.154140+0000","last_clean_scrub_stamp":"2024-05-11T23:04:51.154140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T05:04:21.122468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"196'796","reported_seq":6360,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:56.360638+0000","last_change":"2024-05-11T23:04:56.360707+0000","last_active":"2024-05-11T23:04:56.360638+0000","last_peered":"2024-05-11T23:04:56.360638+0000","last_clean":"2024-05-11T23:04:56.360638+0000","last_became_active":"2024-05-11T23:04:25.806236+0000","last_became_peered":"2024-05-11T23:04:25.806236+0000","last_unstale":"2024-05-11T23:04:56.360638+0000","last_undegraded":"2024-05-11T23:04:56.360638+0000","last_fullsized":"2024-05-11T23:04:56.360638+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'796","last_scrub_stamp":"2024-05-11T23:04:56.360532+0000","last_deep_scrub":"196'796","last_deep_scrub_stamp":"2024-05-11T23:04:56.360532+0000","last_clean_scrub_stamp":"2024-05-11T23:04:56.360532+0000","objects_scrubbed":38,"log_size":796,"log_dups_size":0,"ondisk_log_size":796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T06:55:32.799463+0000","scrub_duration":27,"objects_trimmed":38,"snaptrim_duration":0.081084693999999999,"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":6715,"num_read_kb":25684,"num_write":652,"num_write_kb":13121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":13712581,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"196'1079","reported_seq":11315,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:53.800894+0000","last_change":"2024-05-11T23:04:53.800921+0000","last_active":"2024-05-11T23:04:53.800894+0000","last_peered":"2024-05-11T23:04:53.800894+0000","last_clean":"2024-05-11T23:04:53.800894+0000","last_became_active":"2024-05-11T23:04:34.008456+0000","last_became_peered":"2024-05-11T23:04:34.008456+0000","last_unstale":"2024-05-11T23:04:53.800894+0000","last_undegraded":"2024-05-11T23:04:53.800894+0000","last_fullsized":"2024-05-11T23:04:53.800894+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'1079","last_scrub_stamp":"2024-05-11T23:04:53.800828+0000","last_deep_scrub":"196'1079","last_deep_scrub_stamp":"2024-05-11T23:04:53.800828+0000","last_clean_scrub_stamp":"2024-05-11T23:04:53.800828+0000","objects_scrubbed":57,"log_size":1079,"log_dups_size":0,"ondisk_log_size":1079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:25:10.795627+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0.0017329069999999999,"stat_sum":{"num_bytes":0,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":56,"num_read":10698,"num_read_kb":33938,"num_write":973,"num_write_kb":17254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":898,"num_bytes_recovered":16208786,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":377,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:52.331146+0000","last_change":"2024-05-11T23:04:52.331180+0000","last_active":"2024-05-11T23:04:52.331146+0000","last_peered":"2024-05-11T23:04:52.331146+0000","last_clean":"2024-05-11T23:04:52.331146+0000","last_became_active":"2024-05-11T23:01:19.220855+0000","last_became_peered":"2024-05-11T23:01:19.220855+0000","last_unstale":"2024-05-11T23:04:52.331146+0000","last_undegraded":"2024-05-11T23:04:52.331146+0000","last_fullsized":"2024-05-11T23:04:52.331146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:04:52.331009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:04:52.331009+0000","last_clean_scrub_stamp":"2024-05-11T23:04:52.331009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:23:00.341295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"196'1189","reported_seq":6396,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:49.151090+0000","last_change":"2024-05-11T23:04:49.151145+0000","last_active":"2024-05-11T23:04:49.151090+0000","last_peered":"2024-05-11T23:04:49.151090+0000","last_clean":"2024-05-11T23:04:49.151090+0000","last_became_active":"2024-05-11T23:01:27.459593+0000","last_became_peered":"2024-05-11T23:01:27.459593+0000","last_unstale":"2024-05-11T23:04:49.151090+0000","last_undegraded":"2024-05-11T23:04:49.151090+0000","last_fullsized":"2024-05-11T23:04:49.151090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'1189","last_scrub_stamp":"2024-05-11T23:04:49.151006+0000","last_deep_scrub":"196'1189","last_deep_scrub_stamp":"2024-05-11T23:04:49.151006+0000","last_clean_scrub_stamp":"2024-05-11T23:04:49.151006+0000","objects_scrubbed":41,"log_size":1189,"log_dups_size":0,"ondisk_log_size":1189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:15:58.807439+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0.0017756580000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9995,"num_read_kb":18445,"num_write":1457,"num_write_kb":4408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":395,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:49.642963+0000","last_change":"2024-05-11T23:04:49.642984+0000","last_active":"2024-05-11T23:04:49.642963+0000","last_peered":"2024-05-11T23:04:49.642963+0000","last_clean":"2024-05-11T23:04:49.642963+0000","last_became_active":"2024-05-11T23:04:25.804674+0000","last_became_peered":"2024-05-11T23:04:25.804674+0000","last_unstale":"2024-05-11T23:04:49.642963+0000","last_undegraded":"2024-05-11T23:04:49.642963+0000","last_fullsized":"2024-05-11T23:04:49.642963+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:04:49.642885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:04:49.642885+0000","last_clean_scrub_stamp":"2024-05-11T23:04:49.642885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:58:15.177305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"196'700","reported_seq":15368,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:48.609109+0000","last_change":"2024-05-11T23:04:48.609182+0000","last_active":"2024-05-11T23:04:48.609109+0000","last_peered":"2024-05-11T23:04:48.609109+0000","last_clean":"2024-05-11T23:04:48.609109+0000","last_became_active":"2024-05-11T23:04:26.804248+0000","last_became_peered":"2024-05-11T23:04:26.804248+0000","last_unstale":"2024-05-11T23:04:48.609109+0000","last_undegraded":"2024-05-11T23:04:48.609109+0000","last_fullsized":"2024-05-11T23:04:48.609109+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'700","last_scrub_stamp":"2024-05-11T23:04:48.608991+0000","last_deep_scrub":"196'700","last_deep_scrub_stamp":"2024-05-11T23:04:48.608991+0000","last_clean_scrub_stamp":"2024-05-11T23:04:48.608991+0000","objects_scrubbed":44,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T09:28:40.682780+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0.0014815119999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":15309,"num_read_kb":24676,"num_write":622,"num_write_kb":6352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":299,"num_bytes_recovered":6345105,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":375,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:51.946802+0000","last_change":"2024-05-11T23:04:51.946840+0000","last_active":"2024-05-11T23:04:51.946802+0000","last_peered":"2024-05-11T23:04:51.946802+0000","last_clean":"2024-05-11T23:04:51.946802+0000","last_became_active":"2024-05-11T23:01:19.219471+0000","last_became_peered":"2024-05-11T23:01:19.219471+0000","last_unstale":"2024-05-11T23:04:51.946802+0000","last_undegraded":"2024-05-11T23:04:51.946802+0000","last_fullsized":"2024-05-11T23:04:51.946802+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:04:51.946714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:04:51.946714+0000","last_clean_scrub_stamp":"2024-05-11T23:04:51.946714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:02:12.602945+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"196'738","reported_seq":4360,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:54.112461+0000","last_change":"2024-05-11T23:04:54.112524+0000","last_active":"2024-05-11T23:04:54.112461+0000","last_peered":"2024-05-11T23:04:54.112461+0000","last_clean":"2024-05-11T23:04:54.112461+0000","last_became_active":"2024-05-11T23:04:42.060595+0000","last_became_peered":"2024-05-11T23:04:42.060595+0000","last_unstale":"2024-05-11T23:04:54.112461+0000","last_undegraded":"2024-05-11T23:04:54.112461+0000","last_fullsized":"2024-05-11T23:04:54.112461+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'738","last_scrub_stamp":"2024-05-11T23:04:54.112379+0000","last_deep_scrub":"196'738","last_deep_scrub_stamp":"2024-05-11T23:04:54.112379+0000","last_clean_scrub_stamp":"2024-05-11T23:04:54.112379+0000","objects_scrubbed":48,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:03:53.425062+0000","scrub_duration":35,"objects_trimmed":28,"snaptrim_duration":0.040069711000000001,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":3497,"num_read_kb":16044,"num_write":640,"num_write_kb":10409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":10719266,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":377,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:51.337214+0000","last_change":"2024-05-11T23:04:51.337274+0000","last_active":"2024-05-11T23:04:51.337214+0000","last_peered":"2024-05-11T23:04:51.337214+0000","last_clean":"2024-05-11T23:04:51.337214+0000","last_became_active":"2024-05-11T23:01:19.218846+0000","last_became_peered":"2024-05-11T23:01:19.218846+0000","last_unstale":"2024-05-11T23:04:51.337214+0000","last_undegraded":"2024-05-11T23:04:51.337214+0000","last_fullsized":"2024-05-11T23:04:51.337214+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T23:04:51.337103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T23:04:51.337103+0000","last_clean_scrub_stamp":"2024-05-11T23:04:51.337103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:52:12.387035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"196'119","reported_seq":555,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:59.970128+0000","last_change":"2024-05-11T23:04:59.970192+0000","last_active":"2024-05-11T23:04:59.970128+0000","last_peered":"2024-05-11T23:04:59.970128+0000","last_clean":"2024-05-11T23:04:59.970128+0000","last_became_active":"2024-05-11T23:04:28.806124+0000","last_became_peered":"2024-05-11T23:04:28.806124+0000","last_unstale":"2024-05-11T23:04:59.970128+0000","last_undegraded":"2024-05-11T23:04:59.970128+0000","last_fullsized":"2024-05-11T23:04:59.970128+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'119","last_scrub_stamp":"2024-05-11T23:04:59.970027+0000","last_deep_scrub":"196'119","last_deep_scrub_stamp":"2024-05-11T23:04:59.970027+0000","last_clean_scrub_stamp":"2024-05-11T23:04:59.970027+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:19:50.749197+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"196'1013","reported_seq":5611,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:50.129178+0000","last_change":"2024-05-11T23:04:50.129260+0000","last_active":"2024-05-11T23:04:50.129178+0000","last_peered":"2024-05-11T23:04:50.129178+0000","last_clean":"2024-05-11T23:04:50.129178+0000","last_became_active":"2024-05-11T23:04:25.802390+0000","last_became_peered":"2024-05-11T23:04:25.802390+0000","last_unstale":"2024-05-11T23:04:50.129178+0000","last_undegraded":"2024-05-11T23:04:50.129178+0000","last_fullsized":"2024-05-11T23:04:50.129178+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'1013","last_scrub_stamp":"2024-05-11T23:04:50.129078+0000","last_deep_scrub":"196'1013","last_deep_scrub_stamp":"2024-05-11T23:04:50.129078+0000","last_clean_scrub_stamp":"2024-05-11T23:04:50.129078+0000","objects_scrubbed":47,"log_size":1013,"log_dups_size":0,"ondisk_log_size":1013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:31:35.493127+0000","scrub_duration":35,"objects_trimmed":24,"snaptrim_duration":0.057229434000000003,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":6339,"num_read_kb":20167,"num_write":1151,"num_write_kb":10790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":4220183,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"196'725","reported_seq":9228,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:50.864238+0000","last_change":"2024-05-11T23:04:50.864294+0000","last_active":"2024-05-11T23:04:50.864238+0000","last_peered":"2024-05-11T23:04:50.864238+0000","last_clean":"2024-05-11T23:04:50.864238+0000","last_became_active":"2024-05-11T23:04:34.005569+0000","last_became_peered":"2024-05-11T23:04:34.005569+0000","last_unstale":"2024-05-11T23:04:50.864238+0000","last_undegraded":"2024-05-11T23:04:50.864238+0000","last_fullsized":"2024-05-11T23:04:50.864238+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'725","last_scrub_stamp":"2024-05-11T23:04:50.864145+0000","last_deep_scrub":"196'725","last_deep_scrub_stamp":"2024-05-11T23:04:50.864145+0000","last_clean_scrub_stamp":"2024-05-11T23:04:50.864145+0000","objects_scrubbed":49,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:44:32.944493+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0.00064185299999999995,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":9043,"num_read_kb":47546,"num_write":541,"num_write_kb":29805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":37320801,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"196'830","reported_seq":6714,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:52.034133+0000","last_change":"2024-05-11T23:04:52.034133+0000","last_active":"2024-05-11T23:04:52.034133+0000","last_peered":"2024-05-11T23:04:52.034133+0000","last_clean":"2024-05-11T23:04:52.034133+0000","last_became_active":"2024-05-11T23:03:51.797214+0000","last_became_peered":"2024-05-11T23:03:51.797214+0000","last_unstale":"2024-05-11T23:04:52.034133+0000","last_undegraded":"2024-05-11T23:04:52.034133+0000","last_fullsized":"2024-05-11T23:04:52.034133+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'830","last_scrub_stamp":"2024-05-11T23:04:52.033994+0000","last_deep_scrub":"196'830","last_deep_scrub_stamp":"2024-05-11T23:04:52.033994+0000","last_clean_scrub_stamp":"2024-05-11T23:04:52.033994+0000","objects_scrubbed":45,"log_size":830,"log_dups_size":0,"ondisk_log_size":830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:53:01.219925+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0.001270001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":44,"num_read":5814,"num_read_kb":22550,"num_write":685,"num_write_kb":9625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":538,"num_bytes_recovered":16997094,"num_keys_recovered":140,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"196'692","reported_seq":9990,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:55.134689+0000","last_change":"2024-05-11T23:04:55.134758+0000","last_active":"2024-05-11T23:04:55.134689+0000","last_peered":"2024-05-11T23:04:55.134689+0000","last_clean":"2024-05-11T23:04:55.134689+0000","last_became_active":"2024-05-11T23:04:42.073145+0000","last_became_peered":"2024-05-11T23:04:42.073145+0000","last_unstale":"2024-05-11T23:04:55.134689+0000","last_undegraded":"2024-05-11T23:04:55.134689+0000","last_fullsized":"2024-05-11T23:04:55.134689+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'692","last_scrub_stamp":"2024-05-11T23:04:55.134587+0000","last_deep_scrub":"196'692","last_deep_scrub_stamp":"2024-05-11T23:04:55.134587+0000","last_clean_scrub_stamp":"2024-05-11T23:04:55.134587+0000","objects_scrubbed":35,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T03:02:21.154848+0000","scrub_duration":28,"objects_trimmed":2,"snaptrim_duration":0.00088000000000000003,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":9829,"num_read_kb":52232,"num_write":599,"num_write_kb":42645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":340,"num_bytes_recovered":47434023,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"196'735","reported_seq":5267,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:57.340955+0000","last_change":"2024-05-11T23:04:57.340993+0000","last_active":"2024-05-11T23:04:57.340955+0000","last_peered":"2024-05-11T23:04:57.340955+0000","last_clean":"2024-05-11T23:04:57.340955+0000","last_became_active":"2024-05-11T23:04:25.806852+0000","last_became_peered":"2024-05-11T23:04:25.806852+0000","last_unstale":"2024-05-11T23:04:57.340955+0000","last_undegraded":"2024-05-11T23:04:57.340955+0000","last_fullsized":"2024-05-11T23:04:57.340955+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'735","last_scrub_stamp":"2024-05-11T23:04:57.340861+0000","last_deep_scrub":"196'735","last_deep_scrub_stamp":"2024-05-11T23:04:57.340861+0000","last_clean_scrub_stamp":"2024-05-11T23:04:57.340861+0000","objects_scrubbed":41,"log_size":735,"log_dups_size":0,"ondisk_log_size":735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T08:11:15.263570+0000","scrub_duration":29,"objects_trimmed":20,"snaptrim_duration":0.060182605,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":6324,"num_read_kb":25218,"num_write":659,"num_write_kb":11709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":10615766,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"196'936","reported_seq":21551,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:53.017649+0000","last_change":"2024-05-11T23:04:53.017702+0000","last_active":"2024-05-11T23:04:53.017649+0000","last_peered":"2024-05-11T23:04:53.017649+0000","last_clean":"2024-05-11T23:04:53.017649+0000","last_became_active":"2024-05-11T23:01:27.465570+0000","last_became_peered":"2024-05-11T23:01:27.465570+0000","last_unstale":"2024-05-11T23:04:53.017649+0000","last_undegraded":"2024-05-11T23:04:53.017649+0000","last_fullsized":"2024-05-11T23:04:53.017649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'936","last_scrub_stamp":"2024-05-11T23:04:53.017563+0000","last_deep_scrub":"196'936","last_deep_scrub_stamp":"2024-05-11T23:04:53.017563+0000","last_clean_scrub_stamp":"2024-05-11T23:04:53.017563+0000","objects_scrubbed":39,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T10:58:12.535949+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0.003620968,"stat_sum":{"num_bytes":0,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":39,"num_read":22252,"num_read_kb":40862,"num_write":875,"num_write_kb":12354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"196'1354","reported_seq":14506,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:55.024011+0000","last_change":"2024-05-11T23:04:55.024062+0000","last_active":"2024-05-11T23:04:55.024011+0000","last_peered":"2024-05-11T23:04:55.024011+0000","last_clean":"2024-05-11T23:04:55.024011+0000","last_became_active":"2024-05-11T23:03:52.804360+0000","last_became_peered":"2024-05-11T23:03:52.804360+0000","last_unstale":"2024-05-11T23:04:55.024011+0000","last_undegraded":"2024-05-11T23:04:55.024011+0000","last_fullsized":"2024-05-11T23:04:55.024011+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'1354","last_scrub_stamp":"2024-05-11T23:04:55.023942+0000","last_deep_scrub":"196'1354","last_deep_scrub_stamp":"2024-05-11T23:04:55.023942+0000","last_clean_scrub_stamp":"2024-05-11T23:04:55.023942+0000","objects_scrubbed":46,"log_size":1354,"log_dups_size":0,"ondisk_log_size":1354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T01:26:01.511799+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0.0039667269999999998,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":17126,"num_read_kb":45672,"num_write":1555,"num_write_kb":23766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":18760061,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"196'1167","reported_seq":9590,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:54.043168+0000","last_change":"2024-05-11T23:04:54.043212+0000","last_active":"2024-05-11T23:04:54.043168+0000","last_peered":"2024-05-11T23:04:54.043168+0000","last_clean":"2024-05-11T23:04:54.043168+0000","last_became_active":"2024-05-11T23:03:49.895043+0000","last_became_peered":"2024-05-11T23:03:49.895043+0000","last_unstale":"2024-05-11T23:04:54.043168+0000","last_undegraded":"2024-05-11T23:04:54.043168+0000","last_fullsized":"2024-05-11T23:04:54.043168+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'1167","last_scrub_stamp":"2024-05-11T23:04:54.043092+0000","last_deep_scrub":"196'1167","last_deep_scrub_stamp":"2024-05-11T23:04:54.043092+0000","last_clean_scrub_stamp":"2024-05-11T23:04:54.043092+0000","objects_scrubbed":53,"log_size":1167,"log_dups_size":0,"ondisk_log_size":1167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T11:02:30.465979+0000","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0.0040936139999999998,"stat_sum":{"num_bytes":0,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":53,"num_read":11168,"num_read_kb":37882,"num_write":1390,"num_write_kb":15155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":362,"num_bytes_recovered":15895238,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"196'713","reported_seq":7084,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:53.060417+0000","last_change":"2024-05-11T23:04:53.060417+0000","last_active":"2024-05-11T23:04:53.060417+0000","last_peered":"2024-05-11T23:04:53.060417+0000","last_clean":"2024-05-11T23:04:53.060417+0000","last_became_active":"2024-05-11T23:04:28.806050+0000","last_became_peered":"2024-05-11T23:04:28.806050+0000","last_unstale":"2024-05-11T23:04:53.060417+0000","last_undegraded":"2024-05-11T23:04:53.060417+0000","last_fullsized":"2024-05-11T23:04:53.060417+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'713","last_scrub_stamp":"2024-05-11T23:04:53.060297+0000","last_deep_scrub":"196'713","last_deep_scrub_stamp":"2024-05-11T23:04:53.060297+0000","last_clean_scrub_stamp":"2024-05-11T23:04:53.060297+0000","objects_scrubbed":35,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:43:44.539384+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0.0023348000000000002,"stat_sum":{"num_bytes":0,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":35,"num_read":7071,"num_read_kb":24639,"num_write":633,"num_write_kb":13630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":19437463,"num_keys_recovered":69,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"196'895","reported_seq":6400,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:57.032082+0000","last_change":"2024-05-11T23:04:57.032127+0000","last_active":"2024-05-11T23:04:57.032082+0000","last_peered":"2024-05-11T23:04:57.032082+0000","last_clean":"2024-05-11T23:04:57.032082+0000","last_became_active":"2024-05-11T23:03:53.806713+0000","last_became_peered":"2024-05-11T23:03:53.806713+0000","last_unstale":"2024-05-11T23:04:57.032082+0000","last_undegraded":"2024-05-11T23:04:57.032082+0000","last_fullsized":"2024-05-11T23:04:57.032082+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'895","last_scrub_stamp":"2024-05-11T23:04:57.031997+0000","last_deep_scrub":"196'895","last_deep_scrub_stamp":"2024-05-11T23:04:57.031997+0000","last_clean_scrub_stamp":"2024-05-11T23:04:57.031997+0000","objects_scrubbed":42,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T07:33:29.867855+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0.0011357120000000001,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":7833,"num_read_kb":21923,"num_write":862,"num_write_kb":7573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":450,"num_bytes_recovered":18021482,"num_keys_recovered":78,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"196'1017","reported_seq":5358,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:50.642830+0000","last_change":"2024-05-11T23:04:50.642925+0000","last_active":"2024-05-11T23:04:50.642830+0000","last_peered":"2024-05-11T23:04:50.642830+0000","last_clean":"2024-05-11T23:04:50.642830+0000","last_became_active":"2024-05-11T23:04:27.809030+0000","last_became_peered":"2024-05-11T23:04:27.809030+0000","last_unstale":"2024-05-11T23:04:50.642830+0000","last_undegraded":"2024-05-11T23:04:50.642830+0000","last_fullsized":"2024-05-11T23:04:50.642830+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'1017","last_scrub_stamp":"2024-05-11T23:04:50.642751+0000","last_deep_scrub":"196'1017","last_deep_scrub_stamp":"2024-05-11T23:04:50.642751+0000","last_clean_scrub_stamp":"2024-05-11T23:04:50.642751+0000","objects_scrubbed":42,"log_size":1017,"log_dups_size":0,"ondisk_log_size":1017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T04:06:45.983729+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0.0028179780000000001,"stat_sum":{"num_bytes":0,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":42,"num_read":6275,"num_read_kb":24022,"num_write":1130,"num_write_kb":13297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"196'648","reported_seq":5666,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-05-11T23:04:55.988044+0000","last_change":"2024-05-11T23:04:55.988096+0000","last_active":"2024-05-11T23:04:55.988044+0000","last_peered":"2024-05-11T23:04:55.988044+0000","last_clean":"2024-05-11T23:04:55.988044+0000","last_became_active":"2024-05-11T23:03:52.818374+0000","last_became_peered":"2024-05-11T23:03:52.818374+0000","last_unstale":"2024-05-11T23:04:55.988044+0000","last_undegraded":"2024-05-11T23:04:55.988044+0000","last_fullsized":"2024-05-11T23:04:55.988044+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"196'648","last_scrub_stamp":"2024-05-11T23:04:55.987966+0000","last_deep_scrub":"196'648","last_deep_scrub_stamp":"2024-05-11T23:04:55.987966+0000","last_clean_scrub_stamp":"2024-05-11T23:04:55.987966+0000","objects_scrubbed":49,"log_size":648,"log_dups_size":0,"ondisk_log_size":648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T02:42:50.735682+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0.0031987780000000002,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":5050,"num_read_kb":18194,"num_write":554,"num_write_kb":7419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":4575002,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1465,"num_object_clones":0,"num_object_copies":2930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1465,"num_whiteouts":1461,"num_read":339431,"num_read_kb":1023367,"num_write":28503,"num_write_kb":498784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8670,"num_bytes_recovered":382481677,"num_keys_recovered":951,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16167,"internal_metadata":0},"log_size":28032,"ondisk_log_size":28032,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":1771104,"data_compressed":100800,"data_compressed_allocated":135168,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69640,"kb_used_data":2368,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302200,"statfs":{"total":96636764160,"available":96565452800,"internally_reserved":0,"allocated":2424832,"data_stored":1737431,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509490,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59848,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311992,"statfs":{"total":96636764160,"available":96575479808,"internally_reserved":0,"allocated":2359296,"data_stored":1147063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509490,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87628,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94284212,"statfs":{"total":96636764160,"available":96547033088,"internally_reserved":0,"allocated":2363392,"data_stored":1147082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33480,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338360,"statfs":{"total":96636764160,"available":96602480640,"internally_reserved":0,"allocated":2359296,"data_stored":1147063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":179,"seq":768799145995,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36040,"kb_used_data":2304,"kb_used_omap":49,"kb_used_meta":33678,"kb_avail":94335800,"statfs":{"total":96636764160,"available":96599859200,"internally_reserved":0,"allocated":2359296,"data_stored":1147063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51091,"internal_metadata":34486381},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542194,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77900,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293940,"statfs":{"total":96636764160,"available":96556994560,"internally_reserved":0,"allocated":2363392,"data_stored":1147082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37640,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334200,"statfs":{"total":96636764160,"available":96598220800,"internally_reserved":0,"allocated":2359296,"data_stored":1147063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82104,"kb_used_data":2352,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94289736,"statfs":{"total":96636764160,"available":96552689664,"internally_reserved":0,"allocated":2408448,"data_stored":1737431,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":34145,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32510,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16167,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T23:05:11.997 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-11T23:05:12.313 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-11T23:05:12.313 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-11T23:05:12.313 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-11T23:05:12.313 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T23:05:12.356 INFO:tasks.ceph.osd.0:Stopped 2024-05-11T23:05:12.356 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-11T23:05:12.356 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T23:05:12.398 INFO:tasks.ceph.osd.1:Stopped 2024-05-11T23:05:12.399 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-11T23:05:12.399 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T23:05:12.438 INFO:tasks.ceph.osd.2:Stopped 2024-05-11T23:05:12.438 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-11T23:05:12.438 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T23:05:12.505 INFO:tasks.ceph.osd.3:Stopped 2024-05-11T23:05:12.506 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-11T23:05:12.506 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T23:05:12.546 INFO:tasks.ceph.osd.4:Stopped 2024-05-11T23:05:12.546 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-11T23:05:12.547 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T23:05:12.619 INFO:tasks.ceph.osd.5:Stopped 2024-05-11T23:05:12.619 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-11T23:05:12.619 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T23:05:12.701 INFO:tasks.ceph.osd.6:Stopped 2024-05-11T23:05:12.701 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-11T23:05:12.701 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T23:05:12.736 INFO:tasks.ceph.osd.7:Stopped 2024-05-11T23:05:12.736 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-11T23:05:12.737 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-11T23:05:12.737 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T23:05:12.763 INFO:tasks.ceph.mgr.y:Stopped 2024-05-11T23:05:12.763 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-11T23:05:12.764 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T23:05:12.805 INFO:tasks.ceph.mgr.x:Stopped 2024-05-11T23:05:12.805 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-11T23:05:12.805 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-11T23:05:12.805 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T23:05:12.816 INFO:tasks.ceph.mon.a:Stopped 2024-05-11T23:05:12.817 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-11T23:05:12.817 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T23:05:12.868 INFO:tasks.ceph.mon.c:Stopped 2024-05-11T23:05:12.868 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-11T23:05:12.868 INFO:teuthology.orchestra.run:waiting for 300 2024-05-11T23:05:12.882 INFO:tasks.ceph.mon.b:Stopped 2024-05-11T23:05:12.882 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-11T23:05:12.882 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | head -n 1 2024-05-11T23:05:12.941 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-11T23:05:12.941 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-11T23:05:13.246 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-11T23:05:13.246 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-11T23:05:13.335 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-11T23:05:13.335 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-11T23:05:13.385 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi005.front.sepia.ceph.com 2024-05-11T23:05:13.385 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-11T23:05:13.435 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi115.front.sepia.ceph.com 2024-05-11T23:05:13.435 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-11T23:05:13.813 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi115.front.sepia.ceph.com 2024-05-11T23:05:13.813 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-11T23:05:13.891 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi115.front.sepia.ceph.com 2024-05-11T23:05:13.892 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-11T23:05:13.939 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi115.front.sepia.ceph.com 2024-05-11T23:05:13.939 DEBUG:teuthology.orchestra.run.smithi115:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-11T23:05:13.993 INFO:tasks.ceph:Archiving mon data... 2024-05-11T23:05:13.994 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-05-11_21:16:03-rbd-squid-distro-default-smithi/7702761/data/mon.a.tgz 2024-05-11T23:05:13.994 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-05-11T23:05:14.011 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.QUJcIqktrc 2024-05-11T23:05:14.011 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.QUJcIqktrc 2024-05-11T23:05:14.201 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.QUJcIqktrc 2024-05-11T23:05:14.286 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.QUJcIqktrc is 414KB 2024-05-11T23:05:14.384 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.QUJcIqktrc 2024-05-11T23:05:14.399 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-05-11_21:16:03-rbd-squid-distro-default-smithi/7702761/data/mon.c.tgz 2024-05-11T23:05:14.399 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-05-11T23:05:14.455 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.ORPHIWTz8e 2024-05-11T23:05:14.455 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.ORPHIWTz8e 2024-05-11T23:05:14.640 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.ORPHIWTz8e 2024-05-11T23:05:14.725 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.ORPHIWTz8e is 562KB 2024-05-11T23:05:14.823 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.ORPHIWTz8e 2024-05-11T23:05:14.839 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-05-11_21:16:03-rbd-squid-distro-default-smithi/7702761/data/mon.b.tgz 2024-05-11T23:05:14.839 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-05-11T23:05:14.858 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.dHJZBhTkYC 2024-05-11T23:05:14.858 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.dHJZBhTkYC 2024-05-11T23:05:15.054 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.dHJZBhTkYC 2024-05-11T23:05:15.137 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.dHJZBhTkYC is 562KB 2024-05-11T23:05:15.236 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.dHJZBhTkYC 2024-05-11T23:05:15.251 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-11T23:05:15.251 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-11T23:05:15.254 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-11T23:05:15.334 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-11T23:05:15.372 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-11T23:05:15.372 INFO:tasks.ceph:Archiving crash dumps... 2024-05-11T23:05:15.373 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-05-11_21:16:03-rbd-squid-distro-default-smithi/7702761/remote/smithi005/crash 2024-05-11T23:05:15.374 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-11T23:05:15.400 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-05-11_21:16:03-rbd-squid-distro-default-smithi/7702761/remote/smithi115/crash 2024-05-11T23:05:15.401 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-11T23:05:15.436 INFO:tasks.ceph:Compressing logs... 2024-05-11T23:05:15.436 DEBUG:teuthology.orchestra.run.smithi005:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-11T23:05:15.444 DEBUG:teuthology.orchestra.run.smithi115:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-11T23:05:15.467 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57313.log 2024-05-11T23:05:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62852.log 2024-05-11T23:05:15.513 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33004.log 2024-05-11T23:05:15.513 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-11T23:05:15.514 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.33004.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-11T23:05:15.514 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33004.log.gz 2024-05-11T23:05:15.514 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-11T23:05:15.514 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-11T23:05:15.514 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-11T23:05:15.514 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32924.log 2024-05-11T23:05:15.514 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-osd.4.log: 84.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-11T23:05:15.514 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32964.log 2024-05-11T23:05:15.514 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33084.log 2024-05-11T23:05:15.514 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.32924.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32924.log.gz 2024-05-11T23:05:15.514 INFO:teuthology.orchestra.run.smithi115.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-11T23:05:15.514 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.32964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33044.log 2024-05-11T23:05:15.515 INFO:teuthology.orchestra.run.smithi115.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32964.log.gz 2024-05-11T23:05:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-11T23:05:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.33084.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33084.log.gz 2024-05-11T23:05:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.33044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32833.log 0.0% 2024-05-11T23:05:15.515 INFO:teuthology.orchestra.run.smithi115.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33044.log.gz 2024-05-11T23:05:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-11T23:05:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32793.log 2024-05-11T23:05:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.32833.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32833.log.gz 2024-05-11T23:05:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:gzip/var/log/ceph/ceph-osd.6.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32873.log 2024-05-11T23:05:15.515 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.32793.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32793.log.gz 2024-05-11T23:05:15.515 INFO:teuthology.orchestra.run.smithi115.stderr: -- /var/log/ceph/ceph-mon.b.log 2024-05-11T23:05:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-client.admin.32873.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32873.log.gz 2024-05-11T23:05:15.516 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/ceph-mon.b.log: 93.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-11T23:05:15.628 INFO:teuthology.orchestra.run.smithi115.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-11T23:05:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44396.log 2024-05-11T23:05:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57313.log.gz 2024-05-11T23:05:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:gzip/var/log/ceph/ceph-client.admin.62852.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.63426.log 2024-05-11T23:05:15.659 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62852.log.gz 2024-05-11T23:05:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63384.log 2024-05-11T23:05:15.659 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44396.log.gz 2024-05-11T23:05:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60134.log 2024-05-11T23:05:15.659 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63426.log.gz 2024-05-11T23:05:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42065.log 2024-05-11T23:05:15.659 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63384.log.gz 2024-05-11T23:05:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59266.log 2024-05-11T23:05:15.659 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60134.log.gz 2024-05-11T23:05:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59523.log 2024-05-11T23:05:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42065.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42065.log.gz 2024-05-11T23:05:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52366.log 2024-05-11T23:05:15.660 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59266.log.gz 2024-05-11T23:05:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40002.log 2024-05-11T23:05:15.660 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59523.log.gz 2024-05-11T23:05:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57707.log 2024-05-11T23:05:15.660 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52366.log.gz 2024-05-11T23:05:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58203.log 2024-05-11T23:05:15.660 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40002.log.gz 2024-05-11T23:05:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42924.log 2024-05-11T23:05:15.660 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57707.log.gz 2024-05-11T23:05:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51626.log 2024-05-11T23:05:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58203.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58203.log.gz 2024-05-11T23:05:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62178.log 2024-05-11T23:05:15.661 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42924.log.gz 2024-05-11T23:05:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36821.log 2024-05-11T23:05:15.661 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51626.log.gz 2024-05-11T23:05:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5/var/log/ceph/ceph-client.admin.62178.log: --verbose -- /var/log/ceph/ceph-client.admin.47415.log 2024-05-11T23:05:15.661 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62178.log.gz 2024-05-11T23:05:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62851.log 2024-05-11T23:05:15.661 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36821.log.gz 2024-05-11T23:05:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57387.log 2024-05-11T23:05:15.662 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47415.log.gz 2024-05-11T23:05:15.662 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42765.log 2024-05-11T23:05:15.662 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62851.log.gz 2024-05-11T23:05:15.662 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48611.log 2024-05-11T23:05:15.662 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57387.log.gz 2024-05-11T23:05:15.662 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41507.log 2024-05-11T23:05:15.662 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42765.log.gz 2024-05-11T23:05:15.662 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42745.log 2024-05-11T23:05:15.662 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48611.log.gz 2024-05-11T23:05:15.662 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45892.log 2024-05-11T23:05:15.662 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41507.log.gz 2024-05-11T23:05:15.662 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62849.log 2024-05-11T23:05:15.663 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42745.log.gz 2024-05-11T23:05:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48849.log 2024-05-11T23:05:15.663 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45892.log.gz 2024-05-11T23:05:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32548.log 2024-05-11T23:05:15.663 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62849.log.gz 2024-05-11T23:05:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41926.log 2024-05-11T23:05:15.663 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48849.log.gz 2024-05-11T23:05:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36825.log 2024-05-11T23:05:15.663 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32548.log.gz 2024-05-11T23:05:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61536.log 2024-05-11T23:05:15.663 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41926.log.gz 2024-05-11T23:05:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47106.log 2024-05-11T23:05:15.663 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36825.log.gz 2024-05-11T23:05:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44819.log 2024-05-11T23:05:15.664 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61536.log.gz 2024-05-11T23:05:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46972.log 2024-05-11T23:05:15.664 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47106.log.gz 2024-05-11T23:05:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62503.log 2024-05-11T23:05:15.664 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44819.log.gz 2024-05-11T23:05:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46972.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.54567.log 2024-05-11T23:05:15.664 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46972.log.gz 2024-05-11T23:05:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38150.log 2024-05-11T23:05:15.664 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62503.log.gz 2024-05-11T23:05:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/objectstore_tool.54567.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-11T23:05:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64056.log 2024-05-11T23:05:15.664 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38150.log.gz 2024-05-11T23:05:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58635.log 2024-05-11T23:05:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.64056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62179.log 2024-05-11T23:05:15.665 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64056.log.gz 2024-05-11T23:05:15.665 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58635.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58635.log.gz 2024-05-11T23:05:15.665 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49667.log 2024-05-11T23:05:15.665 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37519.log 2024-05-11T23:05:15.665 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62179.log.gz 2024-05-11T23:05:15.665 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49667.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49667.log.gz 2024-05-11T23:05:15.665 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40625.log 2024-05-11T23:05:15.665 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61788.log 2024-05-11T23:05:15.665 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37519.log.gz 2024-05-11T23:05:15.665 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45653.log 2024-05-11T23:05:15.665 INFO:teuthology.orchestra.run.smithi005.stderr: 91.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-11T23:05:15.665 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40625.log.gz 2024-05-11T23:05:15.665 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61788.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61788.log.gz 2024-05-11T23:05:15.665 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41023.log 2024-05-11T23:05:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:gzip/var/log/ceph/ceph-client.admin.45653.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44616.log 2024-05-11T23:05:15.666 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45653.log.gz 2024-05-11T23:05:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36824.log 2024-05-11T23:05:15.666 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41023.log.gz 2024-05-11T23:05:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38661.log 2024-05-11T23:05:15.666 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44616.log.gz 2024-05-11T23:05:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36824.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-11T23:05:15.666 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36824.log.gz 2024-05-11T23:05:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63510.log 2024-05-11T23:05:15.666 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38661.log.gz 2024-05-11T23:05:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63762.log 2024-05-11T23:05:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61237.log 2024-05-11T23:05:15.666 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63510.log.gz 2024-05-11T23:05:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37780.log 2024-05-11T23:05:15.667 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63762.log.gz 2024-05-11T23:05:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61494.log 2024-05-11T23:05:15.667 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61237.log.gz 2024-05-11T23:05:15.667 INFO:teuthology.orchestra.run.smithi005.stderr: 94.7% -- replaced with /var/log/ceph/objectstore_tool.54567.log.gz 2024-05-11T23:05:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57036.log 2024-05-11T23:05:15.667 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37780.log.gz 2024-05-11T23:05:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49490.log 2024-05-11T23:05:15.667 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61494.log.gz 2024-05-11T23:05:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44876.log 2024-05-11T23:05:15.667 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57036.log.gz 2024-05-11T23:05:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38412.log 2024-05-11T23:05:15.668 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49490.log.gz 2024-05-11T23:05:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40744.log 2024-05-11T23:05:15.668 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44876.log.gz 2024-05-11T23:05:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62847.log 2024-05-11T23:05:15.668 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38412.log.gz 2024-05-11T23:05:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36828.log 2024-05-11T23:05:15.668 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40744.log.gz 2024-05-11T23:05:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61914.log 2024-05-11T23:05:15.668 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62847.log.gz 2024-05-11T23:05:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36828.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.46254.log -- replaced with /var/log/ceph/ceph-client.admin.36828.log.gz 2024-05-11T23:05:15.668 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-11T23:05:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59070.log 2024-05-11T23:05:15.668 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61914.log.gz 2024-05-11T23:05:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43599.log 2024-05-11T23:05:15.669 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46254.log.gz 2024-05-11T23:05:15.669 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45556.log 2024-05-11T23:05:15.669 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59070.log.gz 2024-05-11T23:05:15.669 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56933.log 2024-05-11T23:05:15.669 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43599.log.gz 2024-05-11T23:05:15.669 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45556.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-11T23:05:15.669 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45556.log.gz 2024-05-11T23:05:15.669 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32865.log 2024-05-11T23:05:15.669 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56933.log.gz 2024-05-11T23:05:15.669 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44436.log 2024-05-11T23:05:15.669 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49050.log 2024-05-11T23:05:15.669 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32865.log.gz 2024-05-11T23:05:15.669 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44436.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44436.log.gz 2024-05-11T23:05:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41322.log 2024-05-11T23:05:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61195.log 2024-05-11T23:05:15.670 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49050.log.gz 2024-05-11T23:05:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41322.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41322.log.gz 2024-05-11T23:05:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45215.log 2024-05-11T23:05:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.61195.log: /var/log/ceph/ceph-client.admin.49309.log 2024-05-11T23:05:15.670 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61195.log.gz 2024-05-11T23:05:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45215.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45215.log.gz 2024-05-11T23:05:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50525.log 2024-05-11T23:05:15.670 INFO:teuthology.orchestra.run.smithi005.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-11T23:05:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39736.log 2024-05-11T23:05:15.670 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49309.log.gz 2024-05-11T23:05:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37948.log 2024-05-11T23:05:15.671 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50525.log.gz 2024-05-11T23:05:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62731.log 2024-05-11T23:05:15.671 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39736.log.gz 2024-05-11T23:05:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62846.log 2024-05-11T23:05:15.671 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37948.log.gz 2024-05-11T23:05:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52583.log 2024-05-11T23:05:15.671 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62731.log.gz 2024-05-11T23:05:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63048.log 2024-05-11T23:05:15.671 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62846.log.gz 2024-05-11T23:05:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45235.log 2024-05-11T23:05:15.671 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52583.log.gz 2024-05-11T23:05:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44281.log 2024-05-11T23:05:15.672 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63048.log.gz 2024-05-11T23:05:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38010.log 2024-05-11T23:05:15.672 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45235.log.gz 2024-05-11T23:05:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63132.log 2024-05-11T23:05:15.672 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44281.log.gz 2024-05-11T23:05:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57091.log 2024-05-11T23:05:15.672 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38010.log.gz 2024-05-11T23:05:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41144.log 2024-05-11T23:05:15.672 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63132.log.gz 2024-05-11T23:05:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63258.log 2024-05-11T23:05:15.672 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57091.log.gz 2024-05-11T23:05:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37864.log 2024-05-11T23:05:15.673 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41144.log.gz 2024-05-11T23:05:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63678.log 2024-05-11T23:05:15.673 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63258.log.gz 2024-05-11T23:05:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48593.log 2024-05-11T23:05:15.673 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37864.log.gz 2024-05-11T23:05:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38474.log 2024-05-11T23:05:15.673 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63678.log.gz 2024-05-11T23:05:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48593.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48593.log.gz -5 2024-05-11T23:05:15.673 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41946.log 2024-05-11T23:05:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38474.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-11T23:05:15.674 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38474.log.gz 2024-05-11T23:05:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40188.log 2024-05-11T23:05:15.674 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41946.log.gz 2024-05-11T23:05:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38145.log 2024-05-11T23:05:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40188.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40188.log.gz 2024-05-11T23:05:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52324.log 2024-05-11T23:05:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38537.log 2024-05-11T23:05:15.674 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38145.log.gz 2024-05-11T23:05:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37163.log 2024-05-11T23:05:15.674 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52324.log.gz 2024-05-11T23:05:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57388.log 2024-05-11T23:05:15.675 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38537.log.gz 2024-05-11T23:05:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61998.log 0.0% 2024-05-11T23:05:15.675 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37163.log.gz 2024-05-11T23:05:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37153.log 2024-05-11T23:05:15.675 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57388.log.gz 2024-05-11T23:05:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61998.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61998.log.gz 2024-05-11T23:05:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37462.log 2024-05-11T23:05:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61830.log 2024-05-11T23:05:15.675 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37153.log.gz 2024-05-11T23:05:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37462.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37462.log.gz 2024-05-11T23:05:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59660.log 2024-05-11T23:05:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62176.log 2024-05-11T23:05:15.676 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61830.log.gz 2024-05-11T23:05:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59660.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59660.log.gz 2024-05-11T23:05:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58657.log 2024-05-11T23:05:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38144.log 2024-05-11T23:05:15.676 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62176.log.gz 2024-05-11T23:05:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61284.log 2024-05-11T23:05:15.676 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58657.log.gz 2024-05-11T23:05:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37161.log 2024-05-11T23:05:15.676 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38144.log.gz 2024-05-11T23:05:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60345.log 2024-05-11T23:05:15.676 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61284.log.gz 2024-05-11T23:05:15.677 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36419.log 2024-05-11T23:05:15.677 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37161.log.gz 2024-05-11T23:05:15.677 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63636.log 2024-05-11T23:05:15.677 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60345.log.gz 2024-05-11T23:05:15.677 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36419.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36419.log.gz 2024-05-11T23:05:15.677 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-11T23:05:15.677 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63636.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-11T23:05:15.677 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63636.log.gz 2024-05-11T23:05:15.677 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61620.log 2024-05-11T23:05:15.677 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63972.log 2024-05-11T23:05:15.677 INFO:teuthology.orchestra.run.smithi005.stderr:gzip/var/log/ceph/ceph-client.admin.61620.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.62515.log 2024-05-11T23:05:15.678 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61620.log.gz 2024-05-11T23:05:15.678 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46472.log 2024-05-11T23:05:15.678 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63972.log.gz 2024-05-11T23:05:15.678 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51486.log 2024-05-11T23:05:15.678 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62515.log.gz 2024-05-11T23:05:15.678 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61452.log 2024-05-11T23:05:15.678 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46472.log.gz 2024-05-11T23:05:15.678 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39182.log 2024-05-11T23:05:15.678 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51486.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51486.log.gz 2024-05-11T23:05:15.678 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42364.log 2024-05-11T23:05:15.678 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61452.log.gz 2024-05-11T23:05:15.679 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39182.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.52775.log 2024-05-11T23:05:15.679 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39182.log.gz 2024-05-11T23:05:15.679 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63720.log 2024-05-11T23:05:15.679 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42364.log.gz 2024-05-11T23:05:15.679 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/objectstore_tool.52775.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-11T23:05:15.679 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63720.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63720.log.gz 2024-05-11T23:05:15.679 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60295.log 2024-05-11T23:05:15.679 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62508.log 2024-05-11T23:05:15.679 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.60295.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48691.log 2024-05-11T23:05:15.679 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60295.log.gz 2024-05-11T23:05:15.680 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39521.log 2024-05-11T23:05:15.680 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62508.log: /var/log/ceph/ceph-client.admin.48691.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48691.log.gz 2024-05-11T23:05:15.680 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62508.log.gz 2024-05-11T23:05:15.680 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37360.log 2024-05-11T23:05:15.680 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40966.log 2024-05-11T23:05:15.680 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39521.log.gz 2024-05-11T23:05:15.680 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37360.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37360.log.gz 2024-05-11T23:05:15.680 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63468.log 2024-05-11T23:05:15.680 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40966.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36363.log 2024-05-11T23:05:15.680 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40966.log.gz 2024-05-11T23:05:15.680 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39582.log 2024-05-11T23:05:15.680 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63468.log: /var/log/ceph/ceph-client.admin.36363.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63468.log.gz 2024-05-11T23:05:15.681 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36363.log.gz 2024-05-11T23:05:15.681 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38840.log 2024-05-11T23:05:15.681 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38277.log 2024-05-11T23:05:15.681 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39582.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39582.log.gz 2024-05-11T23:05:15.681 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38840.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38840.log.gz 2024-05-11T23:05:15.681 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46773.log 2024-05-11T23:05:15.681 INFO:teuthology.orchestra.run.smithi005.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.52775.log.gz 2024-05-11T23:05:15.681 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38277.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38277.log.gz 2024-05-11T23:05:15.681 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51363.log 2024-05-11T23:05:15.681 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39244.log 2024-05-11T23:05:15.681 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46773.log: /var/log/ceph/ceph-client.admin.51363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55675.log 2024-05-11T23:05:15.681 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46773.log.gz 2024-05-11T23:05:15.681 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51363.log.gz 2024-05-11T23:05:15.682 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39244.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39244.log.gz 2024-05-11T23:05:15.682 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49789.log 2024-05-11T23:05:15.682 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39898.log 2024-05-11T23:05:15.682 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55675.log: gzip/var/log/ceph/ceph-client.admin.49789.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.59584.log 2024-05-11T23:05:15.682 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55675.log.gz 2024-05-11T23:05:15.682 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49789.log.gz 2024-05-11T23:05:15.682 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39898.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39898.log.gz 2024-05-11T23:05:15.682 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41446.log 2024-05-11T23:05:15.682 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43240.log 2024-05-11T23:05:15.682 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59584.log: /var/log/ceph/ceph-client.admin.41446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61326.log 2024-05-11T23:05:15.682 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59584.log.gz 2024-05-11T23:05:15.682 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41446.log.gz 2024-05-11T23:05:15.683 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43240.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43240.log.gz 2024-05-11T23:05:15.683 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48293.log 2024-05-11T23:05:15.683 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62177.log 2024-05-11T23:05:15.683 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61326.log: /var/log/ceph/ceph-client.admin.48293.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-11T23:05:15.683 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61326.log.gz 2024-05-11T23:05:15.683 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48293.log.gz 2024-05-11T23:05:15.683 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62177.log.gz 2024-05-11T23:05:15.683 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49126.log 2024-05-11T23:05:15.683 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61410.log 2024-05-11T23:05:15.683 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-mon.c.log: gzip/var/log/ceph/ceph-client.admin.49126.log: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49126.log.gz 2024-05-11T23:05:15.683 INFO:teuthology.orchestra.run.smithi005.stderr: /var/log/ceph/ceph-client.admin.64096.log 2024-05-11T23:05:15.684 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58893.log 2024-05-11T23:05:15.684 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64096.log: /var/log/ceph/ceph-client.admin.61410.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64096.log.gz 0.0% 2024-05-11T23:05:15.684 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61410.log.gz 2024-05-11T23:05:15.684 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51965.log 2024-05-11T23:05:15.684 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58893.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41627.log 2024-05-11T23:05:15.684 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58893.log.gz 2024-05-11T23:05:15.684 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51965.log.gz 2024-05-11T23:05:15.684 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63174.log 2024-05-11T23:05:15.684 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41627.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-11T23:05:15.684 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41627.log.gz 2024-05-11T23:05:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62173.log 2024-05-11T23:05:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63174.log: /var/log/ceph/ceph-mon.a.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63174.log.gz 2024-05-11T23:05:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37457.log 2024-05-11T23:05:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62173.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62173.log.gz 2024-05-11T23:05:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37470.log 2024-05-11T23:05:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37457.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37457.log.gz 2024-05-11T23:05:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63090.log 2024-05-11T23:05:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37470.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37470.log.gz 2024-05-11T23:05:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63216.log 2024-05-11T23:05:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63090.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63090.log.gz 2024-05-11T23:05:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53842.log 2024-05-11T23:05:15.685 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63216.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63216.log.gz 2024-05-11T23:05:15.686 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58338.log 2024-05-11T23:05:15.686 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.53842.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53842.log.gz 2024-05-11T23:05:15.686 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37162.log 2024-05-11T23:05:15.686 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58338.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58338.log.gz 2024-05-11T23:05:15.686 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49406.log 2024-05-11T23:05:15.686 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37162.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37162.log.gz 2024-05-11T23:05:15.686 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40585.log 2024-05-11T23:05:15.686 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49406.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49406.log.gz 2024-05-11T23:05:15.686 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43661.log 2024-05-11T23:05:15.686 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40585.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40585.log.gz 2024-05-11T23:05:15.686 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43183.log 2024-05-11T23:05:15.686 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43661.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43661.log.gz 2024-05-11T23:05:15.687 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37906.log 2024-05-11T23:05:15.687 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43183.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43183.log.gz 2024-05-11T23:05:15.687 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62175.log 2024-05-11T23:05:15.687 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37906.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37906.log.gz 2024-05-11T23:05:15.687 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62516.log 2024-05-11T23:05:15.687 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62175.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62175.log.gz 2024-05-11T23:05:15.687 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58989.log 2024-05-11T23:05:15.687 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62516.log.gz 2024-05-11T23:05:15.687 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37151.log 2024-05-11T23:05:15.687 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58989.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58989.log.gz 2024-05-11T23:05:15.687 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64014.log 2024-05-11T23:05:15.687 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37151.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37151.log.gz 2024-05-11T23:05:15.687 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63552.log 2024-05-11T23:05:15.688 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.64014.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64014.log.gz 2024-05-11T23:05:15.688 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37150.log 2024-05-11T23:05:15.688 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63552.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63552.log.gz 2024-05-11T23:05:15.688 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37024.log 2024-05-11T23:05:15.688 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37150.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37150.log.gz 2024-05-11T23:05:15.688 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44522.log 2024-05-11T23:05:15.688 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37024.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37024.log.gz 2024-05-11T23:05:15.688 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47594.log 2024-05-11T23:05:15.688 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44522.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44522.log.gz 2024-05-11T23:05:15.688 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52663.log 2024-05-11T23:05:15.688 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47594.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47594.log.gz 2024-05-11T23:05:15.688 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62180.log 2024-05-11T23:05:15.688 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52663.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52663.log.gz 2024-05-11T23:05:15.688 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42426.log 2024-05-11T23:05:15.689 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62180.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62180.log.gz 2024-05-11T23:05:15.689 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.26741.log 2024-05-11T23:05:15.689 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42426.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42426.log.gz 2024-05-11T23:05:15.689 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63594.log 2024-05-11T23:05:15.689 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph.tmp-client.admin.26741.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.26741.log.gz 2024-05-11T23:05:15.689 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40302.log 2024-05-11T23:05:15.689 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63594.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63594.log.gz 2024-05-11T23:05:15.689 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46153.log 2024-05-11T23:05:15.689 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40302.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40302.log.gz 2024-05-11T23:05:15.689 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50623.log 2024-05-11T23:05:15.689 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46153.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46153.log.gz 2024-05-11T23:05:15.689 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37738.log 2024-05-11T23:05:15.690 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50623.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50623.log.gz 2024-05-11T23:05:15.690 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44217.log 2024-05-11T23:05:15.690 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37738.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37738.log.gz 2024-05-11T23:05:15.690 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51244.log 2024-05-11T23:05:15.690 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.44217.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44217.log.gz 2024-05-11T23:05:15.690 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63888.log 2024-05-11T23:05:15.690 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51244.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51244.log.gz 2024-05-11T23:05:15.690 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41728.log 2024-05-11T23:05:15.690 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63888.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63888.log.gz 2024-05-11T23:05:15.690 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36816.log 2024-05-11T23:05:15.690 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41728.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41728.log.gz 2024-05-11T23:05:15.691 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39740.log 2024-05-11T23:05:15.691 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36816.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36816.log.gz 2024-05-11T23:05:15.691 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61662.log 2024-05-11T23:05:15.691 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39740.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39740.log.gz 2024-05-11T23:05:15.691 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42267.log 2024-05-11T23:05:15.691 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61662.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61662.log.gz 2024-05-11T23:05:15.691 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36646.log 2024-05-11T23:05:15.691 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42267.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42267.log.gz 2024-05-11T23:05:15.691 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56621.log 2024-05-11T23:05:15.691 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36646.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36646.log.gz 2024-05-11T23:05:15.691 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61578.log 2024-05-11T23:05:15.691 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56621.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56621.log.gz 2024-05-11T23:05:15.691 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47191.log 2024-05-11T23:05:15.691 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61578.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61578.log.gz 2024-05-11T23:05:15.691 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48930.log 2024-05-11T23:05:15.692 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47191.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47191.log.gz 2024-05-11T23:05:15.692 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49366.log 2024-05-11T23:05:15.692 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48930.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48930.log.gz 2024-05-11T23:05:15.692 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43025.log 2024-05-11T23:05:15.692 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49366.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49366.log.gz 2024-05-11T23:05:15.692 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47273.log 2024-05-11T23:05:15.692 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43025.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43025.log.gz 2024-05-11T23:05:15.692 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50227.log 2024-05-11T23:05:15.692 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47273.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47273.log.gz 2024-05-11T23:05:15.692 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58379.log 2024-05-11T23:05:15.692 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50227.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50227.log.gz 2024-05-11T23:05:15.692 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37696.log 2024-05-11T23:05:15.692 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58379.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58379.log.gz 2024-05-11T23:05:15.692 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36558.log 2024-05-11T23:05:15.693 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37696.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37696.log.gz 2024-05-11T23:05:15.693 INFO:teuthology.orchestra.run.smithi005.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-11T23:05:15.693 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38350.log 2024-05-11T23:05:15.693 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36558.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36558.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57964.log 2024-05-11T23:05:15.693 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-11T23:05:15.693 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59308.log 2024-05-11T23:05:15.693 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57964.log: /var/log/ceph/ceph-client.admin.38350.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57964.log.gz 2024-05-11T23:05:15.693 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38350.log.gz 2024-05-11T23:05:15.693 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37380.log 2024-05-11T23:05:15.693 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41403.log 2024-05-11T23:05:15.693 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59308.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59308.log.gz 2024-05-11T23:05:15.693 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37380.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37380.log.gz 2024-05-11T23:05:15.694 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63846.log 2024-05-11T23:05:15.694 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41403.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41403.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32968.log 2024-05-11T23:05:15.694 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-11T23:05:15.694 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62732.log 2024-05-11T23:05:15.694 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32968.log: /var/log/ceph/ceph-client.admin.63846.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32968.log.gz 2024-05-11T23:05:15.694 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63846.log.gz 2024-05-11T23:05:15.694 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51908.log 2024-05-11T23:05:15.694 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62732.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62513.log 2024-05-11T23:05:15.694 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62732.log.gz 2024-05-11T23:05:15.694 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57490.log 2024-05-11T23:05:15.694 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51908.log: /var/log/ceph/ceph-client.admin.62513.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62513.log.gz 2024-05-11T23:05:15.694 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51908.log.gz 2024-05-11T23:05:15.694 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48155.log 2024-05-11T23:05:15.695 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57490.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51243.log 2024-05-11T23:05:15.695 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57490.log.gz 2024-05-11T23:05:15.695 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45912.log 2024-05-11T23:05:15.695 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48155.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48155.log.gz 2024-05-11T23:05:15.695 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45038.log 2024-05-11T23:05:15.695 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51243.log.gz 2024-05-11T23:05:15.695 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37822.log 2024-05-11T23:05:15.695 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45038.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45912.log.gz 2024-05-11T23:05:15.695 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45038.log.gz 2024-05-11T23:05:15.695 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49906.log 2024-05-11T23:05:15.695 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48756.log 2024-05-11T23:05:15.695 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37822.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37822.log.gz 2024-05-11T23:05:15.695 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49906.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49906.log.gz 2024-05-11T23:05:15.695 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.53613.log 2024-05-11T23:05:15.696 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48889.log 2024-05-11T23:05:15.696 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48756.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48756.log.gz 2024-05-11T23:05:15.696 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/objectstore_tool.53613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61746.log 2024-05-11T23:05:15.696 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48889.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48889.log.gz 2024-05-11T23:05:15.696 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57687.log 2024-05-11T23:05:15.696 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61746.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61746.log.gz 2024-05-11T23:05:15.696 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61956.log 2024-05-11T23:05:15.696 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.57687.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57687.log.gz 2024-05-11T23:05:15.696 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63342.log 2024-05-11T23:05:15.696 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61956.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61956.log.gz 2024-05-11T23:05:15.696 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62514.log 2024-05-11T23:05:15.696 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63342.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63342.log.gz 2024-05-11T23:05:15.696 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61704.log 2024-05-11T23:05:15.696 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62514.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62514.log.gz 2024-05-11T23:05:15.697 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47813.log 2024-05-11T23:05:15.697 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61704.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61704.log.gz 2024-05-11T23:05:15.697 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56519.log 2024-05-11T23:05:15.697 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47813.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47813.log.gz 2024-05-11T23:05:15.697 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43541.log 2024-05-11T23:05:15.697 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56519.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56519.log.gz 2024-05-11T23:05:15.697 INFO:teuthology.orchestra.run.smithi005.stderr: 95.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61368.log 2024-05-11T23:05:15.697 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/objectstore_tool.53613.log.gz 2024-05-11T23:05:15.697 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38599.log 2024-05-11T23:05:15.697 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43541.log: /var/log/ceph/ceph-client.admin.61368.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43541.log.gz 2024-05-11T23:05:15.697 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61368.log.gz 2024-05-11T23:05:15.697 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46513.log 2024-05-11T23:05:15.697 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.38599.log: /var/log/ceph/ceph-client.admin.56820.log 2024-05-11T23:05:15.698 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38599.log.gz 2024-05-11T23:05:15.698 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62170.log 2024-05-11T23:05:15.698 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46513.log: /var/log/ceph/ceph-client.admin.56820.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46513.log.gz 2024-05-11T23:05:15.698 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56820.log.gz 2024-05-11T23:05:15.698 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38783.log 2024-05-11T23:05:15.698 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62170.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39025.log 2024-05-11T23:05:15.698 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62170.log.gz 2024-05-11T23:05:15.698 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37164.log 2024-05-11T23:05:15.698 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.38783.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38783.log.gz 2024-05-11T23:05:15.698 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39025.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59857.log 2024-05-11T23:05:15.698 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39025.log.gz 2024-05-11T23:05:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47870.log 2024-05-11T23:05:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37164.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37164.log.gz 2024-05-11T23:05:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56367.log 2024-05-11T23:05:15.699 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59857.log.gz 2024-05-11T23:05:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40487.log 2024-05-11T23:05:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.56367.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47870.log.gz 2024-05-11T23:05:15.699 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56367.log.gz 2024-05-11T23:05:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51047.log 2024-05-11T23:05:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51367.log 2024-05-11T23:05:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40487.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40487.log.gz 2024-05-11T23:05:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51047.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51047.log.gz 2024-05-11T23:05:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49926.log 2024-05-11T23:05:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54116.log 2024-05-11T23:05:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51367.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51367.log.gz 2024-05-11T23:05:15.699 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.49926.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49926.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39897.log 2024-05-11T23:05:15.700 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-11T23:05:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51569.log 2024-05-11T23:05:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.54116.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54116.log.gz 2024-05-11T23:05:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.39897.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58019.log 2024-05-11T23:05:15.700 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39897.log.gz 2024-05-11T23:05:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55140.log 2024-05-11T23:05:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.51569.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51569.log.gz 2024-05-11T23:05:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37468.log 2024-05-11T23:05:15.700 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58019.log.gz 2024-05-11T23:05:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.55140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40147.log 2024-05-11T23:05:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37468.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55140.log.gz 2024-05-11T23:05:15.700 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37468.log.gz 2024-05-11T23:05:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36826.log 2024-05-11T23:05:15.700 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48252.log 2024-05-11T23:05:15.701 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40147.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40147.log.gz 2024-05-11T23:05:15.701 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36826.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36826.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46910.log 2024-05-11T23:05:15.701 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-11T23:05:15.701 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36820.log 2024-05-11T23:05:15.701 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48252.log: /var/log/ceph/ceph-client.admin.46910.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48252.log.gz 2024-05-11T23:05:15.701 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46910.log.gz 2024-05-11T23:05:15.701 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47593.log 2024-05-11T23:05:15.701 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63930.log 2024-05-11T23:05:15.701 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36820.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36820.log.gz 2024-05-11T23:05:15.701 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47593.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47593.log.gz 2024-05-11T23:05:15.701 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40084.log 2024-05-11T23:05:15.701 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50944.log 2024-05-11T23:05:15.701 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.40084.log: /var/log/ceph/ceph-client.admin.63930.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63930.log.gz 2024-05-11T23:05:15.701 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40084.log.gz 2024-05-11T23:05:15.702 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36827.log 2024-05-11T23:05:15.702 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48671.log 2024-05-11T23:05:15.702 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50944.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50944.log.gz 2024-05-11T23:05:15.702 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36827.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36827.log.gz 2024-05-11T23:05:15.702 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59959.log 2024-05-11T23:05:15.702 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45695.log 2024-05-11T23:05:15.702 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59959.log: /var/log/ceph/ceph-client.admin.48671.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48671.log.gz 2024-05-11T23:05:15.702 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59959.log.gz 2024-05-11T23:05:15.702 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63804.log 2024-05-11T23:05:15.702 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45296.log 2024-05-11T23:05:15.702 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.45695.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45695.log.gz 2024-05-11T23:05:15.702 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63804.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63804.log.gz 2024-05-11T23:05:15.703 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50747.log 2024-05-11T23:05:15.703 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59761.log 2024-05-11T23:05:15.703 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50747.log: /var/log/ceph/ceph-client.admin.45296.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45296.log.gz 2024-05-11T23:05:15.703 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50747.log.gz 2024-05-11T23:05:15.703 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61872.log 2024-05-11T23:05:15.703 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56757.log 2024-05-11T23:05:15.703 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.59761.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59761.log.gz 2024-05-11T23:05:15.703 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.61872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61872.log.gz 2024-05-11T23:05:15.703 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50248.log 2024-05-11T23:05:15.703 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46076.log 2024-05-11T23:05:15.703 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50248.log: /var/log/ceph/ceph-client.admin.56757.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56757.log.gz 2024-05-11T23:05:15.703 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50248.log.gz 2024-05-11T23:05:15.703 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38064.log 2024-05-11T23:05:15.703 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.46076.log: /var/log/ceph/ceph-client.admin.43998.log 2024-05-11T23:05:15.703 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46076.log.gz 2024-05-11T23:05:15.704 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.38064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63300.log 2024-05-11T23:05:15.704 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43998.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43998.log.gz 2024-05-11T23:05:15.704 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52107.log 2024-05-11T23:05:15.704 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.63300.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63300.log.gz 2024-05-11T23:05:15.704 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42643.log 2024-05-11T23:05:15.704 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.52107.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52107.log.gz 2024-05-11T23:05:15.704 INFO:teuthology.orchestra.run.smithi005.stderr: 95.7% -- replaced with /var/log/ceph/ceph-client.0.38064.log.gz 2024-05-11T23:05:15.704 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36462.log 2024-05-11T23:05:15.704 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.42643.log: 0.0%gzip -5 --verbose -- /var/log/ceph/objectstore_tool.55439.log 2024-05-11T23:05:15.704 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42643.log.gz 2024-05-11T23:05:15.704 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43938.log 2024-05-11T23:05:15.704 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36462.log: /var/log/ceph/objectstore_tool.55439.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36462.log.gz 2024-05-11T23:05:15.704 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62506.log 2024-05-11T23:05:15.704 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.43938.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43938.log.gz 2024-05-11T23:05:15.705 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58615.log 2024-05-11T23:05:15.705 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62506.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62506.log.gz 2024-05-11T23:05:15.705 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50663.log 2024-05-11T23:05:15.705 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58615.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58615.log.gz 2024-05-11T23:05:15.705 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37520.log 2024-05-11T23:05:15.705 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50663.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50663.log.gz 2024-05-11T23:05:15.705 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58039.log 2024-05-11T23:05:15.705 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.37520.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37520.log.gz 2024-05-11T23:05:15.705 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62850.log 2024-05-11T23:05:15.705 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.58039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58039.log.gz 2024-05-11T23:05:15.705 INFO:teuthology.orchestra.run.smithi005.stderr: 94.7% -- replaced with /var/log/ceph/objectstore_tool.55439.log.gz 2024-05-11T23:05:15.705 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41508.log 2024-05-11T23:05:15.705 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46560.log 2024-05-11T23:05:15.705 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62850.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62850.log.gz 2024-05-11T23:05:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.41508.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.36604.log 2024-05-11T23:05:15.706 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41508.log.gz 2024-05-11T23:05:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62512.log 2024-05-11T23:05:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.46560.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46560.log.gz 2024-05-11T23:05:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47322.log 2024-05-11T23:05:15.706 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36604.log.gz 2024-05-11T23:05:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.62512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47890.log 2024-05-11T23:05:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47322.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62512.log.gz 2024-05-11T23:05:15.706 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47322.log.gz 2024-05-11T23:05:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48809.log 2024-05-11T23:05:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50068.log 2024-05-11T23:05:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.47890.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47890.log.gz 2024-05-11T23:05:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.48809.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48809.log.gz 2024-05-11T23:05:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.50068.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50068.log.gz 2024-05-11T23:05:15.736 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-11T23:05:15.921 INFO:teuthology.orchestra.run.smithi005.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-11T23:05:16.773 INFO:teuthology.orchestra.run.smithi005.stderr: 94.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-11T23:05:17.138 INFO:teuthology.orchestra.run.smithi115.stderr: 94.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-11T23:05:19.094 INFO:teuthology.orchestra.run.smithi005.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-11T23:05:20.843 INFO:teuthology.orchestra.run.smithi115.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-11T23:05:20.892 INFO:teuthology.orchestra.run.smithi005.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-11T23:05:20.949 INFO:teuthology.orchestra.run.smithi005.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-11T23:05:20.951 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-05-11T23:05:20.951 INFO:teuthology.orchestra.run.smithi005.stderr:real 0m5.493s 2024-05-11T23:05:20.951 INFO:teuthology.orchestra.run.smithi005.stderr:user 0m15.947s 2024-05-11T23:05:20.951 INFO:teuthology.orchestra.run.smithi005.stderr:sys 0m0.741s 2024-05-11T23:05:21.145 INFO:teuthology.orchestra.run.smithi115.stderr: 95.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-11T23:05:23.198 INFO:teuthology.orchestra.run.smithi115.stderr: 95.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-11T23:05:23.200 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-05-11T23:05:23.200 INFO:teuthology.orchestra.run.smithi115.stderr:real 0m7.707s 2024-05-11T23:05:23.200 INFO:teuthology.orchestra.run.smithi115.stderr:user 0m19.633s 2024-05-11T23:05:23.200 INFO:teuthology.orchestra.run.smithi115.stderr:sys 0m0.798s 2024-05-11T23:05:23.200 INFO:tasks.ceph:Archiving logs... 2024-05-11T23:05:23.201 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-11_21:16:03-rbd-squid-distro-default-smithi/7702761/remote/smithi005/log 2024-05-11T23:05:23.202 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-11T23:05:25.290 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-11_21:16:03-rbd-squid-distro-default-smithi/7702761/remote/smithi115/log 2024-05-11T23:05:25.291 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-11T23:05:27.719 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-11T23:05:27.728 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-11T23:05:27.728 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-11T23:05:27.731 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-11T23:05:27.824 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-11T23:05:27.834 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-11T23:05:27.834 DEBUG:teuthology.orchestra.run.smithi005:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T23:05:27.837 DEBUG:teuthology.orchestra.run.smithi115:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T23:05:27.852 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-05-11T23:05:27.853 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-05-11T23:05:27.857 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T23:05:27.858 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-05-11T23:05:27.858 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 63 +497us[ +490us] +/- 69ms 2024-05-11T23:05:27.858 INFO:teuthology.orchestra.run.smithi005.stdout:^* hv02.front.sepia.ceph.com 3 6 377 61 -1447us[-1455us] +/- 68ms 2024-05-11T23:05:27.858 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 +947us[ +940us] +/- 68ms 2024-05-11T23:05:27.858 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T23:05:27.859 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T23:05:27.859 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-05-11T23:05:27.860 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 61 +513us[ +513us] +/- 69ms 2024-05-11T23:05:27.860 INFO:teuthology.orchestra.run.smithi115.stdout:^* hv02.front.sepia.ceph.com 3 6 377 61 -1442us[-1441us] +/- 68ms 2024-05-11T23:05:27.860 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 +957us[ +957us] +/- 68ms 2024-05-11T23:05:27.860 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T23:05:27.860 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-11T23:05:27.869 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-11T23:05:27.870 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-11T23:05:27.880 DEBUG:teuthology.orchestra.run.smithi005:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-11T23:05:27.987 DEBUG:teuthology.orchestra.run.smithi115:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-11T23:05:28.064 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T23:05:28.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T23:05:28.096 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T23:05:28.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T23:05:28.126 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-11T23:05:28.136 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-11T23:05:28.145 INFO:teuthology.task.internal:Duration was 699.450253 seconds 2024-05-11T23:05:28.146 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-11T23:05:28.177 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-11T23:05:28.177 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-11T23:05:28.180 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-11T23:05:28.220 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T23:05:28.220 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T23:05:28.556 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-11T23:05:28.556 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi005.front.sepia.ceph.com 2024-05-11T23:05:28.556 DEBUG:teuthology.orchestra.run.smithi005:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-11T23:05:28.619 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi115.front.sepia.ceph.com 2024-05-11T23:05:28.620 DEBUG:teuthology.orchestra.run.smithi115:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-11T23:05:28.647 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-11T23:05:28.647 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-11T23:05:28.662 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-11T23:05:28.713 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-11T23:05:28.714 DEBUG:teuthology.orchestra.run.smithi005:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-11T23:05:28.731 DEBUG:teuthology.orchestra.run.smithi115:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-11T23:05:28.868 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-11T23:05:28.881 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-11T23:05:28.881 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-11T23:05:28.907 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-11T23:05:28.932 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-11T23:05:28.942 DEBUG:teuthology.orchestra.run.smithi005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-11T23:05:28.949 DEBUG:teuthology.orchestra.run.smithi115:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-11T23:05:28.971 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = core 2024-05-11T23:05:28.996 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = core 2024-05-11T23:05:29.020 DEBUG:teuthology.orchestra.run.smithi005:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-11T23:05:29.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T23:05:29.042 DEBUG:teuthology.orchestra.run.smithi115:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-11T23:05:29.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T23:05:29.075 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-11T23:05:29.088 INFO:teuthology.task.internal:Transferring archived files... 2024-05-11T23:05:29.089 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-11_21:16:03-rbd-squid-distro-default-smithi/7702761/remote/smithi005 2024-05-11T23:05:29.089 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-11T23:05:29.131 DEBUG:teuthology.misc:Transferring archived files from smithi115:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-11_21:16:03-rbd-squid-distro-default-smithi/7702761/remote/smithi115 2024-05-11T23:05:29.132 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-11T23:05:29.166 INFO:teuthology.task.internal:Removing archive directory... 2024-05-11T23:05:29.166 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-11T23:05:29.169 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-11T23:05:29.216 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-11T23:05:29.229 INFO:teuthology.task.internal:Not uploading archives. 2024-05-11T23:05:29.229 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-11T23:05:29.240 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-11T23:05:29.240 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-11T23:05:29.242 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-11T23:05:29.255 INFO:teuthology.orchestra.run.smithi005.stdout: 266037 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 11 23:05 /home/ubuntu/cephtest 2024-05-11T23:05:29.271 INFO:teuthology.orchestra.run.smithi115.stdout: 265307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 11 23:05 /home/ubuntu/cephtest 2024-05-11T23:05:29.272 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-11T23:05:29.283 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-11T23:05:29.312 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writearound} duration: 699.4502534866333 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-05-11T23:05:29.312 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-11T23:05:29.393 INFO:teuthology.run:pass